LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Yu Zhao <yuzhao@google.com>
To: Konstantin Kharlamov <hi-angel@yandex.ru>
Cc: Linux-MM <linux-mm@kvack.org>, Alex Shi <alexs@kernel.org>,
	Andi Kleen <ak@linux.intel.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Benjamin Manes <ben.manes@gmail.com>,
	Dave Chinner <david@fromorbit.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Hillf Danton <hdanton@sina.com>, Jens Axboe <axboe@kernel.dk>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Joonsoo Kim <iamjoonsoo.kim@lge.com>,
	Matthew Wilcox <willy@infradead.org>,
	Mel Gorman <mgorman@suse.de>, Miaohe Lin <linmiaohe@huawei.com>,
	Michael Larabel <michael@michaellarabel.com>,
	Michal Hocko <mhocko@suse.com>,
	Michel Lespinasse <michel@lespinasse.org>,
	Rik van Riel <riel@surriel.com>, Roman Gushchin <guro@fb.com>,
	Rong Chen <rong.a.chen@intel.com>,
	SeongJae Park <sjpark@amazon.de>,
	Tim Chen <tim.c.chen@linux.intel.com>,
	Vlastimil Babka <vbabka@suse.cz>, Yang Shi <shy828301@gmail.com>,
	Ying Huang <ying.huang@intel.com>, Zi Yan <ziy@nvidia.com>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	lkp@lists.01.org,
	Kernel Page Reclaim v2 <page-reclaim@google.com>
Subject: Re: [PATCH v2 00/16] Multigenerational LRU Framework
Date: Fri, 30 Apr 2021 13:31:57 -0600
Message-ID: <CAOUHufaoGe1=Vigp7cHbvx0G1=zvHQrNc_et4C9UTw4bYOSY8Q@mail.gmail.com> (raw)
In-Reply-To: <8ce5be3df2137e975d7333024b6120b71b214617.camel@yandex.ru>

On Fri, Apr 30, 2021 at 12:38 AM Konstantin Kharlamov
<hi-angel@yandex.ru> wrote:
>
> Btw, I noticed a fun thing, an improvement. I don't know yet if it can be
> attributed to 5.12 (which I didn't try alone yet) or to the LRU patchset, but
> I'd assume the latter, because 5.12 seems didn't to have had anything
> interesting regarding memory performance¹.

I appreciate the testing and the report. They mean a lot to us.

This improvement is to be expected, and it works both ways. There are
cases that swapping is not a good idea, for example, when building
large repos. Without this patchset, some of my browser memory usually
gets swapped out while tons of memory is used to cache files I don't
really care about.

I completely agree with you on the memory cgroup part: theoretically
it could work around the problem but nobody knows how much memory to
reserve for Skype or Firefox :)

I will keep you posted on the following developments.

Thanks!

> I usually have Skype running in background for work purposes, which is only used
> 2-3 times in a week. So one would expect it to be one the first victims to
> memory reclaim. Unfortunately, I never seen this to actually happen (till now,
> that is): all skypeforlinux processes routinely have 0 bytes in SWAP, and the
> only circumstances under which its processes can get into SWAP is after
> experiencing many SWAP-storms. It was so hard for the kernel to move these
> unused processes to SWAP that at some point I even tried to research if there
> are any odd flags a userspace may have set on a process to keep it in RAM, just
> in case that's what happens to Skype (A: no, that wasn't the case, running Skype
> in a memory limited cgroup makes it swap. It's just that kernel decision were
> lacking for some reason).
>
> So, anyway, I am delighted to see now that while testing this patchset, and
> without encountering even a single SWAP-storm yet, skypeforlinux are one of the
> processes residing in SWAP!!
>
>      λ smem -kc "name user pid pss swap" | grep skype
>     skypeforlinux            constantine  1151    60.0K     7.5M
>     skypeforlinux            constantine  1215   195.0K     8.1M
>     skypeforlinux            constantine  1149   706.0K     7.5M
>     skypeforlinux            constantine  1148   743.0K     7.3M
>     skypeforlinux            constantine  1307     1.4M     8.0M
>     skypeforlinux            constantine  1213     2.1M    46.1M
>     skypeforlinux            constantine  1206    14.0M    10.8M
>     skypeforlinux            constantine   818    38.5M    34.3M
>     skypeforlinux            constantine  1242   103.2M    46.8M
>
> !!!
>
> 1: https://kernelnewbies.org/Linux_5.12#Memory_management
>
> On Fri, 2021-04-30 at 02:46 +0300, Konstantin Kharlamov wrote:
> > In case you need it yet, this series is:
> >
> > Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
> >
> > My success story: I have Archlinux with 8G RAM + zswap + swap. While developing,
> > I have lots of apps opened such as multiple LSP-servers for different langs,
> > chats, two browsers, etc… Usually, my system gets quickly to a point of SWAP-
> > storms, where I have to kill LSP-servers, restart browsers to free memory, etc,
> > otherwise the system lags heavily and is barely usable.
> >
> > 1.5 day ago I migrated from 5.11.15 kernel to 5.12 + the LRU patchset, and I
> > started up by opening lots of apps to create memory pressure, and worked for a
> > day like this. Till now I had *not a single SWAP-storm*, and mind you I got 3.4G
> > in SWAP. I was never getting to the point of 3G in SWAP before without a single
> > SWAP-storm.
> >
> > Right now my gf on Fedora 33 also suffers from SWAP-storms on her old Macbook
> > 2013 with 4G RAM + zswap + swap, I think the next week I'll build for her 5.12 +
> > LRU patchset as well. Will see how it goes, I expect it will improve her
> > experience by a lot too.
> >
> > P.S.: upon replying please keep me CCed, I'm not subscribed to the list
>

      reply index

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-13  6:56 Yu Zhao
2021-04-13  6:56 ` [PATCH v2 01/16] include/linux/memcontrol.h: do not warn in page_memcg_rcu() if !CONFIG_MEMCG Yu Zhao
2021-04-13  6:56 ` [PATCH v2 02/16] include/linux/nodemask.h: define next_memory_node() if !CONFIG_NUMA Yu Zhao
2021-04-13  6:56 ` [PATCH v2 03/16] include/linux/huge_mm.h: define is_huge_zero_pmd() if !CONFIG_TRANSPARENT_HUGEPAGE Yu Zhao
2021-04-13  6:56 ` [PATCH v2 04/16] include/linux/cgroup.h: export cgroup_mutex Yu Zhao
2021-04-13  6:56 ` [PATCH v2 05/16] mm/swap.c: export activate_page() Yu Zhao
2021-04-13  6:56 ` [PATCH v2 06/16] mm, x86: support the access bit on non-leaf PMD entries Yu Zhao
2021-04-13  6:56 ` [PATCH v2 07/16] mm/vmscan.c: refactor shrink_node() Yu Zhao
2021-04-13  6:56 ` [PATCH v2 08/16] mm: multigenerational lru: groundwork Yu Zhao
2021-04-13  6:56 ` [PATCH v2 09/16] mm: multigenerational lru: activation Yu Zhao
2021-04-13  6:56 ` [PATCH v2 10/16] mm: multigenerational lru: mm_struct list Yu Zhao
2021-04-14 14:36   ` Matthew Wilcox
2021-04-13  6:56 ` [PATCH v2 11/16] mm: multigenerational lru: aging Yu Zhao
2021-04-13  6:56 ` [PATCH v2 12/16] mm: multigenerational lru: eviction Yu Zhao
2021-04-13  6:56 ` [PATCH v2 13/16] mm: multigenerational lru: page reclaim Yu Zhao
2021-04-13  6:56 ` [PATCH v2 14/16] mm: multigenerational lru: user interface Yu Zhao
2021-04-13  6:56 ` [PATCH v2 15/16] mm: multigenerational lru: Kconfig Yu Zhao
2021-04-13  6:56 ` [PATCH v2 16/16] mm: multigenerational lru: documentation Yu Zhao
2021-04-13  7:51 ` [PATCH v2 00/16] Multigenerational LRU Framework SeongJae Park
2021-04-13 16:13   ` Jens Axboe
2021-04-13 16:42     ` SeongJae Park
2021-04-13 23:14     ` Dave Chinner
2021-04-14  2:29       ` Rik van Riel
     [not found]         ` <CAOUHufafMcaG8sOS=1YMy2P_6p0R1FzP16bCwpUau7g1-PybBQ@mail.gmail.com>
2021-04-14  6:15           ` Huang, Ying
2021-04-14  7:58             ` Yu Zhao
2021-04-14  8:27               ` Huang, Ying
2021-04-14 13:51                 ` Rik van Riel
2021-04-14 15:56                   ` Andi Kleen
2021-04-14 15:58                   ` [page-reclaim] " Shakeel Butt
2021-04-14 18:45                   ` Yu Zhao
2021-04-14 15:51           ` Andi Kleen
2021-04-14 15:58             ` Rik van Riel
2021-04-14 19:14               ` Yu Zhao
2021-04-14 19:41                 ` Rik van Riel
2021-04-14 20:08                   ` Yu Zhao
2021-04-14 19:04             ` Yu Zhao
2021-04-15  3:00               ` Andi Kleen
2021-04-15  7:13                 ` Yu Zhao
2021-04-15  8:19                   ` Huang, Ying
2021-04-15  9:57                   ` Michel Lespinasse
2021-04-24  2:33                     ` Yu Zhao
2021-04-24  3:30                       ` Andi Kleen
2021-04-24  4:16                         ` Yu Zhao
2021-04-14  3:40       ` Yu Zhao
2021-04-14  4:50         ` Dave Chinner
2021-04-14  7:16           ` Yu Zhao
2021-04-14 10:00             ` Yu Zhao
2021-04-15  1:36             ` Dave Chinner
2021-04-24 21:21               ` Yu Zhao
2021-04-14 14:43       ` Jens Axboe
2021-04-14 19:42         ` Yu Zhao
2021-04-15  1:21         ` Dave Chinner
2021-04-14 17:43 ` Johannes Weiner
2021-04-27 10:35   ` Yu Zhao
2021-04-29 23:46 ` Konstantin Kharlamov
2021-04-30  6:37   ` Konstantin Kharlamov
2021-04-30 19:31     ` Yu Zhao [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAOUHufaoGe1=Vigp7cHbvx0G1=zvHQrNc_et4C9UTw4bYOSY8Q@mail.gmail.com' \
    --to=yuzhao@google.com \
    --cc=ak@linux.intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexs@kernel.org \
    --cc=axboe@kernel.dk \
    --cc=ben.manes@gmail.com \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@linux.intel.com \
    --cc=david@fromorbit.com \
    --cc=guro@fb.com \
    --cc=hannes@cmpxchg.org \
    --cc=hdanton@sina.com \
    --cc=hi-angel@yandex.ru \
    --cc=iamjoonsoo.kim@lge.com \
    --cc=linmiaohe@huawei.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lkp@lists.01.org \
    --cc=mgorman@suse.de \
    --cc=mhocko@suse.com \
    --cc=michael@michaellarabel.com \
    --cc=michel@lespinasse.org \
    --cc=page-reclaim@google.com \
    --cc=riel@surriel.com \
    --cc=rong.a.chen@intel.com \
    --cc=shy828301@gmail.com \
    --cc=sjpark@amazon.de \
    --cc=tim.c.chen@linux.intel.com \
    --cc=vbabka@suse.cz \
    --cc=willy@infradead.org \
    --cc=ying.huang@intel.com \
    --cc=ziy@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git