From: Johannes Weiner <hannes@cmpxchg.org> To: Joonsoo Kim <js1304@gmail.com>, Alex Shi <alex.shi@linux.alibaba.com> Cc: Shakeel Butt <shakeelb@google.com>, Hugh Dickins <hughd@google.com>, Michal Hocko <mhocko@suse.com>, "Kirill A. Shutemov" <kirill@shutemov.name>, Roman Gushchin <guro@fb.com>, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 00/18] mm: memcontrol: charge swapin pages on instantiation Date: Mon, 20 Apr 2020 18:11:08 -0400 [thread overview] Message-ID: <20200420221126.341272-1-hannes@cmpxchg.org> (raw) This patch series reworks memcg to charge swapin pages directly at swapin time, rather than at fault time, which may be much later, or not happen at all. The delayed charging scheme we have right now causes problems: - Alex's per-cgroup lru_lock patches rely on pages that have been isolated from the LRU to have a stable page->mem_cgroup; otherwise the lock may change underneath him. Swapcache pages are charged only after they are added to the LRU, and charging doesn't follow the LRU isolation protocol. - Joonsoo's anon workingset patches need a suitable LRU at the time the page enters the swap cache and displaces the non-resident info. But the correct LRU is only available after charging. - It's a containment hole / DoS vector. Users can trigger arbitrarily large swap readahead using MADV_WILLNEED. The memory is never charged unless somebody actually touches it. - It complicates the page->mem_cgroup stabilization rules In order to charge pages directly at swapin time, the memcg code base needs to be prepared, and several overdue cleanups become a necessity: To charge pages at swapin time, we need to always have cgroup ownership tracking of swap records. We also cannot rely on page->mapping to tell apart page types at charge time, because that's only set up during a page fault. To eliminate the page->mapping dependency, memcg needs to ditch its private page type counters (MEMCG_CACHE, MEMCG_RSS, NR_SHMEM) in favor of the generic vmstat counters and accounting sites, such as NR_FILE_PAGES, NR_ANON_MAPPED etc. To switch to generic vmstat counters, the charge sequence must be adjusted such that page->mem_cgroup is set up by the time these counters are modified. The series is structured as follows: 1. Bug fixes 2. Decoupling charging from rmap 3. Swap controller integration into memcg 4. Direct swapin charging The patches survive a simple swapout->swapin test inside a virtual machine. Because this is blocking two major patch sets, I'm sending these out early and will continue testing in parallel to the review. include/linux/memcontrol.h | 53 +---- include/linux/mm.h | 4 +- include/linux/swap.h | 6 +- init/Kconfig | 17 +- kernel/events/uprobes.c | 10 +- mm/filemap.c | 43 ++--- mm/huge_memory.c | 45 ++--- mm/khugepaged.c | 25 +-- mm/memcontrol.c | 448 ++++++++++++++----------------------------- mm/memory.c | 51 ++--- mm/migrate.c | 20 +- mm/rmap.c | 53 +++-- mm/shmem.c | 117 +++++------ mm/swap_cgroup.c | 6 - mm/swap_state.c | 89 +++++---- mm/swapfile.c | 25 +-- mm/userfaultfd.c | 5 +- 17 files changed, 367 insertions(+), 650 deletions(-)
WARNING: multiple messages have this Message-ID (diff)
From: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org> To: Joonsoo Kim <js1304-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Alex Shi <alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org> Cc: Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>, "Kirill A. Shutemov" <kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org>, Roman Gushchin <guro-b10kYP2dOMg@public.gmane.org>, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, kernel-team-b10kYP2dOMg@public.gmane.org Subject: [PATCH 00/18] mm: memcontrol: charge swapin pages on instantiation Date: Mon, 20 Apr 2020 18:11:08 -0400 [thread overview] Message-ID: <20200420221126.341272-1-hannes@cmpxchg.org> (raw) This patch series reworks memcg to charge swapin pages directly at swapin time, rather than at fault time, which may be much later, or not happen at all. The delayed charging scheme we have right now causes problems: - Alex's per-cgroup lru_lock patches rely on pages that have been isolated from the LRU to have a stable page->mem_cgroup; otherwise the lock may change underneath him. Swapcache pages are charged only after they are added to the LRU, and charging doesn't follow the LRU isolation protocol. - Joonsoo's anon workingset patches need a suitable LRU at the time the page enters the swap cache and displaces the non-resident info. But the correct LRU is only available after charging. - It's a containment hole / DoS vector. Users can trigger arbitrarily large swap readahead using MADV_WILLNEED. The memory is never charged unless somebody actually touches it. - It complicates the page->mem_cgroup stabilization rules In order to charge pages directly at swapin time, the memcg code base needs to be prepared, and several overdue cleanups become a necessity: To charge pages at swapin time, we need to always have cgroup ownership tracking of swap records. We also cannot rely on page->mapping to tell apart page types at charge time, because that's only set up during a page fault. To eliminate the page->mapping dependency, memcg needs to ditch its private page type counters (MEMCG_CACHE, MEMCG_RSS, NR_SHMEM) in favor of the generic vmstat counters and accounting sites, such as NR_FILE_PAGES, NR_ANON_MAPPED etc. To switch to generic vmstat counters, the charge sequence must be adjusted such that page->mem_cgroup is set up by the time these counters are modified. The series is structured as follows: 1. Bug fixes 2. Decoupling charging from rmap 3. Swap controller integration into memcg 4. Direct swapin charging The patches survive a simple swapout->swapin test inside a virtual machine. Because this is blocking two major patch sets, I'm sending these out early and will continue testing in parallel to the review. include/linux/memcontrol.h | 53 +---- include/linux/mm.h | 4 +- include/linux/swap.h | 6 +- init/Kconfig | 17 +- kernel/events/uprobes.c | 10 +- mm/filemap.c | 43 ++--- mm/huge_memory.c | 45 ++--- mm/khugepaged.c | 25 +-- mm/memcontrol.c | 448 ++++++++++++++----------------------------- mm/memory.c | 51 ++--- mm/migrate.c | 20 +- mm/rmap.c | 53 +++-- mm/shmem.c | 117 +++++------ mm/swap_cgroup.c | 6 - mm/swap_state.c | 89 +++++---- mm/swapfile.c | 25 +-- mm/userfaultfd.c | 5 +- 17 files changed, 367 insertions(+), 650 deletions(-)
next reply other threads:[~2020-04-20 22:11 UTC|newest] Thread overview: 140+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-04-20 22:11 Johannes Weiner [this message] 2020-04-20 22:11 ` [PATCH 00/18] mm: memcontrol: charge swapin pages on instantiation Johannes Weiner 2020-04-20 22:11 ` [PATCH 01/18] mm: fix NUMA node file count error in replace_page_cache() Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-21 8:28 ` Alex Shi 2020-04-21 8:28 ` Alex Shi 2020-04-21 19:13 ` Shakeel Butt 2020-04-21 19:13 ` Shakeel Butt 2020-04-21 19:13 ` Shakeel Butt 2020-04-22 6:34 ` Joonsoo Kim 2020-04-22 6:34 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 02/18] mm: memcontrol: fix theoretical race in charge moving Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-22 6:36 ` Joonsoo Kim 2020-04-22 6:36 ` Joonsoo Kim 2020-04-22 16:51 ` Shakeel Butt 2020-04-22 16:51 ` Shakeel Butt 2020-04-22 16:51 ` Shakeel Butt 2020-04-22 17:42 ` Johannes Weiner 2020-04-22 17:42 ` Johannes Weiner 2020-04-22 18:01 ` Shakeel Butt 2020-04-22 18:01 ` Shakeel Butt 2020-04-22 18:01 ` Shakeel Butt 2020-04-22 18:02 ` Shakeel Butt 2020-04-22 18:02 ` Shakeel Butt 2020-04-20 22:11 ` [PATCH 03/18] mm: memcontrol: drop @compound parameter from memcg charging API Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-21 9:11 ` Alex Shi 2020-04-21 9:11 ` Alex Shi 2020-04-22 6:37 ` Joonsoo Kim 2020-04-22 6:37 ` Joonsoo Kim 2020-04-22 17:30 ` Shakeel Butt 2020-04-22 17:30 ` Shakeel Butt 2020-04-22 17:30 ` Shakeel Butt 2020-04-20 22:11 ` [PATCH 04/18] mm: memcontrol: move out cgroup swaprate throttling Johannes Weiner 2020-04-21 9:11 ` Alex Shi 2020-04-21 9:11 ` Alex Shi 2020-04-22 6:37 ` Joonsoo Kim 2020-04-22 6:37 ` Joonsoo Kim 2020-04-22 22:20 ` Shakeel Butt 2020-04-22 22:20 ` Shakeel Butt 2020-04-22 22:20 ` Shakeel Butt 2020-04-20 22:11 ` [PATCH 05/18] mm: memcontrol: convert page cache to a new mem_cgroup_charge() API Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-21 9:12 ` Alex Shi 2020-04-21 9:12 ` Alex Shi 2020-04-22 6:40 ` Joonsoo Kim 2020-04-22 6:40 ` Joonsoo Kim 2020-04-22 12:09 ` Johannes Weiner 2020-04-22 12:09 ` Johannes Weiner 2020-04-23 5:25 ` Joonsoo Kim 2020-05-08 16:01 ` Johannes Weiner 2020-05-11 1:57 ` Joonsoo Kim 2020-05-11 7:38 ` Hugh Dickins 2020-05-11 7:38 ` Hugh Dickins 2020-05-11 7:38 ` Hugh Dickins 2020-05-11 15:06 ` Johannes Weiner 2020-05-11 16:32 ` Hugh Dickins 2020-05-11 16:32 ` Hugh Dickins 2020-05-11 16:32 ` Hugh Dickins 2020-05-11 18:10 ` Johannes Weiner 2020-05-11 18:10 ` Johannes Weiner 2020-05-11 18:12 ` Johannes Weiner 2020-05-11 18:44 ` Hugh Dickins 2020-05-11 18:44 ` Hugh Dickins 2020-05-11 18:44 ` Hugh Dickins 2020-04-20 22:11 ` [PATCH 06/18] mm: memcontrol: prepare uncharging for removal of private page type counters Johannes Weiner 2020-04-21 9:12 ` Alex Shi 2020-04-21 9:12 ` Alex Shi 2020-04-22 6:41 ` Joonsoo Kim 2020-04-22 6:41 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 07/18] mm: memcontrol: prepare move_account " Johannes Weiner 2020-04-21 9:13 ` Alex Shi 2020-04-21 9:13 ` Alex Shi 2020-04-22 6:41 ` Joonsoo Kim 2020-04-22 6:41 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 08/18] mm: memcontrol: prepare cgroup vmstat infrastructure for native anon counters Johannes Weiner 2020-04-22 6:42 ` Joonsoo Kim 2020-04-22 6:42 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 09/18] mm: memcontrol: switch to native NR_FILE_PAGES and NR_SHMEM counters Johannes Weiner 2020-04-22 6:42 ` Joonsoo Kim 2020-04-22 6:42 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 10/18] mm: memcontrol: switch to native NR_ANON_MAPPED counter Johannes Weiner 2020-04-22 6:51 ` Joonsoo Kim 2020-04-22 12:28 ` Johannes Weiner 2020-04-23 5:27 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 11/18] mm: memcontrol: switch to native NR_ANON_THPS counter Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-24 0:29 ` Joonsoo Kim 2020-04-24 0:29 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 12/18] mm: memcontrol: convert anon and file-thp to new mem_cgroup_charge() API Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-24 0:29 ` Joonsoo Kim 2020-04-24 0:29 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 13/18] mm: memcontrol: drop unused try/commit/cancel charge API Johannes Weiner 2020-04-24 0:30 ` Joonsoo Kim 2020-04-24 0:30 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 14/18] mm: memcontrol: prepare swap controller setup for integration Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-24 0:30 ` Joonsoo Kim 2020-04-24 0:30 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 15/18] mm: memcontrol: make swap tracking an integral part of memory control Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-21 9:27 ` Alex Shi 2020-04-21 9:27 ` Alex Shi 2020-04-21 14:39 ` Johannes Weiner 2020-04-21 14:39 ` Johannes Weiner 2020-04-22 3:14 ` Alex Shi 2020-04-22 3:14 ` Alex Shi 2020-04-22 13:30 ` Johannes Weiner 2020-04-22 13:30 ` Johannes Weiner 2020-04-22 13:40 ` Alex Shi 2020-04-22 13:43 ` Alex Shi 2020-04-24 0:30 ` Joonsoo Kim 2020-04-24 0:30 ` Joonsoo Kim 2020-04-24 3:01 ` Johannes Weiner 2020-04-20 22:11 ` [PATCH 16/18] mm: memcontrol: charge swapin pages on instantiation Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-21 9:21 ` Alex Shi 2020-04-21 9:21 ` Alex Shi 2020-04-24 0:44 ` Joonsoo Kim 2020-04-24 2:51 ` Johannes Weiner 2020-04-24 2:51 ` Johannes Weiner 2020-04-28 6:49 ` Joonsoo Kim 2020-04-28 6:49 ` Joonsoo Kim 2020-04-28 6:49 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 17/18] mm: memcontrol: delete unused lrucare handling Johannes Weiner 2020-04-20 22:11 ` Johannes Weiner 2020-04-24 0:46 ` Joonsoo Kim 2020-04-24 0:46 ` Joonsoo Kim 2020-04-20 22:11 ` [PATCH 18/18] mm: memcontrol: update page->mem_cgroup stability rules Johannes Weiner 2020-04-21 9:20 ` Alex Shi 2020-04-21 9:20 ` Alex Shi 2020-04-24 0:48 ` Joonsoo Kim 2020-04-24 0:48 ` Joonsoo Kim 2020-04-21 9:10 ` Hillf Danton 2020-04-21 14:34 ` Johannes Weiner 2020-04-21 14:34 ` Johannes Weiner 2020-04-21 9:32 ` [PATCH 00/18] mm: memcontrol: charge swapin pages on instantiation Alex Shi 2020-04-21 9:32 ` Alex Shi
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200420221126.341272-1-hannes@cmpxchg.org \ --to=hannes@cmpxchg.org \ --cc=alex.shi@linux.alibaba.com \ --cc=cgroups@vger.kernel.org \ --cc=guro@fb.com \ --cc=hughd@google.com \ --cc=js1304@gmail.com \ --cc=kernel-team@fb.com \ --cc=kirill@shutemov.name \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mhocko@suse.com \ --cc=shakeelb@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.