From: Yosry Ahmed <yosryahmed@google.com>
To: "Tejun Heo" <tj@kernel.org>, "Josef Bacik" <josef@toxicpanda.com>,
"Jens Axboe" <axboe@kernel.dk>,
"Zefan Li" <lizefan.x@bytedance.com>,
"Johannes Weiner" <hannes@cmpxchg.org>,
"Michal Hocko" <mhocko@kernel.org>,
"Roman Gushchin" <roman.gushchin@linux.dev>,
"Shakeel Butt" <shakeelb@google.com>,
"Muchun Song" <muchun.song@linux.dev>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Michal Koutný" <mkoutny@suse.com>
Cc: Vasily Averin <vasily.averin@linux.dev>,
cgroups@vger.kernel.org, linux-block@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
bpf@vger.kernel.org, Yosry Ahmed <yosryahmed@google.com>
Subject: [PATCH v3 0/8] memcg: avoid flushing stats atomically where possible
Date: Thu, 30 Mar 2023 19:17:53 +0000 [thread overview]
Message-ID: <20230330191801.1967435-1-yosryahmed@google.com> (raw)
rstat flushing is an expensive operation that scales with the number of
cpus and the number of cgroups in the system. The purpose of this series
is to minimize the contexts where we flush stats atomically.
Patches 1 and 2 are cleanups requested during reviews of prior versions
of this series.
Patch 3 makes sure we never try to flush from within an irq context.
Patches 4 to 7 introduce separate variants of mem_cgroup_flush_stats()
for atomic and non-atomic flushing, and make sure we only flush the
stats atomically when necessary.
Patch 8 is a slightly tangential optimization that limits the work done
by rstat flushing in some scenarios.
v2 -> v3:
- Collected more Acks (thanks everyone!).
- Dropped controversial patch 4 from v2.
- Improved commit logs and cover letter (Michal).
v2: https://lore.kernel.org/linux-mm/20230328221644.803272-1-yosryahmed@google.com/
v1 -> v2:
- Added more context in patch 4's commit log.
- Added atomic_read() before atomic_xchg() in patch 5 to avoid
needlessly locking the cache line (Shakeel).
- Refactored patch 6: added a common helper, do_flush_stats(), for
mem_cgroup_flush_stats{_atomic}() (Johannes).
- Renamed mem_cgroup_flush_stats_ratelimited() to
mem_cgroup_flush_stats_atomic_ratelimited() in patch 6. It is restored
in patch 7, but this maintains consistency (Johannes).
- Added line break to keep the lock section visually separated in patch
7 (Johannes).
v1: https://lore.kernel.org/lkml/20230328061638.203420-1-yosryahmed@google.com/
RFC -> v1:
- Dropped patch 1 that attempted to make the global rstat lock a non-irq
lock, will follow up on that separetly (Shakeel).
- Dropped stats_flush_lock entirely, replaced by an atomic (Johannes).
- Renamed cgroup_rstat_flush_irqsafe() to cgroup_rstat_flush_atomic()
instead of removing it (Johannes).
- Added a patch to rename mem_cgroup_flush_stats_delayed() to
mem_cgroup_flush_stats_ratelimited() (Johannes).
- Separate APIs for flushing memcg stats in atomic and non-atomic
contexts instead of a boolean argument (Johannes).
- Added patches 3 & 4 to make sure we never flush from irq context
(Shakeel & Johannes).
RFC: https://lore.kernel.org/lkml/20230323040037.2389095-1-yosryahmed@google.com/
Yosry Ahmed (8):
cgroup: rename cgroup_rstat_flush_"irqsafe" to "atomic"
memcg: rename mem_cgroup_flush_stats_"delayed" to "ratelimited"
memcg: do not flush stats in irq context
memcg: replace stats_flush_lock with an atomic
memcg: sleep during flushing stats in safe contexts
workingset: memcg: sleep when flushing stats in workingset_refault()
vmscan: memcg: sleep when flushing stats during reclaim
memcg: do not modify rstat tree for zero updates
include/linux/cgroup.h | 2 +-
include/linux/memcontrol.h | 9 ++++-
kernel/cgroup/rstat.c | 4 +-
mm/memcontrol.c | 78 ++++++++++++++++++++++++++++++--------
mm/workingset.c | 5 ++-
5 files changed, 76 insertions(+), 22 deletions(-)
--
2.40.0.348.gf938b09366-goog
next reply other threads:[~2023-03-30 19:18 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-30 19:17 Yosry Ahmed [this message]
2023-03-30 19:17 ` [PATCH v3 1/8] cgroup: rename cgroup_rstat_flush_"irqsafe" to "atomic" Yosry Ahmed
2023-03-30 19:17 ` Yosry Ahmed
2023-03-30 19:17 ` [PATCH v3 2/8] memcg: rename mem_cgroup_flush_stats_"delayed" to "ratelimited" Yosry Ahmed
2023-03-30 19:17 ` [PATCH v3 3/8] memcg: do not flush stats in irq context Yosry Ahmed
2023-03-30 19:17 ` Yosry Ahmed
2023-03-30 19:17 ` [PATCH v3 4/8] memcg: replace stats_flush_lock with an atomic Yosry Ahmed
2023-04-04 16:52 ` Michal Koutný
2023-04-04 16:52 ` Michal Koutný
2023-04-04 17:13 ` Shakeel Butt
2023-04-04 17:13 ` Shakeel Butt
2023-04-04 17:21 ` Shakeel Butt
2023-04-04 17:21 ` Shakeel Butt
2023-04-04 17:32 ` Michal Koutný
2023-03-30 19:17 ` [PATCH v3 5/8] memcg: sleep during flushing stats in safe contexts Yosry Ahmed
2023-03-30 19:30 ` Johannes Weiner
2023-04-04 15:09 ` Johannes Weiner
2023-03-30 19:17 ` [PATCH v3 6/8] workingset: memcg: sleep when flushing stats in workingset_refault() Yosry Ahmed
2023-04-04 16:53 ` Michal Koutný
2023-04-04 16:53 ` Michal Koutný
2023-04-04 18:09 ` Yosry Ahmed
2023-04-05 8:00 ` Michal Koutný
2023-04-05 8:00 ` Michal Koutný
2023-04-05 8:02 ` Yosry Ahmed
2023-04-05 8:02 ` Yosry Ahmed
2023-03-30 19:18 ` [PATCH v3 7/8] vmscan: memcg: sleep when flushing stats during reclaim Yosry Ahmed
2023-03-30 19:18 ` Yosry Ahmed
2023-03-30 19:18 ` [PATCH v3 8/8] memcg: do not modify rstat tree for zero updates Yosry Ahmed
2023-03-30 19:18 ` Yosry Ahmed
2023-04-04 16:53 ` Michal Koutný
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230330191801.1967435-1-yosryahmed@google.com \
--to=yosryahmed@google.com \
--cc=akpm@linux-foundation.org \
--cc=axboe@kernel.dk \
--cc=bpf@vger.kernel.org \
--cc=cgroups@vger.kernel.org \
--cc=hannes@cmpxchg.org \
--cc=josef@toxicpanda.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lizefan.x@bytedance.com \
--cc=mhocko@kernel.org \
--cc=mkoutny@suse.com \
--cc=muchun.song@linux.dev \
--cc=roman.gushchin@linux.dev \
--cc=shakeelb@google.com \
--cc=tj@kernel.org \
--cc=vasily.averin@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.