* [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage
@ 2020-12-08 9:51 Muchun Song
2020-12-08 18:20 ` Shakeel Butt
2020-12-09 2:21 ` Roman Gushchin
0 siblings, 2 replies; 7+ messages in thread
From: Muchun Song @ 2020-12-08 9:51 UTC (permalink / raw)
To: hannes, mhocko, vdavydov.dev, akpm, shakeelb, guro, sfr, chris,
laoar.shao, richard.weiyang
Cc: linux-kernel, cgroups, linux-mm, Muchun Song
The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32
of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat
to optimize memory usage.
The size of struct lruvec_stat is 304 bytes on 64 bits system. As it
is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu
bytes per-memcg per-node where ncpu is the number of the possible CPU.
If there are c memory cgroup (include dying cgroup) and n NUMA node in
the system. Finally, we can save (152 * ncpu * c * n) bytes.
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
---
Changes in v1 -> v2:
- Update the commit log to point out how many bytes that we can save.
include/linux/memcontrol.h | 6 +++++-
mm/memcontrol.c | 10 +++++++++-
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 3febf64d1b80..290d6ec8535a 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -92,6 +92,10 @@ struct lruvec_stat {
long count[NR_VM_NODE_STAT_ITEMS];
};
+struct per_cpu_lruvec_stat {
+ s32 count[NR_VM_NODE_STAT_ITEMS];
+};
+
/*
* Bitmap of shrinker::id corresponding to memcg-aware shrinkers,
* which have elements charged to this memcg.
@@ -111,7 +115,7 @@ struct mem_cgroup_per_node {
struct lruvec_stat __percpu *lruvec_stat_local;
/* Subtree VM stats (batched updates) */
- struct lruvec_stat __percpu *lruvec_stat_cpu;
+ struct per_cpu_lruvec_stat __percpu *lruvec_stat_cpu;
atomic_long_t lruvec_stat[NR_VM_NODE_STAT_ITEMS];
unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS];
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index eec44918d373..da6dc6ca388d 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -5198,7 +5198,7 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node)
return 1;
}
- pn->lruvec_stat_cpu = alloc_percpu_gfp(struct lruvec_stat,
+ pn->lruvec_stat_cpu = alloc_percpu_gfp(struct per_cpu_lruvec_stat,
GFP_KERNEL_ACCOUNT);
if (!pn->lruvec_stat_cpu) {
free_percpu(pn->lruvec_stat_local);
@@ -7089,6 +7089,14 @@ static int __init mem_cgroup_init(void)
{
int cpu, node;
+ /*
+ * Currently s32 type (can refer to struct per_cpu_lruvec_stat) is
+ * used for per-memcg-per-cpu caching of per-node statistics. In order
+ * to work fine, we should make sure that the overfill threshold can't
+ * exceed S32_MAX / PAGE_SIZE.
+ */
+ BUILD_BUG_ON(MEMCG_CHARGE_BATCH > S32_MAX / PAGE_SIZE);
+
cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL,
memcg_hotplug_cpu_dead);
--
2.11.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage
2020-12-08 9:51 [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage Muchun Song
@ 2020-12-08 18:20 ` Shakeel Butt
2020-12-09 2:21 ` Roman Gushchin
1 sibling, 0 replies; 7+ messages in thread
From: Shakeel Butt @ 2020-12-08 18:20 UTC (permalink / raw)
To: Muchun Song
Cc: Johannes Weiner, Michal Hocko, Vladimir Davydov, Andrew Morton,
Roman Gushchin, Stephen Rothwell, Chris Down, Yafang Shao,
Wei Yang, LKML, Cgroups, Linux MM
On Tue, Dec 8, 2020 at 1:53 AM Muchun Song <songmuchun@bytedance.com> wrote:
>
> The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32
> of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat
> to optimize memory usage.
>
> The size of struct lruvec_stat is 304 bytes on 64 bits system. As it
> is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu
> bytes per-memcg per-node where ncpu is the number of the possible CPU.
> If there are c memory cgroup (include dying cgroup) and n NUMA node in
> the system. Finally, we can save (152 * ncpu * c * n) bytes.
>
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Few nits below:
Reviewed-by: Shakeel Butt <shakeelb@google.com>
> ---
> Changes in v1 -> v2:
> - Update the commit log to point out how many bytes that we can save.
>
> include/linux/memcontrol.h | 6 +++++-
> mm/memcontrol.c | 10 +++++++++-
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 3febf64d1b80..290d6ec8535a 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -92,6 +92,10 @@ struct lruvec_stat {
> long count[NR_VM_NODE_STAT_ITEMS];
> };
>
> +struct per_cpu_lruvec_stat {
lruvec_stat is also per-cpu, so the name per_cpu_lruvec_stat does not
really tell why it is different from lruvec. Maybe name is
batched_lruvec_stat or something else.
> + s32 count[NR_VM_NODE_STAT_ITEMS];
> +};
> +
> /*
> * Bitmap of shrinker::id corresponding to memcg-aware shrinkers,
> * which have elements charged to this memcg.
> @@ -111,7 +115,7 @@ struct mem_cgroup_per_node {
> struct lruvec_stat __percpu *lruvec_stat_local;
A comment for the above why it still needs to be lruvec_stat.
>
> /* Subtree VM stats (batched updates) */
> - struct lruvec_stat __percpu *lruvec_stat_cpu;
> + struct per_cpu_lruvec_stat __percpu *lruvec_stat_cpu;
> atomic_long_t lruvec_stat[NR_VM_NODE_STAT_ITEMS];
>
> unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS];
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index eec44918d373..da6dc6ca388d 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -5198,7 +5198,7 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node)
> return 1;
> }
>
> - pn->lruvec_stat_cpu = alloc_percpu_gfp(struct lruvec_stat,
> + pn->lruvec_stat_cpu = alloc_percpu_gfp(struct per_cpu_lruvec_stat,
> GFP_KERNEL_ACCOUNT);
> if (!pn->lruvec_stat_cpu) {
> free_percpu(pn->lruvec_stat_local);
> @@ -7089,6 +7089,14 @@ static int __init mem_cgroup_init(void)
> {
> int cpu, node;
>
> + /*
> + * Currently s32 type (can refer to struct per_cpu_lruvec_stat) is
> + * used for per-memcg-per-cpu caching of per-node statistics. In order
> + * to work fine, we should make sure that the overfill threshold can't
> + * exceed S32_MAX / PAGE_SIZE.
> + */
> + BUILD_BUG_ON(MEMCG_CHARGE_BATCH > S32_MAX / PAGE_SIZE);
> +
> cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL,
> memcg_hotplug_cpu_dead);
>
> --
> 2.11.0
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage
2020-12-08 9:51 [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage Muchun Song
2020-12-08 18:20 ` Shakeel Butt
@ 2020-12-09 2:21 ` Roman Gushchin
2020-12-09 2:28 ` Roman Gushchin
2020-12-09 2:31 ` [External] " Muchun Song
1 sibling, 2 replies; 7+ messages in thread
From: Roman Gushchin @ 2020-12-09 2:21 UTC (permalink / raw)
To: Muchun Song
Cc: hannes, mhocko, vdavydov.dev, akpm, shakeelb, sfr, chris,
laoar.shao, richard.weiyang, linux-kernel, cgroups, linux-mm
On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote:
> The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32
> of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat
> to optimize memory usage.
>
> The size of struct lruvec_stat is 304 bytes on 64 bits system. As it
> is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu
> bytes per-memcg per-node where ncpu is the number of the possible CPU.
> If there are c memory cgroup (include dying cgroup) and n NUMA node in
> the system. Finally, we can save (152 * ncpu * c * n) bytes.
Honestly, I'm not convinced.
Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory.
If the machine has 128Gb of RAM, it's .000000003%.
Using longs (s64) allows not to think too much about overflows
and can also be slightly faster on 64-bit machines.
Thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage
2020-12-09 2:21 ` Roman Gushchin
@ 2020-12-09 2:28 ` Roman Gushchin
2020-12-09 2:31 ` [External] " Muchun Song
1 sibling, 0 replies; 7+ messages in thread
From: Roman Gushchin @ 2020-12-09 2:28 UTC (permalink / raw)
To: Muchun Song
Cc: hannes, mhocko, vdavydov.dev, akpm, shakeelb, sfr, chris,
laoar.shao, richard.weiyang, linux-kernel, cgroups, linux-mm
On Tue, Dec 08, 2020 at 06:21:18PM -0800, Roman Gushchin wrote:
> On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote:
> > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32
> > of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat
> > to optimize memory usage.
> >
> > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it
> > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu
> > bytes per-memcg per-node where ncpu is the number of the possible CPU.
> > If there are c memory cgroup (include dying cgroup) and n NUMA node in
> > the system. Finally, we can save (152 * ncpu * c * n) bytes.
>
> Honestly, I'm not convinced.
> Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory.
> If the machine has 128Gb of RAM, it's .000000003%.
My bad, it's actually
(32*2*500*152)/(128*1024*1024*1024) = .0035%
but it's still in the noise.
>
> Using longs (s64) allows not to think too much about overflows
> and can also be slightly faster on 64-bit machines.
>
> Thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [External] Re: [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage
2020-12-09 2:21 ` Roman Gushchin
2020-12-09 2:28 ` Roman Gushchin
@ 2020-12-09 2:31 ` Muchun Song
2020-12-09 3:52 ` Roman Gushchin
1 sibling, 1 reply; 7+ messages in thread
From: Muchun Song @ 2020-12-09 2:31 UTC (permalink / raw)
To: Roman Gushchin
Cc: Johannes Weiner, Michal Hocko, Vladimir Davydov, Andrew Morton,
Shakeel Butt, Stephen Rothwell, Chris Down, Yafang Shao,
richard.weiyang, LKML, Cgroups, Linux Memory Management List
On Wed, Dec 9, 2020 at 10:21 AM Roman Gushchin <guro@fb.com> wrote:
>
> On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote:
> > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32
> > of lruvec_stat_cpu is enough. And introduce struct per_cpu_lruvec_stat
> > to optimize memory usage.
> >
> > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it
> > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu
> > bytes per-memcg per-node where ncpu is the number of the possible CPU.
> > If there are c memory cgroup (include dying cgroup) and n NUMA node in
> > the system. Finally, we can save (152 * ncpu * c * n) bytes.
>
> Honestly, I'm not convinced.
> Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory.
> If the machine has 128Gb of RAM, it's .000000003%.
Hi Roman,
When the cpu hotplug is enabled, the ncpu can be 256 on
some configurations. Also, the c can be more large when
there are many dying cgroup in the system.
So the savings depends on the environment and
configurations. Right?
>
> Using longs (s64) allows not to think too much about overflows
> and can also be slightly faster on 64-bit machines.
>
> Thanks!
--
Yours,
Muchun
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [External] Re: [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage
2020-12-09 2:31 ` [External] " Muchun Song
@ 2020-12-09 3:52 ` Roman Gushchin
2020-12-09 7:05 ` Muchun Song
0 siblings, 1 reply; 7+ messages in thread
From: Roman Gushchin @ 2020-12-09 3:52 UTC (permalink / raw)
To: Muchun Song
Cc: Johannes Weiner, Michal Hocko, Vladimir Davydov, Andrew Morton,
Shakeel Butt, Stephen Rothwell, Chris Down, Yafang Shao,
richard.weiyang, LKML, Cgroups, Linux Memory Management List
On Wed, Dec 09, 2020 at 10:31:55AM +0800, Muchun Song wrote:
> On Wed, Dec 9, 2020 at 10:21 AM Roman Gushchin <guro@fb.com> wrote:
> >
> > On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote:
> > > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32
> > > of lruvec_stat_cpu is enough.
Actually the threshold can be as big as MEMCG_CHARGE_BATCH * PAGE_SIZE.
It still fits into s32, but without explicitly saying it it's hard to
understand why not choosing s8, as in vmstat.c.
> > >
> > > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it
> > > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu
> > > bytes per-memcg per-node where ncpu is the number of the possible CPU.
> > > If there are c memory cgroup (include dying cgroup) and n NUMA node in
> > > the system. Finally, we can save (152 * ncpu * c * n) bytes.
> >
> > Honestly, I'm not convinced.
> > Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory.
> > If the machine has 128Gb of RAM, it's .000000003%.
>
> Hi Roman,
>
> When the cpu hotplug is enabled, the ncpu can be 256 on
> some configurations. Also, the c can be more large when
> there are many dying cgroup in the system.
>
> So the savings depends on the environment and
> configurations. Right?
Of course, but machines with more CPUs tend to have more RAM as well.
Thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [External] Re: [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage
2020-12-09 3:52 ` Roman Gushchin
@ 2020-12-09 7:05 ` Muchun Song
0 siblings, 0 replies; 7+ messages in thread
From: Muchun Song @ 2020-12-09 7:05 UTC (permalink / raw)
To: Roman Gushchin
Cc: Johannes Weiner, Michal Hocko, Vladimir Davydov, Andrew Morton,
Shakeel Butt, Stephen Rothwell, Chris Down, Yafang Shao,
richard.weiyang, LKML, Cgroups, Linux Memory Management List
On Wed, Dec 9, 2020 at 11:52 AM Roman Gushchin <guro@fb.com> wrote:
>
> On Wed, Dec 09, 2020 at 10:31:55AM +0800, Muchun Song wrote:
> > On Wed, Dec 9, 2020 at 10:21 AM Roman Gushchin <guro@fb.com> wrote:
> > >
> > > On Tue, Dec 08, 2020 at 05:51:32PM +0800, Muchun Song wrote:
> > > > The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), so the type of s32
> > > > of lruvec_stat_cpu is enough.
>
> Actually the threshold can be as big as MEMCG_CHARGE_BATCH * PAGE_SIZE.
> It still fits into s32, but without explicitly saying it it's hard to
> understand why not choosing s8, as in vmstat.c.
Yeah, here I need to update the commit log.
>
> > > >
> > > > The size of struct lruvec_stat is 304 bytes on 64 bits system. As it
> > > > is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu
> > > > bytes per-memcg per-node where ncpu is the number of the possible CPU.
> > > > If there are c memory cgroup (include dying cgroup) and n NUMA node in
> > > > the system. Finally, we can save (152 * ncpu * c * n) bytes.
> > >
> > > Honestly, I'm not convinced.
> > > Say, ncpu = 32, n = 2, c = 500. We're saving <5Mb of memory.
> > > If the machine has 128Gb of RAM, it's .000000003%.
> >
> > Hi Roman,
> >
> > When the cpu hotplug is enabled, the ncpu can be 256 on
> > some configurations. Also, the c can be more large when
> > there are many dying cgroup in the system.
> >
> > So the savings depends on the environment and
> > configurations. Right?
>
> Of course, but machines with more CPUs tend to have more RAM as well.
Here I mean possible CPU not online CPU. The number of possible
CPUs may be greater than online CPUs. The per-cpu allocator is based
on the number of possible CPUs. Right?
Thanks.
>
> Thanks!
--
Yours,
Muchun
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-12-09 7:06 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-08 9:51 [PATCH v2] mm: memcontrol: optimize per-lruvec stats counter memory usage Muchun Song
2020-12-08 18:20 ` Shakeel Butt
2020-12-09 2:21 ` Roman Gushchin
2020-12-09 2:28 ` Roman Gushchin
2020-12-09 2:31 ` [External] " Muchun Song
2020-12-09 3:52 ` Roman Gushchin
2020-12-09 7:05 ` Muchun Song
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).