From: Roman Gushchin <roman.gushchin@linux.dev>
To: Vasily Averin <vvs@openvz.org>
Cc: "Michal Koutný" <mkoutny@suse.com>,
"Vlastimil Babka" <vbabka@suse.cz>,
"Shakeel Butt" <shakeelb@google.com>,
kernel@openvz.org, "Florian Westphal" <fw@strlen.de>,
linux-kernel@vger.kernel.org, "Michal Hocko" <mhocko@suse.com>,
cgroups@vger.kernel.org,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Tejun Heo" <tj@kernel.org>
Subject: Re: kernfs memcg accounting
Date: Wed, 11 May 2022 10:46:23 -0700 [thread overview]
Message-ID: <Ynv2b6aN5zMnJuLA@carbon> (raw)
In-Reply-To: <0eec6575-548e-23e0-0d99-4e079a33d338@openvz.org>
On Wed, May 11, 2022 at 09:01:40AM +0300, Vasily Averin wrote:
> On 5/11/22 06:06, Roman Gushchin wrote:
> > On Wed, May 04, 2022 at 12:00:18PM +0300, Vasily Averin wrote:
> >> From my point of view it is most important to account allocated memory
> >> to any cgroup inside container. Select of proper memcg is a secondary goal here.
> >> Frankly speaking I do not see a big difference between memcg of current process,
> >> memcg of newly created child and memcg of its parent.
> >>
> >> As far as I understand, Roman chose the parent memcg because it was a special
> >> case of creating a new memory group. He temporally changed active memcg
> >> in mem_cgroup_css_alloc() and properly accounted all required memcg-specific
> >> allocations.
> >
> > My primary goal was to apply the memory pressure on memory cgroups with a lot
> > of (dying) children cgroups. On a multi-cpu machine a memory cgroup structure
> > is way larger than a page, so a cgroup which looks small can be really large
> > if we calculate the amount of memory taken by all children memcg internals.
> >
> > Applying this pressure to another cgroup (e.g. the one which contains systemd)
> > doesn't help to reclaim any pages which are pinning the dying cgroups.
> >
> > For other controllers (maybe blkcg aside, idk) it shouldn't matter, because
> > there is no such problem there.
> >
> > For consistency reasons I'd suggest to charge all *large* allocations
> > (e.g. percpu) to the parent cgroup. Small allocations can be ignored.
>
> I showed in [1] other large allocation:
> "
> number bytes $1*$2 sum note call_site
> of alloc
> allocs
> ------------------------------------------------------------
> 1 14448 14448 14448 = percpu_alloc_percpu:
> 1 8192 8192 22640 ++ (mem_cgroup_css_alloc+0x54)
> 49 128 6272 28912 ++ (__kernfs_new_node+0x4e)
> 49 96 4704 33616 ? (simple_xattr_alloc+0x2c)
> 49 88 4312 37928 ++ (__kernfs_iattrs+0x56)
> 1 4096 4096 42024 ++ (cgroup_mkdir+0xc7)
> 1 3840 3840 45864 = percpu_alloc_percpu:
> 4 512 2048 47912 + (alloc_fair_sched_group+0x166)
> 4 512 2048 49960 + (alloc_fair_sched_group+0x139)
> 1 2048 2048 52008 ++ (mem_cgroup_css_alloc+0x109)
> "
> [1] https://lore.kernel.org/all/1aa4cd22-fcb6-0e8d-a1c6-23661d618864@openvz.org/
> = already accounted
> ++ to be accounted first
> + to be accounted a bit later
>
> There is no problems with objects allocated in mem_cgroup_alloc(),
> they will be accounted to parent's memcg.
> However I do not understand how to handle other large objects?
>
> We could move set_active_memcg(parent) call from mem_cgroup_css_alloc()
> to cgroup_apply_control_enable() and handle allocation in all .css_alloc()
>
> However I need to handle allocations called from cgroup_mkdir() too and
> badly understand how to do it properly.
I don't think there is a better alternative than just having several
set_active_memcg(parent);...set_active_memcg(old); places in the cgroup.c.
Nesting is fine here, so it shouldn't be a big issue.
next prev parent reply other threads:[~2022-05-11 17:46 UTC|newest]
Thread overview: 139+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-27 10:37 [PATCH] memcg: accounting for objects allocated for new netdevice Vasily Averin
2022-04-27 14:01 ` Michal Koutný
2022-04-27 16:52 ` Shakeel Butt
2022-04-27 22:35 ` Vasily Averin
2022-05-02 12:15 ` [PATCH memcg v2] " Vasily Averin
2022-05-04 20:50 ` Luis Chamberlain
2022-05-05 3:50 ` patchwork-bot+netdevbpf
2022-05-11 2:51 ` Roman Gushchin
2022-05-02 19:37 ` kernfs memcg accounting Vasily Averin
2022-05-02 21:22 ` Michal Koutný
2022-05-04 9:00 ` Vasily Averin
2022-05-04 14:10 ` Michal Koutný
2022-05-04 21:16 ` Vasily Averin
2022-05-05 9:47 ` Michal Koutný
2022-05-06 8:37 ` Vasily Averin
2022-05-11 3:06 ` Roman Gushchin
2022-05-11 6:01 ` Vasily Averin
2022-05-11 16:49 ` Michal Koutný
2022-05-11 17:46 ` Roman Gushchin [this message]
2022-05-11 16:34 ` Michal Koutný
2022-05-11 18:10 ` Roman Gushchin
2022-05-13 15:51 ` [PATCH 0/4] memcg: accounting for objects allocated by mkdir cgroup Vasily Averin
2022-05-13 17:49 ` Roman Gushchin
2022-05-21 16:37 ` [PATCH mm v2 0/9] " Vasily Averin
2022-05-30 11:25 ` [PATCH mm v3 " Vasily Averin
2022-05-30 11:55 ` Michal Hocko
2022-05-30 13:09 ` Vasily Averin
2022-05-30 14:22 ` Michal Hocko
2022-05-30 19:58 ` Vasily Averin
2022-05-31 7:16 ` Michal Hocko
2022-06-01 3:43 ` Vasily Averin
2022-06-01 9:15 ` Michal Koutný
2022-06-01 9:32 ` Michal Hocko
2022-06-01 13:05 ` Michal Hocko
2022-06-01 14:22 ` Roman Gushchin
2022-06-01 15:24 ` Michal Hocko
2022-06-01 9:26 ` Michal Hocko
2022-06-13 5:34 ` [PATCH mm v4 " Vasily Averin
2022-06-23 14:50 ` [PATCH mm v5 0/9] memcg: accounting for objects allocated by mkdir, cgroup Vasily Averin
2022-06-23 15:03 ` Vasily Averin
2022-06-23 16:07 ` Michal Hocko
2022-06-23 16:55 ` Shakeel Butt
2022-06-24 10:40 ` Vasily Averin
2022-06-24 12:26 ` Michal Koutný
2022-06-24 13:59 ` Michal Hocko
2022-06-25 9:43 ` [PATCH RFC] memcg: avoid idr ids space depletion Vasily Averin
2022-06-25 14:04 ` [PATCH RFC] memcg: notify about global mem_cgroup_id " Vasily Averin
2022-06-26 1:56 ` Roman Gushchin
2022-06-26 7:11 ` Vasily Averin
2022-06-27 2:12 ` [PATCH cgroup] cgroup: set the correct return code if hierarchy limits are reached Vasily Averin
2022-06-27 3:33 ` Muchun Song
2022-06-27 9:07 ` Tejun Heo
2022-06-28 0:44 ` Roman Gushchin
2022-06-28 3:59 ` Vasily Averin
2022-06-28 9:16 ` Michal Koutný
2022-06-28 9:22 ` Tejun Heo
2022-06-29 6:13 ` Vasily Averin
2022-06-29 19:25 ` Tejun Heo
2022-07-01 2:42 ` Roman Gushchin
2022-06-27 2:11 ` [PATCH mm v2] memcg: notify about global mem_cgroup_id space depletion Vasily Averin
2022-06-27 3:23 ` Muchun Song
2022-06-27 6:49 ` Vasily Averin
2022-06-28 1:11 ` Roman Gushchin
2022-06-28 3:43 ` Vasily Averin
2022-06-28 9:08 ` Michal Koutný
2022-06-27 16:37 ` [PATCH mm v5 0/9] memcg: accounting for objects allocated by mkdir, cgroup Shakeel Butt
2022-07-01 11:03 ` Michal Hocko
2022-07-10 18:53 ` Vasily Averin
2022-07-11 16:24 ` Michal Hocko
2022-06-23 14:50 ` [PATCH mm v5 1/9] memcg: enable accounting for struct cgroup Vasily Averin
2022-06-23 14:50 ` [PATCH mm v5 2/9] memcg: enable accounting for kernfs nodes Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 3/9] memcg: enable accounting for kernfs iattrs Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 4/9] memcg: enable accounting for struct simple_xattr Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 5/9] memcg: enable accounting for percpu allocation of struct psi_group_cpu Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 6/9] memcg: enable accounting for percpu allocation of struct cgroup_rstat_cpu Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 7/9] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-06-23 14:51 ` [PATCH mm v5 8/9] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-06-23 14:52 ` [PATCH mm v5 9/9] memcg: enable accounting for perpu allocation of struct rt_rq Vasily Averin
2022-06-13 5:34 ` [PATCH mm v4 1/9] memcg: enable accounting for struct cgroup Vasily Averin
2022-06-13 5:34 ` [PATCH mm v4 2/9] memcg: enable accounting for kernfs nodes Vasily Averin
2022-06-13 5:34 ` [PATCH mm v4 3/9] memcg: enable accounting for kernfs iattrs Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 4/9] memcg: enable accounting for struct simple_xattr Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 5/9] memcg: enable accounting for percpu allocation of struct psi_group_cpu Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 6/9] memcg: enable accounting for percpu allocation of struct cgroup_rstat_cpu Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 7/9] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 8/9] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-06-13 5:35 ` [PATCH mm v4 9/9] memcg: enable accounting for perpu allocation of struct rt_rq Vasily Averin
[not found] ` <cover.1653899364.git.vvs@openvz.org>
2022-05-30 11:25 ` [PATCH mm v3 1/9] memcg: enable accounting for struct cgroup Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 2/9] memcg: enable accounting for kernfs nodes Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 3/9] memcg: enable accounting for kernfs iattrs Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 4/9] memcg: enable accounting for struct simple_xattr Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 5/9] memcg: enable accounting for percpu allocation of struct psi_group_cpu Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 6/9] memcg: enable accounting for percpu allocation of struct cgroup_rstat_cpu Vasily Averin
2022-05-30 15:04 ` Muchun Song
2022-05-30 11:26 ` [PATCH mm v3 7/9] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-05-30 11:26 ` [PATCH mm v3 8/9] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-05-30 11:27 ` [PATCH mm v3 9/9] memcg: enable accounting for perpu allocation of struct rt_rq Vasily Averin
2022-05-30 15:06 ` Muchun Song
2022-05-21 16:37 ` [PATCH mm v2 1/9] memcg: enable accounting for struct cgroup Vasily Averin
2022-05-22 6:37 ` Muchun Song
2022-05-21 16:37 ` [PATCH mm v2 2/9] memcg: enable accounting for kernfs nodes Vasily Averin
2022-05-22 6:37 ` Muchun Song
2022-05-21 16:37 ` [PATCH mm v2 3/9] memcg: enable accounting for kernfs iattrs Vasily Averin
2022-05-22 6:38 ` Muchun Song
2022-05-21 16:38 ` [PATCH mm v2 4/9] memcg: enable accounting for struct simple_xattr Vasily Averin
2022-05-22 6:38 ` Muchun Song
2022-05-21 16:38 ` [PATCH mm v2 5/9] memcg: enable accounting for percpu allocation of struct psi_group_cpu Vasily Averin
2022-05-21 21:34 ` Shakeel Butt
2022-05-22 6:40 ` Muchun Song
2022-05-25 1:30 ` Roman Gushchin
2022-05-21 16:38 ` [PATCH mm v2 6/9] memcg: enable accounting for percpu allocation of struct cgroup_rstat_cpu Vasily Averin
2022-05-21 17:58 ` Vasily Averin
2022-05-21 21:35 ` Shakeel Butt
2022-05-21 22:05 ` kernel test robot
2022-05-25 1:31 ` Roman Gushchin
2022-05-21 16:38 ` [PATCH mm v2 7/9] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-05-22 6:47 ` Muchun Song
2022-05-21 16:38 ` [PATCH mm v2 8/9] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-05-22 6:49 ` Muchun Song
2022-05-21 16:39 ` [PATCH mm v2 9/9] memcg: enable accounting for percpu allocation of struct rt_rq Vasily Averin
2022-05-21 21:37 ` Shakeel Butt
2022-05-25 1:31 ` Roman Gushchin
2022-05-13 15:51 ` [PATCH 1/4] memcg: enable accounting for large allocations in mem_cgroup_css_alloc Vasily Averin
2022-05-19 16:46 ` Michal Koutný
2022-05-20 1:07 ` Shakeel Butt
2022-05-13 15:51 ` [PATCH 2/4] memcg: enable accounting for kernfs nodes and iattrs Vasily Averin
2022-05-19 16:33 ` Michal Koutný
2022-05-20 1:12 ` Shakeel Butt
2022-05-13 15:52 ` [PATCH 3/4] memcg: enable accounting for struct cgroup Vasily Averin
2022-05-19 16:53 ` Michal Koutný
2022-05-20 7:24 ` Vasily Averin
2022-05-20 20:16 ` Vasily Averin
2022-05-21 0:55 ` Roman Gushchin
2022-05-21 7:28 ` Vasily Averin
2022-05-23 13:52 ` Michal Koutný
2022-05-20 1:31 ` Shakeel Butt
2022-05-13 15:52 ` [PATCH 4/4] memcg: enable accounting for allocations in alloc_fair_sched_group Vasily Averin
2022-05-19 16:45 ` Michal Koutný
2022-05-20 1:18 ` Shakeel Butt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Ynv2b6aN5zMnJuLA@carbon \
--to=roman.gushchin@linux.dev \
--cc=cgroups@vger.kernel.org \
--cc=fw@strlen.de \
--cc=gregkh@linuxfoundation.org \
--cc=kernel@openvz.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mhocko@suse.com \
--cc=mkoutny@suse.com \
--cc=shakeelb@google.com \
--cc=tj@kernel.org \
--cc=vbabka@suse.cz \
--cc=vvs@openvz.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).