From: Waiman Long <longman@redhat.com>
To: Roman Gushchin <guro@fb.com>, linux-mm@kvack.org
Cc: Michal Hocko <mhocko@kernel.org>,
Johannes Weiner <hannes@cmpxchg.org>,
linux-kernel@vger.kernel.org, kernel-team@fb.com,
Shakeel Butt <shakeelb@google.com>,
Vladimir Davydov <vdavydov.dev@gmail.com>,
Christoph Lameter <cl@linux.com>
Subject: Re: [PATCH 00/16] The new slab memory controller
Date: Fri, 18 Oct 2019 13:03:54 -0400 [thread overview]
Message-ID: <f3eb1843-8f10-1e7e-9cc7-9e0209c837ce@redhat.com> (raw)
In-Reply-To: <20191018002820.307763-1-guro@fb.com>
On 10/17/19 8:28 PM, Roman Gushchin wrote:
> The existing slab memory controller is based on the idea of replicating
> slab allocator internals for each memory cgroup. This approach promises
> a low memory overhead (one pointer per page), and isn't adding too much
> code on hot allocation and release paths. But is has a very serious flaw:
^it^
> it leads to a low slab utilization.
>
> Using a drgn* script I've got an estimation of slab utilization on
> a number of machines running different production workloads. In most
> cases it was between 45% and 65%, and the best number I've seen was
> around 85%. Turning kmem accounting off brings it to high 90s. Also
> it brings back 30-50% of slab memory. It means that the real price
> of the existing slab memory controller is way bigger than a pointer
> per page.
>
> The real reason why the existing design leads to a low slab utilization
> is simple: slab pages are used exclusively by one memory cgroup.
> If there are only few allocations of certain size made by a cgroup,
> or if some active objects (e.g. dentries) are left after the cgroup is
> deleted, or the cgroup contains a single-threaded application which is
> barely allocating any kernel objects, but does it every time on a new CPU:
> in all these cases the resulting slab utilization is very low.
> If kmem accounting is off, the kernel is able to use free space
> on slab pages for other allocations.
In the case of slub memory allocator, it is not just unused space within
a slab. It is also the use of per-cpu slabs that can hold up a lot of
memory, especially if the tasks jump around to different cpus. The
problem is compounded if a lot of memcgs are being used. Memory
utilization can improve quite significantly if per-cpu slabs are
disabled. Of course, it comes with a performance cost.
Cheers,
Longman
next prev parent reply other threads:[~2019-10-18 17:03 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-18 0:28 [PATCH 00/16] The new slab memory controller Roman Gushchin
2019-10-18 0:28 ` [PATCH 01/16] mm: memcg: introduce mem_cgroup_ptr Roman Gushchin
2019-10-18 0:28 ` [PATCH 02/16] mm: vmstat: use s32 for vm_node_stat_diff in struct per_cpu_nodestat Roman Gushchin
2019-10-20 22:44 ` Christopher Lameter
2019-10-21 1:15 ` Roman Gushchin
2019-10-21 18:09 ` Christopher Lameter
2019-10-20 22:51 ` Christopher Lameter
2019-10-21 1:21 ` Roman Gushchin
2019-10-18 0:28 ` [PATCH 03/16] mm: vmstat: convert slab vmstat counter to bytes Roman Gushchin
2019-10-18 0:28 ` [PATCH 04/16] mm: memcg/slab: allocate space for memcg ownership data for non-root slabs Roman Gushchin
2019-10-18 0:28 ` [PATCH 05/16] mm: slub: implement SLUB version of obj_to_index() Roman Gushchin
2019-10-18 0:28 ` [PATCH 06/16] mm: memcg/slab: save memcg ownership data for non-root slab objects Roman Gushchin
2019-10-18 0:28 ` [PATCH 07/16] mm: memcg: move memcg_kmem_bypass() to memcontrol.h Roman Gushchin
2019-10-18 0:28 ` [PATCH 08/16] mm: memcg: introduce __mod_lruvec_memcg_state() Roman Gushchin
2019-10-18 0:28 ` [PATCH 09/16] mm: memcg/slab: charge individual slab objects instead of pages Roman Gushchin
2019-10-25 19:41 ` Johannes Weiner
2019-10-25 20:00 ` Roman Gushchin
2019-10-25 20:52 ` Johannes Weiner
2019-10-31 1:52 ` Roman Gushchin
2019-10-31 14:23 ` Johannes Weiner
2019-10-31 14:41 ` Johannes Weiner
2019-10-31 15:07 ` Roman Gushchin
2019-10-31 18:50 ` Johannes Weiner
2019-10-18 0:28 ` [PATCH 10/16] mm: memcg: move get_mem_cgroup_from_current() to memcontrol.h Roman Gushchin
2019-10-18 0:28 ` [PATCH 11/16] mm: memcg/slab: replace memcg_from_slab_page() with memcg_from_slab_obj() Roman Gushchin
2019-10-18 0:28 ` [PATCH 13/16] mm: memcg/slab: deprecate memory.kmem.slabinfo Roman Gushchin
2019-10-18 0:28 ` [PATCH 14/16] mm: memcg/slab: use one set of kmem_caches for all memory cgroups Roman Gushchin
2019-10-18 0:28 ` [PATCH 15/16] tools/cgroup: make slabinfo.py compatible with new slab controller Roman Gushchin
2019-10-18 0:28 ` [PATCH 16/16] mm: slab: remove redundant check in memcg_accumulate_slabinfo() Roman Gushchin
2019-10-18 17:03 ` Waiman Long [this message]
2019-10-18 17:12 ` [PATCH 00/16] The new slab memory controller Roman Gushchin
2019-10-22 13:22 ` Michal Hocko
2019-10-22 13:28 ` Michal Hocko
2019-10-22 15:48 ` Roman Gushchin
2019-10-22 13:31 ` Michal Hocko
2019-10-22 15:59 ` Roman Gushchin
-- strict thread matches above, loose matches on Subject: below --
2019-09-05 21:45 [PATCH RFC 00/14] " Roman Gushchin
2019-12-09 9:17 ` [PATCH 00/16] " Bharata B Rao
2019-12-09 11:56 ` Bharata B Rao
2019-12-09 18:04 ` Roman Gushchin
2019-12-10 6:23 ` Bharata B Rao
2019-12-10 18:05 ` Roman Gushchin
2020-01-13 8:47 ` Bharata B Rao
2020-01-13 15:31 ` Roman Gushchin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f3eb1843-8f10-1e7e-9cc7-9e0209c837ce@redhat.com \
--to=longman@redhat.com \
--cc=cl@linux.com \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=shakeelb@google.com \
--cc=vdavydov.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).