From: Imran Khan <imran.f.khan@oracle.com>
To: Roman Gushchin <guro@fb.com>,
Andrew Morton <akpm@linux-foundation.org>,
linux-mm@kvack.org
Cc: Michal Hocko <mhocko@kernel.org>,
Johannes Weiner <hannes@cmpxchg.org>,
Shakeel Butt <shakeelb@google.com>,
linux-kernel@vger.kernel.org, kernel-team@fb.com
Subject: Re: [PATCH] mm: memcg/slab: optimize objcg stock draining
Date: Wed, 6 Jan 2021 17:19:33 +1100 [thread overview]
Message-ID: <92c34535-57de-39aa-7984-48faaa02c9e9@oracle.com> (raw)
In-Reply-To: <20210106042239.2860107-1-guro@fb.com>
On 6/1/21 3:22 pm, Roman Gushchin wrote:
> Imran Khan reported a regression in hackbench results caused by the
> commit f2fe7b09a52b ("mm: memcg/slab: charge individual slab objects
> instead of pages"). The regression is noticeable in the case of
> a consequent allocation of several relatively large slab objects,
> e.g. skb's. As soon as the amount of stocked bytes exceeds PAGE_SIZE,
> drain_obj_stock() and __memcg_kmem_uncharge() are called, and it leads
> to a number of atomic operations in page_counter_uncharge().
>
> The corresponding call graph is below (provided by Imran Khan):
> |__alloc_skb
> | |
> | |__kmalloc_reserve.isra.61
> | | |
> | | |__kmalloc_node_track_caller
> | | | |
> | | | |slab_pre_alloc_hook.constprop.88
> | | | obj_cgroup_charge
> | | | | |
> | | | | |__memcg_kmem_charge
> | | | | | |
> | | | | | |page_counter_try_charge
> | | | | |
> | | | | |refill_obj_stock
> | | | | | |
> | | | | | |drain_obj_stock.isra.68
> | | | | | | |
> | | | | | | |__memcg_kmem_uncharge
> | | | | | | | |
> | | | | | | | |page_counter_uncharge
> | | | | | | | | |
> | | | | | | | | |page_counter_cancel
> | | | |
> | | | |
> | | | |__slab_alloc
> | | | | |
> | | | | |___slab_alloc
> | | | | |
> | | | |slab_post_alloc_hook
>
> Instead of directly uncharging the accounted kernel memory, it's
> possible to refill the generic page-sized per-cpu stock instead.
> It's a much faster operation, especially on a default hierarchy.
> As a bonus, __memcg_kmem_uncharge_page() will also get faster,
> so the freeing of page-sized kernel allocations (e.g. large kmallocs)
> will become faster.
>
> A similar change has been done earlier for the socket memory by
> the commit 475d0487a2ad ("mm: memcontrol: use per-cpu stocks for
> socket memory uncharging").
>
> Signed-off-by: Roman Gushchin <guro@fb.com>
> Reported-by: Imran Khan <imran.f.khan@oracle.com>
Tested-by: Imran Khan <imran.f.khan@oracle.com>
> ---
> mm/memcontrol.c | 4 +---
> 1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 0d74b80fa4de..8148c1df3aff 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -3122,9 +3122,7 @@ void __memcg_kmem_uncharge(struct mem_cgroup *memcg, unsigned int nr_pages)
> if (!cgroup_subsys_on_dfl(memory_cgrp_subsys))
> page_counter_uncharge(&memcg->kmem, nr_pages);
>
> - page_counter_uncharge(&memcg->memory, nr_pages);
> - if (do_memsw_account())
> - page_counter_uncharge(&memcg->memsw, nr_pages);
> + refill_stock(memcg, nr_pages);
> }
>
> /**
>
next prev parent reply other threads:[~2021-01-06 6:23 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-06 4:22 [PATCH] mm: memcg/slab: optimize objcg stock draining Roman Gushchin
2021-01-06 6:05 ` Shakeel Butt
2021-01-06 16:42 ` Roman Gushchin
2021-01-06 6:19 ` Imran Khan [this message]
2021-01-06 19:50 ` Andrew Morton
2021-01-06 20:04 ` Roman Gushchin
2021-01-20 16:34 ` Michal Koutný
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=92c34535-57de-39aa-7984-48faaa02c9e9@oracle.com \
--to=imran.f.khan@oracle.com \
--cc=akpm@linux-foundation.org \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=shakeelb@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).