Linux-mm Archive on lore.kernel.org
 help / color / Atom feed
From: Michal Hocko <mhocko@kernel.org>
To: Roman Gushchin <guro@fb.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Kernel Team <Kernel-team@fb.com>
Subject: Re: [PATCH 2/2] mm: memcontrol: flush percpu slab vmstats on kmem offlining
Date: Thu, 15 Aug 2019 10:35:36 +0200
Message-ID: <20190815083536.GD9477@dhcp22.suse.cz> (raw)
In-Reply-To: <20190814215408.GA5584@tower.dhcp.thefacebook.com>

On Wed 14-08-19 21:54:12, Roman Gushchin wrote:
> On Wed, Aug 14, 2019 at 01:32:42PM +0200, Michal Hocko wrote:
> > On Mon 12-08-19 15:29:11, Roman Gushchin wrote:
> > > I've noticed that the "slab" value in memory.stat is sometimes 0,
> > > even if some children memory cgroups have a non-zero "slab" value.
> > > The following investigation showed that this is the result
> > > of the kmem_cache reparenting in combination with the per-cpu
> > > batching of slab vmstats.
> > > 
> > > At the offlining some vmstat value may leave in the percpu cache,
> > > not being propagated upwards by the cgroup hierarchy. It means
> > > that stats on ancestor levels are lower than actual. Later when
> > > slab pages are released, the precise number of pages is substracted
> > > on the parent level, making the value negative. We don't show negative
> > > values, 0 is printed instead.
> > 
> > So the difference with other counters is that slab ones are reparented
> > and that's why we have treat them specially? I guess that is what the
> > comment in the code suggest but being explicit in the changelog would be
> > nice.
> 
> Right. And I believe the list can be extended further. Objects which
> are often outliving the origin memory cgroup (e.g. pagecache pages)
> are pinning dead cgroups, so it will be cool to reparent them all.
> 
> > 
> > [...]
> > > -static void memcg_flush_percpu_vmstats(struct mem_cgroup *memcg)
> > > +static void memcg_flush_percpu_vmstats(struct mem_cgroup *memcg, bool slab_only)
> > >  {
> > >  	unsigned long stat[MEMCG_NR_STAT];
> > >  	struct mem_cgroup *mi;
> > >  	int node, cpu, i;
> > > +	int min_idx, max_idx;
> > >  
> > > -	for (i = 0; i < MEMCG_NR_STAT; i++)
> > > +	if (slab_only) {
> > > +		min_idx = NR_SLAB_RECLAIMABLE;
> > > +		max_idx = NR_SLAB_UNRECLAIMABLE;
> > > +	} else {
> > > +		min_idx = 0;
> > > +		max_idx = MEMCG_NR_STAT;
> > > +	}
> > 
> > This is just ugly has hell! I really detest how this implicitly makes
> > counters value very special without any note in the node_stat_item
> > definition. Is it such a big deal to have a per counter flush and do
> > the loop over all counters resp. specific counters around it so much
> > worse? This should be really a slow path to safe few instructions or
> > cache misses, no?
> 
> I believe that it is a big deal, because it's
> NR_VMSTAT_ITEMS * all memory cgroups * online cpus * numa nodes.

I am not sure I follow. I just meant to remove all for (i = 0; i < MEMCG_NR_STAT; i++)
from flushing and do that loop around the flushing function. That would
mean that the NR_SLAB_$FOO wouldn't have to play tricks and simply call
the flushing for the two counters.

> If the goal is to merge it with cpu hotplug code, I'd think about passing
> cpumask to it, and do the opposite. Also I'm not sure I understand
> why reordering loops will make it less ugly.

And adding a cpu/nodemasks would just work with that as well, right.

> 
> But you're right, a comment nearby NR_SLAB_(UN)RECLAIMABLE definition
> is totaly worth it. How about something like:
> 
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 8b5f758942a2..231bcbe5dcc6 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -215,8 +215,9 @@ enum node_stat_item {
>         NR_INACTIVE_FILE,       /*  "     "     "   "       "         */
>         NR_ACTIVE_FILE,         /*  "     "     "   "       "         */
>         NR_UNEVICTABLE,         /*  "     "     "   "       "         */
> -       NR_SLAB_RECLAIMABLE,
> -       NR_SLAB_UNRECLAIMABLE,
> +       NR_SLAB_RECLAIMABLE,    /* Please, do not reorder this item */
> +       NR_SLAB_UNRECLAIMABLE,  /* and this one without looking at
> +                                * memcg_flush_percpu_vmstats() first. */
>         NR_ISOLATED_ANON,       /* Temporary isolated pages from anon lru */
>         NR_ISOLATED_FILE,       /* Temporary isolated pages from file lru */
>         WORKINGSET_NODES,

Thanks, that is an improvement.
-- 
Michal Hocko
SUSE Labs


      reply index

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-12 22:29 [PATCH 0/2] flush percpu vmstats Roman Gushchin
2019-08-12 22:29 ` [PATCH 1/2] mm: memcontrol: flush percpu vmstats before releasing memcg Roman Gushchin
2019-08-13 21:27   ` Andrew Morton
2019-08-13 21:46     ` Roman Gushchin
2019-08-14 11:26   ` Michal Hocko
2019-08-12 22:29 ` [PATCH 2/2] mm: memcontrol: flush percpu slab vmstats on kmem offlining Roman Gushchin
2019-08-14 11:32   ` Michal Hocko
2019-08-14 21:54     ` Roman Gushchin
2019-08-15  8:35       ` Michal Hocko [this message]

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190815083536.GD9477@dhcp22.suse.cz \
    --to=mhocko@kernel.org \
    --cc=Kernel-team@fb.com \
    --cc=akpm@linux-foundation.org \
    --cc=guro@fb.com \
    --cc=hannes@cmpxchg.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mm Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \
		linux-mm@kvack.org linux-mm@archiver.kernel.org
	public-inbox-index linux-mm


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kvack.linux-mm


AGPL code for this site: git clone https://public-inbox.org/ public-inbox