From: Michal Hocko <mhocko@kernel.org>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Tejun Heo <tj@kernel.org>,
"David S. Miller" <davem@davemloft.net>,
linux-mm@kvack.org, cgroups@vger.kernel.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
kernel-team@fb.com, Vladimir Davydov <vdavydov.dev@gmail.com>
Subject: Re: [PATCH 1/3] mm: memcontrol: make per-cpu charge cache IRQ-safe for socket accounting
Date: Mon, 19 Sep 2016 14:01:34 +0200 [thread overview]
Message-ID: <20160919120133.GM10785@dhcp22.suse.cz> (raw)
In-Reply-To: <20160914194846.11153-1-hannes@cmpxchg.org>
[Fixup Vladimir's email]
On Wed 14-09-16 15:48:44, Johannes Weiner wrote:
> From: Johannes Weiner <jweiner@fb.com>
>
> During cgroup2 rollout into production, we started encountering css
> refcount underflows and css access crashes in the memory controller.
> Splitting the heavily shared css reference counter into logical users
> narrowed the imbalance down to the cgroup2 socket memory accounting.
>
> The problem turns out to be the per-cpu charge cache. Cgroup1 had a
> separate socket counter, but the new cgroup2 socket accounting goes
> through the common charge path that uses a shared per-cpu cache for
> all memory that is being tracked. Those caches are safe against
> scheduling preemption, but not against interrupts - such as the newly
> added packet receive path. When cache draining is interrupted by
> network RX taking pages out of the cache, the resuming drain operation
> will put references of in-use pages, thus causing the imbalance.
>
> Disable IRQs during all per-cpu charge cache operations.
>
> Fixes: f7e1cb6ec51b ("mm: memcontrol: account socket memory in unified hierarchy memory controller")
> Cc: <stable@vger.kernel.org> # 4.5+
> Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> mm/memcontrol.c | 31 ++++++++++++++++++++++---------
> 1 file changed, 22 insertions(+), 9 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 7a8d6624758a..60bb830abc34 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -1710,17 +1710,22 @@ static DEFINE_MUTEX(percpu_charge_mutex);
> static bool consume_stock(struct mem_cgroup *memcg, unsigned int nr_pages)
> {
> struct memcg_stock_pcp *stock;
> + unsigned long flags;
> bool ret = false;
>
> if (nr_pages > CHARGE_BATCH)
> return ret;
>
> - stock = &get_cpu_var(memcg_stock);
> + local_irq_save(flags);
> +
> + stock = this_cpu_ptr(&memcg_stock);
> if (memcg == stock->cached && stock->nr_pages >= nr_pages) {
> stock->nr_pages -= nr_pages;
> ret = true;
> }
> - put_cpu_var(memcg_stock);
> +
> + local_irq_restore(flags);
> +
> return ret;
> }
>
> @@ -1741,15 +1746,18 @@ static void drain_stock(struct memcg_stock_pcp *stock)
> stock->cached = NULL;
> }
>
> -/*
> - * This must be called under preempt disabled or must be called by
> - * a thread which is pinned to local cpu.
> - */
> static void drain_local_stock(struct work_struct *dummy)
> {
> - struct memcg_stock_pcp *stock = this_cpu_ptr(&memcg_stock);
> + struct memcg_stock_pcp *stock;
> + unsigned long flags;
> +
> + local_irq_save(flags);
> +
> + stock = this_cpu_ptr(&memcg_stock);
> drain_stock(stock);
> clear_bit(FLUSHING_CACHED_CHARGE, &stock->flags);
> +
> + local_irq_restore(flags);
> }
>
> /*
> @@ -1758,14 +1766,19 @@ static void drain_local_stock(struct work_struct *dummy)
> */
> static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages)
> {
> - struct memcg_stock_pcp *stock = &get_cpu_var(memcg_stock);
> + struct memcg_stock_pcp *stock;
> + unsigned long flags;
> +
> + local_irq_save(flags);
>
> + stock = this_cpu_ptr(&memcg_stock);
> if (stock->cached != memcg) { /* reset if necessary */
> drain_stock(stock);
> stock->cached = memcg;
> }
> stock->nr_pages += nr_pages;
> - put_cpu_var(memcg_stock);
> +
> + local_irq_restore(flags);
> }
>
> /*
> --
> 2.9.3
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-09-19 12:01 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-14 19:48 [PATCH 1/3] mm: memcontrol: make per-cpu charge cache IRQ-safe for socket accounting Johannes Weiner
2016-09-14 19:48 ` [PATCH 2/3] cgroup: duplicate cgroup reference when cloning sockets Johannes Weiner
2016-09-19 12:03 ` Michal Hocko
2016-09-19 15:43 ` Vladimir Davydov
2016-09-14 19:48 ` [PATCH 3/3] mm: memcontrol: consolidate cgroup socket tracking Johannes Weiner
2016-09-14 20:45 ` Tejun Heo
2016-09-14 21:12 ` kbuild test robot
2016-09-14 21:12 ` kbuild test robot
2016-09-15 5:34 ` kbuild test robot
2016-09-14 22:17 ` Andrew Morton
2016-09-15 14:27 ` Johannes Weiner
2016-09-19 12:04 ` Michal Hocko
2016-09-19 12:01 ` Michal Hocko [this message]
2016-09-19 15:35 ` [PATCH 1/3] mm: memcontrol: make per-cpu charge cache IRQ-safe for socket accounting Vladimir Davydov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160919120133.GM10785@dhcp22.suse.cz \
--to=mhocko@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=davem@davemloft.net \
--cc=hannes@cmpxchg.org \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=netdev@vger.kernel.org \
--cc=tj@kernel.org \
--cc=vdavydov.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).