From: Michal Hocko <mhocko@suse.cz>
To: Glauber Costa <glommer@parallels.com>
Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org,
kamezawa.hiroyu@jp.fujitsu.com, devel@openvz.org,
Tejun Heo <tj@kernel.org>,
linux-mm@kvack.org, Suleiman Souhlal <suleiman@google.com>,
Frederic Weisbecker <fweisbec@gmail.com>,
Mel Gorman <mgorman@suse.de>,
David Rientjes <rientjes@google.com>,
Johannes Weiner <hannes@cmpxchg.org>
Subject: Re: [PATCH v3 12/13] execute the whole memcg freeing in rcu callback
Date: Mon, 1 Oct 2012 15:27:11 +0200 [thread overview]
Message-ID: <20121001132711.GL8622@dhcp22.suse.cz> (raw)
In-Reply-To: <1347977050-29476-13-git-send-email-glommer@parallels.com>
On Tue 18-09-12 18:04:09, Glauber Costa wrote:
> A lot of the initialization we do in mem_cgroup_create() is done with softirqs
> enabled. This include grabbing a css id, which holds &ss->id_lock->rlock, and
> the per-zone trees, which holds rtpz->lock->rlock. All of those signal to the
> lockdep mechanism that those locks can be used in SOFTIRQ-ON-W context. This
> means that the freeing of memcg structure must happen in a compatible context,
> otherwise we'll get a deadlock.
Maybe I am missing something obvious but why cannot we simply disble
(soft)irqs in mem_cgroup_create rather than make the free path much more
complicated. It really feels strange to defer everything (e.g. soft
reclaim tree cleanup which should be a no-op at the time because there
shouldn't be any user pages in the group).
> The reference counting mechanism we use allows the memcg structure to be freed
> later and outlive the actual memcg destruction from the filesystem. However, we
> have little, if any, means to guarantee in which context the last memcg_put
> will happen. The best we can do is test it and try to make sure no invalid
> context releases are happening. But as we add more code to memcg, the possible
> interactions grow in number and expose more ways to get context conflicts.
>
> We already moved a part of the freeing to a worker thread to be context-safe
> for the static branches disabling. I see no reason not to do it for the whole
> freeing action. I consider this to be the safe choice.
>
> Signed-off-by: Glauber Costa <glommer@parallels.com>
> Tested-by: Greg Thelen <gthelen@google.com>
> CC: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> CC: Michal Hocko <mhocko@suse.cz>
> CC: Johannes Weiner <hannes@cmpxchg.org>
> ---
> mm/memcontrol.c | 66 +++++++++++++++++++++++++++++----------------------------
> 1 file changed, 34 insertions(+), 32 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index b05ecac..74654f0 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -5082,16 +5082,29 @@ out_free:
> }
>
> /*
> - * Helpers for freeing a kmalloc()ed/vzalloc()ed mem_cgroup by RCU,
> - * but in process context. The work_freeing structure is overlaid
> - * on the rcu_freeing structure, which itself is overlaid on memsw.
> + * At destroying mem_cgroup, references from swap_cgroup can remain.
> + * (scanning all at force_empty is too costly...)
> + *
> + * Instead of clearing all references at force_empty, we remember
> + * the number of reference from swap_cgroup and free mem_cgroup when
> + * it goes down to 0.
> + *
> + * Removal of cgroup itself succeeds regardless of refs from swap.
> */
> -static void free_work(struct work_struct *work)
> +
> +static void __mem_cgroup_free(struct mem_cgroup *memcg)
> {
> - struct mem_cgroup *memcg;
> + int node;
> int size = sizeof(struct mem_cgroup);
>
> - memcg = container_of(work, struct mem_cgroup, work_freeing);
> + mem_cgroup_remove_from_trees(memcg);
> + free_css_id(&mem_cgroup_subsys, &memcg->css);
> +
> + for_each_node(node)
> + free_mem_cgroup_per_zone_info(memcg, node);
> +
> + free_percpu(memcg->stat);
> +
> /*
> * We need to make sure that (at least for now), the jump label
> * destruction code runs outside of the cgroup lock. This is because
> @@ -5110,38 +5123,27 @@ static void free_work(struct work_struct *work)
> vfree(memcg);
> }
>
> -static void free_rcu(struct rcu_head *rcu_head)
> -{
> - struct mem_cgroup *memcg;
> -
> - memcg = container_of(rcu_head, struct mem_cgroup, rcu_freeing);
> - INIT_WORK(&memcg->work_freeing, free_work);
> - schedule_work(&memcg->work_freeing);
> -}
>
> /*
> - * At destroying mem_cgroup, references from swap_cgroup can remain.
> - * (scanning all at force_empty is too costly...)
> - *
> - * Instead of clearing all references at force_empty, we remember
> - * the number of reference from swap_cgroup and free mem_cgroup when
> - * it goes down to 0.
> - *
> - * Removal of cgroup itself succeeds regardless of refs from swap.
> + * Helpers for freeing a kmalloc()ed/vzalloc()ed mem_cgroup by RCU,
> + * but in process context. The work_freeing structure is overlaid
> + * on the rcu_freeing structure, which itself is overlaid on memsw.
> */
> -
> -static void __mem_cgroup_free(struct mem_cgroup *memcg)
> +static void free_work(struct work_struct *work)
> {
> - int node;
> + struct mem_cgroup *memcg;
>
> - mem_cgroup_remove_from_trees(memcg);
> - free_css_id(&mem_cgroup_subsys, &memcg->css);
> + memcg = container_of(work, struct mem_cgroup, work_freeing);
> + __mem_cgroup_free(memcg);
> +}
>
> - for_each_node(node)
> - free_mem_cgroup_per_zone_info(memcg, node);
> +static void free_rcu(struct rcu_head *rcu_head)
> +{
> + struct mem_cgroup *memcg;
>
> - free_percpu(memcg->stat);
> - call_rcu(&memcg->rcu_freeing, free_rcu);
> + memcg = container_of(rcu_head, struct mem_cgroup, rcu_freeing);
> + INIT_WORK(&memcg->work_freeing, free_work);
> + schedule_work(&memcg->work_freeing);
> }
>
> static void mem_cgroup_get(struct mem_cgroup *memcg)
> @@ -5153,7 +5155,7 @@ static void __mem_cgroup_put(struct mem_cgroup *memcg, int count)
> {
> if (atomic_sub_and_test(count, &memcg->refcnt)) {
> struct mem_cgroup *parent = parent_mem_cgroup(memcg);
> - __mem_cgroup_free(memcg);
> + call_rcu(&memcg->rcu_freeing, free_rcu);
> if (parent)
> mem_cgroup_put(parent);
> }
> --
> 1.7.11.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe cgroups" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Michal Hocko
SUSE Labs
next prev parent reply other threads:[~2012-10-01 13:27 UTC|newest]
Thread overview: 127+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-18 14:03 [PATCH v3 00/13] kmem controller for memcg Glauber Costa
2012-09-18 14:03 ` [PATCH v3 01/13] memcg: Make it possible to use the stock for more than one page Glauber Costa
2012-10-01 18:48 ` Johannes Weiner
2012-09-18 14:03 ` [PATCH v3 02/13] memcg: Reclaim when more than one page needed Glauber Costa
2012-10-01 19:00 ` Johannes Weiner
2012-09-18 14:04 ` [PATCH v3 03/13] memcg: change defines to an enum Glauber Costa
2012-10-01 19:06 ` Johannes Weiner
2012-10-02 9:10 ` Glauber Costa
2012-09-18 14:04 ` [PATCH v3 04/13] kmem accounting basic infrastructure Glauber Costa
2012-09-21 16:34 ` Tejun Heo
2012-09-24 8:09 ` Glauber Costa
2012-09-26 14:03 ` Michal Hocko
2012-09-26 14:33 ` Glauber Costa
2012-09-26 16:01 ` Michal Hocko
2012-09-26 17:34 ` Glauber Costa
2012-09-26 16:36 ` Tejun Heo
2012-09-26 17:36 ` Glauber Costa
2012-09-26 17:44 ` Tejun Heo
2012-09-26 17:53 ` Glauber Costa
2012-09-26 18:01 ` Tejun Heo
2012-09-26 18:56 ` Glauber Costa
2012-09-26 19:34 ` Tejun Heo
2012-09-26 19:46 ` Glauber Costa
2012-09-26 19:56 ` Tejun Heo
2012-09-26 20:02 ` Glauber Costa
2012-09-26 20:16 ` Tejun Heo
2012-09-26 21:24 ` Glauber Costa
2012-09-26 22:10 ` Tejun Heo
2012-09-26 22:29 ` Glauber Costa
2012-09-26 22:42 ` Tejun Heo
2012-09-26 22:54 ` Glauber Costa
2012-09-26 23:08 ` Tejun Heo
2012-09-26 23:20 ` Glauber Costa
2012-09-26 23:33 ` Tejun Heo
2012-09-27 12:15 ` Michal Hocko
2012-09-27 12:20 ` Glauber Costa
2012-09-27 12:40 ` Michal Hocko
2012-09-27 12:40 ` Glauber Costa
2012-09-27 12:54 ` Michal Hocko
2012-09-27 14:28 ` Mel Gorman
2012-09-27 14:49 ` Tejun Heo
2012-09-27 14:57 ` Glauber Costa
2012-09-27 17:46 ` Tejun Heo
2012-09-27 17:56 ` Michal Hocko
2012-09-27 18:45 ` Glauber Costa
2012-09-30 7:57 ` Tejun Heo
2012-09-30 8:02 ` Tejun Heo
2012-09-30 8:56 ` James Bottomley
2012-09-30 10:37 ` Tejun Heo
2012-09-30 11:25 ` James Bottomley
2012-10-01 0:57 ` Tejun Heo
2012-10-01 8:43 ` Glauber Costa
2012-10-01 8:46 ` Glauber Costa
2012-10-03 22:59 ` Tejun Heo
2012-10-01 8:36 ` Glauber Costa
2012-09-27 12:08 ` Michal Hocko
2012-09-27 12:11 ` Glauber Costa
2012-09-27 14:33 ` Tejun Heo
2012-09-27 14:43 ` Mel Gorman
2012-09-27 14:58 ` Tejun Heo
2012-09-27 18:30 ` Glauber Costa
2012-09-30 8:23 ` Tejun Heo
2012-10-01 8:45 ` Glauber Costa
2012-10-03 22:54 ` Tejun Heo
2012-10-04 11:55 ` Glauber Costa
2012-10-06 2:19 ` Tejun Heo
2012-09-27 15:09 ` Michal Hocko
2012-09-30 8:47 ` Tejun Heo
2012-10-01 9:27 ` Michal Hocko
2012-10-03 22:43 ` Tejun Heo
2012-10-05 13:47 ` Michal Hocko
2012-09-26 22:11 ` Johannes Weiner
2012-09-26 22:45 ` Glauber Costa
2012-09-18 14:04 ` [PATCH v3 05/13] Add a __GFP_KMEMCG flag Glauber Costa
2012-09-18 14:15 ` Rik van Riel
2012-09-18 15:06 ` Christoph Lameter
2012-09-19 7:39 ` Glauber Costa
2012-09-19 14:07 ` Christoph Lameter
2012-09-27 13:34 ` Mel Gorman
2012-09-27 13:41 ` Glauber Costa
2012-10-01 19:09 ` Johannes Weiner
2012-09-18 14:04 ` [PATCH v3 06/13] memcg: kmem controller infrastructure Glauber Costa
2012-09-20 16:05 ` JoonSoo Kim
2012-09-21 8:41 ` Glauber Costa
2012-09-21 9:14 ` JoonSoo Kim
2012-09-26 15:51 ` Michal Hocko
2012-09-27 11:31 ` Glauber Costa
2012-09-27 13:44 ` Michal Hocko
2012-09-28 11:34 ` Glauber Costa
2012-09-30 8:25 ` Tejun Heo
2012-10-01 8:28 ` Glauber Costa
2012-10-03 22:11 ` Tejun Heo
2012-10-01 9:44 ` Michal Hocko
2012-10-01 9:48 ` Michal Hocko
2012-10-01 10:09 ` Glauber Costa
2012-10-01 11:51 ` Michal Hocko
2012-10-01 11:51 ` Glauber Costa
2012-10-01 11:58 ` Michal Hocko
2012-10-01 12:04 ` Glauber Costa
2012-09-18 14:04 ` [PATCH v3 07/13] mm: Allocate kernel pages to the right memcg Glauber Costa
2012-09-27 13:50 ` Mel Gorman
2012-09-28 9:43 ` Glauber Costa
2012-09-28 13:28 ` Mel Gorman
2012-09-27 13:52 ` Michal Hocko
2012-09-18 14:04 ` [PATCH v3 08/13] res_counter: return amount of charges after res_counter_uncharge Glauber Costa
2012-10-01 10:00 ` Michal Hocko
2012-10-01 10:01 ` Glauber Costa
2012-09-18 14:04 ` [PATCH v3 09/13] memcg: kmem accounting lifecycle management Glauber Costa
2012-10-01 12:15 ` Michal Hocko
2012-10-01 12:29 ` Glauber Costa
2012-10-01 12:36 ` Michal Hocko
2012-10-01 12:43 ` Glauber Costa
2012-09-18 14:04 ` [PATCH v3 10/13] memcg: use static branches when code not in use Glauber Costa
2012-10-01 12:25 ` Michal Hocko
2012-10-01 12:27 ` Glauber Costa
2012-09-18 14:04 ` [PATCH v3 11/13] memcg: allow a memcg with kmem charges to be destructed Glauber Costa
2012-10-01 12:30 ` Michal Hocko
2012-09-18 14:04 ` [PATCH v3 12/13] execute the whole memcg freeing in rcu callback Glauber Costa
2012-09-21 17:23 ` Tejun Heo
2012-09-24 8:48 ` Glauber Costa
2012-10-01 13:27 ` Michal Hocko [this message]
2012-10-04 10:53 ` Glauber Costa
2012-10-04 14:20 ` Glauber Costa
2012-10-05 15:31 ` Johannes Weiner
2012-10-08 9:45 ` Glauber Costa
2012-09-18 14:04 ` [PATCH v3 13/13] protect architectures where THREAD_SIZE >= PAGE_SIZE against fork bombs Glauber Costa
2012-10-01 13:17 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121001132711.GL8622@dhcp22.suse.cz \
--to=mhocko@suse.cz \
--cc=cgroups@vger.kernel.org \
--cc=devel@openvz.org \
--cc=fweisbec@gmail.com \
--cc=glommer@parallels.com \
--cc=hannes@cmpxchg.org \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=rientjes@google.com \
--cc=suleiman@google.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).