From: Glauber Costa <glommer@parallels.com> To: <linux-kernel@vger.kernel.org> Cc: <cgroups@vger.kernel.org>, <linux-mm@kvack.org>, <kamezawa.hiroyu@jp.fujitsu.com>, Tejun Heo <tj@kernel.org>, Li Zefan <lizefan@huawei.com>, Greg Thelen <gthelen@google.com>, Suleiman Souhlal <suleiman@google.com>, Michal Hocko <mhocko@suse.cz>, Johannes Weiner <hannes@cmpxchg.org>, <devel@openvz.org>, Glauber Costa <glommer@parallels.com>, Christoph Lameter <cl@linux.com>, Pekka Enberg <penberg@cs.helsinki.fi> Subject: [PATCH v2 21/29] slab: per-memcg accounting of slab caches Date: Fri, 11 May 2012 14:44:23 -0300 [thread overview] Message-ID: <1336758272-24284-22-git-send-email-glommer@parallels.com> (raw) In-Reply-To: <1336758272-24284-1-git-send-email-glommer@parallels.com> This patch charges allocation of a slab object to a particular memcg. The cache is selected with mem_cgroup_get_kmem_cache(), which is the biggest overhead we pay here, because it happens at all allocations. However, other than forcing a function call, this function is not very expensive, and try to return as soon as we realize we are not a memcg cache. The charge/uncharge functions are heavier, but are only called for new page allocations. Code is heavily inspired by Suleiman's, with adaptations to the patchset and minor simplifications by me. Signed-off-by: Glauber Costa <glommer@parallels.com> CC: Christoph Lameter <cl@linux.com> CC: Pekka Enberg <penberg@cs.helsinki.fi> CC: Michal Hocko <mhocko@suse.cz> CC: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> CC: Johannes Weiner <hannes@cmpxchg.org> CC: Suleiman Souhlal <suleiman@google.com> --- include/linux/slab_def.h | 62 ++++++++++++++++++++++++++++ mm/slab.c | 102 +++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 154 insertions(+), 10 deletions(-) diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h index 06e4a3e..ed8c43c 100644 --- a/include/linux/slab_def.h +++ b/include/linux/slab_def.h @@ -218,4 +218,66 @@ found: #endif /* CONFIG_NUMA */ +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM + +void kmem_cache_drop_ref(struct kmem_cache *cachep); + +static inline void +kmem_cache_get_ref(struct kmem_cache *cachep) +{ + if (cachep->memcg_params.id == -1 && + unlikely(!atomic_add_unless(&cachep->memcg_params.refcnt, 1, 0))) + BUG(); +} + +static inline void +mem_cgroup_put_kmem_cache(struct kmem_cache *cachep) +{ + rcu_read_unlock(); +} + +static inline void +mem_cgroup_kmem_cache_prepare_sleep(struct kmem_cache *cachep) +{ + /* + * Make sure the cache doesn't get freed while we have interrupts + * enabled. + */ + kmem_cache_get_ref(cachep); +} + +static inline void +mem_cgroup_kmem_cache_finish_sleep(struct kmem_cache *cachep) +{ + kmem_cache_drop_ref(cachep); +} + +#else /* CONFIG_CGROUP_MEM_RES_CTLR_KMEM */ + +static inline void +kmem_cache_get_ref(struct kmem_cache *cachep) +{ +} + +static inline void +kmem_cache_drop_ref(struct kmem_cache *cachep) +{ +} + +static inline void +mem_cgroup_put_kmem_cache(struct kmem_cache *cachep) +{ +} + +static inline void +mem_cgroup_kmem_cache_prepare_sleep(struct kmem_cache *cachep) +{ +} + +static inline void +mem_cgroup_kmem_cache_finish_sleep(struct kmem_cache *cachep) +{ +} +#endif /* CONFIG_CGROUP_MEM_RES_CTLR_KMEM */ + #endif /* _LINUX_SLAB_DEF_H */ diff --git a/mm/slab.c b/mm/slab.c index 985714a..7022f86 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1821,20 +1821,28 @@ static void *kmem_getpages(struct kmem_cache *cachep, gfp_t flags, int nodeid) if (cachep->flags & SLAB_RECLAIM_ACCOUNT) flags |= __GFP_RECLAIMABLE; + nr_pages = (1 << cachep->gfporder); + if (!mem_cgroup_charge_slab(cachep, flags, nr_pages * PAGE_SIZE)) + return NULL; + page = alloc_pages_exact_node(nodeid, flags | __GFP_NOTRACK, cachep->gfporder); if (!page) { if (!(flags & __GFP_NOWARN) && printk_ratelimit()) slab_out_of_memory(cachep, flags, nodeid); + + mem_cgroup_uncharge_slab(cachep, nr_pages * PAGE_SIZE); return NULL; } - nr_pages = (1 << cachep->gfporder); if (cachep->flags & SLAB_RECLAIM_ACCOUNT) add_zone_page_state(page_zone(page), NR_SLAB_RECLAIMABLE, nr_pages); else add_zone_page_state(page_zone(page), NR_SLAB_UNRECLAIMABLE, nr_pages); + + kmem_cache_get_ref(cachep); + for (i = 0; i < nr_pages; i++) __SetPageSlab(page + i); @@ -1850,6 +1858,14 @@ static void *kmem_getpages(struct kmem_cache *cachep, gfp_t flags, int nodeid) return page_address(page); } +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM +void kmem_cache_drop_ref(struct kmem_cache *cachep) +{ + if (cachep->memcg_params.id == -1) + atomic_dec(&cachep->memcg_params.refcnt); +} +#endif /* CONFIG_CGROUP_MEM_RES_CTLR_KMEM */ + /* * Interface to system's page release. */ @@ -1867,6 +1883,8 @@ static void kmem_freepages(struct kmem_cache *cachep, void *addr) else sub_zone_page_state(page_zone(page), NR_SLAB_UNRECLAIMABLE, nr_freed); + mem_cgroup_uncharge_slab(cachep, i * PAGE_SIZE); + kmem_cache_drop_ref(cachep); while (i--) { BUG_ON(!PageSlab(page)); __ClearPageSlab(page); @@ -2854,10 +2872,11 @@ void kmem_cache_destroy(struct kmem_cache *cachep) #ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM /* Not a memcg cache */ - if (cachep->memcg_params.id != -1) + if (cachep->memcg_params.id != -1) { mem_cgroup_release_cache(cachep); + mem_cgroup_flush_cache_create_queue(); + } #endif - __kmem_cache_destroy(cachep); mutex_unlock(&cache_chain_mutex); put_online_cpus(); @@ -3063,8 +3082,10 @@ static int cache_grow(struct kmem_cache *cachep, offset *= cachep->colour_off; - if (local_flags & __GFP_WAIT) + if (local_flags & __GFP_WAIT) { local_irq_enable(); + mem_cgroup_kmem_cache_prepare_sleep(cachep); + } /* * The test for missing atomic flag is performed here, rather than @@ -3093,8 +3114,10 @@ static int cache_grow(struct kmem_cache *cachep, cache_init_objs(cachep, slabp); - if (local_flags & __GFP_WAIT) + if (local_flags & __GFP_WAIT) { local_irq_disable(); + mem_cgroup_kmem_cache_finish_sleep(cachep); + } check_irq_off(); spin_lock(&l3->list_lock); @@ -3107,8 +3130,10 @@ static int cache_grow(struct kmem_cache *cachep, opps1: kmem_freepages(cachep, objp); failed: - if (local_flags & __GFP_WAIT) + if (local_flags & __GFP_WAIT) { local_irq_disable(); + mem_cgroup_kmem_cache_finish_sleep(cachep); + } return 0; } @@ -3869,11 +3894,15 @@ static inline void __cache_free(struct kmem_cache *cachep, void *objp, */ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { - void *ret = __cache_alloc(cachep, flags, __builtin_return_address(0)); + void *ret; + + rcu_read_lock(); + cachep = mem_cgroup_get_kmem_cache(cachep, flags); + rcu_read_unlock(); + ret = __cache_alloc(cachep, flags, __builtin_return_address(0)); trace_kmem_cache_alloc(_RET_IP_, ret, obj_size(cachep), cachep->buffer_size, flags); - return ret; } EXPORT_SYMBOL(kmem_cache_alloc); @@ -3884,6 +3913,10 @@ kmem_cache_alloc_trace(size_t size, struct kmem_cache *cachep, gfp_t flags) { void *ret; + rcu_read_lock(); + cachep = mem_cgroup_get_kmem_cache(cachep, flags); + rcu_read_unlock(); + ret = __cache_alloc(cachep, flags, __builtin_return_address(0)); trace_kmalloc(_RET_IP_, ret, @@ -3896,13 +3929,17 @@ EXPORT_SYMBOL(kmem_cache_alloc_trace); #ifdef CONFIG_NUMA void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { - void *ret = __cache_alloc_node(cachep, flags, nodeid, + void *ret; + + rcu_read_lock(); + cachep = mem_cgroup_get_kmem_cache(cachep, flags); + rcu_read_unlock(); + ret = __cache_alloc_node(cachep, flags, nodeid, __builtin_return_address(0)); trace_kmem_cache_alloc_node(_RET_IP_, ret, obj_size(cachep), cachep->buffer_size, flags, nodeid); - return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node); @@ -3915,6 +3952,9 @@ void *kmem_cache_alloc_node_trace(size_t size, { void *ret; + rcu_read_lock(); + cachep = mem_cgroup_get_kmem_cache(cachep, flags); + rcu_read_unlock(); ret = __cache_alloc_node(cachep, flags, nodeid, __builtin_return_address(0)); trace_kmalloc_node(_RET_IP_, ret, @@ -4023,9 +4063,33 @@ void kmem_cache_free(struct kmem_cache *cachep, void *objp) local_irq_save(flags); debug_check_no_locks_freed(objp, obj_size(cachep)); + +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM + { + struct kmem_cache *actual_cachep; + + actual_cachep = virt_to_cache(objp); + if (actual_cachep != cachep) { + VM_BUG_ON(actual_cachep->memcg_params.id != -1); + cachep = actual_cachep; + } + /* + * Grab a reference so that the cache is guaranteed to stay + * around. + * If we are freeing the last object of a dead memcg cache, + * the kmem_cache_drop_ref() at the end of this function + * will end up freeing the cache. + */ + kmem_cache_get_ref(cachep); + } +#endif + if (!(cachep->flags & SLAB_DEBUG_OBJECTS)) debug_check_no_obj_freed(objp, obj_size(cachep)); __cache_free(cachep, objp, __builtin_return_address(0)); + + kmem_cache_drop_ref(cachep); + local_irq_restore(flags); trace_kmem_cache_free(_RET_IP_, objp); @@ -4053,9 +4117,19 @@ void kfree(const void *objp) local_irq_save(flags); kfree_debugcheck(objp); c = virt_to_cache(objp); + + /* + * Grab a reference so that the cache is guaranteed to stay around. + * If we are freeing the last object of a dead memcg cache, the + * kmem_cache_drop_ref() at the end of this function will end up + * freeing the cache. + */ + kmem_cache_get_ref(c); + debug_check_no_locks_freed(objp, obj_size(c)); debug_check_no_obj_freed(objp, obj_size(c)); __cache_free(c, (void *)objp, __builtin_return_address(0)); + kmem_cache_drop_ref(c); local_irq_restore(flags); } EXPORT_SYMBOL(kfree); @@ -4324,6 +4398,13 @@ static void cache_reap(struct work_struct *w) list_for_each_entry(searchp, &cache_chain, next) { check_irq_on(); +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM + /* For memcg caches, make sure we only reap the active ones. */ + if (searchp->memcg_params.id == -1 && + !atomic_add_unless(&searchp->memcg_params.refcnt, 1, 0)) + continue; +#endif + /* * We only take the l3 lock if absolutely necessary and we * have established with reasonable certainty that @@ -4356,6 +4437,7 @@ static void cache_reap(struct work_struct *w) STATS_ADD_REAPED(searchp, freed); } next: + kmem_cache_drop_ref(searchp); cond_resched(); } check_irq_on(); -- 1.7.7.6
WARNING: multiple messages have this Message-ID (diff)
From: Glauber Costa <glommer@parallels.com> To: linux-kernel@vger.kernel.org Cc: cgroups@vger.kernel.org, linux-mm@kvack.org, kamezawa.hiroyu@jp.fujitsu.com, Tejun Heo <tj@kernel.org>, Li Zefan <lizefan@huawei.com>, Greg Thelen <gthelen@google.com>, Suleiman Souhlal <suleiman@google.com>, Michal Hocko <mhocko@suse.cz>, Johannes Weiner <hannes@cmpxchg.org>, devel@openvz.org, Glauber Costa <glommer@parallels.com>, Christoph Lameter <cl@linux.com>, Pekka Enberg <penberg@cs.helsinki.fi> Subject: [PATCH v2 21/29] slab: per-memcg accounting of slab caches Date: Fri, 11 May 2012 14:44:23 -0300 [thread overview] Message-ID: <1336758272-24284-22-git-send-email-glommer@parallels.com> (raw) In-Reply-To: <1336758272-24284-1-git-send-email-glommer@parallels.com> This patch charges allocation of a slab object to a particular memcg. The cache is selected with mem_cgroup_get_kmem_cache(), which is the biggest overhead we pay here, because it happens at all allocations. However, other than forcing a function call, this function is not very expensive, and try to return as soon as we realize we are not a memcg cache. The charge/uncharge functions are heavier, but are only called for new page allocations. Code is heavily inspired by Suleiman's, with adaptations to the patchset and minor simplifications by me. Signed-off-by: Glauber Costa <glommer@parallels.com> CC: Christoph Lameter <cl@linux.com> CC: Pekka Enberg <penberg@cs.helsinki.fi> CC: Michal Hocko <mhocko@suse.cz> CC: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> CC: Johannes Weiner <hannes@cmpxchg.org> CC: Suleiman Souhlal <suleiman@google.com> --- include/linux/slab_def.h | 62 ++++++++++++++++++++++++++++ mm/slab.c | 102 +++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 154 insertions(+), 10 deletions(-) diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h index 06e4a3e..ed8c43c 100644 --- a/include/linux/slab_def.h +++ b/include/linux/slab_def.h @@ -218,4 +218,66 @@ found: #endif /* CONFIG_NUMA */ +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM + +void kmem_cache_drop_ref(struct kmem_cache *cachep); + +static inline void +kmem_cache_get_ref(struct kmem_cache *cachep) +{ + if (cachep->memcg_params.id == -1 && + unlikely(!atomic_add_unless(&cachep->memcg_params.refcnt, 1, 0))) + BUG(); +} + +static inline void +mem_cgroup_put_kmem_cache(struct kmem_cache *cachep) +{ + rcu_read_unlock(); +} + +static inline void +mem_cgroup_kmem_cache_prepare_sleep(struct kmem_cache *cachep) +{ + /* + * Make sure the cache doesn't get freed while we have interrupts + * enabled. + */ + kmem_cache_get_ref(cachep); +} + +static inline void +mem_cgroup_kmem_cache_finish_sleep(struct kmem_cache *cachep) +{ + kmem_cache_drop_ref(cachep); +} + +#else /* CONFIG_CGROUP_MEM_RES_CTLR_KMEM */ + +static inline void +kmem_cache_get_ref(struct kmem_cache *cachep) +{ +} + +static inline void +kmem_cache_drop_ref(struct kmem_cache *cachep) +{ +} + +static inline void +mem_cgroup_put_kmem_cache(struct kmem_cache *cachep) +{ +} + +static inline void +mem_cgroup_kmem_cache_prepare_sleep(struct kmem_cache *cachep) +{ +} + +static inline void +mem_cgroup_kmem_cache_finish_sleep(struct kmem_cache *cachep) +{ +} +#endif /* CONFIG_CGROUP_MEM_RES_CTLR_KMEM */ + #endif /* _LINUX_SLAB_DEF_H */ diff --git a/mm/slab.c b/mm/slab.c index 985714a..7022f86 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1821,20 +1821,28 @@ static void *kmem_getpages(struct kmem_cache *cachep, gfp_t flags, int nodeid) if (cachep->flags & SLAB_RECLAIM_ACCOUNT) flags |= __GFP_RECLAIMABLE; + nr_pages = (1 << cachep->gfporder); + if (!mem_cgroup_charge_slab(cachep, flags, nr_pages * PAGE_SIZE)) + return NULL; + page = alloc_pages_exact_node(nodeid, flags | __GFP_NOTRACK, cachep->gfporder); if (!page) { if (!(flags & __GFP_NOWARN) && printk_ratelimit()) slab_out_of_memory(cachep, flags, nodeid); + + mem_cgroup_uncharge_slab(cachep, nr_pages * PAGE_SIZE); return NULL; } - nr_pages = (1 << cachep->gfporder); if (cachep->flags & SLAB_RECLAIM_ACCOUNT) add_zone_page_state(page_zone(page), NR_SLAB_RECLAIMABLE, nr_pages); else add_zone_page_state(page_zone(page), NR_SLAB_UNRECLAIMABLE, nr_pages); + + kmem_cache_get_ref(cachep); + for (i = 0; i < nr_pages; i++) __SetPageSlab(page + i); @@ -1850,6 +1858,14 @@ static void *kmem_getpages(struct kmem_cache *cachep, gfp_t flags, int nodeid) return page_address(page); } +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM +void kmem_cache_drop_ref(struct kmem_cache *cachep) +{ + if (cachep->memcg_params.id == -1) + atomic_dec(&cachep->memcg_params.refcnt); +} +#endif /* CONFIG_CGROUP_MEM_RES_CTLR_KMEM */ + /* * Interface to system's page release. */ @@ -1867,6 +1883,8 @@ static void kmem_freepages(struct kmem_cache *cachep, void *addr) else sub_zone_page_state(page_zone(page), NR_SLAB_UNRECLAIMABLE, nr_freed); + mem_cgroup_uncharge_slab(cachep, i * PAGE_SIZE); + kmem_cache_drop_ref(cachep); while (i--) { BUG_ON(!PageSlab(page)); __ClearPageSlab(page); @@ -2854,10 +2872,11 @@ void kmem_cache_destroy(struct kmem_cache *cachep) #ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM /* Not a memcg cache */ - if (cachep->memcg_params.id != -1) + if (cachep->memcg_params.id != -1) { mem_cgroup_release_cache(cachep); + mem_cgroup_flush_cache_create_queue(); + } #endif - __kmem_cache_destroy(cachep); mutex_unlock(&cache_chain_mutex); put_online_cpus(); @@ -3063,8 +3082,10 @@ static int cache_grow(struct kmem_cache *cachep, offset *= cachep->colour_off; - if (local_flags & __GFP_WAIT) + if (local_flags & __GFP_WAIT) { local_irq_enable(); + mem_cgroup_kmem_cache_prepare_sleep(cachep); + } /* * The test for missing atomic flag is performed here, rather than @@ -3093,8 +3114,10 @@ static int cache_grow(struct kmem_cache *cachep, cache_init_objs(cachep, slabp); - if (local_flags & __GFP_WAIT) + if (local_flags & __GFP_WAIT) { local_irq_disable(); + mem_cgroup_kmem_cache_finish_sleep(cachep); + } check_irq_off(); spin_lock(&l3->list_lock); @@ -3107,8 +3130,10 @@ static int cache_grow(struct kmem_cache *cachep, opps1: kmem_freepages(cachep, objp); failed: - if (local_flags & __GFP_WAIT) + if (local_flags & __GFP_WAIT) { local_irq_disable(); + mem_cgroup_kmem_cache_finish_sleep(cachep); + } return 0; } @@ -3869,11 +3894,15 @@ static inline void __cache_free(struct kmem_cache *cachep, void *objp, */ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { - void *ret = __cache_alloc(cachep, flags, __builtin_return_address(0)); + void *ret; + + rcu_read_lock(); + cachep = mem_cgroup_get_kmem_cache(cachep, flags); + rcu_read_unlock(); + ret = __cache_alloc(cachep, flags, __builtin_return_address(0)); trace_kmem_cache_alloc(_RET_IP_, ret, obj_size(cachep), cachep->buffer_size, flags); - return ret; } EXPORT_SYMBOL(kmem_cache_alloc); @@ -3884,6 +3913,10 @@ kmem_cache_alloc_trace(size_t size, struct kmem_cache *cachep, gfp_t flags) { void *ret; + rcu_read_lock(); + cachep = mem_cgroup_get_kmem_cache(cachep, flags); + rcu_read_unlock(); + ret = __cache_alloc(cachep, flags, __builtin_return_address(0)); trace_kmalloc(_RET_IP_, ret, @@ -3896,13 +3929,17 @@ EXPORT_SYMBOL(kmem_cache_alloc_trace); #ifdef CONFIG_NUMA void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { - void *ret = __cache_alloc_node(cachep, flags, nodeid, + void *ret; + + rcu_read_lock(); + cachep = mem_cgroup_get_kmem_cache(cachep, flags); + rcu_read_unlock(); + ret = __cache_alloc_node(cachep, flags, nodeid, __builtin_return_address(0)); trace_kmem_cache_alloc_node(_RET_IP_, ret, obj_size(cachep), cachep->buffer_size, flags, nodeid); - return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node); @@ -3915,6 +3952,9 @@ void *kmem_cache_alloc_node_trace(size_t size, { void *ret; + rcu_read_lock(); + cachep = mem_cgroup_get_kmem_cache(cachep, flags); + rcu_read_unlock(); ret = __cache_alloc_node(cachep, flags, nodeid, __builtin_return_address(0)); trace_kmalloc_node(_RET_IP_, ret, @@ -4023,9 +4063,33 @@ void kmem_cache_free(struct kmem_cache *cachep, void *objp) local_irq_save(flags); debug_check_no_locks_freed(objp, obj_size(cachep)); + +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM + { + struct kmem_cache *actual_cachep; + + actual_cachep = virt_to_cache(objp); + if (actual_cachep != cachep) { + VM_BUG_ON(actual_cachep->memcg_params.id != -1); + cachep = actual_cachep; + } + /* + * Grab a reference so that the cache is guaranteed to stay + * around. + * If we are freeing the last object of a dead memcg cache, + * the kmem_cache_drop_ref() at the end of this function + * will end up freeing the cache. + */ + kmem_cache_get_ref(cachep); + } +#endif + if (!(cachep->flags & SLAB_DEBUG_OBJECTS)) debug_check_no_obj_freed(objp, obj_size(cachep)); __cache_free(cachep, objp, __builtin_return_address(0)); + + kmem_cache_drop_ref(cachep); + local_irq_restore(flags); trace_kmem_cache_free(_RET_IP_, objp); @@ -4053,9 +4117,19 @@ void kfree(const void *objp) local_irq_save(flags); kfree_debugcheck(objp); c = virt_to_cache(objp); + + /* + * Grab a reference so that the cache is guaranteed to stay around. + * If we are freeing the last object of a dead memcg cache, the + * kmem_cache_drop_ref() at the end of this function will end up + * freeing the cache. + */ + kmem_cache_get_ref(c); + debug_check_no_locks_freed(objp, obj_size(c)); debug_check_no_obj_freed(objp, obj_size(c)); __cache_free(c, (void *)objp, __builtin_return_address(0)); + kmem_cache_drop_ref(c); local_irq_restore(flags); } EXPORT_SYMBOL(kfree); @@ -4324,6 +4398,13 @@ static void cache_reap(struct work_struct *w) list_for_each_entry(searchp, &cache_chain, next) { check_irq_on(); +#ifdef CONFIG_CGROUP_MEM_RES_CTLR_KMEM + /* For memcg caches, make sure we only reap the active ones. */ + if (searchp->memcg_params.id == -1 && + !atomic_add_unless(&searchp->memcg_params.refcnt, 1, 0)) + continue; +#endif + /* * We only take the l3 lock if absolutely necessary and we * have established with reasonable certainty that @@ -4356,6 +4437,7 @@ static void cache_reap(struct work_struct *w) STATS_ADD_REAPED(searchp, freed); } next: + kmem_cache_drop_ref(searchp); cond_resched(); } check_irq_on(); -- 1.7.7.6 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2012-05-11 17:51 UTC|newest] Thread overview: 167+ messages / expand[flat|nested] mbox.gz Atom feed top 2012-05-11 17:44 [PATCH v2 00/29] kmem limitation for memcg Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 01/29] slab: dup name string Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-15 22:04 ` David Rientjes 2012-05-15 22:04 ` David Rientjes 2012-05-16 6:12 ` Glauber Costa 2012-05-16 6:12 ` Glauber Costa 2012-05-16 6:12 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 02/29] slub: fix slab_state for slub Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:51 ` Christoph Lameter 2012-05-11 17:51 ` Christoph Lameter 2012-05-15 21:55 ` David Rientjes 2012-05-15 21:55 ` David Rientjes 2012-05-15 21:55 ` David Rientjes 2012-05-16 6:10 ` Glauber Costa 2012-05-16 6:10 ` Glauber Costa 2012-05-16 6:10 ` Glauber Costa 2012-05-17 10:14 ` Glauber Costa 2012-05-17 10:14 ` Glauber Costa 2012-05-17 10:14 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 03/29] memcg: Always free struct memcg through schedule_work() Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 04/29] slub: always get the cache from its page in kfree Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:53 ` Christoph Lameter 2012-05-11 17:53 ` Christoph Lameter 2012-05-11 17:57 ` Glauber Costa 2012-05-11 17:57 ` Glauber Costa 2012-05-11 17:57 ` Glauber Costa 2012-05-11 18:06 ` Christoph Lameter 2012-05-11 18:06 ` Christoph Lameter 2012-05-11 18:11 ` Glauber Costa 2012-05-11 18:11 ` Glauber Costa 2012-05-11 18:11 ` Glauber Costa 2012-05-11 18:17 ` Christoph Lameter 2012-05-11 18:17 ` Christoph Lameter 2012-05-11 18:20 ` Glauber Costa 2012-05-11 18:20 ` Glauber Costa 2012-05-11 18:20 ` Glauber Costa 2012-05-11 18:32 ` Christoph Lameter 2012-05-11 18:32 ` Christoph Lameter 2012-05-11 18:32 ` Christoph Lameter 2012-05-11 18:42 ` Glauber Costa 2012-05-11 18:42 ` Glauber Costa 2012-05-11 18:42 ` Glauber Costa 2012-05-11 18:56 ` Christoph Lameter 2012-05-11 18:56 ` Christoph Lameter 2012-05-11 18:56 ` Christoph Lameter 2012-05-11 18:58 ` Glauber Costa 2012-05-11 18:58 ` Glauber Costa 2012-05-11 18:58 ` Glauber Costa 2012-05-11 19:09 ` Christoph Lameter 2012-05-11 19:09 ` Christoph Lameter 2012-05-11 19:11 ` Glauber Costa 2012-05-11 19:11 ` Glauber Costa 2012-05-11 19:11 ` Glauber Costa 2012-05-11 19:20 ` Christoph Lameter 2012-05-11 19:20 ` Christoph Lameter 2012-05-11 19:24 ` Glauber Costa 2012-05-11 19:24 ` Glauber Costa 2012-05-11 19:24 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 05/29] slab: rename gfpflags to allocflags Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:54 ` Christoph Lameter 2012-05-11 17:54 ` Christoph Lameter 2012-05-15 21:57 ` David Rientjes 2012-05-15 21:57 ` David Rientjes 2012-05-11 17:44 ` [PATCH v2 06/29] memcg: Make it possible to use the stock for more than one page Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 07/29] memcg: Reclaim when more than one page needed Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 08/29] slab: use obj_size field of struct kmem_cache when not debugging Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 09/29] memcg: change defines to an enum Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 10/29] res_counter: don't force return value checking in res_counter_charge_nofail Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 11/29] cgroups: ability to stop res charge propagation on bounded ancestor Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-15 2:59 ` KAMEZAWA Hiroyuki 2012-05-15 2:59 ` KAMEZAWA Hiroyuki 2012-05-16 6:16 ` Glauber Costa 2012-05-16 6:16 ` Glauber Costa 2012-05-16 6:16 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 12/29] kmem slab accounting basic infrastructure Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 13/29] slab/slub: struct memcg_params Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 14/29] slub: consider a memcg parameter in kmem_create_cache Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 15/29] slab: pass memcg parameter to kmem_cache_create Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 16/29] slub: create duplicate cache Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 17/29] slab: " Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 18/29] memcg: kmem controller charge/uncharge infrastructure Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-15 2:57 ` KAMEZAWA Hiroyuki 2012-05-15 2:57 ` KAMEZAWA Hiroyuki 2012-05-16 6:42 ` Glauber Costa 2012-05-16 6:42 ` Glauber Costa 2012-05-16 8:18 ` KAMEZAWA Hiroyuki 2012-05-16 8:18 ` KAMEZAWA Hiroyuki 2012-05-16 8:25 ` Glauber Costa 2012-05-16 8:25 ` Glauber Costa 2012-05-16 8:25 ` Glauber Costa 2012-05-16 9:15 ` KAMEZAWA Hiroyuki 2012-05-16 9:15 ` KAMEZAWA Hiroyuki 2012-05-16 9:15 ` KAMEZAWA Hiroyuki 2012-05-11 17:44 ` [PATCH v2 19/29] skip memcg kmem allocations in specified code regions Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-15 2:46 ` KAMEZAWA Hiroyuki 2012-05-15 2:46 ` KAMEZAWA Hiroyuki 2012-05-16 6:19 ` Glauber Costa 2012-05-16 6:19 ` Glauber Costa 2012-05-16 6:19 ` Glauber Costa 2012-05-16 7:55 ` KAMEZAWA Hiroyuki 2012-05-16 7:55 ` KAMEZAWA Hiroyuki 2012-05-16 7:55 ` KAMEZAWA Hiroyuki 2012-05-11 17:44 ` [PATCH v2 20/29] slub: charge allocation to a memcg Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa [this message] 2012-05-11 17:44 ` [PATCH v2 21/29] slab: per-memcg accounting of slab caches Glauber Costa 2012-05-11 17:44 ` [PATCH v2 22/29] memcg: disable kmem code when not in use Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 23/29] memcg: destroy memcg caches Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 24/29] memcg/slub: shrink dead caches Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 25/29] memcg: Track all the memcg children of a kmem_cache Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 26/29] memcg: Per-memcg memory.kmem.slabinfo file Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 27/29] slub: create slabinfo file for memcg Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 28/29] slub: track all children of a kmem cache Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 17:44 ` [PATCH v2 29/29] Documentation: add documentation for slab tracker for memcg Glauber Costa 2012-05-11 17:44 ` Glauber Costa 2012-05-11 18:05 ` [PATCH v2 00/29] kmem limitation " Glauber Costa 2012-05-11 18:05 ` Glauber Costa
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1336758272-24284-22-git-send-email-glommer@parallels.com \ --to=glommer@parallels.com \ --cc=cgroups@vger.kernel.org \ --cc=cl@linux.com \ --cc=devel@openvz.org \ --cc=gthelen@google.com \ --cc=hannes@cmpxchg.org \ --cc=kamezawa.hiroyu@jp.fujitsu.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lizefan@huawei.com \ --cc=mhocko@suse.cz \ --cc=penberg@cs.helsinki.fi \ --cc=suleiman@google.com \ --cc=tj@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.