[PATCH v8 0/2] Directed kmem charging

[PATCH v8 0/2] Directed kmem charging

[Shakeel Butt]

The Linux kernel's memory cgroup allows limiting the memory usage of
the jobs running on the system to provide isolation between the jobs.
All the kernel memory allocated in the context of the job and marked
with __GFP_ACCOUNT will also be included in the memory usage and be
limited by the job's limit.

The kernel memory can only be charged to the memcg of the process in
whose context kernel memory was allocated. However there are cases where
the allocated kernel memory should be charged to the memcg different
from the current processes's memcg. This patch series contains two such
concrete use-cases i.e. fsnotify and buffer_head.

The fsnotify event objects can consume a lot of system memory for large
or unlimited queues if there is either no or slow listener. The events
are allocated in the context of the event producer. However they should
be charged to the event consumer. Similarly the buffer_head objects can
be allocated in a memcg different from the memcg of the page for which
buffer_head objects are being allocated.

To solve this issue, this patch series introduces mechanism to charge
kernel memory to a given memcg. In case of fsnotify events, the memcg of
the consumer can be used for charging and for buffer_head, the memcg of
the page can be charged. For directed charging, the caller can use the
scope API memalloc_[un]use_memcg() to specify the memcg to charge for
all the __GFP_ACCOUNT allocations within the scope.

Shakeel Butt (2):
  fs: fsnotify: account fsnotify metadata to kmemcg
  fs, mm: account buffer_head to kmemcg

 fs/buffer.c                          | 15 ++++++-
 fs/notify/dnotify/dnotify.c          |  5 ++-
 fs/notify/fanotify/fanotify.c        | 17 ++++++--
 fs/notify/fanotify/fanotify_user.c   |  5 ++-
 fs/notify/group.c                    |  4 ++
 fs/notify/inotify/inotify_fsnotify.c | 15 ++++++-
 fs/notify/inotify/inotify_user.c     |  5 ++-
 include/linux/fsnotify_backend.h     | 12 ++++--
 include/linux/memcontrol.h           | 14 +++++++
 include/linux/sched.h                |  3 ++
 include/linux/sched/mm.h             | 41 +++++++++++++++++++
 kernel/fork.c                        |  3 ++
 mm/memcontrol.c                      | 60 ++++++++++++++++++++++++++--
 13 files changed, 182 insertions(+), 17 deletions(-)

-- 
2.18.0.rc2.346.g013aa6912e-goog

[PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Shakeel Butt]

A lot of memory can be consumed by the events generated for the huge or
unlimited queues if there is either no or slow listener.  This can cause
system level memory pressure or OOMs.  So, it's better to account the
fsnotify kmem caches to the memcg of the listener.

However the listener can be in a different memcg than the memcg of the
producer and these allocations happen in the context of the event
producer. This patch introduces remote memcg charging API which the
producer can use to charge the allocations to the memcg of the listener.

There are seven fsnotify kmem caches and among them allocations from
dnotify_struct_cache, dnotify_mark_cache, fanotify_mark_cache and
inotify_inode_mark_cachep happens in the context of syscall from the
listener.  So, SLAB_ACCOUNT is enough for these caches.

The objects from fsnotify_mark_connector_cachep are not accounted as they
are small compared to the notification mark or events and it is unclear
whom to account connector to since it is shared by all events attached to
the inode.

The allocations from the event caches happen in the context of the event
producer.  For such caches we will need to remote charge the allocations
to the listener's memcg.  Thus we save the memcg reference in the
fsnotify_group structure of the listener.

This patch has also moved the members of fsnotify_group to keep the size
same, at least for 64 bit build, even with additional member by filling
the holes.

Signed-off-by: Shakeel Butt <shakeelb@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Amir Goldstein <amir73il@gmail.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Roman Gushchin <guro@fb.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
---
Changelog since v7:
- Simplify the remote memcg charging API
- get_mem_cgroup_from_current() returns root_mem_cgroup if
  current->active_memcg is not NULL but css_tryget_online() fails.

Changelog since v6:
- Removed Jan's ACK as the code has changed a lot
- Squashed the separate remote charging API path into this one
- Removed kmalloc* & kmem_cache_alloc* APIs and only kept the scope API
- Changed fsnotify remote charging code to use scope API

Changelog since v5:
- None

Changelog since v4:
- Fixed the build for CONFIG_MEMCG=n

Changelog since v3:
- Rebased over Jan's patches.
- Some cleanup based on Amir's comments.

Changelog since v2:
- None

Changelog since v1:
- no more charging fsnotify_mark_connector objects
- Fixed the build for SLOB

 fs/notify/dnotify/dnotify.c          |  5 ++--
 fs/notify/fanotify/fanotify.c        | 14 ++++++++---
 fs/notify/fanotify/fanotify_user.c   |  5 +++-
 fs/notify/group.c                    |  3 +++
 fs/notify/inotify/inotify_fsnotify.c |  7 +++++-
 fs/notify/inotify/inotify_user.c     |  5 +++-
 include/linux/fsnotify_backend.h     | 12 ++++++---
 include/linux/memcontrol.h           | 10 +++++++-
 include/linux/sched.h                |  3 +++
 include/linux/sched/mm.h             | 37 ++++++++++++++++++++++++++++
 kernel/fork.c                        |  3 +++
 mm/memcontrol.c                      | 37 +++++++++++++++++++++++++---
 12 files changed, 123 insertions(+), 18 deletions(-)

diff --git a/fs/notify/dnotify/dnotify.c b/fs/notify/dnotify/dnotify.c
index e2bea2ac5dfb..a6365e6bc047 100644
--- a/fs/notify/dnotify/dnotify.c
+++ b/fs/notify/dnotify/dnotify.c
@@ -384,8 +384,9 @@ int fcntl_dirnotify(int fd, struct file *filp, unsigned long arg)
 
 static int __init dnotify_init(void)
 {
-	dnotify_struct_cache = KMEM_CACHE(dnotify_struct, SLAB_PANIC);
-	dnotify_mark_cache = KMEM_CACHE(dnotify_mark, SLAB_PANIC);
+	dnotify_struct_cache = KMEM_CACHE(dnotify_struct,
+					  SLAB_PANIC|SLAB_ACCOUNT);
+	dnotify_mark_cache = KMEM_CACHE(dnotify_mark, SLAB_PANIC|SLAB_ACCOUNT);
 
 	dnotify_group = fsnotify_alloc_group(&dnotify_fsnotify_ops);
 	if (IS_ERR(dnotify_group))
diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index f90842efea13..6ff1f75d156d 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -11,6 +11,7 @@
 #include <linux/types.h>
 #include <linux/wait.h>
 #include <linux/audit.h>
+#include <linux/sched/mm.h>
 
 #include "fanotify.h"
 
@@ -140,8 +141,8 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
 						 struct inode *inode, u32 mask,
 						 const struct path *path)
 {
-	struct fanotify_event_info *event;
-	gfp_t gfp = GFP_KERNEL;
+	struct fanotify_event_info *event = NULL;
+	gfp_t gfp = GFP_KERNEL | __GFP_ACCOUNT;
 
 	/*
 	 * For queues with unlimited length lost events are not expected and
@@ -151,19 +152,22 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
 	if (group->max_events == UINT_MAX)
 		gfp |= __GFP_NOFAIL;
 
+	/* Whoever is interested in the event, pays for the allocation. */
+	memalloc_use_memcg(group->memcg);
+
 	if (fanotify_is_perm_event(mask)) {
 		struct fanotify_perm_event_info *pevent;
 
 		pevent = kmem_cache_alloc(fanotify_perm_event_cachep, gfp);
 		if (!pevent)
-			return NULL;
+			goto out;
 		event = &pevent->fae;
 		pevent->response = 0;
 		goto init;
 	}
 	event = kmem_cache_alloc(fanotify_event_cachep, gfp);
 	if (!event)
-		return NULL;
+		goto out;
 init: __maybe_unused
 	fsnotify_init_event(&event->fse, inode, mask);
 	event->tgid = get_pid(task_tgid(current));
@@ -174,6 +178,8 @@ init: __maybe_unused
 		event->path.mnt = NULL;
 		event->path.dentry = NULL;
 	}
+out:
+	memalloc_unuse_memcg();
 	return event;
 }
 
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index ec4d8c59d0e3..0cf45041dc32 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -16,6 +16,7 @@
 #include <linux/uaccess.h>
 #include <linux/compat.h>
 #include <linux/sched/signal.h>
+#include <linux/memcontrol.h>
 
 #include <asm/ioctls.h>
 
@@ -756,6 +757,7 @@ SYSCALL_DEFINE2(fanotify_init, unsigned int, flags, unsigned int, event_f_flags)
 
 	group->fanotify_data.user = user;
 	atomic_inc(&user->fanotify_listeners);
+	group->memcg = get_mem_cgroup_from_mm(current->mm);
 
 	oevent = fanotify_alloc_event(group, NULL, FS_Q_OVERFLOW, NULL);
 	if (unlikely(!oevent)) {
@@ -957,7 +959,8 @@ COMPAT_SYSCALL_DEFINE6(fanotify_mark,
  */
 static int __init fanotify_user_setup(void)
 {
-	fanotify_mark_cache = KMEM_CACHE(fsnotify_mark, SLAB_PANIC);
+	fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
+					 SLAB_PANIC|SLAB_ACCOUNT);
 	fanotify_event_cachep = KMEM_CACHE(fanotify_event_info, SLAB_PANIC);
 	if (IS_ENABLED(CONFIG_FANOTIFY_ACCESS_PERMISSIONS)) {
 		fanotify_perm_event_cachep =
diff --git a/fs/notify/group.c b/fs/notify/group.c
index aa5468f23e45..c03b83662876 100644
--- a/fs/notify/group.c
+++ b/fs/notify/group.c
@@ -22,6 +22,7 @@
 #include <linux/srcu.h>
 #include <linux/rculist.h>
 #include <linux/wait.h>
+#include <linux/memcontrol.h>
 
 #include <linux/fsnotify_backend.h>
 #include "fsnotify.h"
@@ -36,6 +37,8 @@ static void fsnotify_final_destroy_group(struct fsnotify_group *group)
 	if (group->ops->free_group_priv)
 		group->ops->free_group_priv(group);
 
+	mem_cgroup_put(group->memcg);
+
 	kfree(group);
 }
 
diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
index 9ab6dde38a14..52e167d04b11 100644
--- a/fs/notify/inotify/inotify_fsnotify.c
+++ b/fs/notify/inotify/inotify_fsnotify.c
@@ -31,6 +31,7 @@
 #include <linux/types.h>
 #include <linux/sched.h>
 #include <linux/sched/user.h>
+#include <linux/sched/mm.h>
 
 #include "inotify.h"
 
@@ -98,7 +99,11 @@ int inotify_handle_event(struct fsnotify_group *group,
 	i_mark = container_of(inode_mark, struct inotify_inode_mark,
 			      fsn_mark);
 
-	event = kmalloc(alloc_len, GFP_KERNEL);
+	/* Whoever is interested in the event, pays for the allocation. */
+	memalloc_use_memcg(group->memcg);
+	event = kmalloc(alloc_len, GFP_KERNEL | __GFP_ACCOUNT);
+	memalloc_unuse_memcg();
+
 	if (unlikely(!event)) {
 		/*
 		 * Treat lost event due to ENOMEM the same way as queue
diff --git a/fs/notify/inotify/inotify_user.c b/fs/notify/inotify/inotify_user.c
index 1cf5b779d862..749c46ababa0 100644
--- a/fs/notify/inotify/inotify_user.c
+++ b/fs/notify/inotify/inotify_user.c
@@ -38,6 +38,7 @@
 #include <linux/uaccess.h>
 #include <linux/poll.h>
 #include <linux/wait.h>
+#include <linux/memcontrol.h>
 
 #include "inotify.h"
 #include "../fdinfo.h"
@@ -636,6 +637,7 @@ static struct fsnotify_group *inotify_new_group(unsigned int max_events)
 	oevent->name_len = 0;
 
 	group->max_events = max_events;
+	group->memcg = get_mem_cgroup_from_mm(current->mm);
 
 	spin_lock_init(&group->inotify_data.idr_lock);
 	idr_init(&group->inotify_data.idr);
@@ -808,7 +810,8 @@ static int __init inotify_user_setup(void)
 
 	BUG_ON(hweight32(ALL_INOTIFY_BITS) != 21);
 
-	inotify_inode_mark_cachep = KMEM_CACHE(inotify_inode_mark, SLAB_PANIC);
+	inotify_inode_mark_cachep = KMEM_CACHE(inotify_inode_mark,
+					       SLAB_PANIC|SLAB_ACCOUNT);
 
 	inotify_max_queued_events = 16384;
 	init_user_ns.ucount_max[UCOUNT_INOTIFY_INSTANCES] = 128;
diff --git a/include/linux/fsnotify_backend.h b/include/linux/fsnotify_backend.h
index b38964a7a521..a0c4790c5302 100644
--- a/include/linux/fsnotify_backend.h
+++ b/include/linux/fsnotify_backend.h
@@ -84,6 +84,8 @@ struct fsnotify_event_private_data;
 struct fsnotify_fname;
 struct fsnotify_iter_info;
 
+struct mem_cgroup;
+
 /*
  * Each group much define these ops.  The fsnotify infrastructure will call
  * these operations for each relevant group.
@@ -127,6 +129,8 @@ struct fsnotify_event {
  * everything will be cleaned up.
  */
 struct fsnotify_group {
+	const struct fsnotify_ops *ops;	/* how this group handles things */
+
 	/*
 	 * How the refcnt is used is up to each group.  When the refcnt hits 0
 	 * fsnotify will clean up all of the resources associated with this group.
@@ -137,8 +141,6 @@ struct fsnotify_group {
 	 */
 	refcount_t refcnt;		/* things with interest in this group */
 
-	const struct fsnotify_ops *ops;	/* how this group handles things */
-
 	/* needed to send notification to userspace */
 	spinlock_t notification_lock;		/* protect the notification_list */
 	struct list_head notification_list;	/* list of event_holder this group needs to send to userspace */
@@ -160,6 +162,8 @@ struct fsnotify_group {
 	atomic_t num_marks;		/* 1 for each mark and 1 for not being
 					 * past the point of no return when freeing
 					 * a group */
+	atomic_t user_waits;		/* Number of tasks waiting for user
+					 * response */
 	struct list_head marks_list;	/* all inode marks for this group */
 
 	struct fasync_struct *fsn_fa;    /* async notification */
@@ -167,8 +171,8 @@ struct fsnotify_group {
 	struct fsnotify_event *overflow_event;	/* Event we queue when the
 						 * notification list is too
 						 * full */
-	atomic_t user_waits;		/* Number of tasks waiting for user
-					 * response */
+
+	struct mem_cgroup *memcg;	/* memcg to charge allocations */
 
 	/* groups can define private fields here or use the void *private */
 	union {
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 3607913032be..cb04b382c8d2 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -378,6 +378,8 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *, struct pglist_data *);
 bool task_in_mem_cgroup(struct task_struct *task, struct mem_cgroup *memcg);
 struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
 
+struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
+
 static inline
 struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css){
 	return css ? container_of(css, struct mem_cgroup, css) : NULL;
@@ -385,7 +387,8 @@ struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css){
 
 static inline void mem_cgroup_put(struct mem_cgroup *memcg)
 {
-	css_put(&memcg->css);
+	if (memcg)
+		css_put(&memcg->css);
 }
 
 #define mem_cgroup_from_counter(counter, member)	\
@@ -857,6 +860,11 @@ static inline bool task_in_mem_cgroup(struct task_struct *task,
 	return true;
 }
 
+static inline struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
+{
+	return NULL;
+}
+
 static inline void mem_cgroup_put(struct mem_cgroup *memcg)
 {
 }
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 87bf02d93a27..9cba7f874443 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1149,6 +1149,9 @@ struct task_struct {
 
 	/* Number of pages to reclaim on returning to userland: */
 	unsigned int			memcg_nr_pages_over_high;
+
+	/* Used by memcontrol for targeted memcg charge: */
+	struct mem_cgroup		*active_memcg;
 #endif
 
 #ifdef CONFIG_UPROBES
diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h
index 44d356f5e47c..2ac55340cb73 100644
--- a/include/linux/sched/mm.h
+++ b/include/linux/sched/mm.h
@@ -248,6 +248,43 @@ static inline void memalloc_noreclaim_restore(unsigned int flags)
 	current->flags = (current->flags & ~PF_MEMALLOC) | flags;
 }
 
+#ifdef CONFIG_MEMCG
+/**
+ * memalloc_use_memcg - Starts the remote memcg charging scope.
+ * @memcg: memcg to charge.
+ *
+ * This function marks the beginning of the remote memcg charging scope. All the
+ * __GFP_ACCOUNT allocations till the end of the scope will be charged to the
+ * given memcg.
+ *
+ * NOTE: This function is not nesting safe.
+ */
+static inline void memalloc_use_memcg(struct mem_cgroup *memcg)
+{
+	WARN_ON_ONCE(current->active_memcg);
+	current->active_memcg = memcg;
+}
+
+/**
+ * memalloc_unuse_memcg - Ends the remote memcg charging scope.
+ *
+ * This function marks the end of the remote memcg charging scope started by
+ * memalloc_use_memcg().
+ */
+static inline void memalloc_unuse_memcg(void)
+{
+	current->active_memcg = NULL;
+}
+#else
+static inline void memalloc_use_memcg(struct mem_cgroup *memcg)
+{
+}
+
+static inline void memalloc_unuse_memcg(void)
+{
+}
+#endif
+
 #ifdef CONFIG_MEMBARRIER
 enum {
 	MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY		= (1U << 0),
diff --git a/kernel/fork.c b/kernel/fork.c
index a64d0a19f174..0b07db6fef06 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -843,6 +843,9 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node)
 	tsk->fail_nth = 0;
 #endif
 
+#ifdef CONFIG_MEMCG
+	tsk->active_memcg = NULL;
+#endif
 	return tsk;
 
 free_stack:
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 02e77c88967a..b25ca5c13196 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -678,9 +678,20 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p)
 }
 EXPORT_SYMBOL(mem_cgroup_from_task);
 
-static struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
+/**
+ * get_mem_cgroup_from_mm: Obtain a reference on given mm_struct's memcg.
+ * @mm: mm from which memcg should be extracted. It can be NULL.
+ *
+ * Obtain a reference on mm->memcg and returns it if successful. Otherwise
+ * root_mem_cgroup is returned. However if mem_cgroup is disabled, NULL is
+ * returned.
+ */
+struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
 {
-	struct mem_cgroup *memcg = NULL;
+	struct mem_cgroup *memcg;
+
+	if (mem_cgroup_disabled())
+		return NULL;
 
 	rcu_read_lock();
 	do {
@@ -700,6 +711,24 @@ static struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
 	rcu_read_unlock();
 	return memcg;
 }
+EXPORT_SYMBOL(get_mem_cgroup_from_mm);
+
+/**
+ * If current->active_memcg is non-NULL, do not fallback to current->mm->memcg.
+ */
+static __always_inline struct mem_cgroup *get_mem_cgroup_from_current(void)
+{
+	if (unlikely(current->active_memcg)) {
+		struct mem_cgroup *memcg = root_mem_cgroup;
+
+		rcu_read_lock();
+		if (css_tryget_online(&current->active_memcg->css))
+			memcg = current->active_memcg;
+		rcu_read_unlock();
+		return memcg;
+	}
+	return get_mem_cgroup_from_mm(current->mm);
+}
 
 /**
  * mem_cgroup_iter - iterate over memory cgroup hierarchy
@@ -2260,7 +2289,7 @@ struct kmem_cache *memcg_kmem_get_cache(struct kmem_cache *cachep)
 	if (current->memcg_kmem_skip_account)
 		return cachep;
 
-	memcg = get_mem_cgroup_from_mm(current->mm);
+	memcg = get_mem_cgroup_from_current();
 	kmemcg_id = READ_ONCE(memcg->kmemcg_id);
 	if (kmemcg_id < 0)
 		goto out;
@@ -2344,7 +2373,7 @@ int memcg_kmem_charge(struct page *page, gfp_t gfp, int order)
 	if (memcg_kmem_bypass())
 		return 0;
 
-	memcg = get_mem_cgroup_from_mm(current->mm);
+	memcg = get_mem_cgroup_from_current();
 	if (!mem_cgroup_is_root(memcg)) {
 		ret = memcg_kmem_charge_memcg(page, gfp, order, memcg);
 		if (!ret)
-- 
2.18.0.rc2.346.g013aa6912e-goog

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Jan Kara]

On Wed 27-06-18 12:12:49, Shakeel Butt wrote:
> A lot of memory can be consumed by the events generated for the huge or
> unlimited queues if there is either no or slow listener.  This can cause
> system level memory pressure or OOMs.  So, it's better to account the
> fsnotify kmem caches to the memcg of the listener.
> 
> However the listener can be in a different memcg than the memcg of the
> producer and these allocations happen in the context of the event
> producer. This patch introduces remote memcg charging API which the
> producer can use to charge the allocations to the memcg of the listener.
> 
> There are seven fsnotify kmem caches and among them allocations from
> dnotify_struct_cache, dnotify_mark_cache, fanotify_mark_cache and
> inotify_inode_mark_cachep happens in the context of syscall from the
> listener.  So, SLAB_ACCOUNT is enough for these caches.
> 
> The objects from fsnotify_mark_connector_cachep are not accounted as they
> are small compared to the notification mark or events and it is unclear
> whom to account connector to since it is shared by all events attached to
> the inode.
> 
> The allocations from the event caches happen in the context of the event
> producer.  For such caches we will need to remote charge the allocations
> to the listener's memcg.  Thus we save the memcg reference in the
> fsnotify_group structure of the listener.
> 
> This patch has also moved the members of fsnotify_group to keep the size
> same, at least for 64 bit build, even with additional member by filling
> the holes.

...

>  static int __init fanotify_user_setup(void)
>  {
> -	fanotify_mark_cache = KMEM_CACHE(fsnotify_mark, SLAB_PANIC);
> +	fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
> +					 SLAB_PANIC|SLAB_ACCOUNT);
>  	fanotify_event_cachep = KMEM_CACHE(fanotify_event_info, SLAB_PANIC);
>  	if (IS_ENABLED(CONFIG_FANOTIFY_ACCESS_PERMISSIONS)) {
>  		fanotify_perm_event_cachep =

Why don't you setup also fanotify_event_cachep and
fanotify_perm_event_cachep caches with SLAB_ACCOUNT and instead specify
__GFP_ACCOUNT manually? Otherwise the patch looks good to me.

								Honza

-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Shakeel Butt]

On Thu, Jun 28, 2018 at 12:03 PM Jan Kara <jack@suse.cz> wrote:
>
> On Wed 27-06-18 12:12:49, Shakeel Butt wrote:
> > A lot of memory can be consumed by the events generated for the huge or
> > unlimited queues if there is either no or slow listener.  This can cause
> > system level memory pressure or OOMs.  So, it's better to account the
> > fsnotify kmem caches to the memcg of the listener.
> >
> > However the listener can be in a different memcg than the memcg of the
> > producer and these allocations happen in the context of the event
> > producer. This patch introduces remote memcg charging API which the
> > producer can use to charge the allocations to the memcg of the listener.
> >
> > There are seven fsnotify kmem caches and among them allocations from
> > dnotify_struct_cache, dnotify_mark_cache, fanotify_mark_cache and
> > inotify_inode_mark_cachep happens in the context of syscall from the
> > listener.  So, SLAB_ACCOUNT is enough for these caches.
> >
> > The objects from fsnotify_mark_connector_cachep are not accounted as they
> > are small compared to the notification mark or events and it is unclear
> > whom to account connector to since it is shared by all events attached to
> > the inode.
> >
> > The allocations from the event caches happen in the context of the event
> > producer.  For such caches we will need to remote charge the allocations
> > to the listener's memcg.  Thus we save the memcg reference in the
> > fsnotify_group structure of the listener.
> >
> > This patch has also moved the members of fsnotify_group to keep the size
> > same, at least for 64 bit build, even with additional member by filling
> > the holes.
>
> ...
>
> >  static int __init fanotify_user_setup(void)
> >  {
> > -     fanotify_mark_cache = KMEM_CACHE(fsnotify_mark, SLAB_PANIC);
> > +     fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
> > +                                      SLAB_PANIC|SLAB_ACCOUNT);
> >       fanotify_event_cachep = KMEM_CACHE(fanotify_event_info, SLAB_PANIC);
> >       if (IS_ENABLED(CONFIG_FANOTIFY_ACCESS_PERMISSIONS)) {
> >               fanotify_perm_event_cachep =
>
> Why don't you setup also fanotify_event_cachep and
> fanotify_perm_event_cachep caches with SLAB_ACCOUNT and instead specify
> __GFP_ACCOUNT manually? Otherwise the patch looks good to me.
>

Hi Jan, IMHO having a visible __GFP_ACCOUNT along with
memalloc_use_memcg() makes the code more explicit and readable that we
want to targeted/remote memcg charging. However if you think
otherwise, I will replace __GFP_ACCOUNT with SLAB_ACCOUNT.

thanks,
Shakeel

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Michal Hocko]

On Thu 28-06-18 12:21:26, Shakeel Butt wrote:
> On Thu, Jun 28, 2018 at 12:03 PM Jan Kara <jack@suse.cz> wrote:
> >
> > On Wed 27-06-18 12:12:49, Shakeel Butt wrote:
> > > A lot of memory can be consumed by the events generated for the huge or
> > > unlimited queues if there is either no or slow listener.  This can cause
> > > system level memory pressure or OOMs.  So, it's better to account the
> > > fsnotify kmem caches to the memcg of the listener.
> > >
> > > However the listener can be in a different memcg than the memcg of the
> > > producer and these allocations happen in the context of the event
> > > producer. This patch introduces remote memcg charging API which the
> > > producer can use to charge the allocations to the memcg of the listener.
> > >
> > > There are seven fsnotify kmem caches and among them allocations from
> > > dnotify_struct_cache, dnotify_mark_cache, fanotify_mark_cache and
> > > inotify_inode_mark_cachep happens in the context of syscall from the
> > > listener.  So, SLAB_ACCOUNT is enough for these caches.
> > >
> > > The objects from fsnotify_mark_connector_cachep are not accounted as they
> > > are small compared to the notification mark or events and it is unclear
> > > whom to account connector to since it is shared by all events attached to
> > > the inode.
> > >
> > > The allocations from the event caches happen in the context of the event
> > > producer.  For such caches we will need to remote charge the allocations
> > > to the listener's memcg.  Thus we save the memcg reference in the
> > > fsnotify_group structure of the listener.
> > >
> > > This patch has also moved the members of fsnotify_group to keep the size
> > > same, at least for 64 bit build, even with additional member by filling
> > > the holes.
> >
> > ...
> >
> > >  static int __init fanotify_user_setup(void)
> > >  {
> > > -     fanotify_mark_cache = KMEM_CACHE(fsnotify_mark, SLAB_PANIC);
> > > +     fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
> > > +                                      SLAB_PANIC|SLAB_ACCOUNT);
> > >       fanotify_event_cachep = KMEM_CACHE(fanotify_event_info, SLAB_PANIC);
> > >       if (IS_ENABLED(CONFIG_FANOTIFY_ACCESS_PERMISSIONS)) {
> > >               fanotify_perm_event_cachep =
> >
> > Why don't you setup also fanotify_event_cachep and
> > fanotify_perm_event_cachep caches with SLAB_ACCOUNT and instead specify
> > __GFP_ACCOUNT manually? Otherwise the patch looks good to me.
> >
> 
> Hi Jan, IMHO having a visible __GFP_ACCOUNT along with
> memalloc_use_memcg() makes the code more explicit and readable that we
> want to targeted/remote memcg charging.

Agreed. If you had an implicit SLAB_ACCOUNT then you could get
inconsistencies when some allocations would get charged to the current
task while others would not.
-- 
Michal Hocko
SUSE Labs

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Jan Kara]

On Thu 28-06-18 12:21:26, Shakeel Butt wrote:
> On Thu, Jun 28, 2018 at 12:03 PM Jan Kara <jack@suse.cz> wrote:
> >
> > On Wed 27-06-18 12:12:49, Shakeel Butt wrote:
> > > A lot of memory can be consumed by the events generated for the huge or
> > > unlimited queues if there is either no or slow listener.  This can cause
> > > system level memory pressure or OOMs.  So, it's better to account the
> > > fsnotify kmem caches to the memcg of the listener.
> > >
> > > However the listener can be in a different memcg than the memcg of the
> > > producer and these allocations happen in the context of the event
> > > producer. This patch introduces remote memcg charging API which the
> > > producer can use to charge the allocations to the memcg of the listener.
> > >
> > > There are seven fsnotify kmem caches and among them allocations from
> > > dnotify_struct_cache, dnotify_mark_cache, fanotify_mark_cache and
> > > inotify_inode_mark_cachep happens in the context of syscall from the
> > > listener.  So, SLAB_ACCOUNT is enough for these caches.
> > >
> > > The objects from fsnotify_mark_connector_cachep are not accounted as they
> > > are small compared to the notification mark or events and it is unclear
> > > whom to account connector to since it is shared by all events attached to
> > > the inode.
> > >
> > > The allocations from the event caches happen in the context of the event
> > > producer.  For such caches we will need to remote charge the allocations
> > > to the listener's memcg.  Thus we save the memcg reference in the
> > > fsnotify_group structure of the listener.
> > >
> > > This patch has also moved the members of fsnotify_group to keep the size
> > > same, at least for 64 bit build, even with additional member by filling
> > > the holes.
> >
> > ...
> >
> > >  static int __init fanotify_user_setup(void)
> > >  {
> > > -     fanotify_mark_cache = KMEM_CACHE(fsnotify_mark, SLAB_PANIC);
> > > +     fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
> > > +                                      SLAB_PANIC|SLAB_ACCOUNT);
> > >       fanotify_event_cachep = KMEM_CACHE(fanotify_event_info, SLAB_PANIC);
> > >       if (IS_ENABLED(CONFIG_FANOTIFY_ACCESS_PERMISSIONS)) {
> > >               fanotify_perm_event_cachep =
> >
> > Why don't you setup also fanotify_event_cachep and
> > fanotify_perm_event_cachep caches with SLAB_ACCOUNT and instead specify
> > __GFP_ACCOUNT manually? Otherwise the patch looks good to me.
> >
> 
> Hi Jan, IMHO having a visible __GFP_ACCOUNT along with
> memalloc_use_memcg() makes the code more explicit and readable that we
> want to targeted/remote memcg charging. However if you think
> otherwise, I will replace __GFP_ACCOUNT with SLAB_ACCOUNT.

OK, fair enough.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

[PATCH] fs-fsnotify-account-fsnotify-metadata-to-kmemcg.patch.cleanup

[Shakeel Butt]

Hi Andres, this is a small cleanup to the patch "fs: fsnotify: account
fsnotify metadata to kmemcg". Please squash.

Signed-off-by: Shakeel Butt <shakeelb@google.com>
---
 fs/notify/fanotify/fanotify.c        | 2 +-
 fs/notify/inotify/inotify_fsnotify.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index 6ff1f75d156d..eb4e75175cfb 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -142,7 +142,7 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
 						 const struct path *path)
 {
 	struct fanotify_event_info *event = NULL;
-	gfp_t gfp = GFP_KERNEL | __GFP_ACCOUNT;
+	gfp_t gfp = GFP_KERNEL_ACCOUNT;
 
 	/*
 	 * For queues with unlimited length lost events are not expected and
diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
index 52e167d04b11..f4184b4f3815 100644
--- a/fs/notify/inotify/inotify_fsnotify.c
+++ b/fs/notify/inotify/inotify_fsnotify.c
@@ -101,7 +101,7 @@ int inotify_handle_event(struct fsnotify_group *group,
 
 	/* Whoever is interested in the event, pays for the allocation. */
 	memalloc_use_memcg(group->memcg);
-	event = kmalloc(alloc_len, GFP_KERNEL | __GFP_ACCOUNT);
+	event = kmalloc(alloc_len, GFP_KERNEL_ACCOUNT);
 	memalloc_unuse_memcg();
 
 	if (unlikely(!event)) {
-- 
2.18.0.399.gad0ab374a1-goog

Re: [PATCH] fs-fsnotify-account-fsnotify-metadata-to-kmemcg.patch.cleanup

[Shakeel Butt]

On Mon, Jul 2, 2018 at 2:54 PM Shakeel Butt <shakeelb@google.com> wrote:
>
> Hi Andres, this is a small cleanup to the patch "fs: fsnotify: account

*Andrew*

> fsnotify metadata to kmemcg". Please squash.
>
> Signed-off-by: Shakeel Butt <shakeelb@google.com>
> ---
>  fs/notify/fanotify/fanotify.c        | 2 +-
>  fs/notify/inotify/inotify_fsnotify.c | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
> index 6ff1f75d156d..eb4e75175cfb 100644
> --- a/fs/notify/fanotify/fanotify.c
> +++ b/fs/notify/fanotify/fanotify.c
> @@ -142,7 +142,7 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
>                                                  const struct path *path)
>  {
>         struct fanotify_event_info *event = NULL;
> -       gfp_t gfp = GFP_KERNEL | __GFP_ACCOUNT;
> +       gfp_t gfp = GFP_KERNEL_ACCOUNT;
>
>         /*
>          * For queues with unlimited length lost events are not expected and
> diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
> index 52e167d04b11..f4184b4f3815 100644
> --- a/fs/notify/inotify/inotify_fsnotify.c
> +++ b/fs/notify/inotify/inotify_fsnotify.c
> @@ -101,7 +101,7 @@ int inotify_handle_event(struct fsnotify_group *group,
>
>         /* Whoever is interested in the event, pays for the allocation. */
>         memalloc_use_memcg(group->memcg);
> -       event = kmalloc(alloc_len, GFP_KERNEL | __GFP_ACCOUNT);
> +       event = kmalloc(alloc_len, GFP_KERNEL_ACCOUNT);
>         memalloc_unuse_memcg();
>
>         if (unlikely(!event)) {
> --
> 2.18.0.399.gad0ab374a1-goog
>

[PATCH 2/2] fs, mm: account buffer_head to kmemcg

[Shakeel Butt]

The buffer_head can consume a significant amount of system memory and
is directly related to the amount of page cache. In our production
environment we have observed that a lot of machines are spending a
significant amount of memory as buffer_head and can not be left as
system memory overhead.

Charging buffer_head is not as simple as adding __GFP_ACCOUNT to the
allocation. The buffer_heads can be allocated in a memcg different from
the memcg of the page for which buffer_heads are being allocated. One
concrete example is memory reclaim. The reclaim can trigger I/O of pages
of any memcg on the system. So, the right way to charge buffer_head is
to extract the memcg from the page for which buffer_heads are being
allocated and then use targeted memcg charging API.

Signed-off-by: Shakeel Butt <shakeelb@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Amir Goldstein <amir73il@gmail.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Roman Gushchin <guro@fb.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
---
Changelog since v2:
- get_mem_cgroup_from_page() returns root_mem_cgroup if page->memcg is
  either NULL or css_tryget_online fails.

Changelog since v1:
- simple code cleanups

 fs/buffer.c                | 10 +++++++++-
 include/linux/memcontrol.h |  7 +++++++
 mm/memcontrol.c            | 22 ++++++++++++++++++++++
 3 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/fs/buffer.c b/fs/buffer.c
index 8194e3049fc5..235826333936 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -45,6 +45,7 @@
 #include <linux/mpage.h>
 #include <linux/bit_spinlock.h>
 #include <linux/pagevec.h>
+#include <linux/sched/mm.h>
 #include <trace/events/block.h>
 
 static int fsync_buffers_list(spinlock_t *lock, struct list_head *list);
@@ -815,10 +816,14 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size,
 	struct buffer_head *bh, *head;
 	gfp_t gfp = GFP_NOFS;
 	long offset;
+	struct mem_cgroup *memcg;
 
 	if (retry)
 		gfp |= __GFP_NOFAIL;
 
+	memcg = get_mem_cgroup_from_page(page);
+	memalloc_use_memcg(memcg);
+
 	head = NULL;
 	offset = PAGE_SIZE;
 	while ((offset -= size) >= 0) {
@@ -835,6 +840,9 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size,
 		/* Link the buffer to its page */
 		set_bh_page(bh, page, offset);
 	}
+out:
+	memalloc_unuse_memcg();
+	mem_cgroup_put(memcg);
 	return head;
 /*
  * In case anything failed, we just free everything we got.
@@ -848,7 +856,7 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size,
 		} while (head);
 	}
 
-	return NULL;
+	goto out;
 }
 EXPORT_SYMBOL_GPL(alloc_page_buffers);
 
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index cb04b382c8d2..919b98ddda45 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -380,6 +380,8 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
 
 struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
 
+struct mem_cgroup *get_mem_cgroup_from_page(struct page *page);
+
 static inline
 struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css){
 	return css ? container_of(css, struct mem_cgroup, css) : NULL;
@@ -865,6 +867,11 @@ static inline struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
 	return NULL;
 }
 
+static inline struct mem_cgroup *get_mem_cgroup_from_page(struct page *page)
+{
+	return NULL;
+}
+
 static inline void mem_cgroup_put(struct mem_cgroup *memcg)
 {
 }
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index b25ca5c13196..21a7c2fb8097 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -713,6 +713,28 @@ struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
 }
 EXPORT_SYMBOL(get_mem_cgroup_from_mm);
 
+/**
+ * get_mem_cgroup_from_page: Obtain a reference on given page's memcg.
+ * @page: page from which memcg should be extracted.
+ *
+ * Obtain a reference on page->memcg and returns it if successful. Otherwise
+ * root_mem_cgroup is returned.
+ */
+struct mem_cgroup *get_mem_cgroup_from_page(struct page *page)
+{
+	struct mem_cgroup *memcg = page->mem_cgroup;
+
+	if (mem_cgroup_disabled())
+		return NULL;
+
+	rcu_read_lock();
+	if (!memcg || !css_tryget_online(&memcg->css))
+		memcg = root_mem_cgroup;
+	rcu_read_unlock();
+	return memcg;
+}
+EXPORT_SYMBOL(get_mem_cgroup_from_page);
+
 /**
  * If current->active_memcg is non-NULL, do not fallback to current->mm->memcg.
  */
-- 
2.18.0.rc2.346.g013aa6912e-goog

[PATCH] fs-mm-account-buffer_head-to-kmemcg.patch.fix

[Shakeel Butt]

The patch "fs, mm: account buffer_head to kmemcg" missed to add
__GFP_ACCOUNT flag into the gfp mask for directed memcg charging. So,
adding it. Andrew, please squash this into the original patch.

Signed-off-by: Shakeel Butt <shakeelb@google.com>
---
 fs/buffer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/buffer.c b/fs/buffer.c
index e08863af56f6..405d4723ed3d 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -814,7 +814,7 @@ struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size,
 		bool retry)
 {
 	struct buffer_head *bh, *head;
-	gfp_t gfp = GFP_NOFS;
+	gfp_t gfp = GFP_NOFS | __GFP_ACCOUNT;
 	long offset;
 	struct mem_cgroup *memcg;
 
-- 
2.18.0.399.gad0ab374a1-goog

Re: [PATCH v8 0/2] Directed kmem charging

[Andrew Morton]

On Wed, 27 Jun 2018 12:12:48 -0700 Shakeel Butt <shakeelb@google.com> wrote:

> The Linux kernel's memory cgroup allows limiting the memory usage of
> the jobs running on the system to provide isolation between the jobs.
> All the kernel memory allocated in the context of the job and marked
> with __GFP_ACCOUNT will also be included in the memory usage and be
> limited by the job's limit.
> 
> The kernel memory can only be charged to the memcg of the process in
> whose context kernel memory was allocated. However there are cases where
> the allocated kernel memory should be charged to the memcg different
> from the current processes's memcg. This patch series contains two such
> concrete use-cases i.e. fsnotify and buffer_head.
> 
> The fsnotify event objects can consume a lot of system memory for large
> or unlimited queues if there is either no or slow listener. The events
> are allocated in the context of the event producer. However they should
> be charged to the event consumer. Similarly the buffer_head objects can
> be allocated in a memcg different from the memcg of the page for which
> buffer_head objects are being allocated.
> 
> To solve this issue, this patch series introduces mechanism to charge
> kernel memory to a given memcg. In case of fsnotify events, the memcg of
> the consumer can be used for charging and for buffer_head, the memcg of
> the page can be charged. For directed charging, the caller can use the
> scope API memalloc_[un]use_memcg() to specify the memcg to charge for
> all the __GFP_ACCOUNT allocations within the scope.

This patchset is not showing signs of having been well reviewed at
this time.  Could people please take another look?

Re: [PATCH v8 0/2] Directed kmem charging

[Johannes Weiner]

On Wed, Aug 15, 2018 at 03:25:11PM -0700, Andrew Morton wrote:
> On Wed, 27 Jun 2018 12:12:48 -0700 Shakeel Butt <shakeelb@google.com> wrote:
> 
> > The Linux kernel's memory cgroup allows limiting the memory usage of
> > the jobs running on the system to provide isolation between the jobs.
> > All the kernel memory allocated in the context of the job and marked
> > with __GFP_ACCOUNT will also be included in the memory usage and be
> > limited by the job's limit.
> > 
> > The kernel memory can only be charged to the memcg of the process in
> > whose context kernel memory was allocated. However there are cases where
> > the allocated kernel memory should be charged to the memcg different
> > from the current processes's memcg. This patch series contains two such
> > concrete use-cases i.e. fsnotify and buffer_head.
> > 
> > The fsnotify event objects can consume a lot of system memory for large
> > or unlimited queues if there is either no or slow listener. The events
> > are allocated in the context of the event producer. However they should
> > be charged to the event consumer. Similarly the buffer_head objects can
> > be allocated in a memcg different from the memcg of the page for which
> > buffer_head objects are being allocated.
> > 
> > To solve this issue, this patch series introduces mechanism to charge
> > kernel memory to a given memcg. In case of fsnotify events, the memcg of
> > the consumer can be used for charging and for buffer_head, the memcg of
> > the page can be charged. For directed charging, the caller can use the
> > scope API memalloc_[un]use_memcg() to specify the memcg to charge for
> > all the __GFP_ACCOUNT allocations within the scope.
> 
> This patchset is not showing signs of having been well reviewed at
> this time.  Could people please take another look?

I don't have the mailing list archives for this anymore, but the
series as it stands in mmots looks good to me and incorporates all the
feedback I remember giving.

[ My only gripe really is that it applies current->active_memcg only
  to kmem charges, not others as well. Right now it doesn't matter,
  but I can see this costing a kernel developer implementing remote
  charges for something other than kmem some time to realize. ]

Anyway, please feel free to add

Acked-by: Johannes Weiner <hannes@cmpxchg.org>

for 1/2 and 2/2 plus their two fixlets.

[PATCH v7 0/2] Directed kmem charging

[Shakeel Butt]

The Linux kernel's memory cgroup allows limiting the memory usage of
the jobs running on the system to provide isolation between the jobs.
All the kernel memory allocated in the context of the job and marked
with __GFP_ACCOUNT will also be included in the memory usage and be
limited by the job's limit.

The kernel memory can only be charged to the memcg of the process in
whose context kernel memory was allocated. However there are cases where
the allocated kernel memory should be charged to the memcg different
from the current processes's memcg. This patch series contains two such
concrete use-cases i.e. fsnotify and buffer_head.

The fsnotify event objects can consume a lot of system memory for large
or unlimited queues if there is either no or slow listener. The events
are allocated in the context of the event producer. However they should
be charged to the event consumer. Similarly the buffer_head objects can
be allocated in a memcg different from the memcg of the page for which
buffer_head objects are being allocated.

To solve this issue, this patch series introduces mechanism to charge
kernel memory to a given memcg. In case of fsnotify events, the memcg of
the consumer can be used for charging and for buffer_head, the memcg of
the page can be charged. For directed charging, the caller can use the
scope API memalloc_[un]use_memcg() to specify the memcg to charge for
all the __GFP_ACCOUNT allocations within the scope.

Shakeel Butt (2):
  fs: fsnotify: account fsnotify metadata to kmemcg
  fs, mm: account buffer_head to kmemcg

 fs/buffer.c                          | 15 ++++++-
 fs/notify/dnotify/dnotify.c          |  5 ++-
 fs/notify/fanotify/fanotify.c        | 17 ++++++--
 fs/notify/fanotify/fanotify_user.c   |  5 ++-
 fs/notify/group.c                    |  4 ++
 fs/notify/inotify/inotify_fsnotify.c | 15 ++++++-
 fs/notify/inotify/inotify_user.c     |  5 ++-
 include/linux/fsnotify_backend.h     | 12 ++++--
 include/linux/memcontrol.h           | 14 +++++++
 include/linux/sched.h                |  3 ++
 include/linux/sched/mm.h             | 41 +++++++++++++++++++
 kernel/fork.c                        |  3 ++
 mm/memcontrol.c                      | 60 ++++++++++++++++++++++++++--
 13 files changed, 182 insertions(+), 17 deletions(-)

-- 
2.18.0.rc2.346.g013aa6912e-goog

[PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Shakeel Butt]

A lot of memory can be consumed by the events generated for the huge or
unlimited queues if there is either no or slow listener.  This can cause
system level memory pressure or OOMs.  So, it's better to account the
fsnotify kmem caches to the memcg of the listener.

However the listener can be in a different memcg than the memcg of the
producer and these allocations happen in the context of the event
producer. This patch introduces remote memcg charging scope API which the
producer can use to charge the allocations to the memcg of the listener.

There are seven fsnotify kmem caches and among them allocations from
dnotify_struct_cache, dnotify_mark_cache, fanotify_mark_cache and
inotify_inode_mark_cachep happens in the context of syscall from the
listener.  So, SLAB_ACCOUNT is enough for these caches.

The objects from fsnotify_mark_connector_cachep are not accounted as they
are small compared to the notification mark or events and it is unclear
whom to account connector to since it is shared by all events attached to
the inode.

The allocations from the event caches happen in the context of the event
producer.  For such caches we will need to remote charge the allocations
to the listener's memcg.  Thus we save the memcg reference in the
fsnotify_group structure of the listener.

This patch has also moved the members of fsnotify_group to keep the size
same, at least for 64 bit build, even with additional member by filling
the holes.

Signed-off-by: Shakeel Butt <shakeelb@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Amir Goldstein <amir73il@gmail.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Roman Gushchin <guro@fb.com>
---
Changelog since v6:
- Removed Jan's ACK as the code has changed a lot
- Squashed the separate remote charging API path into this one
- Removed kmalloc* & kmem_cache_alloc* APIs and only kept the scope API
- Changed fsnotify remote charging code to use scope API

Changelog since v5:
- None

Changelog since v4:
- Fixed the build for CONFIG_MEMCG=n

Changelog since v3:
- Rebased over Jan's patches.
- Some cleanup based on Amir's comments.

Changelog since v2:
- None

Changelog since v1:
- no more charging fsnotify_mark_connector objects
- Fixed the build for SLOB

 fs/notify/dnotify/dnotify.c          |  5 ++--
 fs/notify/fanotify/fanotify.c        | 17 ++++++++++--
 fs/notify/fanotify/fanotify_user.c   |  5 +++-
 fs/notify/group.c                    |  4 +++
 fs/notify/inotify/inotify_fsnotify.c | 15 +++++++++-
 fs/notify/inotify/inotify_user.c     |  5 +++-
 include/linux/fsnotify_backend.h     | 12 +++++---
 include/linux/memcontrol.h           |  7 +++++
 include/linux/sched.h                |  3 ++
 include/linux/sched/mm.h             | 41 ++++++++++++++++++++++++++++
 kernel/fork.c                        |  3 ++
 mm/memcontrol.c                      | 38 +++++++++++++++++++++++---
 12 files changed, 139 insertions(+), 16 deletions(-)

diff --git a/fs/notify/dnotify/dnotify.c b/fs/notify/dnotify/dnotify.c
index e2bea2ac5dfb..a6365e6bc047 100644
--- a/fs/notify/dnotify/dnotify.c
+++ b/fs/notify/dnotify/dnotify.c
@@ -384,8 +384,9 @@ int fcntl_dirnotify(int fd, struct file *filp, unsigned long arg)
 
 static int __init dnotify_init(void)
 {
-	dnotify_struct_cache = KMEM_CACHE(dnotify_struct, SLAB_PANIC);
-	dnotify_mark_cache = KMEM_CACHE(dnotify_mark, SLAB_PANIC);
+	dnotify_struct_cache = KMEM_CACHE(dnotify_struct,
+					  SLAB_PANIC|SLAB_ACCOUNT);
+	dnotify_mark_cache = KMEM_CACHE(dnotify_mark, SLAB_PANIC|SLAB_ACCOUNT);
 
 	dnotify_group = fsnotify_alloc_group(&dnotify_fsnotify_ops);
 	if (IS_ERR(dnotify_group))
diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index f90842efea13..d6dfcf0ec21f 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -11,6 +11,7 @@
 #include <linux/types.h>
 #include <linux/wait.h>
 #include <linux/audit.h>
+#include <linux/sched/mm.h>
 
 #include "fanotify.h"
 
@@ -140,8 +141,9 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
 						 struct inode *inode, u32 mask,
 						 const struct path *path)
 {
-	struct fanotify_event_info *event;
+	struct fanotify_event_info *event = NULL;
 	gfp_t gfp = GFP_KERNEL;
+	struct mem_cgroup *old_memcg = NULL;
 
 	/*
 	 * For queues with unlimited length lost events are not expected and
@@ -151,19 +153,25 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
 	if (group->max_events == UINT_MAX)
 		gfp |= __GFP_NOFAIL;
 
+	/* Whoever is interested in the event, pays for the allocation. */
+	if (group->memcg) {
+		gfp |= __GFP_ACCOUNT;
+		old_memcg = memalloc_use_memcg(group->memcg);
+	}
+
 	if (fanotify_is_perm_event(mask)) {
 		struct fanotify_perm_event_info *pevent;
 
 		pevent = kmem_cache_alloc(fanotify_perm_event_cachep, gfp);
 		if (!pevent)
-			return NULL;
+			goto out;
 		event = &pevent->fae;
 		pevent->response = 0;
 		goto init;
 	}
 	event = kmem_cache_alloc(fanotify_event_cachep, gfp);
 	if (!event)
-		return NULL;
+		goto out;
 init: __maybe_unused
 	fsnotify_init_event(&event->fse, inode, mask);
 	event->tgid = get_pid(task_tgid(current));
@@ -174,6 +182,9 @@ init: __maybe_unused
 		event->path.mnt = NULL;
 		event->path.dentry = NULL;
 	}
+out:
+	if (group->memcg)
+		memalloc_unuse_memcg(old_memcg);
 	return event;
 }
 
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index ec4d8c59d0e3..0cf45041dc32 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -16,6 +16,7 @@
 #include <linux/uaccess.h>
 #include <linux/compat.h>
 #include <linux/sched/signal.h>
+#include <linux/memcontrol.h>
 
 #include <asm/ioctls.h>
 
@@ -756,6 +757,7 @@ SYSCALL_DEFINE2(fanotify_init, unsigned int, flags, unsigned int, event_f_flags)
 
 	group->fanotify_data.user = user;
 	atomic_inc(&user->fanotify_listeners);
+	group->memcg = get_mem_cgroup_from_mm(current->mm);
 
 	oevent = fanotify_alloc_event(group, NULL, FS_Q_OVERFLOW, NULL);
 	if (unlikely(!oevent)) {
@@ -957,7 +959,8 @@ COMPAT_SYSCALL_DEFINE6(fanotify_mark,
  */
 static int __init fanotify_user_setup(void)
 {
-	fanotify_mark_cache = KMEM_CACHE(fsnotify_mark, SLAB_PANIC);
+	fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
+					 SLAB_PANIC|SLAB_ACCOUNT);
 	fanotify_event_cachep = KMEM_CACHE(fanotify_event_info, SLAB_PANIC);
 	if (IS_ENABLED(CONFIG_FANOTIFY_ACCESS_PERMISSIONS)) {
 		fanotify_perm_event_cachep =
diff --git a/fs/notify/group.c b/fs/notify/group.c
index aa5468f23e45..cbcda1cb9a74 100644
--- a/fs/notify/group.c
+++ b/fs/notify/group.c
@@ -22,6 +22,7 @@
 #include <linux/srcu.h>
 #include <linux/rculist.h>
 #include <linux/wait.h>
+#include <linux/memcontrol.h>
 
 #include <linux/fsnotify_backend.h>
 #include "fsnotify.h"
@@ -36,6 +37,9 @@ static void fsnotify_final_destroy_group(struct fsnotify_group *group)
 	if (group->ops->free_group_priv)
 		group->ops->free_group_priv(group);
 
+	if (group->memcg)
+		mem_cgroup_put(group->memcg);
+
 	kfree(group);
 }
 
diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
index 9ab6dde38a14..73b4d6c55497 100644
--- a/fs/notify/inotify/inotify_fsnotify.c
+++ b/fs/notify/inotify/inotify_fsnotify.c
@@ -31,6 +31,7 @@
 #include <linux/types.h>
 #include <linux/sched.h>
 #include <linux/sched/user.h>
+#include <linux/sched/mm.h>
 
 #include "inotify.h"
 
@@ -73,9 +74,11 @@ int inotify_handle_event(struct fsnotify_group *group,
 	struct inotify_inode_mark *i_mark;
 	struct inotify_event_info *event;
 	struct fsnotify_event *fsn_event;
+	struct mem_cgroup *old_memcg = NULL;
 	int ret;
 	int len = 0;
 	int alloc_len = sizeof(struct inotify_event_info);
+	gfp_t gfp = GFP_KERNEL;
 
 	if (WARN_ON(fsnotify_iter_vfsmount_mark(iter_info)))
 		return 0;
@@ -98,7 +101,17 @@ int inotify_handle_event(struct fsnotify_group *group,
 	i_mark = container_of(inode_mark, struct inotify_inode_mark,
 			      fsn_mark);
 
-	event = kmalloc(alloc_len, GFP_KERNEL);
+	/* Whoever is interested in the event, pays for the allocation. */
+	if (group->memcg) {
+		gfp |= __GFP_ACCOUNT;
+		old_memcg = memalloc_use_memcg(group->memcg);
+	}
+
+	event = kmalloc(alloc_len, gfp);
+
+	if (group->memcg)
+		memalloc_unuse_memcg(old_memcg);
+
 	if (unlikely(!event)) {
 		/*
 		 * Treat lost event due to ENOMEM the same way as queue
diff --git a/fs/notify/inotify/inotify_user.c b/fs/notify/inotify/inotify_user.c
index 1cf5b779d862..749c46ababa0 100644
--- a/fs/notify/inotify/inotify_user.c
+++ b/fs/notify/inotify/inotify_user.c
@@ -38,6 +38,7 @@
 #include <linux/uaccess.h>
 #include <linux/poll.h>
 #include <linux/wait.h>
+#include <linux/memcontrol.h>
 
 #include "inotify.h"
 #include "../fdinfo.h"
@@ -636,6 +637,7 @@ static struct fsnotify_group *inotify_new_group(unsigned int max_events)
 	oevent->name_len = 0;
 
 	group->max_events = max_events;
+	group->memcg = get_mem_cgroup_from_mm(current->mm);
 
 	spin_lock_init(&group->inotify_data.idr_lock);
 	idr_init(&group->inotify_data.idr);
@@ -808,7 +810,8 @@ static int __init inotify_user_setup(void)
 
 	BUG_ON(hweight32(ALL_INOTIFY_BITS) != 21);
 
-	inotify_inode_mark_cachep = KMEM_CACHE(inotify_inode_mark, SLAB_PANIC);
+	inotify_inode_mark_cachep = KMEM_CACHE(inotify_inode_mark,
+					       SLAB_PANIC|SLAB_ACCOUNT);
 
 	inotify_max_queued_events = 16384;
 	init_user_ns.ucount_max[UCOUNT_INOTIFY_INSTANCES] = 128;
diff --git a/include/linux/fsnotify_backend.h b/include/linux/fsnotify_backend.h
index b38964a7a521..a0c4790c5302 100644
--- a/include/linux/fsnotify_backend.h
+++ b/include/linux/fsnotify_backend.h
@@ -84,6 +84,8 @@ struct fsnotify_event_private_data;
 struct fsnotify_fname;
 struct fsnotify_iter_info;
 
+struct mem_cgroup;
+
 /*
  * Each group much define these ops.  The fsnotify infrastructure will call
  * these operations for each relevant group.
@@ -127,6 +129,8 @@ struct fsnotify_event {
  * everything will be cleaned up.
  */
 struct fsnotify_group {
+	const struct fsnotify_ops *ops;	/* how this group handles things */
+
 	/*
 	 * How the refcnt is used is up to each group.  When the refcnt hits 0
 	 * fsnotify will clean up all of the resources associated with this group.
@@ -137,8 +141,6 @@ struct fsnotify_group {
 	 */
 	refcount_t refcnt;		/* things with interest in this group */
 
-	const struct fsnotify_ops *ops;	/* how this group handles things */
-
 	/* needed to send notification to userspace */
 	spinlock_t notification_lock;		/* protect the notification_list */
 	struct list_head notification_list;	/* list of event_holder this group needs to send to userspace */
@@ -160,6 +162,8 @@ struct fsnotify_group {
 	atomic_t num_marks;		/* 1 for each mark and 1 for not being
 					 * past the point of no return when freeing
 					 * a group */
+	atomic_t user_waits;		/* Number of tasks waiting for user
+					 * response */
 	struct list_head marks_list;	/* all inode marks for this group */
 
 	struct fasync_struct *fsn_fa;    /* async notification */
@@ -167,8 +171,8 @@ struct fsnotify_group {
 	struct fsnotify_event *overflow_event;	/* Event we queue when the
 						 * notification list is too
 						 * full */
-	atomic_t user_waits;		/* Number of tasks waiting for user
-					 * response */
+
+	struct mem_cgroup *memcg;	/* memcg to charge allocations */
 
 	/* groups can define private fields here or use the void *private */
 	union {
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 3607913032be..6c857be8a9b7 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -378,6 +378,8 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *, struct pglist_data *);
 bool task_in_mem_cgroup(struct task_struct *task, struct mem_cgroup *memcg);
 struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
 
+struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
+
 static inline
 struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css){
 	return css ? container_of(css, struct mem_cgroup, css) : NULL;
@@ -857,6 +859,11 @@ static inline bool task_in_mem_cgroup(struct task_struct *task,
 	return true;
 }
 
+static inline struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
+{
+	return NULL;
+}
+
 static inline void mem_cgroup_put(struct mem_cgroup *memcg)
 {
 }
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 87bf02d93a27..9cba7f874443 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1149,6 +1149,9 @@ struct task_struct {
 
 	/* Number of pages to reclaim on returning to userland: */
 	unsigned int			memcg_nr_pages_over_high;
+
+	/* Used by memcontrol for targeted memcg charge: */
+	struct mem_cgroup		*active_memcg;
 #endif
 
 #ifdef CONFIG_UPROBES
diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h
index 44d356f5e47c..75f2d6ee2b72 100644
--- a/include/linux/sched/mm.h
+++ b/include/linux/sched/mm.h
@@ -248,6 +248,47 @@ static inline void memalloc_noreclaim_restore(unsigned int flags)
 	current->flags = (current->flags & ~PF_MEMALLOC) | flags;
 }
 
+#ifdef CONFIG_MEMCG
+/**
+ * memalloc_use_memcg - Starts the remote memcg charging scope.
+ * @memcg: memcg to charge.
+ *
+ * This function marks the beginning of the remote memcg charging scope. All the
+ * __GFP_ACCOUNT allocations till the end of the scope will be charged to the
+ * given memcg. Passing NULL will disable the remote memcg charging of the outer
+ * scope.
+ */
+static inline struct mem_cgroup *memalloc_use_memcg(struct mem_cgroup *memcg)
+{
+	struct mem_cgroup *old_memcg = current->active_memcg;
+
+	current->active_memcg = memcg;
+	return old_memcg;
+}
+
+/**
+ * memalloc_unuse_memcg - Ends the remote memcg charging scope.
+ * @memcg: outer scope memcg to restore.
+ *
+ * This function marks the end of the remote memcg charging scope started by
+ * memalloc_use_memcg(). Always make sure the given memcg is the return valure
+ * from the pairing memalloc_use_memcg call.
+ */
+static inline void memalloc_unuse_memcg(struct mem_cgroup *memcg)
+{
+	current->active_memcg = memcg;
+}
+#else
+static inline struct mem_cgroup *memalloc_use_memcg(struct mem_cgroup *memcg)
+{
+	return NULL;
+}
+
+static inline void memalloc_unuse_memcg(struct mem_cgroup *memcg)
+{
+}
+#endif
+
 #ifdef CONFIG_MEMBARRIER
 enum {
 	MEMBARRIER_STATE_PRIVATE_EXPEDITED_READY		= (1U << 0),
diff --git a/kernel/fork.c b/kernel/fork.c
index a64d0a19f174..0b07db6fef06 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -843,6 +843,9 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node)
 	tsk->fail_nth = 0;
 #endif
 
+#ifdef CONFIG_MEMCG
+	tsk->active_memcg = NULL;
+#endif
 	return tsk;
 
 free_stack:
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 02e77c88967a..6b1a8f8e0a82 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -678,9 +678,20 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p)
 }
 EXPORT_SYMBOL(mem_cgroup_from_task);
 
-static struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
+/**
+ * get_mem_cgroup_from_mm: Obtain a reference on given mm_struct's memcg.
+ * @mm: mm from which memcg should be extracted. It can be NULL.
+ *
+ * Obtain a reference on mm->memcg and returns it if successful. Otherwise
+ * root_mem_cgroup is returned. However if mem_cgroup is disabled, NULL is
+ * returned.
+ */
+struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
 {
-	struct mem_cgroup *memcg = NULL;
+	struct mem_cgroup *memcg;
+
+	if (mem_cgroup_disabled())
+		return NULL;
 
 	rcu_read_lock();
 	do {
@@ -700,6 +711,25 @@ static struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
 	rcu_read_unlock();
 	return memcg;
 }
+EXPORT_SYMBOL(get_mem_cgroup_from_mm);
+
+/**
+ * First try to obtain reference on current->active_memcg. On failure, try to
+ * obtain reference on current->mm->memcg. On further failure root_mem_cgroup
+ * is returned.
+ */
+static __always_inline struct mem_cgroup *get_mem_cgroup_from_current(void)
+{
+	if (unlikely(current->active_memcg)) {
+		rcu_read_lock();
+		if (css_tryget_online(&current->active_memcg->css)) {
+			rcu_read_unlock();
+			return current->active_memcg;
+		}
+		rcu_read_unlock();
+	}
+	return get_mem_cgroup_from_mm(current->mm);
+}
 
 /**
  * mem_cgroup_iter - iterate over memory cgroup hierarchy
@@ -2260,7 +2290,7 @@ struct kmem_cache *memcg_kmem_get_cache(struct kmem_cache *cachep)
 	if (current->memcg_kmem_skip_account)
 		return cachep;
 
-	memcg = get_mem_cgroup_from_mm(current->mm);
+	memcg = get_mem_cgroup_from_current();
 	kmemcg_id = READ_ONCE(memcg->kmemcg_id);
 	if (kmemcg_id < 0)
 		goto out;
@@ -2344,7 +2374,7 @@ int memcg_kmem_charge(struct page *page, gfp_t gfp, int order)
 	if (memcg_kmem_bypass())
 		return 0;
 
-	memcg = get_mem_cgroup_from_mm(current->mm);
+	memcg = get_mem_cgroup_from_current();
 	if (!mem_cgroup_is_root(memcg)) {
 		ret = memcg_kmem_charge_memcg(page, gfp, order, memcg);
 		if (!ret)
-- 
2.18.0.rc2.346.g013aa6912e-goog

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Amir Goldstein]

On Tue, Jun 26, 2018 at 2:06 AM, Shakeel Butt <shakeelb@google.com> wrote:
> A lot of memory can be consumed by the events generated for the huge or
> unlimited queues if there is either no or slow listener.  This can cause
> system level memory pressure or OOMs.  So, it's better to account the
> fsnotify kmem caches to the memcg of the listener.
>
> However the listener can be in a different memcg than the memcg of the
> producer and these allocations happen in the context of the event
> producer. This patch introduces remote memcg charging scope API which the
> producer can use to charge the allocations to the memcg of the listener.
>
> There are seven fsnotify kmem caches and among them allocations from
> dnotify_struct_cache, dnotify_mark_cache, fanotify_mark_cache and
> inotify_inode_mark_cachep happens in the context of syscall from the
> listener.  So, SLAB_ACCOUNT is enough for these caches.
>
> The objects from fsnotify_mark_connector_cachep are not accounted as they
> are small compared to the notification mark or events and it is unclear
> whom to account connector to since it is shared by all events attached to
> the inode.
>
> The allocations from the event caches happen in the context of the event
> producer.  For such caches we will need to remote charge the allocations
> to the listener's memcg.  Thus we save the memcg reference in the
> fsnotify_group structure of the listener.
>
> This patch has also moved the members of fsnotify_group to keep the size
> same, at least for 64 bit build, even with additional member by filling
> the holes.
>
> Signed-off-by: Shakeel Butt <shakeelb@google.com>
> Cc: Michal Hocko <mhocko@kernel.org>
> Cc: Jan Kara <jack@suse.cz>
> Cc: Amir Goldstein <amir73il@gmail.com>
> Cc: Greg Thelen <gthelen@google.com>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
> Cc: Roman Gushchin <guro@fb.com>
> ---
> Changelog since v6:
> - Removed Jan's ACK as the code has changed a lot
> - Squashed the separate remote charging API path into this one
> - Removed kmalloc* & kmem_cache_alloc* APIs and only kept the scope API
> - Changed fsnotify remote charging code to use scope API
>
> Changelog since v5:
> - None
>
> Changelog since v4:
> - Fixed the build for CONFIG_MEMCG=n
>
> Changelog since v3:
> - Rebased over Jan's patches.
> - Some cleanup based on Amir's comments.
>
> Changelog since v2:
> - None
>
> Changelog since v1:
> - no more charging fsnotify_mark_connector objects
> - Fixed the build for SLOB
>
>  fs/notify/dnotify/dnotify.c          |  5 ++--
>  fs/notify/fanotify/fanotify.c        | 17 ++++++++++--
>  fs/notify/fanotify/fanotify_user.c   |  5 +++-
>  fs/notify/group.c                    |  4 +++
>  fs/notify/inotify/inotify_fsnotify.c | 15 +++++++++-
>  fs/notify/inotify/inotify_user.c     |  5 +++-
>  include/linux/fsnotify_backend.h     | 12 +++++---
>  include/linux/memcontrol.h           |  7 +++++
>  include/linux/sched.h                |  3 ++
>  include/linux/sched/mm.h             | 41 ++++++++++++++++++++++++++++
>  kernel/fork.c                        |  3 ++
>  mm/memcontrol.c                      | 38 +++++++++++++++++++++++---
>  12 files changed, 139 insertions(+), 16 deletions(-)
>
> diff --git a/fs/notify/dnotify/dnotify.c b/fs/notify/dnotify/dnotify.c
> index e2bea2ac5dfb..a6365e6bc047 100644
> --- a/fs/notify/dnotify/dnotify.c
> +++ b/fs/notify/dnotify/dnotify.c
> @@ -384,8 +384,9 @@ int fcntl_dirnotify(int fd, struct file *filp, unsigned long arg)
>
>  static int __init dnotify_init(void)
>  {
> -       dnotify_struct_cache = KMEM_CACHE(dnotify_struct, SLAB_PANIC);
> -       dnotify_mark_cache = KMEM_CACHE(dnotify_mark, SLAB_PANIC);
> +       dnotify_struct_cache = KMEM_CACHE(dnotify_struct,
> +                                         SLAB_PANIC|SLAB_ACCOUNT);
> +       dnotify_mark_cache = KMEM_CACHE(dnotify_mark, SLAB_PANIC|SLAB_ACCOUNT);
>
>         dnotify_group = fsnotify_alloc_group(&dnotify_fsnotify_ops);
>         if (IS_ERR(dnotify_group))
> diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
> index f90842efea13..d6dfcf0ec21f 100644
> --- a/fs/notify/fanotify/fanotify.c
> +++ b/fs/notify/fanotify/fanotify.c
> @@ -11,6 +11,7 @@
>  #include <linux/types.h>
>  #include <linux/wait.h>
>  #include <linux/audit.h>
> +#include <linux/sched/mm.h>
>
>  #include "fanotify.h"
>
> @@ -140,8 +141,9 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
>                                                  struct inode *inode, u32 mask,
>                                                  const struct path *path)
>  {
> -       struct fanotify_event_info *event;
> +       struct fanotify_event_info *event = NULL;
>         gfp_t gfp = GFP_KERNEL;
> +       struct mem_cgroup *old_memcg = NULL;
>
>         /*
>          * For queues with unlimited length lost events are not expected and
> @@ -151,19 +153,25 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
>         if (group->max_events == UINT_MAX)
>                 gfp |= __GFP_NOFAIL;
>
> +       /* Whoever is interested in the event, pays for the allocation. */
> +       if (group->memcg) {
> +               gfp |= __GFP_ACCOUNT;
> +               old_memcg = memalloc_use_memcg(group->memcg);
> +       }
> +
>         if (fanotify_is_perm_event(mask)) {
>                 struct fanotify_perm_event_info *pevent;
>
>                 pevent = kmem_cache_alloc(fanotify_perm_event_cachep, gfp);
>                 if (!pevent)
> -                       return NULL;
> +                       goto out;
>                 event = &pevent->fae;
>                 pevent->response = 0;
>                 goto init;
>         }
>         event = kmem_cache_alloc(fanotify_event_cachep, gfp);
>         if (!event)
> -               return NULL;
> +               goto out;
>  init: __maybe_unused
>         fsnotify_init_event(&event->fse, inode, mask);
>         event->tgid = get_pid(task_tgid(current));
> @@ -174,6 +182,9 @@ init: __maybe_unused
>                 event->path.mnt = NULL;
>                 event->path.dentry = NULL;
>         }
> +out:
> +       if (group->memcg)
> +               memalloc_unuse_memcg(old_memcg);
>         return event;
>  }
>
> diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
> index ec4d8c59d0e3..0cf45041dc32 100644
> --- a/fs/notify/fanotify/fanotify_user.c
> +++ b/fs/notify/fanotify/fanotify_user.c
> @@ -16,6 +16,7 @@
>  #include <linux/uaccess.h>
>  #include <linux/compat.h>
>  #include <linux/sched/signal.h>
> +#include <linux/memcontrol.h>
>
>  #include <asm/ioctls.h>
>
> @@ -756,6 +757,7 @@ SYSCALL_DEFINE2(fanotify_init, unsigned int, flags, unsigned int, event_f_flags)
>
>         group->fanotify_data.user = user;
>         atomic_inc(&user->fanotify_listeners);
> +       group->memcg = get_mem_cgroup_from_mm(current->mm);
>
>         oevent = fanotify_alloc_event(group, NULL, FS_Q_OVERFLOW, NULL);
>         if (unlikely(!oevent)) {
> @@ -957,7 +959,8 @@ COMPAT_SYSCALL_DEFINE6(fanotify_mark,
>   */
>  static int __init fanotify_user_setup(void)
>  {
> -       fanotify_mark_cache = KMEM_CACHE(fsnotify_mark, SLAB_PANIC);
> +       fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
> +                                        SLAB_PANIC|SLAB_ACCOUNT);
>         fanotify_event_cachep = KMEM_CACHE(fanotify_event_info, SLAB_PANIC);
>         if (IS_ENABLED(CONFIG_FANOTIFY_ACCESS_PERMISSIONS)) {
>                 fanotify_perm_event_cachep =
> diff --git a/fs/notify/group.c b/fs/notify/group.c
> index aa5468f23e45..cbcda1cb9a74 100644
> --- a/fs/notify/group.c
> +++ b/fs/notify/group.c
> @@ -22,6 +22,7 @@
>  #include <linux/srcu.h>
>  #include <linux/rculist.h>
>  #include <linux/wait.h>
> +#include <linux/memcontrol.h>
>
>  #include <linux/fsnotify_backend.h>
>  #include "fsnotify.h"
> @@ -36,6 +37,9 @@ static void fsnotify_final_destroy_group(struct fsnotify_group *group)
>         if (group->ops->free_group_priv)
>                 group->ops->free_group_priv(group);
>
> +       if (group->memcg)
> +               mem_cgroup_put(group->memcg);
> +
>         kfree(group);
>  }
>
> diff --git a/fs/notify/inotify/inotify_fsnotify.c b/fs/notify/inotify/inotify_fsnotify.c
> index 9ab6dde38a14..73b4d6c55497 100644
> --- a/fs/notify/inotify/inotify_fsnotify.c
> +++ b/fs/notify/inotify/inotify_fsnotify.c
> @@ -31,6 +31,7 @@
>  #include <linux/types.h>
>  #include <linux/sched.h>
>  #include <linux/sched/user.h>
> +#include <linux/sched/mm.h>
>
>  #include "inotify.h"
>
> @@ -73,9 +74,11 @@ int inotify_handle_event(struct fsnotify_group *group,
>         struct inotify_inode_mark *i_mark;
>         struct inotify_event_info *event;
>         struct fsnotify_event *fsn_event;
> +       struct mem_cgroup *old_memcg = NULL;
>         int ret;
>         int len = 0;
>         int alloc_len = sizeof(struct inotify_event_info);
> +       gfp_t gfp = GFP_KERNEL;
>
>         if (WARN_ON(fsnotify_iter_vfsmount_mark(iter_info)))
>                 return 0;
> @@ -98,7 +101,17 @@ int inotify_handle_event(struct fsnotify_group *group,
>         i_mark = container_of(inode_mark, struct inotify_inode_mark,
>                               fsn_mark);
>
> -       event = kmalloc(alloc_len, GFP_KERNEL);
> +       /* Whoever is interested in the event, pays for the allocation. */
> +       if (group->memcg) {
> +               gfp |= __GFP_ACCOUNT;
> +               old_memcg = memalloc_use_memcg(group->memcg);
> +       }
> +
> +       event = kmalloc(alloc_len, gfp);
> +
> +       if (group->memcg)
> +               memalloc_unuse_memcg(old_memcg);
> +
>         if (unlikely(!event)) {
>                 /*
>                  * Treat lost event due to ENOMEM the same way as queue
> diff --git a/fs/notify/inotify/inotify_user.c b/fs/notify/inotify/inotify_user.c
> index 1cf5b779d862..749c46ababa0 100644
> --- a/fs/notify/inotify/inotify_user.c
> +++ b/fs/notify/inotify/inotify_user.c
> @@ -38,6 +38,7 @@
>  #include <linux/uaccess.h>
>  #include <linux/poll.h>
>  #include <linux/wait.h>
> +#include <linux/memcontrol.h>
>
>  #include "inotify.h"
>  #include "../fdinfo.h"
> @@ -636,6 +637,7 @@ static struct fsnotify_group *inotify_new_group(unsigned int max_events)
>         oevent->name_len = 0;
>
>         group->max_events = max_events;
> +       group->memcg = get_mem_cgroup_from_mm(current->mm);
>
>         spin_lock_init(&group->inotify_data.idr_lock);
>         idr_init(&group->inotify_data.idr);
> @@ -808,7 +810,8 @@ static int __init inotify_user_setup(void)
>
>         BUG_ON(hweight32(ALL_INOTIFY_BITS) != 21);
>
> -       inotify_inode_mark_cachep = KMEM_CACHE(inotify_inode_mark, SLAB_PANIC);
> +       inotify_inode_mark_cachep = KMEM_CACHE(inotify_inode_mark,
> +                                              SLAB_PANIC|SLAB_ACCOUNT);
>
>         inotify_max_queued_events = 16384;
>         init_user_ns.ucount_max[UCOUNT_INOTIFY_INSTANCES] = 128;
> diff --git a/include/linux/fsnotify_backend.h b/include/linux/fsnotify_backend.h
> index b38964a7a521..a0c4790c5302 100644
> --- a/include/linux/fsnotify_backend.h
> +++ b/include/linux/fsnotify_backend.h
> @@ -84,6 +84,8 @@ struct fsnotify_event_private_data;
>  struct fsnotify_fname;
>  struct fsnotify_iter_info;
>
> +struct mem_cgroup;
> +
>  /*
>   * Each group much define these ops.  The fsnotify infrastructure will call
>   * these operations for each relevant group.
> @@ -127,6 +129,8 @@ struct fsnotify_event {
>   * everything will be cleaned up.
>   */
>  struct fsnotify_group {
> +       const struct fsnotify_ops *ops; /* how this group handles things */
> +
>         /*
>          * How the refcnt is used is up to each group.  When the refcnt hits 0
>          * fsnotify will clean up all of the resources associated with this group.
> @@ -137,8 +141,6 @@ struct fsnotify_group {
>          */
>         refcount_t refcnt;              /* things with interest in this group */
>
> -       const struct fsnotify_ops *ops; /* how this group handles things */
> -
>         /* needed to send notification to userspace */
>         spinlock_t notification_lock;           /* protect the notification_list */
>         struct list_head notification_list;     /* list of event_holder this group needs to send to userspace */
> @@ -160,6 +162,8 @@ struct fsnotify_group {
>         atomic_t num_marks;             /* 1 for each mark and 1 for not being
>                                          * past the point of no return when freeing
>                                          * a group */
> +       atomic_t user_waits;            /* Number of tasks waiting for user
> +                                        * response */
>         struct list_head marks_list;    /* all inode marks for this group */
>
>         struct fasync_struct *fsn_fa;    /* async notification */
> @@ -167,8 +171,8 @@ struct fsnotify_group {
>         struct fsnotify_event *overflow_event;  /* Event we queue when the
>                                                  * notification list is too
>                                                  * full */
> -       atomic_t user_waits;            /* Number of tasks waiting for user
> -                                        * response */
> +
> +       struct mem_cgroup *memcg;       /* memcg to charge allocations */
>
>         /* groups can define private fields here or use the void *private */
>         union {
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 3607913032be..6c857be8a9b7 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -378,6 +378,8 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *, struct pglist_data *);
>  bool task_in_mem_cgroup(struct task_struct *task, struct mem_cgroup *memcg);
>  struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
>
> +struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
> +
>  static inline
>  struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css){
>         return css ? container_of(css, struct mem_cgroup, css) : NULL;
> @@ -857,6 +859,11 @@ static inline bool task_in_mem_cgroup(struct task_struct *task,
>         return true;
>  }
>
> +static inline struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm)
> +{
> +       return NULL;
> +}
> +
>  static inline void mem_cgroup_put(struct mem_cgroup *memcg)
>  {
>  }
> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index 87bf02d93a27..9cba7f874443 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -1149,6 +1149,9 @@ struct task_struct {
>
>         /* Number of pages to reclaim on returning to userland: */
>         unsigned int                    memcg_nr_pages_over_high;
> +
> +       /* Used by memcontrol for targeted memcg charge: */
> +       struct mem_cgroup               *active_memcg;
>  #endif
>
>  #ifdef CONFIG_UPROBES
> diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h
> index 44d356f5e47c..75f2d6ee2b72 100644
> --- a/include/linux/sched/mm.h
> +++ b/include/linux/sched/mm.h
> @@ -248,6 +248,47 @@ static inline void memalloc_noreclaim_restore(unsigned int flags)
>         current->flags = (current->flags & ~PF_MEMALLOC) | flags;
>  }
>
> +#ifdef CONFIG_MEMCG
> +/**
> + * memalloc_use_memcg - Starts the remote memcg charging scope.
> + * @memcg: memcg to charge.
> + *
> + * This function marks the beginning of the remote memcg charging scope. All the
> + * __GFP_ACCOUNT allocations till the end of the scope will be charged to the
> + * given memcg. Passing NULL will disable the remote memcg charging of the outer
> + * scope.
> + */
> +static inline struct mem_cgroup *memalloc_use_memcg(struct mem_cgroup *memcg)
> +{
> +       struct mem_cgroup *old_memcg = current->active_memcg;
> +
> +       current->active_memcg = memcg;
> +       return old_memcg;
> +}
> +
> +/**
> + * memalloc_unuse_memcg - Ends the remote memcg charging scope.
> + * @memcg: outer scope memcg to restore.
> + *
> + * This function marks the end of the remote memcg charging scope started by
> + * memalloc_use_memcg(). Always make sure the given memcg is the return valure
> + * from the pairing memalloc_use_memcg call.
> + */
> +static inline void memalloc_unuse_memcg(struct mem_cgroup *memcg)
> +{
> +       current->active_memcg = memcg;
> +}

The verb 'unuse' takes an argument memcg and 'uses' it - too weird.
You can use 'override'/'revert' verbs like override_creds or just call
memalloc_use_memcg(old_memcg) since there is no reference taken
anyway in use_memcg and no reference released in unuse_memcg.

Otherwise looks good to me.

Thanks,
Amir,

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Shakeel Butt]

On Mon, Jun 25, 2018 at 10:49 PM Amir Goldstein <amir73il@gmail.com> wrote:
>
...
>
> The verb 'unuse' takes an argument memcg and 'uses' it - too weird.
> You can use 'override'/'revert' verbs like override_creds or just call
> memalloc_use_memcg(old_memcg) since there is no reference taken
> anyway in use_memcg and no reference released in unuse_memcg.
>
> Otherwise looks good to me.
>

Thanks for your feedback. Just using memalloc_use_memcg(old_memcg) and
ignoring the return seems more simple. I will wait for feedback from
other before changing anything.

thanks,
Shakeel

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Johannes Weiner]

On Tue, Jun 26, 2018 at 11:00:53AM -0700, Shakeel Butt wrote:
> On Mon, Jun 25, 2018 at 10:49 PM Amir Goldstein <amir73il@gmail.com> wrote:
> >
> ...
> >
> > The verb 'unuse' takes an argument memcg and 'uses' it - too weird.
> > You can use 'override'/'revert' verbs like override_creds or just call
> > memalloc_use_memcg(old_memcg) since there is no reference taken
> > anyway in use_memcg and no reference released in unuse_memcg.
> >
> > Otherwise looks good to me.
> >
> 
> Thanks for your feedback. Just using memalloc_use_memcg(old_memcg) and
> ignoring the return seems more simple. I will wait for feedback from
> other before changing anything.

We're not nesting calls to memalloc_use_memcg(), right? So we don't
have to return old_memcg and don't have to pass anything to unuse, it
can always set current->active_memcg to NULL.

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Shakeel Butt]

On Tue, Jun 26, 2018 at 11:55 AM Johannes Weiner <hannes@cmpxchg.org> wrote:
>
> On Tue, Jun 26, 2018 at 11:00:53AM -0700, Shakeel Butt wrote:
> > On Mon, Jun 25, 2018 at 10:49 PM Amir Goldstein <amir73il@gmail.com> wrote:
> > >
> > ...
> > >
> > > The verb 'unuse' takes an argument memcg and 'uses' it - too weird.
> > > You can use 'override'/'revert' verbs like override_creds or just call
> > > memalloc_use_memcg(old_memcg) since there is no reference taken
> > > anyway in use_memcg and no reference released in unuse_memcg.
> > >
> > > Otherwise looks good to me.
> > >
> >
> > Thanks for your feedback. Just using memalloc_use_memcg(old_memcg) and
> > ignoring the return seems more simple. I will wait for feedback from
> > other before changing anything.
>
> We're not nesting calls to memalloc_use_memcg(), right? So we don't
> have to return old_memcg and don't have to pass anything to unuse, it
> can always set current->active_memcg to NULL.

For buffer_head, the allocation is done with GFP_NOFS. So, I think
there is no chance of nesting. The fsnotify uses GFP_KERNEL but based
on my limited understanding of fsnotify, there should not be any
nesting i.e. the allocation triggering reclaim which trigger fsnotify
events. Though I would like Amir or Jan to confirm there is no nesting
possible.

thanks,
Shakeel

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Jan Kara]

On Tue 26-06-18 12:07:57, Shakeel Butt wrote:
> On Tue, Jun 26, 2018 at 11:55 AM Johannes Weiner <hannes@cmpxchg.org> wrote:
> >
> > On Tue, Jun 26, 2018 at 11:00:53AM -0700, Shakeel Butt wrote:
> > > On Mon, Jun 25, 2018 at 10:49 PM Amir Goldstein <amir73il@gmail.com> wrote:
> > > >
> > > ...
> > > >
> > > > The verb 'unuse' takes an argument memcg and 'uses' it - too weird.
> > > > You can use 'override'/'revert' verbs like override_creds or just call
> > > > memalloc_use_memcg(old_memcg) since there is no reference taken
> > > > anyway in use_memcg and no reference released in unuse_memcg.
> > > >
> > > > Otherwise looks good to me.
> > > >
> > >
> > > Thanks for your feedback. Just using memalloc_use_memcg(old_memcg) and
> > > ignoring the return seems more simple. I will wait for feedback from
> > > other before changing anything.
> >
> > We're not nesting calls to memalloc_use_memcg(), right? So we don't
> > have to return old_memcg and don't have to pass anything to unuse, it
> > can always set current->active_memcg to NULL.
> 
> For buffer_head, the allocation is done with GFP_NOFS. So, I think
> there is no chance of nesting. The fsnotify uses GFP_KERNEL but based
> on my limited understanding of fsnotify, there should not be any
> nesting i.e. the allocation triggering reclaim which trigger fsnotify
> events. Though I would like Amir or Jan to confirm there is no nesting
> possible.

You are correct. Fsnotify events are generated only as a result of some
syscall, not due to reclaim or stuff like that.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Johannes Weiner]

On Mon, Jun 25, 2018 at 04:06:58PM -0700, Shakeel Butt wrote:
> @@ -140,8 +141,9 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
>  						 struct inode *inode, u32 mask,
>  						 const struct path *path)
>  {
> -	struct fanotify_event_info *event;
> +	struct fanotify_event_info *event = NULL;
>  	gfp_t gfp = GFP_KERNEL;
> +	struct mem_cgroup *old_memcg = NULL;
>  
>  	/*
>  	 * For queues with unlimited length lost events are not expected and
> @@ -151,19 +153,25 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
>  	if (group->max_events == UINT_MAX)
>  		gfp |= __GFP_NOFAIL;
>  
> +	/* Whoever is interested in the event, pays for the allocation. */
> +	if (group->memcg) {
> +		gfp |= __GFP_ACCOUNT;
> +		old_memcg = memalloc_use_memcg(group->memcg);
> +	}

group->memcg is only NULL when memcg is disabled or there is some
offlining race. Can you make memalloc_use_memcg(NULL) mean that it
should charge root_mem_cgroup instead of current->mm->memcg? That way
we can make this site unconditional while retaining the behavior:

	gfp_t gfp = GFP_KERNEL | __GFP_ACCOUNT;

	memalloc_use_memcg(group->memcg);
	kmem_cache_alloc(..., gfp);
out:
	memalloc_unuse_memcg();

(dropping old_memcg and the unuse parameter as per the other mail)

>  	if (fanotify_is_perm_event(mask)) {
>  		struct fanotify_perm_event_info *pevent;
>  
>  		pevent = kmem_cache_alloc(fanotify_perm_event_cachep, gfp);
>  		if (!pevent)
> -			return NULL;
> +			goto out;
>  		event = &pevent->fae;
>  		pevent->response = 0;
>  		goto init;
>  	}
>  	event = kmem_cache_alloc(fanotify_event_cachep, gfp);
>  	if (!event)
> -		return NULL;
> +		goto out;
>  init: __maybe_unused
>  	fsnotify_init_event(&event->fse, inode, mask);
>  	event->tgid = get_pid(task_tgid(current));
> @@ -174,6 +182,9 @@ init: __maybe_unused
>  		event->path.mnt = NULL;
>  		event->path.dentry = NULL;
>  	}
> +out:
> +	if (group->memcg)
> +		memalloc_unuse_memcg(old_memcg);
>  	return event;
>  }

Thanks,
Johannes

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Shakeel Butt]

On Tue, Jun 26, 2018 at 12:03 PM Johannes Weiner <hannes@cmpxchg.org> wrote:
>
> On Mon, Jun 25, 2018 at 04:06:58PM -0700, Shakeel Butt wrote:
> > @@ -140,8 +141,9 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
> >                                                struct inode *inode, u32 mask,
> >                                                const struct path *path)
> >  {
> > -     struct fanotify_event_info *event;
> > +     struct fanotify_event_info *event = NULL;
> >       gfp_t gfp = GFP_KERNEL;
> > +     struct mem_cgroup *old_memcg = NULL;
> >
> >       /*
> >        * For queues with unlimited length lost events are not expected and
> > @@ -151,19 +153,25 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
> >       if (group->max_events == UINT_MAX)
> >               gfp |= __GFP_NOFAIL;
> >
> > +     /* Whoever is interested in the event, pays for the allocation. */
> > +     if (group->memcg) {
> > +             gfp |= __GFP_ACCOUNT;
> > +             old_memcg = memalloc_use_memcg(group->memcg);
> > +     }
>
> group->memcg is only NULL when memcg is disabled or there is some
> offlining race. Can you make memalloc_use_memcg(NULL) mean that it
> should charge root_mem_cgroup instead of current->mm->memcg? That way
> we can make this site unconditional while retaining the behavior:
>
>         gfp_t gfp = GFP_KERNEL | __GFP_ACCOUNT;
>
>         memalloc_use_memcg(group->memcg);
>         kmem_cache_alloc(..., gfp);
> out:
>         memalloc_unuse_memcg();
>
> (dropping old_memcg and the unuse parameter as per the other mail)
>

group->memcg is only NULL when memcg is disabled (i.e.
get_mem_cgroup_from_mm() returns root_mem_cgroup for offlined
mm->memcg). Though group->memcg can point to an offlined memcg.

If I understand you correctly this is what we want:

1. If group->memcg is NULL then __GFP_ACCOUNT is a noop i.e. memcg is disabled.
2. If group->memcg is root_mem_cgroup, then __GFP_ACCOUNT again is a
kind of noop (charges to root_mem_cgroups are bypassed).
3. If group->memcg is offlined memcg, then make __GFP_ACCOUNT noop by
returning root_mem_cgroup from get_mem_cgroup_from_current().
4. Else charge group->memcg.

This seems reasonable. After your Ack and Amir's or Jan's answer to
the nesting query, I will resend the next version of this patch
series.

In future if we find any use-cases of memalloc_use_memcg nesting then
we can make it work for nesting.

thanks,
Shakeel

Re: [PATCH 1/2] fs: fsnotify: account fsnotify metadata to kmemcg

[Amir Goldstein]

On Tue, Jun 26, 2018 at 11:05 PM, Shakeel Butt <shakeelb@google.com> wrote:
> On Tue, Jun 26, 2018 at 12:03 PM Johannes Weiner <hannes@cmpxchg.org> wrote:
>>
>> On Mon, Jun 25, 2018 at 04:06:58PM -0700, Shakeel Butt wrote:
>> > @@ -140,8 +141,9 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
>> >                                                struct inode *inode, u32 mask,
>> >                                                const struct path *path)
>> >  {
>> > -     struct fanotify_event_info *event;
>> > +     struct fanotify_event_info *event = NULL;
>> >       gfp_t gfp = GFP_KERNEL;
>> > +     struct mem_cgroup *old_memcg = NULL;
>> >
>> >       /*
>> >        * For queues with unlimited length lost events are not expected and
>> > @@ -151,19 +153,25 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
>> >       if (group->max_events == UINT_MAX)
>> >               gfp |= __GFP_NOFAIL;
>> >
>> > +     /* Whoever is interested in the event, pays for the allocation. */
>> > +     if (group->memcg) {
>> > +             gfp |= __GFP_ACCOUNT;
>> > +             old_memcg = memalloc_use_memcg(group->memcg);
>> > +     }
>>
>> group->memcg is only NULL when memcg is disabled or there is some
>> offlining race. Can you make memalloc_use_memcg(NULL) mean that it
>> should charge root_mem_cgroup instead of current->mm->memcg? That way
>> we can make this site unconditional while retaining the behavior:
>>
>>         gfp_t gfp = GFP_KERNEL | __GFP_ACCOUNT;
>>
>>         memalloc_use_memcg(group->memcg);
>>         kmem_cache_alloc(..., gfp);
>> out:
>>         memalloc_unuse_memcg();
>>
>> (dropping old_memcg and the unuse parameter as per the other mail)
>>
>
> group->memcg is only NULL when memcg is disabled (i.e.
> get_mem_cgroup_from_mm() returns root_mem_cgroup for offlined
> mm->memcg). Though group->memcg can point to an offlined memcg.
>
> If I understand you correctly this is what we want:
>
> 1. If group->memcg is NULL then __GFP_ACCOUNT is a noop i.e. memcg is disabled.
> 2. If group->memcg is root_mem_cgroup, then __GFP_ACCOUNT again is a
> kind of noop (charges to root_mem_cgroups are bypassed).
> 3. If group->memcg is offlined memcg, then make __GFP_ACCOUNT noop by
> returning root_mem_cgroup from get_mem_cgroup_from_current().
> 4. Else charge group->memcg.
>
> This seems reasonable. After your Ack and Amir's or Jan's answer to
> the nesting query, I will resend the next version of this patch
> series.
>
> In future if we find any use-cases of memalloc_use_memcg nesting then
> we can make it work for nesting.
>

For the fsnotify use case memalloc_use_memcg() certainly doesn't
need to nest, but I wonder, if that facility becomes popular among different
subsystems, how exactly do you intend to monitor that it doesn't grow
nested use cases? I would suggest that you at least leave a
WARN_ON_ONCE if memalloc_use_memcg() is called and
active_memcg is already set.

Thanks,
Amir.