[PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Kirill Tkhai]

On nodes without memory overcommit, it's common a situation,
when memcg exceeds its limit and pages from pagecache are
shrinked on reclaim, while node has a lot of free memory.
Further access to the pages requires real device IO, while
IO causes time delays, worse powerusage, worse throughput
for other users of the device, etc.

Cleancache is not a good solution for this problem, since
it implies copying of page on every cleancache_put_page()
and cleancache_get_page(). Also, it requires introduction
of internal per-cleancache_ops data structures to manage
cached pages and their inodes relationships, which again
introduces overhead.

This patchset introduces another solution. It introduces
a new scheme for evicting memcg pages:

  1)__remove_mapping() uncharges unmapped page memcg
    and leaves page in pagecache on memcg reclaim;

  2)putback_lru_page() places page into root_mem_cgroup
    list, since its memcg is NULL. Page may be evicted
    on global reclaim (and this will be easily, as
    page is not mapped, so shrinker will shrink it
    with 100% probability of success);

  3)pagecache_get_page() charges page into memcg of
    a task, which takes it first.

Below is small test, which shows profit of the patchset.

Create memcg with limit 20M (exact value does not matter much):
  $ mkdir /sys/fs/cgroup/memory/ct
  $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
  $ echo $$ > /sys/fs/cgroup/memory/ct/tasks

Then twice read 1GB file:
  $ time cat file_1gb > /dev/null

Before (2 iterations):
  1)0.01user 0.82system 0:11.16elapsed 7%CPU
  2)0.01user 0.91system 0:11.16elapsed 8%CPU

After (2 iterations):
  1)0.01user 0.57system 0:11.31elapsed 5%CPU
  2)0.00user 0.28system 0:00.28elapsed 100%CPU

With the patch set applied, we have file pages are cached
during the second read, so the result is 39 times faster.

This may be useful for slow disks, NFS, nodes without
overcommit by memory, in case of two memcg access the same
files, etc.

---

Kirill Tkhai (3):
      mm: Uncharge and keep page in pagecache on memcg reclaim
      mm: Recharge page memcg on first get from pagecache
      mm: Pass FGP_NOWAIT in generic_file_buffered_read and enable ext4


 fs/ext4/inode.c         |    1 +
 include/linux/pagemap.h |    1 +
 mm/filemap.c            |   38 ++++++++++++++++++++++++++++++++++++--
 mm/vmscan.c             |   22 ++++++++++++++++++----
 4 files changed, 56 insertions(+), 6 deletions(-)

--
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>

[PATCH 1/3] mm: Uncharge and keep page in pagecache on memcg reclaim

[Kirill Tkhai]

This patch makes __remove_mapping() not remove a page from
pagecache on memcg reclaim. After all mappings are removed
and refcounter is freezed, we uncharge page memcg. Further
putback_lru_page() places page into root_mem_cgroup, so it
remains in pagecache till global reclaim. This gives memcg
tasks extra possibility to obtain page from pagecache
instead of launching IO.

Next patch makes pagecache_get_page() to recharge a page
in case of its memcg is NULL (i.e., on first access after
uncharging). It looks to be the only function, which is
used by filesystems to obtain a pagecache page. Here we
introduce AS_KEEP_MEMCG_RECLAIM flag to mark the filesystems,
which are reviewed, that they really follow this way. It
has a sense to keep pages in __remove_mapping() only for
them. Later, we remove this flags after all filesystems are
reviewed.

Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
---
 include/linux/pagemap.h |    1 +
 mm/vmscan.c             |   22 ++++++++++++++++++----
 2 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 1020e6f40880..1b880da85868 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -29,6 +29,7 @@ enum mapping_flags {
 	AS_EXITING	= 4, 	/* final truncate in progress */
 	/* writeback related tags are not used */
 	AS_NO_WRITEBACK_TAGS = 5,
+	AS_KEEP_MEMCG_RECLAIM = 6,
 };
 
 /**
diff --git a/mm/vmscan.c b/mm/vmscan.c
index a714c4f800e9..7237603c8973 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -887,7 +887,7 @@ static pageout_t pageout(struct page *page, struct address_space *mapping,
  * gets returned with a refcount of 0.
  */
 static int __remove_mapping(struct address_space *mapping, struct page *page,
-			    bool reclaimed)
+			    bool reclaimed, bool memcg_reclaim)
 {
 	unsigned long flags;
 	int refcount;
@@ -963,7 +963,20 @@ static int __remove_mapping(struct address_space *mapping, struct page *page,
 		if (reclaimed && page_is_file_cache(page) &&
 		    !mapping_exiting(mapping) && !dax_mapping(mapping))
 			shadow = workingset_eviction(mapping, page);
-		__delete_from_page_cache(page, shadow);
+#ifdef CONFIG_MEMCG
+		if (memcg_reclaim &&
+		    test_bit(AS_KEEP_MEMCG_RECLAIM, &mapping->flags)) {
+			/*
+			 * Page is not dirty/writeback/mapped, so we may avoid
+			 * taking mem_cgroup::move_lock for changing its memcg.
+			 * See mem_cgroup_move_account() for details.
+			 */
+			mem_cgroup_uncharge(page);
+			page_ref_unfreeze(page, refcount);
+			goto cannot_free;
+		} else
+#endif
+			__delete_from_page_cache(page, shadow);
 		xa_unlock_irqrestore(&mapping->i_pages, flags);
 
 		if (freepage != NULL)
@@ -985,7 +998,7 @@ static int __remove_mapping(struct address_space *mapping, struct page *page,
  */
 int remove_mapping(struct address_space *mapping, struct page *page)
 {
-	if (__remove_mapping(mapping, page, false)) {
+	if (__remove_mapping(mapping, page, false, false)) {
 		/*
 		 * Unfreezing the refcount with 1 rather than 2 effectively
 		 * drops the pagecache ref for us without requiring another
@@ -1458,7 +1471,8 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 
 			count_vm_event(PGLAZYFREED);
 			count_memcg_page_event(page, PGLAZYFREED);
-		} else if (!mapping || !__remove_mapping(mapping, page, true))
+		} else if (!mapping || !__remove_mapping(mapping, page, true,
+							!global_reclaim(sc)))
 			goto keep_locked;
 
 		unlock_page(page);

[PATCH 2/3] mm: Recharge page memcg on first get from pagecache

[Kirill Tkhai]

This patch makes pagecache_get_page() to charge uncharged
page into memcg of process, which accesses the page first.
Page will be returned in case of it's charged only, so
memcg tasks can't use pages, which was left by __remove_mapping(),
without accounting them. In case of accounting is not possible,
pages remain in pagecache, and further global reclaim will
remove them (and this will be easily, since pages are not
mapped by any task).

Also, note that uncharged page can't be dirty or under
writeback, since it was able to be isolated in __remove_mapping()
earlier.

Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
---
 mm/filemap.c |   30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/mm/filemap.c b/mm/filemap.c
index 65c85c47bdb1..2603c44fc74a 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1576,15 +1576,18 @@ EXPORT_SYMBOL(find_lock_entry);
 struct page *pagecache_get_page(struct address_space *mapping, pgoff_t offset,
 	int fgp_flags, gfp_t gfp_mask)
 {
+	struct mem_cgroup *memcg;
 	struct page *page;
+	bool drop_lock;
 
 repeat:
+	drop_lock = false;
 	page = find_get_entry(mapping, offset);
 	if (xa_is_value(page))
 		page = NULL;
 	if (!page)
 		goto no_page;
-
+lock:
 	if (fgp_flags & FGP_LOCK) {
 		if (fgp_flags & FGP_NOWAIT) {
 			if (!trylock_page(page)) {
@@ -1604,6 +1607,31 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t offset,
 		VM_BUG_ON_PAGE(page->index != offset, page);
 	}
 
+	if (!mem_cgroup_disabled() && !PageHuge(page) &&
+	    !page_memcg(page) && !page_mapped(page) &&
+	    test_bit(AS_KEEP_MEMCG_RECLAIM, &mapping->flags)) {
+		if (!(fgp_flags & FGP_LOCK)) {
+			drop_lock = true;
+			fgp_flags |= FGP_LOCK;
+			goto lock;
+		}
+
+		if (!WARN_ON(PageDirty(page) || PageWriteback(page))) {
+			if (mem_cgroup_try_charge(page, current->mm,
+					gfp_mask, &memcg, false)) {
+				unlock_page(page);
+				put_page(page);
+				return NULL;
+			}
+			mem_cgroup_commit_charge(page, memcg, true, false);
+			if (!isolate_lru_page(page))
+				putback_lru_page(page);
+		}
+	}
+
+	if (drop_lock)
+		unlock_page(page);
+
 	if (fgp_flags & FGP_ACCESSED)
 		mark_page_accessed(page);
 

[PATCH 3/3] mm: Pass FGP_NOWAIT in generic_file_buffered_read and enable ext4

[Kirill Tkhai]

All page-obtaining functions, which are used by ext4, look
to go thru pagecache_get_page() path, so all taken uncharged
pages will be properly charged. Thus, we enable AS_KEEP_MEMCG_RECLAIM
for ext4 regular files.

Since memcg accounting requires page lock, and function
generic_file_buffered_read() is the only of ext4-used
functions, which does not care about FGP_NOWAIT, we make
it use find_get_page_flags() and pass the flag. This allows
pagecache_get_page() to use lock_page(), when it's possible.

Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
---
 fs/ext4/inode.c |    1 +
 mm/filemap.c    |    8 +++++++-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index b1d7ddd70eee..2fc9e4a7c0db 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -5065,6 +5065,7 @@ struct inode *__ext4_iget(struct super_block *sb, unsigned long ino,
 		inode->i_op = &ext4_file_inode_operations;
 		inode->i_fop = &ext4_file_operations;
 		ext4_set_aops(inode);
+		set_bit(AS_KEEP_MEMCG_RECLAIM, &inode->i_mapping->flags);
 	} else if (S_ISDIR(inode->i_mode)) {
 		inode->i_op = &ext4_dir_inode_operations;
 		inode->i_fop = &ext4_dir_operations;
diff --git a/mm/filemap.c b/mm/filemap.c
index 2603c44fc74a..46922003811f 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -2061,6 +2061,11 @@ static void shrink_readahead_size_eio(struct file *filp,
 	ra->ra_pages /= 4;
 }
 
+static int kiocb_fgp_flags(struct kiocb *iocb)
+{
+	return (iocb->ki_flags & IOCB_NOWAIT) ? FGP_NOWAIT : 0;
+}
+
 /**
  * generic_file_buffered_read - generic file read routine
  * @iocb:	the iocb to read
@@ -2111,7 +2116,8 @@ static ssize_t generic_file_buffered_read(struct kiocb *iocb,
 			goto out;
 		}
 
-		page = find_get_page(mapping, index);
+		page = find_get_page_flags(mapping, index,
+					   kiocb_fgp_flags(iocb));
 		if (!page) {
 			if (iocb->ki_flags & IOCB_NOWAIT)
 				goto would_block;

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Michal Hocko]

On Wed 09-01-19 15:20:18, Kirill Tkhai wrote:
> On nodes without memory overcommit, it's common a situation,
> when memcg exceeds its limit and pages from pagecache are
> shrinked on reclaim, while node has a lot of free memory.

Yes, that is the semantic of the hard limit. If the system is not
overcommitted then the hard limit can be used to prevent unexpected
direct reclaim from unrelated activity.

> Further access to the pages requires real device IO, while
> IO causes time delays, worse powerusage, worse throughput
> for other users of the device, etc.

It is to be expected that a memory throttled usage will have this side
effect IMO.

> Cleancache is not a good solution for this problem, since
> it implies copying of page on every cleancache_put_page()
> and cleancache_get_page(). Also, it requires introduction
> of internal per-cleancache_ops data structures to manage
> cached pages and their inodes relationships, which again
> introduces overhead.
> 
> This patchset introduces another solution. It introduces
> a new scheme for evicting memcg pages:
> 
>   1)__remove_mapping() uncharges unmapped page memcg
>     and leaves page in pagecache on memcg reclaim;
> 
>   2)putback_lru_page() places page into root_mem_cgroup
>     list, since its memcg is NULL. Page may be evicted
>     on global reclaim (and this will be easily, as
>     page is not mapped, so shrinker will shrink it
>     with 100% probability of success);
> 
>   3)pagecache_get_page() charges page into memcg of
>     a task, which takes it first.

But this also means that any hard limited memcg can fill up all the
memory and break the above assumption about the isolation from direct
reclaim. Not to mention the OOM or is there anything you do anything
about preventing that?

That beig said, I do not think we want to or even can change the
semantic of the hard limit and break existing setups. I am still
interested to hear more about more detailed/specific usecases that might
benefit from this behavior. Why do those users even use hard limit at
all? To protect from anon memory leaks? Do different memcgs share the
page cache heavily?
-- 
Michal Hocko
SUSE Labs

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Kirill Tkhai]

Hi, Michal,

On 09.01.2019 17:11, Michal Hocko wrote:
> On Wed 09-01-19 15:20:18, Kirill Tkhai wrote:
>> On nodes without memory overcommit, it's common a situation,
>> when memcg exceeds its limit and pages from pagecache are
>> shrinked on reclaim, while node has a lot of free memory.
> 
> Yes, that is the semantic of the hard limit. If the system is not
> overcommitted then the hard limit can be used to prevent unexpected
> direct reclaim from unrelated activity.

According to Documentation/admin-guide/cgroup-v2.rst:

  memory.max
        Memory usage hard limit.  This is the final protection
        mechanism.  If a cgroup's memory usage reaches this limit and
        can't be reduced, the OOM killer is invoked in the cgroup.
        Under certain circumstances, the usage may go over the limit
        temporarily.

There is nothing about direct reclaim in another memcg. I don't think
we break something here.

File pages are accounted to memcg, and this guarantees, that single
memcg won't occupy all system memory by its unevictible page cache.
But the suggested patchset follows the same way. Pages, which remain
in pagecache, are easy-to-be-evicted, since they are not dirty and
not under writeback. System can drop them fast and in foreseeable time.
This is cardinal thing about the patchset: remained pages do not
introduce principal burden on system memory or reclaim time.

>> Further access to the pages requires real device IO, while
>> IO causes time delays, worse powerusage, worse throughput
>> for other users of the device, etc.
> 
> It is to be expected that a memory throttled usage will have this side
> effect IMO.
> 
>> Cleancache is not a good solution for this problem, since
>> it implies copying of page on every cleancache_put_page()
>> and cleancache_get_page(). Also, it requires introduction
>> of internal per-cleancache_ops data structures to manage
>> cached pages and their inodes relationships, which again
>> introduces overhead.
>>
>> This patchset introduces another solution. It introduces
>> a new scheme for evicting memcg pages:
>>
>>   1)__remove_mapping() uncharges unmapped page memcg
>>     and leaves page in pagecache on memcg reclaim;
>>
>>   2)putback_lru_page() places page into root_mem_cgroup
>>     list, since its memcg is NULL. Page may be evicted
>>     on global reclaim (and this will be easily, as
>>     page is not mapped, so shrinker will shrink it
>>     with 100% probability of success);
>>
>>   3)pagecache_get_page() charges page into memcg of
>>     a task, which takes it first.
> 
> But this also means that any hard limited memcg can fill up all the
> memory and break the above assumption about the isolation from direct
> reclaim. Not to mention the OOM or is there anything you do anything
> about preventing that?

This is discussed thing. We may add such the pages into tail of LRU list
instead of head. We may introduce one more separate list to link such
the pages only, and fastly evict them in case of global reclaim. I don't
think there is a problem.
 
> That beig said, I do not think we want to or even can change the
> semantic of the hard limit and break existing setups.

Using the original description and the comments I gave in this message,
could you please to clarify the way we break existing setups?

> I am still
> interested to hear more about more detailed/specific usecases that might
> benefit from this behavior. Why do those users even use hard limit at
> all? To protect from anon memory leaks?

In multi-user machine people want to have size of available to container
memory equal to the size, which they pay. So, hard limit is needed to prevent
one container to occupy all system memory via slowly-evictible writeback
pages, unevictible anon pages, etc. You can't fastly allocate a page,
in case of many pages are under writeback, this operation is very slow.

(But unmapped pagecache pages introduced by patchset is another thing:
 you just need to take not sleeping spinlock to call __delete_from_page_cache()
 only. This is fast)

Multi-user machine may have more memory, than sum of all containers hard
limit. This may be used as an optimization just to reduce disk IO. There
is no contradiction to sane sense here. And it's not a rare situation.
In our kernel we have cleancache driver for handling this situation, but
cleancache is not the best solution like I wrote.

Not overcommited system is likely case for the patchset, while the below
is a little less likely:

> Do different memcgs share the page cache heavily?

People may use NFS server from different containers. They may want
to have low priority userspace workers and high priority main task,
and they may want to reduce the workers memory consumption. These
are the cases.

Kirill

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Michal Hocko]

On Wed 09-01-19 18:43:05, Kirill Tkhai wrote:
> Hi, Michal,
> 
> On 09.01.2019 17:11, Michal Hocko wrote:
> > On Wed 09-01-19 15:20:18, Kirill Tkhai wrote:
> >> On nodes without memory overcommit, it's common a situation,
> >> when memcg exceeds its limit and pages from pagecache are
> >> shrinked on reclaim, while node has a lot of free memory.
> > 
> > Yes, that is the semantic of the hard limit. If the system is not
> > overcommitted then the hard limit can be used to prevent unexpected
> > direct reclaim from unrelated activity.
> 
> According to Documentation/admin-guide/cgroup-v2.rst:
> 
>   memory.max
>         Memory usage hard limit.  This is the final protection
>         mechanism.  If a cgroup's memory usage reaches this limit and
>         can't be reduced, the OOM killer is invoked in the cgroup.
>         Under certain circumstances, the usage may go over the limit
>         temporarily.
> 
> There is nothing about direct reclaim in another memcg. I don't think
> we break something here.

Others in the thread have pointed that out already. What is a hard limit
in one memcg is an isolateion protection in another one. Especially when
the system is not overcommited.

> File pages are accounted to memcg, and this guarantees, that single
> memcg won't occupy all system memory by its unevictible page cache.
> But the suggested patchset follows the same way. Pages, which remain
> in pagecache, are easy-to-be-evicted, since they are not dirty and
> not under writeback. System can drop them fast and in foreseeable time.
> This is cardinal thing about the patchset: remained pages do not
> introduce principal burden on system memory or reclaim time.

What does prevent that the page cache is easily reclaimable? Aka clean
and ready to be dropped? Not to mention that even when the reclaim is
fast it is not free. Especially when you do not expect that because you
haven't reached your hard limit and the admin made sure that hard limits
do not overcommit.

[...]

> > But this also means that any hard limited memcg can fill up all the
> > memory and break the above assumption about the isolation from direct
> > reclaim. Not to mention the OOM or is there anything you do anything
> > about preventing that?
> 
> This is discussed thing. We may add such the pages into tail of LRU list
> instead of head. We may introduce one more separate list to link such
> the pages only, and fastly evict them in case of global reclaim. I don't
> think there is a problem.
>  
> > That beig said, I do not think we want to or even can change the
> > semantic of the hard limit and break existing setups.
> 
> Using the original description and the comments I gave in this message,
> could you please to clarify the way we break existing setups?

isolation as explained above.

> > I am still
> > interested to hear more about more detailed/specific usecases that might
> > benefit from this behavior. Why do those users even use hard limit at
> > all? To protect from anon memory leaks?
> 
> In multi-user machine people want to have size of available to container
> memory equal to the size, which they pay. So, hard limit is needed to prevent
> one container to occupy all system memory via slowly-evictible writeback
> pages, unevictible anon pages, etc. You can't fastly allocate a page,
> in case of many pages are under writeback, this operation is very slow.
> 
> (But unmapped pagecache pages introduced by patchset is another thing:
>  you just need to take not sleeping spinlock to call __delete_from_page_cache()
>  only. This is fast)
> 
> Multi-user machine may have more memory, than sum of all containers hard
> limit. This may be used as an optimization just to reduce disk IO. There
> is no contradiction to sane sense here. And it's not a rare situation.
> In our kernel we have cleancache driver for handling this situation, but
> cleancache is not the best solution like I wrote.
> 
> Not overcommited system is likely case for the patchset, while the below
> is a little less likely:

I beliave Johannes has explained that you are trying to use the hard
limit in a wrong way for something it is not designed for.

-- 
Michal Hocko
SUSE Labs

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Kirill Tkhai]

On 09.01.2019 20:10, Michal Hocko wrote:
> On Wed 09-01-19 18:43:05, Kirill Tkhai wrote:
>> Hi, Michal,
>>
>> On 09.01.2019 17:11, Michal Hocko wrote:
>>> On Wed 09-01-19 15:20:18, Kirill Tkhai wrote:
>>>> On nodes without memory overcommit, it's common a situation,
>>>> when memcg exceeds its limit and pages from pagecache are
>>>> shrinked on reclaim, while node has a lot of free memory.
>>>
>>> Yes, that is the semantic of the hard limit. If the system is not
>>> overcommitted then the hard limit can be used to prevent unexpected
>>> direct reclaim from unrelated activity.
>>
>> According to Documentation/admin-guide/cgroup-v2.rst:
>>
>>   memory.max
>>         Memory usage hard limit.  This is the final protection
>>         mechanism.  If a cgroup's memory usage reaches this limit and
>>         can't be reduced, the OOM killer is invoked in the cgroup.
>>         Under certain circumstances, the usage may go over the limit
>>         temporarily.
>>
>> There is nothing about direct reclaim in another memcg. I don't think
>> we break something here.
> 
> Others in the thread have pointed that out already. What is a hard limit
> in one memcg is an isolateion protection in another one. Especially when
> the system is not overcommited.
> 
>> File pages are accounted to memcg, and this guarantees, that single
>> memcg won't occupy all system memory by its unevictible page cache.
>> But the suggested patchset follows the same way. Pages, which remain
>> in pagecache, are easy-to-be-evicted, since they are not dirty and
>> not under writeback. System can drop them fast and in foreseeable time.
>> This is cardinal thing about the patchset: remained pages do not
>> introduce principal burden on system memory or reclaim time.
> 
> What does prevent that the page cache is easily reclaimable? Aka clean
> and ready to be dropped? Not to mention that even when the reclaim is
> fast it is not free. Especially when you do not expect that because you
> haven't reached your hard limit and the admin made sure that hard limits
> do not overcommit.

Yes, I mean it's clean and ready to drop.

I understand the problem, so in case of people worry about reclaim speed
increasing, this does not mean we should completely forget this way. This
means we possible may find a compromise, which is suitable for everybody.

>>> But this also means that any hard limited memcg can fill up all the
>>> memory and break the above assumption about the isolation from direct
>>> reclaim. Not to mention the OOM or is there anything you do anything
>>> about preventing that?
>>
>> This is discussed thing. We may add such the pages into tail of LRU list
>> instead of head. We may introduce one more separate list to link such
>> the pages only, and fastly evict them in case of global reclaim. I don't
>> think there is a problem.
>>  
>>> That beig said, I do not think we want to or even can change the
>>> semantic of the hard limit and break existing setups.
>>
>> Using the original description and the comments I gave in this message,
>> could you please to clarify the way we break existing setups?
> 
> isolation as explained above.
> 
>>> I am still
>>> interested to hear more about more detailed/specific usecases that might
>>> benefit from this behavior. Why do those users even use hard limit at
>>> all? To protect from anon memory leaks?
>>
>> In multi-user machine people want to have size of available to container
>> memory equal to the size, which they pay. So, hard limit is needed to prevent
>> one container to occupy all system memory via slowly-evictible writeback
>> pages, unevictible anon pages, etc. You can't fastly allocate a page,
>> in case of many pages are under writeback, this operation is very slow.
>>
>> (But unmapped pagecache pages introduced by patchset is another thing:
>>  you just need to take not sleeping spinlock to call __delete_from_page_cache()
>>  only. This is fast)
>>
>> Multi-user machine may have more memory, than sum of all containers hard
>> limit. This may be used as an optimization just to reduce disk IO. There
>> is no contradiction to sane sense here. And it's not a rare situation.
>> In our kernel we have cleancache driver for handling this situation, but
>> cleancache is not the best solution like I wrote.
>>
>> Not overcommited system is likely case for the patchset, while the below
>> is a little less likely:
> 
> I beliave Johannes has explained that you are trying to use the hard
> limit in a wrong way for something it is not designed for.

In general, I think a some time useful design is not a Bible, that nobody
is allowed to change. We should not limit us in something, in case of this
has a sense and may be useful. This is just a note in general.

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Michal Hocko]

On Thu 10-01-19 12:42:02, Kirill Tkhai wrote:
[...]
> In general, I think a some time useful design is not a Bible, that nobody
> is allowed to change. We should not limit us in something, in case of this
> has a sense and may be useful. This is just a note in general.

But any semantic exported to the userspace and real application
depending on it is carved in stone for ever. And this is the case here I
am afraid. So if we really need some sort of soft unmapping or
reparenting a memory from a memcg then we really need to find a
different way. I do not see a straightforward way right now TBH.
-- 
Michal Hocko
SUSE Labs

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Josef Bacik]

On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote:
> On nodes without memory overcommit, it's common a situation,
> when memcg exceeds its limit and pages from pagecache are
> shrinked on reclaim, while node has a lot of free memory.
> Further access to the pages requires real device IO, while
> IO causes time delays, worse powerusage, worse throughput
> for other users of the device, etc.
> 
> Cleancache is not a good solution for this problem, since
> it implies copying of page on every cleancache_put_page()
> and cleancache_get_page(). Also, it requires introduction
> of internal per-cleancache_ops data structures to manage
> cached pages and their inodes relationships, which again
> introduces overhead.
> 
> This patchset introduces another solution. It introduces
> a new scheme for evicting memcg pages:
> 
>   1)__remove_mapping() uncharges unmapped page memcg
>     and leaves page in pagecache on memcg reclaim;
> 
>   2)putback_lru_page() places page into root_mem_cgroup
>     list, since its memcg is NULL. Page may be evicted
>     on global reclaim (and this will be easily, as
>     page is not mapped, so shrinker will shrink it
>     with 100% probability of success);
> 
>   3)pagecache_get_page() charges page into memcg of
>     a task, which takes it first.
> 
> Below is small test, which shows profit of the patchset.
> 
> Create memcg with limit 20M (exact value does not matter much):
>   $ mkdir /sys/fs/cgroup/memory/ct
>   $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
>   $ echo $$ > /sys/fs/cgroup/memory/ct/tasks
> 
> Then twice read 1GB file:
>   $ time cat file_1gb > /dev/null
> 
> Before (2 iterations):
>   1)0.01user 0.82system 0:11.16elapsed 7%CPU
>   2)0.01user 0.91system 0:11.16elapsed 8%CPU
> 
> After (2 iterations):
>   1)0.01user 0.57system 0:11.31elapsed 5%CPU
>   2)0.00user 0.28system 0:00.28elapsed 100%CPU
> 
> With the patch set applied, we have file pages are cached
> during the second read, so the result is 39 times faster.
> 
> This may be useful for slow disks, NFS, nodes without
> overcommit by memory, in case of two memcg access the same
> files, etc.
> 

This isn't going to work for us (Facebook).  The whole reason the hard limit
exists is to keep different groups from messing up other groups.  Page cache
reclaim is not free, most of our pain and most of the reason we use cgroups
is to limit the effect of flooding the machine with pagecache from different
groups.  Memory leaks happen few and far between, but chef doing a yum
update in the system container happens regularly.  If you talk about suddenly
orphaning these pages to the root container it still creates pressure on the
main workload, pressure that results in it having to take time from what it's
doing and free up memory instead.  Thanks,

Josef

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Kirill Tkhai]

Hi, Josef,

On 09.01.2019 18:49, Josef Bacik wrote:
> On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote:
>> On nodes without memory overcommit, it's common a situation,
>> when memcg exceeds its limit and pages from pagecache are
>> shrinked on reclaim, while node has a lot of free memory.
>> Further access to the pages requires real device IO, while
>> IO causes time delays, worse powerusage, worse throughput
>> for other users of the device, etc.
>>
>> Cleancache is not a good solution for this problem, since
>> it implies copying of page on every cleancache_put_page()
>> and cleancache_get_page(). Also, it requires introduction
>> of internal per-cleancache_ops data structures to manage
>> cached pages and their inodes relationships, which again
>> introduces overhead.
>>
>> This patchset introduces another solution. It introduces
>> a new scheme for evicting memcg pages:
>>
>>   1)__remove_mapping() uncharges unmapped page memcg
>>     and leaves page in pagecache on memcg reclaim;
>>
>>   2)putback_lru_page() places page into root_mem_cgroup
>>     list, since its memcg is NULL. Page may be evicted
>>     on global reclaim (and this will be easily, as
>>     page is not mapped, so shrinker will shrink it
>>     with 100% probability of success);
>>
>>   3)pagecache_get_page() charges page into memcg of
>>     a task, which takes it first.
>>
>> Below is small test, which shows profit of the patchset.
>>
>> Create memcg with limit 20M (exact value does not matter much):
>>   $ mkdir /sys/fs/cgroup/memory/ct
>>   $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
>>   $ echo $$ > /sys/fs/cgroup/memory/ct/tasks
>>
>> Then twice read 1GB file:
>>   $ time cat file_1gb > /dev/null
>>
>> Before (2 iterations):
>>   1)0.01user 0.82system 0:11.16elapsed 7%CPU
>>   2)0.01user 0.91system 0:11.16elapsed 8%CPU
>>
>> After (2 iterations):
>>   1)0.01user 0.57system 0:11.31elapsed 5%CPU
>>   2)0.00user 0.28system 0:00.28elapsed 100%CPU
>>
>> With the patch set applied, we have file pages are cached
>> during the second read, so the result is 39 times faster.
>>
>> This may be useful for slow disks, NFS, nodes without
>> overcommit by memory, in case of two memcg access the same
>> files, etc.
>>
> 
> This isn't going to work for us (Facebook).  The whole reason the hard limit
> exists is to keep different groups from messing up other groups.  Page cache
> reclaim is not free, most of our pain and most of the reason we use cgroups
> is to limit the effect of flooding the machine with pagecache from different
> groups.

I understand the problem.

> Memory leaks happen few and far between, but chef doing a yum
> update in the system container happens regularly.  If you talk about suddenly
> orphaning these pages to the root container it still creates pressure on the
> main workload, pressure that results in it having to take time from what it's
> doing and free up memory instead.

Could you please to clarify additional pressure, which introduces the patchset?
The number of actions, which are needed to evict a pagecache page, remain almost
the same: we just delay __delete_from_page_cache() to global reclaim. Global
reclaim should not introduce much pressure, since it's the iteration on a single
memcg (we should not dive into hell of children memcg, since root memcg reclaim
should be successful and free enough pages, should't we?).

Also, what is about implementing this as static key option? What about linking
orphaned pagecache pages into separate list, which is easy-to-iterate?

Thanks,
Kirill

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Josef Bacik]

On Wed, Jan 09, 2019 at 07:08:09PM +0300, Kirill Tkhai wrote:
> Hi, Josef,
> 
> On 09.01.2019 18:49, Josef Bacik wrote:
> > On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote:
> >> On nodes without memory overcommit, it's common a situation,
> >> when memcg exceeds its limit and pages from pagecache are
> >> shrinked on reclaim, while node has a lot of free memory.
> >> Further access to the pages requires real device IO, while
> >> IO causes time delays, worse powerusage, worse throughput
> >> for other users of the device, etc.
> >>
> >> Cleancache is not a good solution for this problem, since
> >> it implies copying of page on every cleancache_put_page()
> >> and cleancache_get_page(). Also, it requires introduction
> >> of internal per-cleancache_ops data structures to manage
> >> cached pages and their inodes relationships, which again
> >> introduces overhead.
> >>
> >> This patchset introduces another solution. It introduces
> >> a new scheme for evicting memcg pages:
> >>
> >>   1)__remove_mapping() uncharges unmapped page memcg
> >>     and leaves page in pagecache on memcg reclaim;
> >>
> >>   2)putback_lru_page() places page into root_mem_cgroup
> >>     list, since its memcg is NULL. Page may be evicted
> >>     on global reclaim (and this will be easily, as
> >>     page is not mapped, so shrinker will shrink it
> >>     with 100% probability of success);
> >>
> >>   3)pagecache_get_page() charges page into memcg of
> >>     a task, which takes it first.
> >>
> >> Below is small test, which shows profit of the patchset.
> >>
> >> Create memcg with limit 20M (exact value does not matter much):
> >>   $ mkdir /sys/fs/cgroup/memory/ct
> >>   $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
> >>   $ echo $$ > /sys/fs/cgroup/memory/ct/tasks
> >>
> >> Then twice read 1GB file:
> >>   $ time cat file_1gb > /dev/null
> >>
> >> Before (2 iterations):
> >>   1)0.01user 0.82system 0:11.16elapsed 7%CPU
> >>   2)0.01user 0.91system 0:11.16elapsed 8%CPU
> >>
> >> After (2 iterations):
> >>   1)0.01user 0.57system 0:11.31elapsed 5%CPU
> >>   2)0.00user 0.28system 0:00.28elapsed 100%CPU
> >>
> >> With the patch set applied, we have file pages are cached
> >> during the second read, so the result is 39 times faster.
> >>
> >> This may be useful for slow disks, NFS, nodes without
> >> overcommit by memory, in case of two memcg access the same
> >> files, etc.
> >>
> > 
> > This isn't going to work for us (Facebook).  The whole reason the hard limit
> > exists is to keep different groups from messing up other groups.  Page cache
> > reclaim is not free, most of our pain and most of the reason we use cgroups
> > is to limit the effect of flooding the machine with pagecache from different
> > groups.
> 
> I understand the problem.
> 
> > Memory leaks happen few and far between, but chef doing a yum
> > update in the system container happens regularly.  If you talk about suddenly
> > orphaning these pages to the root container it still creates pressure on the
> > main workload, pressure that results in it having to take time from what it's
> > doing and free up memory instead.
> 
> Could you please to clarify additional pressure, which introduces the patchset?
> The number of actions, which are needed to evict a pagecache page, remain almost
> the same: we just delay __delete_from_page_cache() to global reclaim. Global
> reclaim should not introduce much pressure, since it's the iteration on a single
> memcg (we should not dive into hell of children memcg, since root memcg reclaim
> should be successful and free enough pages, should't we?).

If we go into global reclaim at all.  If we're unable to allocate a page as the
most important cgroup we start shrinking ourselves first right?  And then
eventually end up in global reclaim, right?  So it may be easily enough
reclaimed, but we're going to waste a lot of time getting there in the meantime,
which means latency that's hard to pin down.

And secondly this allows hard limited cgroups to essentially leak pagecache into
the whole system, creating waaaaaaay more memory pressure than what I think you
intend.  Your logic is that we'll exceed our limit, evict some pagecache to the
root cgroup, and we avoid a OOM and everything is ok.  However what will really
happen is some user is going to do dd if=/dev/zero of=file and we'll just
happily keep shoving these pages off into the root cg and suddenly we have 100gb
of useless pagecache that we have to reclaim.  Yeah we just have to delete it
from the root, but thats only once we get to that part, before that there's a
bunch of latency inducing work that has to be done to get to deleting the pages.

> 
> Also, what is about implementing this as static key option? What about linking
> orphaned pagecache pages into separate list, which is easy-to-iterate?

Yeah if we have a way to short-circuit the normal reclaim path and just go to
evicting these easily evicted pages then that would make it more palatable.  But
I'd like to see testing to verify that this faster way really is faster and
doesn't induce latency on other protected workloads.  We put hard limits on
groups we don't care about, we want those things to die in a fire.  The excess
IO from re-reading those pages is mitigated with io.latency, and eventually
io.weight for proportional control, so really isn't an argument for keeping
pages around.  Thanks,

Josef

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Kirill Tkhai]

On 09.01.2019 19:33, Josef Bacik wrote:
> On Wed, Jan 09, 2019 at 07:08:09PM +0300, Kirill Tkhai wrote:
>> Hi, Josef,
>>
>> On 09.01.2019 18:49, Josef Bacik wrote:
>>> On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote:
>>>> On nodes without memory overcommit, it's common a situation,
>>>> when memcg exceeds its limit and pages from pagecache are
>>>> shrinked on reclaim, while node has a lot of free memory.
>>>> Further access to the pages requires real device IO, while
>>>> IO causes time delays, worse powerusage, worse throughput
>>>> for other users of the device, etc.
>>>>
>>>> Cleancache is not a good solution for this problem, since
>>>> it implies copying of page on every cleancache_put_page()
>>>> and cleancache_get_page(). Also, it requires introduction
>>>> of internal per-cleancache_ops data structures to manage
>>>> cached pages and their inodes relationships, which again
>>>> introduces overhead.
>>>>
>>>> This patchset introduces another solution. It introduces
>>>> a new scheme for evicting memcg pages:
>>>>
>>>>   1)__remove_mapping() uncharges unmapped page memcg
>>>>     and leaves page in pagecache on memcg reclaim;
>>>>
>>>>   2)putback_lru_page() places page into root_mem_cgroup
>>>>     list, since its memcg is NULL. Page may be evicted
>>>>     on global reclaim (and this will be easily, as
>>>>     page is not mapped, so shrinker will shrink it
>>>>     with 100% probability of success);
>>>>
>>>>   3)pagecache_get_page() charges page into memcg of
>>>>     a task, which takes it first.
>>>>
>>>> Below is small test, which shows profit of the patchset.
>>>>
>>>> Create memcg with limit 20M (exact value does not matter much):
>>>>   $ mkdir /sys/fs/cgroup/memory/ct
>>>>   $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
>>>>   $ echo $$ > /sys/fs/cgroup/memory/ct/tasks
>>>>
>>>> Then twice read 1GB file:
>>>>   $ time cat file_1gb > /dev/null
>>>>
>>>> Before (2 iterations):
>>>>   1)0.01user 0.82system 0:11.16elapsed 7%CPU
>>>>   2)0.01user 0.91system 0:11.16elapsed 8%CPU
>>>>
>>>> After (2 iterations):
>>>>   1)0.01user 0.57system 0:11.31elapsed 5%CPU
>>>>   2)0.00user 0.28system 0:00.28elapsed 100%CPU
>>>>
>>>> With the patch set applied, we have file pages are cached
>>>> during the second read, so the result is 39 times faster.
>>>>
>>>> This may be useful for slow disks, NFS, nodes without
>>>> overcommit by memory, in case of two memcg access the same
>>>> files, etc.
>>>>
>>>
>>> This isn't going to work for us (Facebook).  The whole reason the hard limit
>>> exists is to keep different groups from messing up other groups.  Page cache
>>> reclaim is not free, most of our pain and most of the reason we use cgroups
>>> is to limit the effect of flooding the machine with pagecache from different
>>> groups.
>>
>> I understand the problem.
>>
>>> Memory leaks happen few and far between, but chef doing a yum
>>> update in the system container happens regularly.  If you talk about suddenly
>>> orphaning these pages to the root container it still creates pressure on the
>>> main workload, pressure that results in it having to take time from what it's
>>> doing and free up memory instead.
>>
>> Could you please to clarify additional pressure, which introduces the patchset?
>> The number of actions, which are needed to evict a pagecache page, remain almost
>> the same: we just delay __delete_from_page_cache() to global reclaim. Global
>> reclaim should not introduce much pressure, since it's the iteration on a single
>> memcg (we should not dive into hell of children memcg, since root memcg reclaim
>> should be successful and free enough pages, should't we?).
> 
> If we go into global reclaim at all.  If we're unable to allocate a page as the
> most important cgroup we start shrinking ourselves first right?  And then
> eventually end up in global reclaim, right?  So it may be easily enough
> reclaimed, but we're going to waste a lot of time getting there in the meantime,
> which means latency that's hard to pin down.
> 
> And secondly this allows hard limited cgroups to essentially leak pagecache into
> the whole system, creating waaaaaaay more memory pressure than what I think you
> intend.  Your logic is that we'll exceed our limit, evict some pagecache to the
> root cgroup, and we avoid a OOM and everything is ok.  However what will really
> happen is some user is going to do dd if=/dev/zero of=file and we'll just
> happily keep shoving these pages off into the root cg and suddenly we have 100gb
> of useless pagecache that we have to reclaim.  Yeah we just have to delete it
> from the root, but thats only once we get to that part, before that there's a
> bunch of latency inducing work that has to be done to get to deleting the pages.

Yeah, but what does introduce the most latency in setup? Do I understand correctly
that hard limit on your setup allows all alloc_pages() calls to go thru get_page_from_freelist()
path, so most allocations do not dive into __alloc_pages_slowpath()?

>>
>> Also, what is about implementing this as static key option? What about linking
>> orphaned pagecache pages into separate list, which is easy-to-iterate?
> 
> Yeah if we have a way to short-circuit the normal reclaim path and just go to
> evicting these easily evicted pages then that would make it more palatable.  But
> I'd like to see testing to verify that this faster way really is faster and
> doesn't induce latency on other protected workloads.  We put hard limits on
> groups we don't care about, we want those things to die in a fire.  The excess
> IO from re-reading those pages is mitigated with io.latency, and eventually
> io.weight for proportional control, so really isn't an argument for keeping
> pages around.  Thanks,
> 
> Josef
> 

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Johannes Weiner]

On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote:
> On nodes without memory overcommit, it's common a situation,
> when memcg exceeds its limit and pages from pagecache are
> shrinked on reclaim, while node has a lot of free memory.
> Further access to the pages requires real device IO, while
> IO causes time delays, worse powerusage, worse throughput
> for other users of the device, etc.
> 
> Cleancache is not a good solution for this problem, since
> it implies copying of page on every cleancache_put_page()
> and cleancache_get_page(). Also, it requires introduction
> of internal per-cleancache_ops data structures to manage
> cached pages and their inodes relationships, which again
> introduces overhead.
> 
> This patchset introduces another solution. It introduces
> a new scheme for evicting memcg pages:
> 
>   1)__remove_mapping() uncharges unmapped page memcg
>     and leaves page in pagecache on memcg reclaim;
> 
>   2)putback_lru_page() places page into root_mem_cgroup
>     list, since its memcg is NULL. Page may be evicted
>     on global reclaim (and this will be easily, as
>     page is not mapped, so shrinker will shrink it
>     with 100% probability of success);
> 
>   3)pagecache_get_page() charges page into memcg of
>     a task, which takes it first.
> 
> Below is small test, which shows profit of the patchset.
> 
> Create memcg with limit 20M (exact value does not matter much):
>   $ mkdir /sys/fs/cgroup/memory/ct
>   $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
>   $ echo $$ > /sys/fs/cgroup/memory/ct/tasks
> 
> Then twice read 1GB file:
>   $ time cat file_1gb > /dev/null
> 
> Before (2 iterations):
>   1)0.01user 0.82system 0:11.16elapsed 7%CPU
>   2)0.01user 0.91system 0:11.16elapsed 8%CPU
> 
> After (2 iterations):
>   1)0.01user 0.57system 0:11.31elapsed 5%CPU
>   2)0.00user 0.28system 0:00.28elapsed 100%CPU
> 
> With the patch set applied, we have file pages are cached
> during the second read, so the result is 39 times faster.
> 
> This may be useful for slow disks, NFS, nodes without
> overcommit by memory, in case of two memcg access the same
> files, etc.

What you're implementing is work conservation: avoid causing IO work,
unless it's physically necessary, not when the memcg limit says so.

This is a great idea, but we already have that in the form of the
memory.low setting (or softlimit in cgroup v1).

Say you have a 100M system and two cgroups. Instead of setting the 20M
limit on group A as you did, you set 80M memory.low on group B. If B
is not using its share and there is no physical memory pressure, group
A can consume as much memory as it wants. If B starts and consumes its
80M, A will get pushed back to 20M. (And when B grows beyond 80M, they
compete fairly over the remaining 20M, just like they would if A had
the 20M limit setting).

At FB we use protection like this for most group allocations. ISTR
Google does too with a modified softlimit implementation in v1.

We do use hard limits for failsafes. I.e. "I don't care if we're not
using all available memory for this one workload, it's already 2x its
expected size, something is wrong with it anyway" -> apply reclaim
pressure and kill if necessary. So we actually do NOT want the work
conservation aspect in this case, because we don't want a likely buggy
workload to compete over memory with well-behaved jobs.

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Shakeel Butt]

Hi Johannes,

On Wed, Jan 9, 2019 at 8:45 AM Johannes Weiner <hannes@cmpxchg.org> wrote:
>
> On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote:
> > On nodes without memory overcommit, it's common a situation,
> > when memcg exceeds its limit and pages from pagecache are
> > shrinked on reclaim, while node has a lot of free memory.
> > Further access to the pages requires real device IO, while
> > IO causes time delays, worse powerusage, worse throughput
> > for other users of the device, etc.
> >
> > Cleancache is not a good solution for this problem, since
> > it implies copying of page on every cleancache_put_page()
> > and cleancache_get_page(). Also, it requires introduction
> > of internal per-cleancache_ops data structures to manage
> > cached pages and their inodes relationships, which again
> > introduces overhead.
> >
> > This patchset introduces another solution. It introduces
> > a new scheme for evicting memcg pages:
> >
> >   1)__remove_mapping() uncharges unmapped page memcg
> >     and leaves page in pagecache on memcg reclaim;
> >
> >   2)putback_lru_page() places page into root_mem_cgroup
> >     list, since its memcg is NULL. Page may be evicted
> >     on global reclaim (and this will be easily, as
> >     page is not mapped, so shrinker will shrink it
> >     with 100% probability of success);
> >
> >   3)pagecache_get_page() charges page into memcg of
> >     a task, which takes it first.
> >
> > Below is small test, which shows profit of the patchset.
> >
> > Create memcg with limit 20M (exact value does not matter much):
> >   $ mkdir /sys/fs/cgroup/memory/ct
> >   $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
> >   $ echo $$ > /sys/fs/cgroup/memory/ct/tasks
> >
> > Then twice read 1GB file:
> >   $ time cat file_1gb > /dev/null
> >
> > Before (2 iterations):
> >   1)0.01user 0.82system 0:11.16elapsed 7%CPU
> >   2)0.01user 0.91system 0:11.16elapsed 8%CPU
> >
> > After (2 iterations):
> >   1)0.01user 0.57system 0:11.31elapsed 5%CPU
> >   2)0.00user 0.28system 0:00.28elapsed 100%CPU
> >
> > With the patch set applied, we have file pages are cached
> > during the second read, so the result is 39 times faster.
> >
> > This may be useful for slow disks, NFS, nodes without
> > overcommit by memory, in case of two memcg access the same
> > files, etc.
>
> What you're implementing is work conservation: avoid causing IO work,
> unless it's physically necessary, not when the memcg limit says so.
>
> This is a great idea, but we already have that in the form of the
> memory.low setting (or softlimit in cgroup v1).
>
> Say you have a 100M system and two cgroups. Instead of setting the 20M
> limit on group A as you did, you set 80M memory.low on group B. If B
> is not using its share and there is no physical memory pressure, group
> A can consume as much memory as it wants. If B starts and consumes its
> 80M, A will get pushed back to 20M. (And when B grows beyond 80M, they
> compete fairly over the remaining 20M, just like they would if A had
> the 20M limit setting).

There is one difference between the example you give and the proposal.
In your example when B starts and consumes its 80M and pushes back A
to 20M, the direct reclaim can be very expensive and
non-deterministic. While in the proposal, the B's direct reclaim will
be very fast and deterministic (assuming no overcommit on hard limits)
as it will always first reclaim unmapped clean pages which were
charged to A.

thanks,
Shakeel

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Johannes Weiner]

On Wed, Jan 09, 2019 at 09:44:28AM -0800, Shakeel Butt wrote:
> Hi Johannes,
> 
> On Wed, Jan 9, 2019 at 8:45 AM Johannes Weiner <hannes@cmpxchg.org> wrote:
> >
> > On Wed, Jan 09, 2019 at 03:20:18PM +0300, Kirill Tkhai wrote:
> > > On nodes without memory overcommit, it's common a situation,
> > > when memcg exceeds its limit and pages from pagecache are
> > > shrinked on reclaim, while node has a lot of free memory.
> > > Further access to the pages requires real device IO, while
> > > IO causes time delays, worse powerusage, worse throughput
> > > for other users of the device, etc.
> > >
> > > Cleancache is not a good solution for this problem, since
> > > it implies copying of page on every cleancache_put_page()
> > > and cleancache_get_page(). Also, it requires introduction
> > > of internal per-cleancache_ops data structures to manage
> > > cached pages and their inodes relationships, which again
> > > introduces overhead.
> > >
> > > This patchset introduces another solution. It introduces
> > > a new scheme for evicting memcg pages:
> > >
> > >   1)__remove_mapping() uncharges unmapped page memcg
> > >     and leaves page in pagecache on memcg reclaim;
> > >
> > >   2)putback_lru_page() places page into root_mem_cgroup
> > >     list, since its memcg is NULL. Page may be evicted
> > >     on global reclaim (and this will be easily, as
> > >     page is not mapped, so shrinker will shrink it
> > >     with 100% probability of success);
> > >
> > >   3)pagecache_get_page() charges page into memcg of
> > >     a task, which takes it first.
> > >
> > > Below is small test, which shows profit of the patchset.
> > >
> > > Create memcg with limit 20M (exact value does not matter much):
> > >   $ mkdir /sys/fs/cgroup/memory/ct
> > >   $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
> > >   $ echo $$ > /sys/fs/cgroup/memory/ct/tasks
> > >
> > > Then twice read 1GB file:
> > >   $ time cat file_1gb > /dev/null
> > >
> > > Before (2 iterations):
> > >   1)0.01user 0.82system 0:11.16elapsed 7%CPU
> > >   2)0.01user 0.91system 0:11.16elapsed 8%CPU
> > >
> > > After (2 iterations):
> > >   1)0.01user 0.57system 0:11.31elapsed 5%CPU
> > >   2)0.00user 0.28system 0:00.28elapsed 100%CPU
> > >
> > > With the patch set applied, we have file pages are cached
> > > during the second read, so the result is 39 times faster.
> > >
> > > This may be useful for slow disks, NFS, nodes without
> > > overcommit by memory, in case of two memcg access the same
> > > files, etc.
> >
> > What you're implementing is work conservation: avoid causing IO work,
> > unless it's physically necessary, not when the memcg limit says so.
> >
> > This is a great idea, but we already have that in the form of the
> > memory.low setting (or softlimit in cgroup v1).
> >
> > Say you have a 100M system and two cgroups. Instead of setting the 20M
> > limit on group A as you did, you set 80M memory.low on group B. If B
> > is not using its share and there is no physical memory pressure, group
> > A can consume as much memory as it wants. If B starts and consumes its
> > 80M, A will get pushed back to 20M. (And when B grows beyond 80M, they
> > compete fairly over the remaining 20M, just like they would if A had
> > the 20M limit setting).
> 
> There is one difference between the example you give and the proposal.
> In your example when B starts and consumes its 80M and pushes back A
> to 20M, the direct reclaim can be very expensive and
> non-deterministic. While in the proposal, the B's direct reclaim will
> be very fast and deterministic (assuming no overcommit on hard limits)
> as it will always first reclaim unmapped clean pages which were
> charged to A.

That struck me more as a side-effect of the implementation having to
unmap the pages to be able to change their page->mem_cgroup.

But regardless, we cannot fundamentally change the memory isolation
semantics of the hard limit like these patches propose, so it's a moot
point. A scheme to prepare likely reclaim candidates in advance for a
low-latency workload startup would have to come in a different form.

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Shakeel Butt]

Hi Kirill,

On Wed, Jan 9, 2019 at 4:20 AM Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
>
> On nodes without memory overcommit, it's common a situation,
> when memcg exceeds its limit and pages from pagecache are
> shrinked on reclaim, while node has a lot of free memory.
> Further access to the pages requires real device IO, while
> IO causes time delays, worse powerusage, worse throughput
> for other users of the device, etc.
>
> Cleancache is not a good solution for this problem, since
> it implies copying of page on every cleancache_put_page()
> and cleancache_get_page(). Also, it requires introduction
> of internal per-cleancache_ops data structures to manage
> cached pages and their inodes relationships, which again
> introduces overhead.
>
> This patchset introduces another solution. It introduces
> a new scheme for evicting memcg pages:
>
>   1)__remove_mapping() uncharges unmapped page memcg
>     and leaves page in pagecache on memcg reclaim;
>
>   2)putback_lru_page() places page into root_mem_cgroup
>     list, since its memcg is NULL. Page may be evicted
>     on global reclaim (and this will be easily, as
>     page is not mapped, so shrinker will shrink it
>     with 100% probability of success);
>
>   3)pagecache_get_page() charges page into memcg of
>     a task, which takes it first.
>

From what I understand from the proposal, on memcg reclaim, the file
pages are uncharged but kept in the memory and if they are accessed
again (either through mmap or syscall), they will be charged again but
to the requesting memcg. Also it is assumed that the global reclaim of
such uncharged file pages is very fast and deterministic. Is that
right?

Shakeel

> Below is small test, which shows profit of the patchset.
>
> Create memcg with limit 20M (exact value does not matter much):
>   $ mkdir /sys/fs/cgroup/memory/ct
>   $ echo 20M > /sys/fs/cgroup/memory/ct/memory.limit_in_bytes
>   $ echo $$ > /sys/fs/cgroup/memory/ct/tasks
>
> Then twice read 1GB file:
>   $ time cat file_1gb > /dev/null
>
> Before (2 iterations):
>   1)0.01user 0.82system 0:11.16elapsed 7%CPU
>   2)0.01user 0.91system 0:11.16elapsed 8%CPU
>
> After (2 iterations):
>   1)0.01user 0.57system 0:11.31elapsed 5%CPU
>   2)0.00user 0.28system 0:00.28elapsed 100%CPU
>
> With the patch set applied, we have file pages are cached
> during the second read, so the result is 39 times faster.
>
> This may be useful for slow disks, NFS, nodes without
> overcommit by memory, in case of two memcg access the same
> files, etc.
>
> ---
>
> Kirill Tkhai (3):
>       mm: Uncharge and keep page in pagecache on memcg reclaim
>       mm: Recharge page memcg on first get from pagecache
>       mm: Pass FGP_NOWAIT in generic_file_buffered_read and enable ext4
>
>
>  fs/ext4/inode.c         |    1 +
>  include/linux/pagemap.h |    1 +
>  mm/filemap.c            |   38 ++++++++++++++++++++++++++++++++++++--
>  mm/vmscan.c             |   22 ++++++++++++++++++----
>  4 files changed, 56 insertions(+), 6 deletions(-)
>
> --
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Kirill Tkhai]

Hi, Shakeel,

On 09.01.2019 20:37, Shakeel Butt wrote:
> Hi Kirill,
> 
> On Wed, Jan 9, 2019 at 4:20 AM Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
>>
>> On nodes without memory overcommit, it's common a situation,
>> when memcg exceeds its limit and pages from pagecache are
>> shrinked on reclaim, while node has a lot of free memory.
>> Further access to the pages requires real device IO, while
>> IO causes time delays, worse powerusage, worse throughput
>> for other users of the device, etc.
>>
>> Cleancache is not a good solution for this problem, since
>> it implies copying of page on every cleancache_put_page()
>> and cleancache_get_page(). Also, it requires introduction
>> of internal per-cleancache_ops data structures to manage
>> cached pages and their inodes relationships, which again
>> introduces overhead.
>>
>> This patchset introduces another solution. It introduces
>> a new scheme for evicting memcg pages:
>>
>>   1)__remove_mapping() uncharges unmapped page memcg
>>     and leaves page in pagecache on memcg reclaim;
>>
>>   2)putback_lru_page() places page into root_mem_cgroup
>>     list, since its memcg is NULL. Page may be evicted
>>     on global reclaim (and this will be easily, as
>>     page is not mapped, so shrinker will shrink it
>>     with 100% probability of success);
>>
>>   3)pagecache_get_page() charges page into memcg of
>>     a task, which takes it first.
>>
> 
> From what I understand from the proposal, on memcg reclaim, the file
> pages are uncharged but kept in the memory and if they are accessed
> again (either through mmap or syscall), they will be charged again but
> to the requesting memcg. Also it is assumed that the global reclaim of
> such uncharged file pages is very fast and deterministic. Is that
> right?

Yes, this was my assumption. But Michal, Josef and Johannes pointed a diving
into reclaim in general is not fast. So, maybe we need some more creativity
here to minimize the effect of this diving..

Thanks,
Kirill

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Shakeel Butt]

On Thu, Jan 10, 2019 at 1:46 AM Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
>
> Hi, Shakeel,
>
> On 09.01.2019 20:37, Shakeel Butt wrote:
> > Hi Kirill,
> >
> > On Wed, Jan 9, 2019 at 4:20 AM Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
> >>
> >> On nodes without memory overcommit, it's common a situation,
> >> when memcg exceeds its limit and pages from pagecache are
> >> shrinked on reclaim, while node has a lot of free memory.
> >> Further access to the pages requires real device IO, while
> >> IO causes time delays, worse powerusage, worse throughput
> >> for other users of the device, etc.
> >>
> >> Cleancache is not a good solution for this problem, since
> >> it implies copying of page on every cleancache_put_page()
> >> and cleancache_get_page(). Also, it requires introduction
> >> of internal per-cleancache_ops data structures to manage
> >> cached pages and their inodes relationships, which again
> >> introduces overhead.
> >>
> >> This patchset introduces another solution. It introduces
> >> a new scheme for evicting memcg pages:
> >>
> >>   1)__remove_mapping() uncharges unmapped page memcg
> >>     and leaves page in pagecache on memcg reclaim;
> >>
> >>   2)putback_lru_page() places page into root_mem_cgroup
> >>     list, since its memcg is NULL. Page may be evicted
> >>     on global reclaim (and this will be easily, as
> >>     page is not mapped, so shrinker will shrink it
> >>     with 100% probability of success);
> >>
> >>   3)pagecache_get_page() charges page into memcg of
> >>     a task, which takes it first.
> >>
> >
> > From what I understand from the proposal, on memcg reclaim, the file
> > pages are uncharged but kept in the memory and if they are accessed
> > again (either through mmap or syscall), they will be charged again but
> > to the requesting memcg. Also it is assumed that the global reclaim of
> > such uncharged file pages is very fast and deterministic. Is that
> > right?
>
> Yes, this was my assumption. But Michal, Josef and Johannes pointed a diving
> into reclaim in general is not fast. So, maybe we need some more creativity
> here to minimize the effect of this diving..
>

I kind of disagree that this patchset is breaking the API semantics as
the charged memory of a memcg will never go over max/limit_in_bytes.
However the concern I have is the performance isolation. The
performance of a pagecache heavy job with a private mount can be
impacted by other jobs running on the system. This might be fine for
some customers but not for Google. One use-case I can tell is the
auto-tuner which adjusts the limits of the jobs based on their
performance and history. So, to make the auto-tuning deterministic we
have to disable the proposed optimization for the jobs with
auto-tuning enabled. Beside that there are internal non-auto-tuned
customers who prefer deterministic performance.

Also I am a bit skeptical that the allocation from the pool of such
(clean unmapped uncharged) file pages can be made as efficient as
fastpath of page allocator. Even if these pages are stored in a
separate list instead of root's LRU, on allocation, the pages need to
be unlinked from their mapping and has to be cleared.

BTW does this optimization have any impact on workingset mechanism?

thanks,
Shakeel

Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg pagecache pages eviction

[Kirill Tkhai]

On 10.01.2019 22:19, Shakeel Butt wrote:
> On Thu, Jan 10, 2019 at 1:46 AM Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
>>
>> Hi, Shakeel,
>>
>> On 09.01.2019 20:37, Shakeel Butt wrote:
>>> Hi Kirill,
>>>
>>> On Wed, Jan 9, 2019 at 4:20 AM Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
>>>>
>>>> On nodes without memory overcommit, it's common a situation,
>>>> when memcg exceeds its limit and pages from pagecache are
>>>> shrinked on reclaim, while node has a lot of free memory.
>>>> Further access to the pages requires real device IO, while
>>>> IO causes time delays, worse powerusage, worse throughput
>>>> for other users of the device, etc.
>>>>
>>>> Cleancache is not a good solution for this problem, since
>>>> it implies copying of page on every cleancache_put_page()
>>>> and cleancache_get_page(). Also, it requires introduction
>>>> of internal per-cleancache_ops data structures to manage
>>>> cached pages and their inodes relationships, which again
>>>> introduces overhead.
>>>>
>>>> This patchset introduces another solution. It introduces
>>>> a new scheme for evicting memcg pages:
>>>>
>>>>   1)__remove_mapping() uncharges unmapped page memcg
>>>>     and leaves page in pagecache on memcg reclaim;
>>>>
>>>>   2)putback_lru_page() places page into root_mem_cgroup
>>>>     list, since its memcg is NULL. Page may be evicted
>>>>     on global reclaim (and this will be easily, as
>>>>     page is not mapped, so shrinker will shrink it
>>>>     with 100% probability of success);
>>>>
>>>>   3)pagecache_get_page() charges page into memcg of
>>>>     a task, which takes it first.
>>>>
>>>
>>> From what I understand from the proposal, on memcg reclaim, the file
>>> pages are uncharged but kept in the memory and if they are accessed
>>> again (either through mmap or syscall), they will be charged again but
>>> to the requesting memcg. Also it is assumed that the global reclaim of
>>> such uncharged file pages is very fast and deterministic. Is that
>>> right?
>>
>> Yes, this was my assumption. But Michal, Josef and Johannes pointed a diving
>> into reclaim in general is not fast. So, maybe we need some more creativity
>> here to minimize the effect of this diving..
>>
> 
> I kind of disagree that this patchset is breaking the API semantics as
> the charged memory of a memcg will never go over max/limit_in_bytes.
> However the concern I have is the performance isolation. The
> performance of a pagecache heavy job with a private mount can be
> impacted by other jobs running on the system. This might be fine for
> some customers but not for Google. One use-case I can tell is the
> auto-tuner which adjusts the limits of the jobs based on their
> performance and history. So, to make the auto-tuning deterministic we
> have to disable the proposed optimization for the jobs with
> auto-tuning enabled. Beside that there are internal non-auto-tuned
> customers who prefer deterministic performance.
> 
> Also I am a bit skeptical that the allocation from the pool of such
> (clean unmapped uncharged) file pages can be made as efficient as
> fastpath of page allocator. Even if these pages are stored in a
> separate list instead of root's LRU, on allocation, the pages need to
> be unlinked from their mapping and has to be cleared.

I'd said, we move this unlinking from one place to another, so the unlinking
itself does not introduce more pressure on node. The difference to current
behavior is 1)we need to get through all shrinker functions once again,
2)this forces caller to do global reclaim and iterate all memcgs.
But it looks like these two things may be solved in some way.

> BTW does this optimization have any impact on workingset mechanism?

I won't say exactly, but since pages are unmapped, we should not call
any workingset handlers, so I don't know what we may break there.

Kirill