* [PATCH] mm thp: shrink deferred split THPs harder
@ 2019-10-01 23:56 Yang Shi
2019-10-02 8:40 ` Michal Hocko
0 siblings, 1 reply; 3+ messages in thread
From: Yang Shi @ 2019-10-01 23:56 UTC (permalink / raw)
To: kirill.shutemov, ktkhai, hannes, mhocko, hughd, shakeelb, rientjes, akpm
Cc: yang.shi, linux-mm, linux-kernel
The deferred split THPs may get accumulated with some workloads, they
would get shrunk when memory pressure is hit. Now we use DEFAULT_SEEKS
to determine how many objects would get scanned then split if possible,
but actually they are not like other system cache objects, i.e. inode
cache which would incur extra I/O if over reclaimed, the unmapped pages
will not be accessed anymore, so we could shrink them more aggressively.
We could shrink THPs more pro-actively even though memory pressure is not
hit, however, IMHO waiting for memory pressure is still a good
compromise and trade-off. And, we do have simpler ways to shrink these
objects harder until we have to take other means do pro-actively drain.
Change shrinker->seeks to 0 to shrink deferred split THPs harder.
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: David Rientjes <rientjes@google.com>
Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
---
mm/huge_memory.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 3b78910..1d6b1f1 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2955,7 +2955,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
static struct shrinker deferred_split_shrinker = {
.count_objects = deferred_split_count,
.scan_objects = deferred_split_scan,
- .seeks = DEFAULT_SEEKS,
+ .seeks = 0,
.flags = SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE |
SHRINKER_NONSLAB,
};
--
1.8.3.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] mm thp: shrink deferred split THPs harder
2019-10-01 23:56 [PATCH] mm thp: shrink deferred split THPs harder Yang Shi
@ 2019-10-02 8:40 ` Michal Hocko
2019-10-02 17:26 ` Yang Shi
0 siblings, 1 reply; 3+ messages in thread
From: Michal Hocko @ 2019-10-02 8:40 UTC (permalink / raw)
To: Yang Shi
Cc: kirill.shutemov, ktkhai, hannes, hughd, shakeelb, rientjes, akpm,
linux-mm, linux-kernel
On Wed 02-10-19 07:56:50, Yang Shi wrote:
> The deferred split THPs may get accumulated with some workloads, they
> would get shrunk when memory pressure is hit. Now we use DEFAULT_SEEKS
> to determine how many objects would get scanned then split if possible,
> but actually they are not like other system cache objects, i.e. inode
> cache which would incur extra I/O if over reclaimed, the unmapped pages
> will not be accessed anymore, so we could shrink them more aggressively.
>
> We could shrink THPs more pro-actively even though memory pressure is not
> hit, however, IMHO waiting for memory pressure is still a good
> compromise and trade-off. And, we do have simpler ways to shrink these
> objects harder until we have to take other means do pro-actively drain.
>
> Change shrinker->seeks to 0 to shrink deferred split THPs harder.
Do you have any numbers on the effect of this patch.
Btw. the whole thing is getting more and more complex and I still
believe the approach is just wrong. We are tunning for something that
doesn't really belong to the memory reclaim in the first place IMHO.
> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: Hugh Dickins <hughd@google.com>
> Cc: Shakeel Butt <shakeelb@google.com>
> Cc: David Rientjes <rientjes@google.com>
> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
> ---
> mm/huge_memory.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 3b78910..1d6b1f1 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2955,7 +2955,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
> static struct shrinker deferred_split_shrinker = {
> .count_objects = deferred_split_count,
> .scan_objects = deferred_split_scan,
> - .seeks = DEFAULT_SEEKS,
> + .seeks = 0,
> .flags = SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE |
> SHRINKER_NONSLAB,
> };
> --
> 1.8.3.1
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] mm thp: shrink deferred split THPs harder
2019-10-02 8:40 ` Michal Hocko
@ 2019-10-02 17:26 ` Yang Shi
0 siblings, 0 replies; 3+ messages in thread
From: Yang Shi @ 2019-10-02 17:26 UTC (permalink / raw)
To: Michal Hocko
Cc: kirill.shutemov, ktkhai, hannes, hughd, shakeelb, rientjes, akpm,
linux-mm, linux-kernel
On 10/2/19 1:40 AM, Michal Hocko wrote:
> On Wed 02-10-19 07:56:50, Yang Shi wrote:
>> The deferred split THPs may get accumulated with some workloads, they
>> would get shrunk when memory pressure is hit. Now we use DEFAULT_SEEKS
>> to determine how many objects would get scanned then split if possible,
>> but actually they are not like other system cache objects, i.e. inode
>> cache which would incur extra I/O if over reclaimed, the unmapped pages
>> will not be accessed anymore, so we could shrink them more aggressively.
>>
>> We could shrink THPs more pro-actively even though memory pressure is not
>> hit, however, IMHO waiting for memory pressure is still a good
>> compromise and trade-off. And, we do have simpler ways to shrink these
>> objects harder until we have to take other means do pro-actively drain.
>>
>> Change shrinker->seeks to 0 to shrink deferred split THPs harder.
> Do you have any numbers on the effect of this patch.
Yes, this patch would make THPs get split earlier.
For example, I have a test case which generates around 4G deferred split
THPs (2K huge pages). With the default seeks, THPs would start to get
split when priority reaches 6 since nr_to_scan depends on priority and
shrinker->seeks. With this patch it would start to get split at the very
beginning (priority 12).
IMHO, somehow this would achieve the similar effect with pro-actively
draining.
>
> Btw. the whole thing is getting more and more complex and I still
> believe the approach is just wrong. We are tunning for something that
> doesn't really belong to the memory reclaim in the first place IMHO.
Maybe, but it is not clear to me that other approaches would be
universally better than the current one unless we could split the page
right away.
>
>> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>> Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
>> Cc: Johannes Weiner <hannes@cmpxchg.org>
>> Cc: Michal Hocko <mhocko@suse.com>
>> Cc: Hugh Dickins <hughd@google.com>
>> Cc: Shakeel Butt <shakeelb@google.com>
>> Cc: David Rientjes <rientjes@google.com>
>> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
>> ---
>> mm/huge_memory.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index 3b78910..1d6b1f1 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -2955,7 +2955,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
>> static struct shrinker deferred_split_shrinker = {
>> .count_objects = deferred_split_count,
>> .scan_objects = deferred_split_scan,
>> - .seeks = DEFAULT_SEEKS,
>> + .seeks = 0,
>> .flags = SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE |
>> SHRINKER_NONSLAB,
>> };
>> --
>> 1.8.3.1
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-10-02 17:26 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-01 23:56 [PATCH] mm thp: shrink deferred split THPs harder Yang Shi
2019-10-02 8:40 ` Michal Hocko
2019-10-02 17:26 ` Yang Shi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).