[PATCH] mm: Allocate shrinker_map on appropriate NUMA node

[PATCH] mm: Allocate shrinker_map on appropriate NUMA node

[Kirill Tkhai]

Despite shrinker_map may be touched from any cpu
(e.g., a bit there may be set by a task running
everywhere); kswapd is always bound to specific
node. So, we will allocate shrinker_map from
related NUMA node to respect its NUMA locality.
Also, this follows generic way we use for allocation
memcg's per-node data.

Two hunks node_state() patterns are borrowed from
alloc_mem_cgroup_per_node_info().

Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
---
 mm/memcontrol.c |   13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6f6dc8712e39..8ccc8ceb1b17 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -323,7 +323,7 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
 					 int size, int old_size)
 {
 	struct memcg_shrinker_map *new, *old;
-	int nid;
+	int nid, tmp;
 
 	lockdep_assert_held(&memcg_shrinker_map_mutex);
 
@@ -333,8 +333,9 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
 		/* Not yet online memcg */
 		if (!old)
 			return 0;
-
-		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
+		/* See comment in alloc_mem_cgroup_per_node_info()*/
+		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : - 1;
+		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, tmp);
 		if (!new)
 			return -ENOMEM;
 
@@ -370,7 +371,7 @@ static void memcg_free_shrinker_maps(struct mem_cgroup *memcg)
 static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
 {
 	struct memcg_shrinker_map *map;
-	int nid, size, ret = 0;
+	int nid, size, tmp, ret = 0;
 
 	if (mem_cgroup_is_root(memcg))
 		return 0;
@@ -378,7 +379,9 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
 	mutex_lock(&memcg_shrinker_map_mutex);
 	size = memcg_shrinker_map_size;
 	for_each_node(nid) {
-		map = kvzalloc(sizeof(*map) + size, GFP_KERNEL);
+		/* See comment in alloc_mem_cgroup_per_node_info()*/
+		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : - 1;
+		map = kvzalloc_node(sizeof(*map) + size, GFP_KERNEL, tmp);
 		if (!map) {
 			memcg_free_shrinker_maps(memcg);
 			ret = -ENOMEM;

Re: [PATCH] mm: Allocate shrinker_map on appropriate NUMA node

[David Hildenbrand]

On 31.01.20 13:09, Kirill Tkhai wrote:
> Despite shrinker_map may be touched from any cpu
> (e.g., a bit there may be set by a task running
> everywhere); kswapd is always bound to specific
> node. So, we will allocate shrinker_map from
> related NUMA node to respect its NUMA locality.
> Also, this follows generic way we use for allocation
> memcg's per-node data.
> 
> Two hunks node_state() patterns are borrowed from
> alloc_mem_cgroup_per_node_info().
> 
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
> ---
>  mm/memcontrol.c |   13 ++++++++-----
>  1 file changed, 8 insertions(+), 5 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 6f6dc8712e39..8ccc8ceb1b17 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -323,7 +323,7 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
>  					 int size, int old_size)
>  {
>  	struct memcg_shrinker_map *new, *old;
> -	int nid;
> +	int nid, tmp;
>  
>  	lockdep_assert_held(&memcg_shrinker_map_mutex);
>  
> @@ -333,8 +333,9 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
>  		/* Not yet online memcg */
>  		if (!old)
>  			return 0;
> -
> -		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
> +		/* See comment in alloc_mem_cgroup_per_node_info()*/
> +		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : - 1;
> +		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, tmp);
>  		if (!new)
>  			return -ENOMEM;
>  
> @@ -370,7 +371,7 @@ static void memcg_free_shrinker_maps(struct mem_cgroup *memcg)
>  static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
>  {
>  	struct memcg_shrinker_map *map;
> -	int nid, size, ret = 0;
> +	int nid, size, tmp, ret = 0;
>  
>  	if (mem_cgroup_is_root(memcg))
>  		return 0;
> @@ -378,7 +379,9 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
>  	mutex_lock(&memcg_shrinker_map_mutex);
>  	size = memcg_shrinker_map_size;
>  	for_each_node(nid) {
> -		map = kvzalloc(sizeof(*map) + size, GFP_KERNEL);
> +		/* See comment in alloc_mem_cgroup_per_node_info()*/
> +		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : - 1;
> +		map = kvzalloc_node(sizeof(*map) + size, GFP_KERNEL, tmp);
>  		if (!map) {
>  			memcg_free_shrinker_maps(memcg);
>  			ret = -ENOMEM;
> 

I think it is preferred to use NUMA_NO_NODE instead of -1.


-- 
Thanks,

David / dhildenb

[PATCH v2] mm: Allocate shrinker_map on appropriate NUMA node

[Kirill Tkhai]

mm: Allocate shrinker_map on appropriate NUMA node

From: Kirill Tkhai <ktkhai@virtuozzo.com>

Despite shrinker_map may be touched from any cpu
(e.g., a bit there may be set by a task running
everywhere); kswapd is always bound to specific
node. So, we will allocate shrinker_map from
related NUMA node to respect its NUMA locality.
Also, this follows generic way we use for allocation
memcg's per-node data.

Two hunks node_state() patterns are borrowed from
alloc_mem_cgroup_per_node_info().

v2: Use NUMA_NO_NODE instead of -1

Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
---
 mm/memcontrol.c |   13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6f6dc8712e39..20700ad25373 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -323,7 +323,7 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
 					 int size, int old_size)
 {
 	struct memcg_shrinker_map *new, *old;
-	int nid;
+	int nid, tmp;
 
 	lockdep_assert_held(&memcg_shrinker_map_mutex);
 
@@ -333,8 +333,9 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
 		/* Not yet online memcg */
 		if (!old)
 			return 0;
-
-		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
+		/* See comment in alloc_mem_cgroup_per_node_info()*/
+		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : NUMA_NO_NODE;
+		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, tmp);
 		if (!new)
 			return -ENOMEM;
 
@@ -370,7 +371,7 @@ static void memcg_free_shrinker_maps(struct mem_cgroup *memcg)
 static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
 {
 	struct memcg_shrinker_map *map;
-	int nid, size, ret = 0;
+	int nid, size, tmp, ret = 0;
 
 	if (mem_cgroup_is_root(memcg))
 		return 0;
@@ -378,7 +379,9 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
 	mutex_lock(&memcg_shrinker_map_mutex);
 	size = memcg_shrinker_map_size;
 	for_each_node(nid) {
-		map = kvzalloc(sizeof(*map) + size, GFP_KERNEL);
+		/* See comment in alloc_mem_cgroup_per_node_info()*/
+		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : NUMA_NO_NODE;
+		map = kvzalloc_node(sizeof(*map) + size, GFP_KERNEL, tmp);
 		if (!map) {
 			memcg_free_shrinker_maps(memcg);
 			ret = -ENOMEM;

Re: [PATCH v2] mm: Allocate shrinker_map on appropriate NUMA node

[Michal Hocko]

On Fri 31-01-20 18:00:51, Kirill Tkhai wrote:
[...]
> @@ -333,8 +333,9 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
>  		/* Not yet online memcg */
>  		if (!old)
>  			return 0;
> -
> -		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
> +		/* See comment in alloc_mem_cgroup_per_node_info()*/
> +		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : NUMA_NO_NODE;
> +		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, tmp);
>  		if (!new)
>  			return -ENOMEM;

I do not think this is a good pattern to copy. Why cannot you simply use
kvmalloc_node with the given node? The allocator should fallback to the
closest node if the given one doesn't have any memory.
-- 
Michal Hocko
SUSE Labs

Re: [PATCH v2] mm: Allocate shrinker_map on appropriate NUMA node

[Kirill Tkhai]

On 31.01.2020 18:47, Michal Hocko wrote:
> On Fri 31-01-20 18:00:51, Kirill Tkhai wrote:
> [...]
>> @@ -333,8 +333,9 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
>>  		/* Not yet online memcg */
>>  		if (!old)
>>  			return 0;
>> -
>> -		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
>> +		/* See comment in alloc_mem_cgroup_per_node_info()*/
>> +		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : NUMA_NO_NODE;
>> +		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, tmp);
>>  		if (!new)
>>  			return -ENOMEM;
> 
> I do not think this is a good pattern to copy. Why cannot you simply use
> kvmalloc_node with the given node? The allocator should fallback to the
> closest node if the given one doesn't have any memory.

Hm, why isn't the same scheme used in alloc_mem_cgroup_per_node_info() then?

Kirill

Re: [PATCH v2] mm: Allocate shrinker_map on appropriate NUMA node

[Michal Hocko]

On Fri 31-01-20 18:49:24, Kirill Tkhai wrote:
> On 31.01.2020 18:47, Michal Hocko wrote:
> > On Fri 31-01-20 18:00:51, Kirill Tkhai wrote:
> > [...]
> >> @@ -333,8 +333,9 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
> >>  		/* Not yet online memcg */
> >>  		if (!old)
> >>  			return 0;
> >> -
> >> -		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
> >> +		/* See comment in alloc_mem_cgroup_per_node_info()*/
> >> +		tmp = node_state(nid, N_NORMAL_MEMORY) ? nid : NUMA_NO_NODE;
> >> +		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, tmp);
> >>  		if (!new)
> >>  			return -ENOMEM;
> > 
> > I do not think this is a good pattern to copy. Why cannot you simply use
> > kvmalloc_node with the given node? The allocator should fallback to the
> > closest node if the given one doesn't have any memory.
> 
> Hm, why isn't the same scheme used in alloc_mem_cgroup_per_node_info() then?

Dunno, it's an old code. Probably worth cleaning up.
-- 
Michal Hocko
SUSE Labs

[PATCH v3] mm: Allocate shrinker_map on appropriate NUMA node

[Kirill Tkhai]

mm: Allocate shrinker_map on appropriate NUMA node

From: Kirill Tkhai <ktkhai@virtuozzo.com>

Despite shrinker_map may be touched from any cpu
(e.g., a bit there may be set by a task running
everywhere); kswapd is always bound to specific
node. So, we will allocate shrinker_map from
related NUMA node to respect its NUMA locality.
Also, this follows generic way we use for allocation
memcg's per-node data.

Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>

v3: Remove node_state() patterns.
v2: Use NUMA_NO_NODE instead of -1.
---
 mm/memcontrol.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6f6dc8712e39..c37382f5a43c 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -334,7 +334,7 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
 		if (!old)
 			return 0;
 
-		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
+		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, nid);
 		if (!new)
 			return -ENOMEM;
 
@@ -378,7 +378,7 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
 	mutex_lock(&memcg_shrinker_map_mutex);
 	size = memcg_shrinker_map_size;
 	for_each_node(nid) {
-		map = kvzalloc(sizeof(*map) + size, GFP_KERNEL);
+		map = kvzalloc_node(sizeof(*map) + size, GFP_KERNEL, nid);
 		if (!map) {
 			memcg_free_shrinker_maps(memcg);
 			ret = -ENOMEM;

Re: [PATCH v3] mm: Allocate shrinker_map on appropriate NUMA node

[Michal Hocko]

On Fri 31-01-20 19:08:49, Kirill Tkhai wrote:
> mm: Allocate shrinker_map on appropriate NUMA node
> 
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> 
> Despite shrinker_map may be touched from any cpu
> (e.g., a bit there may be set by a task running
> everywhere); kswapd is always bound to specific
> node. So, we will allocate shrinker_map from
> related NUMA node to respect its NUMA locality.
> Also, this follows generic way we use for allocation
> memcg's per-node data.

I would just drop the last sentence.
 
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>

Acked-by: Michal Hocko <mhocko@suse.com>
> 
> v3: Remove node_state() patterns.
> v2: Use NUMA_NO_NODE instead of -1.
> ---
>  mm/memcontrol.c |    4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 6f6dc8712e39..c37382f5a43c 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -334,7 +334,7 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
>  		if (!old)
>  			return 0;
>  
> -		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
> +		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, nid);
>  		if (!new)
>  			return -ENOMEM;
>  
> @@ -378,7 +378,7 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
>  	mutex_lock(&memcg_shrinker_map_mutex);
>  	size = memcg_shrinker_map_size;
>  	for_each_node(nid) {
> -		map = kvzalloc(sizeof(*map) + size, GFP_KERNEL);
> +		map = kvzalloc_node(sizeof(*map) + size, GFP_KERNEL, nid);
>  		if (!map) {
>  			memcg_free_shrinker_maps(memcg);
>  			ret = -ENOMEM;

-- 
Michal Hocko
SUSE Labs

Re: [PATCH v3] mm: Allocate shrinker_map on appropriate NUMA node

[David Hildenbrand]

On 31.01.20 17:08, Kirill Tkhai wrote:
> mm: Allocate shrinker_map on appropriate NUMA node
> 
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> 
> Despite shrinker_map may be touched from any cpu
> (e.g., a bit there may be set by a task running
> everywhere); kswapd is always bound to specific
> node. So, we will allocate shrinker_map from
> related NUMA node to respect its NUMA locality.
> Also, this follows generic way we use for allocation
> memcg's per-node data.

You can go up to 72 characters in your patch description :)

Reviewed-by: David Hildenbrand <david@redhat.com>

> 
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
> 
> v3: Remove node_state() patterns.
> v2: Use NUMA_NO_NODE instead of -1.
> ---
>  mm/memcontrol.c |    4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 6f6dc8712e39..c37382f5a43c 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -334,7 +334,7 @@ static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg,
>  		if (!old)
>  			return 0;
>  
> -		new = kvmalloc(sizeof(*new) + size, GFP_KERNEL);
> +		new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, nid);
>  		if (!new)
>  			return -ENOMEM;
>  
> @@ -378,7 +378,7 @@ static int memcg_alloc_shrinker_maps(struct mem_cgroup *memcg)
>  	mutex_lock(&memcg_shrinker_map_mutex);
>  	size = memcg_shrinker_map_size;
>  	for_each_node(nid) {
> -		map = kvzalloc(sizeof(*map) + size, GFP_KERNEL);
> +		map = kvzalloc_node(sizeof(*map) + size, GFP_KERNEL, nid);
>  		if (!map) {
>  			memcg_free_shrinker_maps(memcg);
>  			ret = -ENOMEM;
> 


-- 
Thanks,

David / dhildenb

Re: [PATCH v3] mm: Allocate shrinker_map on appropriate NUMA node

[Shakeel Butt]

On Fri, Jan 31, 2020 at 8:09 AM Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
>
> mm: Allocate shrinker_map on appropriate NUMA node
>
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>
> Despite shrinker_map may be touched from any cpu
> (e.g., a bit there may be set by a task running
> everywhere); kswapd is always bound to specific
> node. So, we will allocate shrinker_map from
> related NUMA node to respect its NUMA locality.
> Also, this follows generic way we use for allocation
> memcg's per-node data.
>
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>

Reviewed-by: Shakeel Butt <shakeelb@google.com>

Re: [PATCH v3] mm: Allocate shrinker_map on appropriate NUMA node

[Roman Gushchin]

On Fri, Jan 31, 2020 at 07:08:49PM +0300, Kirill Tkhai wrote:
> mm: Allocate shrinker_map on appropriate NUMA node
> 
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> 
> Despite shrinker_map may be touched from any cpu
> (e.g., a bit there may be set by a task running
> everywhere); kswapd is always bound to specific
> node. So, we will allocate shrinker_map from
> related NUMA node to respect its NUMA locality.
> Also, this follows generic way we use for allocation
> memcg's per-node data.
> 
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>

Reviewed-by: Roman Gushchin <guro@fb.com>

Thanks!

Re: [PATCH v3] mm: Allocate shrinker_map on appropriate NUMA node

[Kirill Tkhai]

On 31.01.2020 19:18, Michal Hocko wrote:
> On Fri 31-01-20 19:08:49, Kirill Tkhai wrote:
>> mm: Allocate shrinker_map on appropriate NUMA node
>>
>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>>
>> Despite shrinker_map may be touched from any cpu
>> (e.g., a bit there may be set by a task running
>> everywhere); kswapd is always bound to specific
>> node. So, we will allocate shrinker_map from
>> related NUMA node to respect its NUMA locality.
>> Also, this follows generic way we use for allocation
>> memcg's per-node data.
> 
> I would just drop the last sentence.

I mean we allocate memcg->nodeinfo from specific node,
so shrinker_map also should follow this rule. Though,
I have no objections to remove this on patch merge.

>> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
> 
> Acked-by: Michal Hocko <mhocko@suse.com>

Thanks,
Kirill.