[PATCH] blk-mq: remove the calling of local_memory_node()

[PATCH] blk-mq: remove the calling of local_memory_node()

[Xianting Tian]

We don't need to check whether the node is memoryless numa node before
calling allocator interface. SLUB(and SLAB,SLOB) relies on the page
allocator to pick a node. Page allocator should deal with memoryless
nodes just fine. It has zonelists constructed for each possible nodes.
And it will automatically fall back into a node which is closest to the
requested node. As long as __GFP_THISNODE is not enforced of course.

The code comments of kmem_cache_alloc_node() of SLAB also showed this:
 * Fallback to other node is possible if __GFP_THISNODE is not set.

blk-mq code doesn't set __GFP_THISNODE, so we can remove the calling
of local_memory_node().

Fixes: bffed457160ab ("blk-mq: Avoid memoryless numa node encoded in hctx numa_node")

Signed-off-by: Xianting Tian <tian.xianting@h3c.com>
---
 block/blk-mq-cpumap.c | 2 +-
 block/blk-mq.c        | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c
index 0157f2b34..3db84d319 100644
--- a/block/blk-mq-cpumap.c
+++ b/block/blk-mq-cpumap.c
@@ -89,7 +89,7 @@ int blk_mq_hw_queue_to_node(struct blk_mq_queue_map *qmap, unsigned int index)
 
 	for_each_possible_cpu(i) {
 		if (index == qmap->mq_map[i])
-			return local_memory_node(cpu_to_node(i));
+			return cpu_to_node(i);
 	}
 
 	return NUMA_NO_NODE;
diff --git a/block/blk-mq.c b/block/blk-mq.c
index cdced4aca..48f8366b2 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2737,7 +2737,7 @@ static void blk_mq_init_cpu_queues(struct request_queue *q,
 		for (j = 0; j < set->nr_maps; j++) {
 			hctx = blk_mq_map_queue_type(q, j, i);
 			if (nr_hw_queues > 1 && hctx->numa_node == NUMA_NO_NODE)
-				hctx->numa_node = local_memory_node(cpu_to_node(i));
+				hctx->numa_node = cpu_to_node(i);
 		}
 	}
 }
-- 
2.17.1

Re: [PATCH] blk-mq: remove the calling of local_memory_node()

[Michal Hocko]

On Mon 19-10-20 16:20:47, Xianting Tian wrote:
> We don't need to check whether the node is memoryless numa node before
> calling allocator interface. SLUB(and SLAB,SLOB) relies on the page
> allocator to pick a node. Page allocator should deal with memoryless
> nodes just fine. It has zonelists constructed for each possible nodes.
> And it will automatically fall back into a node which is closest to the
> requested node. As long as __GFP_THISNODE is not enforced of course.
> 
> The code comments of kmem_cache_alloc_node() of SLAB also showed this:
>  * Fallback to other node is possible if __GFP_THISNODE is not set.
> 
> blk-mq code doesn't set __GFP_THISNODE, so we can remove the calling
> of local_memory_node().

yes, this is indeed the case. I cannot really judge the blg-mq code but
it seems to be unnecessary. Maybe there are some subtle details not
explained by bffed457160ab though.

> Fixes: bffed457160ab ("blk-mq: Avoid memoryless numa node encoded in hctx numa_node")

But the existing code is not broken. It just overdoes what needs to be
done. So effectively bffed457160ab was not needed. I do not think that
Fixes is really necessary.

> Signed-off-by: Xianting Tian <tian.xianting@h3c.com>
> ---
>  block/blk-mq-cpumap.c | 2 +-
>  block/blk-mq.c        | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c
> index 0157f2b34..3db84d319 100644
> --- a/block/blk-mq-cpumap.c
> +++ b/block/blk-mq-cpumap.c
> @@ -89,7 +89,7 @@ int blk_mq_hw_queue_to_node(struct blk_mq_queue_map *qmap, unsigned int index)
>  
>  	for_each_possible_cpu(i) {
>  		if (index == qmap->mq_map[i])
> -			return local_memory_node(cpu_to_node(i));
> +			return cpu_to_node(i);
>  	}
>  
>  	return NUMA_NO_NODE;
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index cdced4aca..48f8366b2 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -2737,7 +2737,7 @@ static void blk_mq_init_cpu_queues(struct request_queue *q,
>  		for (j = 0; j < set->nr_maps; j++) {
>  			hctx = blk_mq_map_queue_type(q, j, i);
>  			if (nr_hw_queues > 1 && hctx->numa_node == NUMA_NO_NODE)
> -				hctx->numa_node = local_memory_node(cpu_to_node(i));
> +				hctx->numa_node = cpu_to_node(i);
>  		}
>  	}
>  }
> -- 
> 2.17.1

-- 
Michal Hocko
SUSE Labs

RE: [PATCH] blk-mq: remove the calling of local_memory_node()

[Tianxianting]

Thanks Michal,
Hi, raghavendra, Jens
Could you help comment this issue? Thanks in advance.

-----Original Message-----
From: Michal Hocko [mailto:mhocko@suse.com] 
Sent: Monday, October 19, 2020 7:40 PM
To: tianxianting (RD) <tian.xianting@h3c.com>
Cc: axboe@kernel.dk; raghavendra.kt@linux.vnet.ibm.com; linux-block@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: [PATCH] blk-mq: remove the calling of local_memory_node()

On Mon 19-10-20 16:20:47, Xianting Tian wrote:
> We don't need to check whether the node is memoryless numa node before 
> calling allocator interface. SLUB(and SLAB,SLOB) relies on the page 
> allocator to pick a node. Page allocator should deal with memoryless 
> nodes just fine. It has zonelists constructed for each possible nodes.
> And it will automatically fall back into a node which is closest to 
> the requested node. As long as __GFP_THISNODE is not enforced of course.
> 
> The code comments of kmem_cache_alloc_node() of SLAB also showed this:
>  * Fallback to other node is possible if __GFP_THISNODE is not set.
> 
> blk-mq code doesn't set __GFP_THISNODE, so we can remove the calling 
> of local_memory_node().

yes, this is indeed the case. I cannot really judge the blg-mq code but it seems to be unnecessary. Maybe there are some subtle details not explained by bffed457160ab though.

> Fixes: bffed457160ab ("blk-mq: Avoid memoryless numa node encoded in 
> hctx numa_node")

But the existing code is not broken. It just overdoes what needs to be done. So effectively bffed457160ab was not needed. I do not think that Fixes is really necessary.

> Signed-off-by: Xianting Tian <tian.xianting@h3c.com>
> ---
>  block/blk-mq-cpumap.c | 2 +-
>  block/blk-mq.c        | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c index 
> 0157f2b34..3db84d319 100644
> --- a/block/blk-mq-cpumap.c
> +++ b/block/blk-mq-cpumap.c
> @@ -89,7 +89,7 @@ int blk_mq_hw_queue_to_node(struct blk_mq_queue_map 
> *qmap, unsigned int index)
>  
>  	for_each_possible_cpu(i) {
>  		if (index == qmap->mq_map[i])
> -			return local_memory_node(cpu_to_node(i));
> +			return cpu_to_node(i);
>  	}
>  
>  	return NUMA_NO_NODE;
> diff --git a/block/blk-mq.c b/block/blk-mq.c index 
> cdced4aca..48f8366b2 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -2737,7 +2737,7 @@ static void blk_mq_init_cpu_queues(struct request_queue *q,
>  		for (j = 0; j < set->nr_maps; j++) {
>  			hctx = blk_mq_map_queue_type(q, j, i);
>  			if (nr_hw_queues > 1 && hctx->numa_node == NUMA_NO_NODE)
> -				hctx->numa_node = local_memory_node(cpu_to_node(i));
> +				hctx->numa_node = cpu_to_node(i);
>  		}
>  	}
>  }
> --
> 2.17.1

--
Michal Hocko
SUSE Labs

Re: [PATCH] blk-mq: remove the calling of local_memory_node()

[Jens Axboe]

On 10/19/20 2:20 AM, Xianting Tian wrote:
> We don't need to check whether the node is memoryless numa node before
> calling allocator interface. SLUB(and SLAB,SLOB) relies on the page
> allocator to pick a node. Page allocator should deal with memoryless
> nodes just fine. It has zonelists constructed for each possible nodes.
> And it will automatically fall back into a node which is closest to the
> requested node. As long as __GFP_THISNODE is not enforced of course.
> 
> The code comments of kmem_cache_alloc_node() of SLAB also showed this:
>  * Fallback to other node is possible if __GFP_THISNODE is not set.
> 
> blk-mq code doesn't set __GFP_THISNODE, so we can remove the calling
> of local_memory_node().

Applied, thanks.

-- 
Jens Axboe