Linux-RDMA Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] IB/hfi1: Avoid allocing memory on memoryless numa node
@ 2020-10-10  8:57 Xianting Tian
  2020-10-12 12:36 ` Dennis Dalessandro
  0 siblings, 1 reply; 5+ messages in thread
From: Xianting Tian @ 2020-10-10  8:57 UTC (permalink / raw)
  To: mike.marciniszyn, dennis.dalessandro, dledford, jgg
  Cc: linux-rdma, linux-kernel, Xianting Tian

In architecture like powerpc, we can have cpus without any local memory
attached to it. In such cases the node does not have real memory.

Use local_memory_node(), which is guaranteed to have memory.
local_memory_node is a noop in other architectures that does not support
memoryless nodes.

Signed-off-by: Xianting Tian <tian.xianting@h3c.com>
---
 drivers/infiniband/hw/hfi1/file_ops.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c
index 8ca51e43c..79fa22cc7 100644
--- a/drivers/infiniband/hw/hfi1/file_ops.c
+++ b/drivers/infiniband/hw/hfi1/file_ops.c
@@ -965,7 +965,7 @@ static int allocate_ctxt(struct hfi1_filedata *fd, struct hfi1_devdata *dd,
 	 */
 	fd->rec_cpu_num = hfi1_get_proc_affinity(dd->node);
 	if (fd->rec_cpu_num != -1)
-		numa = cpu_to_node(fd->rec_cpu_num);
+		numa = local_memory_node(cpu_to_node(fd->rec_cpu_num));
 	else
 		numa = numa_node_id();
 	ret = hfi1_create_ctxtdata(dd->pport, numa, &uctxt);
-- 
2.17.1


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH] IB/hfi1: Avoid allocing memory on memoryless numa node
  2020-10-10  8:57 [PATCH] IB/hfi1: Avoid allocing memory on memoryless numa node Xianting Tian
@ 2020-10-12 12:36 ` Dennis Dalessandro
  2020-10-12 13:24   ` Tianxianting
  2020-10-16 14:11   ` Jason Gunthorpe
  0 siblings, 2 replies; 5+ messages in thread
From: Dennis Dalessandro @ 2020-10-12 12:36 UTC (permalink / raw)
  To: Xianting Tian, mike.marciniszyn, dennis.dalessandro, dledford, jgg
  Cc: linux-rdma, linux-kernel

On 10/10/2020 4:57 AM, Xianting Tian wrote:
> In architecture like powerpc, we can have cpus without any local memory
> attached to it. In such cases the node does not have real memory.
> 
> Use local_memory_node(), which is guaranteed to have memory.
> local_memory_node is a noop in other architectures that does not support
> memoryless nodes.
> 
> Signed-off-by: Xianting Tian <tian.xianting@h3c.com>
> ---
>   drivers/infiniband/hw/hfi1/file_ops.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c
> index 8ca51e43c..79fa22cc7 100644
> --- a/drivers/infiniband/hw/hfi1/file_ops.c
> +++ b/drivers/infiniband/hw/hfi1/file_ops.c
> @@ -965,7 +965,7 @@ static int allocate_ctxt(struct hfi1_filedata *fd, struct hfi1_devdata *dd,
>   	 */
>   	fd->rec_cpu_num = hfi1_get_proc_affinity(dd->node);
>   	if (fd->rec_cpu_num != -1)
> -		numa = cpu_to_node(fd->rec_cpu_num);
> +		numa = local_memory_node(cpu_to_node(fd->rec_cpu_num));
>   	else
>   		numa = numa_node_id();
>   	ret = hfi1_create_ctxtdata(dd->pport, numa, &uctxt);
> 

The hfi1 driver depends on X86_64. I'm not sure what this patch buys, 
can you expand a bit?

-Denny

^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: [PATCH] IB/hfi1: Avoid allocing memory on memoryless numa node
  2020-10-12 12:36 ` Dennis Dalessandro
@ 2020-10-12 13:24   ` Tianxianting
  2020-10-16 14:11   ` Jason Gunthorpe
  1 sibling, 0 replies; 5+ messages in thread
From: Tianxianting @ 2020-10-12 13:24 UTC (permalink / raw)
  To: Dennis Dalessandro, mike.marciniszyn, dennis.dalessandro, dledford, jgg
  Cc: linux-rdma, linux-kernel

Hi Dennis
Thanks for the comments
If it depends on x86_64, I think this issue doesn't exist.
Sorry to disturb you.

-----Original Message-----
From: Dennis Dalessandro [mailto:dennis.dalessandro@cornelisnetworks.com] 
Sent: Monday, October 12, 2020 8:37 PM
To: tianxianting (RD) <tian.xianting@h3c.com>; mike.marciniszyn@intel.com; dennis.dalessandro@intel.com; dledford@redhat.com; jgg@ziepe.ca
Cc: linux-rdma@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: [PATCH] IB/hfi1: Avoid allocing memory on memoryless numa node

On 10/10/2020 4:57 AM, Xianting Tian wrote:
> In architecture like powerpc, we can have cpus without any local 
> memory attached to it. In such cases the node does not have real memory.
> 
> Use local_memory_node(), which is guaranteed to have memory.
> local_memory_node is a noop in other architectures that does not 
> support memoryless nodes.
> 
> Signed-off-by: Xianting Tian <tian.xianting@h3c.com>
> ---
>   drivers/infiniband/hw/hfi1/file_ops.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/hw/hfi1/file_ops.c 
> b/drivers/infiniband/hw/hfi1/file_ops.c
> index 8ca51e43c..79fa22cc7 100644
> --- a/drivers/infiniband/hw/hfi1/file_ops.c
> +++ b/drivers/infiniband/hw/hfi1/file_ops.c
> @@ -965,7 +965,7 @@ static int allocate_ctxt(struct hfi1_filedata *fd, struct hfi1_devdata *dd,
>   	 */
>   	fd->rec_cpu_num = hfi1_get_proc_affinity(dd->node);
>   	if (fd->rec_cpu_num != -1)
> -		numa = cpu_to_node(fd->rec_cpu_num);
> +		numa = local_memory_node(cpu_to_node(fd->rec_cpu_num));
>   	else
>   		numa = numa_node_id();
>   	ret = hfi1_create_ctxtdata(dd->pport, numa, &uctxt);
> 

The hfi1 driver depends on X86_64. I'm not sure what this patch buys, can you expand a bit?

-Denny

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH] IB/hfi1: Avoid allocing memory on memoryless numa node
  2020-10-12 12:36 ` Dennis Dalessandro
  2020-10-12 13:24   ` Tianxianting
@ 2020-10-16 14:11   ` Jason Gunthorpe
  2020-10-16 15:14     ` Dennis Dalessandro
  1 sibling, 1 reply; 5+ messages in thread
From: Jason Gunthorpe @ 2020-10-16 14:11 UTC (permalink / raw)
  To: Dennis Dalessandro
  Cc: Xianting Tian, mike.marciniszyn, dennis.dalessandro, dledford,
	linux-rdma, linux-kernel

On Mon, Oct 12, 2020 at 08:36:57AM -0400, Dennis Dalessandro wrote:
> On 10/10/2020 4:57 AM, Xianting Tian wrote:
> > In architecture like powerpc, we can have cpus without any local memory
> > attached to it. In such cases the node does not have real memory.
> > 
> > Use local_memory_node(), which is guaranteed to have memory.
> > local_memory_node is a noop in other architectures that does not support
> > memoryless nodes.
> > 
> > Signed-off-by: Xianting Tian <tian.xianting@h3c.com>
> >   drivers/infiniband/hw/hfi1/file_ops.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c
> > index 8ca51e43c..79fa22cc7 100644
> > +++ b/drivers/infiniband/hw/hfi1/file_ops.c
> > @@ -965,7 +965,7 @@ static int allocate_ctxt(struct hfi1_filedata *fd, struct hfi1_devdata *dd,
> >   	 */
> >   	fd->rec_cpu_num = hfi1_get_proc_affinity(dd->node);
> >   	if (fd->rec_cpu_num != -1)
> > -		numa = cpu_to_node(fd->rec_cpu_num);
> > +		numa = local_memory_node(cpu_to_node(fd->rec_cpu_num));
> >   	else
> >   		numa = numa_node_id();
> >   	ret = hfi1_create_ctxtdata(dd->pport, numa, &uctxt);
> > 
> 
> The hfi1 driver depends on X86_64. I'm not sure what this patch buys, can
> you expand a bit?

Yikes, that is strongly discouraged.

Jason

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH] IB/hfi1: Avoid allocing memory on memoryless numa node
  2020-10-16 14:11   ` Jason Gunthorpe
@ 2020-10-16 15:14     ` Dennis Dalessandro
  0 siblings, 0 replies; 5+ messages in thread
From: Dennis Dalessandro @ 2020-10-16 15:14 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Xianting Tian, mike.marciniszyn, dledford, linux-rdma, linux-kernel

On 10/16/2020 10:11 AM, Jason Gunthorpe wrote:
> On Mon, Oct 12, 2020 at 08:36:57AM -0400, Dennis Dalessandro wrote:
>> On 10/10/2020 4:57 AM, Xianting Tian wrote:
>>> In architecture like powerpc, we can have cpus without any local memory
>>> attached to it. In such cases the node does not have real memory.
>>>
>>> Use local_memory_node(), which is guaranteed to have memory.
>>> local_memory_node is a noop in other architectures that does not support
>>> memoryless nodes.
>>>
>>> Signed-off-by: Xianting Tian <tian.xianting@h3c.com>
>>>    drivers/infiniband/hw/hfi1/file_ops.c | 2 +-
>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c
>>> index 8ca51e43c..79fa22cc7 100644
>>> +++ b/drivers/infiniband/hw/hfi1/file_ops.c
>>> @@ -965,7 +965,7 @@ static int allocate_ctxt(struct hfi1_filedata *fd, struct hfi1_devdata *dd,
>>>    	 */
>>>    	fd->rec_cpu_num = hfi1_get_proc_affinity(dd->node);
>>>    	if (fd->rec_cpu_num != -1)
>>> -		numa = cpu_to_node(fd->rec_cpu_num);
>>> +		numa = local_memory_node(cpu_to_node(fd->rec_cpu_num));
>>>    	else
>>>    		numa = numa_node_id();
>>>    	ret = hfi1_create_ctxtdata(dd->pport, numa, &uctxt);
>>>
>>
>> The hfi1 driver depends on X86_64. I'm not sure what this patch buys, can
>> you expand a bit?
> 
> Yikes, that is strongly discouraged.

Hmm. This was never raised as an issue before. Regardless I can't recall 
why we did this in the first place. I'll do some digging, try to jog my 
memory.

-Denny

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-10  8:57 [PATCH] IB/hfi1: Avoid allocing memory on memoryless numa node Xianting Tian
2020-10-12 12:36 ` Dennis Dalessandro
2020-10-12 13:24   ` Tianxianting
2020-10-16 14:11   ` Jason Gunthorpe
2020-10-16 15:14     ` Dennis Dalessandro

Linux-RDMA Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rdma/0 linux-rdma/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rdma linux-rdma/ https://lore.kernel.org/linux-rdma \
		linux-rdma@vger.kernel.org
	public-inbox-index linux-rdma

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rdma


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git