[PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Hoan Tran]

Some NUMA nodes have memory ranges that span other nodes.
Even though a pfn is valid and between a node's start and end pfns,
it may not reside on that node.

This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
this type of NUMA layout.

Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
---
 arch/arm64/Kconfig | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index e688dfa..939d28f 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -959,6 +959,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
 config HOLES_IN_ZONE
 	def_bool y
 
+# Some NUMA nodes have memory ranges that span other nodes.
+# Even though a pfn is valid and between a node's start and end pfns,
+# it may not reside on that node.
+config NODES_SPAN_OTHER_NODES
+	def_bool y
+	depends on ACPI_NUMA
+
 source "kernel/Kconfig.hz"
 
 config ARCH_SUPPORTS_DEBUG_PAGEALLOC
-- 
1.8.3.1

Re: [PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Will Deacon]

On Mon, Feb 03, 2020 at 11:55:14AM -0800, Hoan Tran wrote:
> Some NUMA nodes have memory ranges that span other nodes.
> Even though a pfn is valid and between a node's start and end pfns,
> it may not reside on that node.
> 
> This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
> this type of NUMA layout.
> 
> Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
> ---
>  arch/arm64/Kconfig | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index e688dfa..939d28f 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -959,6 +959,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
>  config HOLES_IN_ZONE
>  	def_bool y
>  
> +# Some NUMA nodes have memory ranges that span other nodes.
> +# Even though a pfn is valid and between a node's start and end pfns,
> +# it may not reside on that node.
> +config NODES_SPAN_OTHER_NODES
> +	def_bool y
> +	depends on ACPI_NUMA
> +

I thought we agreed to do this in the core code?

https://lore.kernel.org/lkml/1562887528-5896-1-git-send-email-Hoan@os.amperecomputing.com

Will

Re: [PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Hoan Tran]

Hi Will,

On 2/6/20 2:23 AM, Will Deacon wrote:
> On Mon, Feb 03, 2020 at 11:55:14AM -0800, Hoan Tran wrote:
>> Some NUMA nodes have memory ranges that span other nodes.
>> Even though a pfn is valid and between a node's start and end pfns,
>> it may not reside on that node.
>>
>> This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
>> this type of NUMA layout.
>>
>> Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
>> ---
>>   arch/arm64/Kconfig | 7 +++++++
>>   1 file changed, 7 insertions(+)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index e688dfa..939d28f 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -959,6 +959,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
>>   config HOLES_IN_ZONE
>>   	def_bool y
>>   
>> +# Some NUMA nodes have memory ranges that span other nodes.
>> +# Even though a pfn is valid and between a node's start and end pfns,
>> +# it may not reside on that node.
>> +config NODES_SPAN_OTHER_NODES
>> +	def_bool y
>> +	depends on ACPI_NUMA
>> +
> 
> I thought we agreed to do this in the core code?
> 
> https://lore.kernel.org/lkml/1562887528-5896-1-git-send-email-Hoan@os.amperecomputing.com

Yes, but it looks like Thomas didn't agree to apply this patch into x86.

https://lore.kernel.org/lkml/alpine.DEB.2.21.1907152042110.1767@nanos.tec.linutronix.de/

Regards
Hoan

> 
> Will
> 

Re: [PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Hoan Tran]

Hi Will,

Do you have any comments?

Thanks
Hoan

On 2/6/20 12:01 PM, Hoan Tran wrote:
> Hi Will,
> 
> On 2/6/20 2:23 AM, Will Deacon wrote:
>> On Mon, Feb 03, 2020 at 11:55:14AM -0800, Hoan Tran wrote:
>>> Some NUMA nodes have memory ranges that span other nodes.
>>> Even though a pfn is valid and between a node's start and end pfns,
>>> it may not reside on that node.
>>>
>>> This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
>>> this type of NUMA layout.
>>>
>>> Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
>>> ---
>>>   arch/arm64/Kconfig | 7 +++++++
>>>   1 file changed, 7 insertions(+)
>>>
>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>>> index e688dfa..939d28f 100644
>>> --- a/arch/arm64/Kconfig
>>> +++ b/arch/arm64/Kconfig
>>> @@ -959,6 +959,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
>>>   config HOLES_IN_ZONE
>>>       def_bool y
>>> +# Some NUMA nodes have memory ranges that span other nodes.
>>> +# Even though a pfn is valid and between a node's start and end pfns,
>>> +# it may not reside on that node.
>>> +config NODES_SPAN_OTHER_NODES
>>> +    def_bool y
>>> +    depends on ACPI_NUMA
>>> +
>>
>> I thought we agreed to do this in the core code?
>>
>> https://lore.kernel.org/lkml/1562887528-5896-1-git-send-email-Hoan@os.amperecomputing.com 
>>
> 
> Yes, but it looks like Thomas didn't agree to apply this patch into x86.
> 
> https://lore.kernel.org/lkml/alpine.DEB.2.21.1907152042110.1767@nanos.tec.linutronix.de/ 
> 
> 
> Regards
> Hoan
> 
>>
>> Will
>>

Re: [PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Hoan Tran]

Hi Will and All,

Any chance you have time to take a look.

Thanks and regards
Hoan

On 2/25/20 3:20 PM, Hoan Tran wrote:
> Hi Will,
> 
> Do you have any comments?
> 
> Thanks
> Hoan
> 
> On 2/6/20 12:01 PM, Hoan Tran wrote:
>> Hi Will,
>>
>> On 2/6/20 2:23 AM, Will Deacon wrote:
>>> On Mon, Feb 03, 2020 at 11:55:14AM -0800, Hoan Tran wrote:
>>>> Some NUMA nodes have memory ranges that span other nodes.
>>>> Even though a pfn is valid and between a node's start and end pfns,
>>>> it may not reside on that node.
>>>>
>>>> This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
>>>> this type of NUMA layout.
>>>>
>>>> Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
>>>> ---
>>>>   arch/arm64/Kconfig | 7 +++++++
>>>>   1 file changed, 7 insertions(+)
>>>>
>>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>>>> index e688dfa..939d28f 100644
>>>> --- a/arch/arm64/Kconfig
>>>> +++ b/arch/arm64/Kconfig
>>>> @@ -959,6 +959,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
>>>>   config HOLES_IN_ZONE
>>>>       def_bool y
>>>> +# Some NUMA nodes have memory ranges that span other nodes.
>>>> +# Even though a pfn is valid and between a node's start and end pfns,
>>>> +# it may not reside on that node.
>>>> +config NODES_SPAN_OTHER_NODES
>>>> +    def_bool y
>>>> +    depends on ACPI_NUMA
>>>> +
>>>
>>> I thought we agreed to do this in the core code?
>>>
>>> https://lore.kernel.org/lkml/1562887528-5896-1-git-send-email-Hoan@os.amperecomputing.com 
>>>
>>
>> Yes, but it looks like Thomas didn't agree to apply this patch into x86.
>>
>> https://lore.kernel.org/lkml/alpine.DEB.2.21.1907152042110.1767@nanos.tec.linutronix.de/ 
>>
>>
>> Regards
>> Hoan
>>
>>>
>>> Will
>>>

Re: [PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Catalin Marinas]

On Thu, Feb 06, 2020 at 12:01:19PM -0800, Hoan Tran wrote:
> Hi Will,
> 
> On 2/6/20 2:23 AM, Will Deacon wrote:
> > On Mon, Feb 03, 2020 at 11:55:14AM -0800, Hoan Tran wrote:
> > > Some NUMA nodes have memory ranges that span other nodes.
> > > Even though a pfn is valid and between a node's start and end pfns,
> > > it may not reside on that node.
> > > 
> > > This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
> > > this type of NUMA layout.
> > > 
> > > Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
> > > ---
> > >   arch/arm64/Kconfig | 7 +++++++
> > >   1 file changed, 7 insertions(+)
> > > 
> > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > > index e688dfa..939d28f 100644
> > > --- a/arch/arm64/Kconfig
> > > +++ b/arch/arm64/Kconfig
> > > @@ -959,6 +959,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
> > >   config HOLES_IN_ZONE
> > >   	def_bool y
> > > +# Some NUMA nodes have memory ranges that span other nodes.
> > > +# Even though a pfn is valid and between a node's start and end pfns,
> > > +# it may not reside on that node.
> > > +config NODES_SPAN_OTHER_NODES
> > > +	def_bool y
> > > +	depends on ACPI_NUMA
> > > +
> > 
> > I thought we agreed to do this in the core code?
> > 
> > https://lore.kernel.org/lkml/1562887528-5896-1-git-send-email-Hoan@os.amperecomputing.com
> 
> Yes, but it looks like Thomas didn't agree to apply this patch into
> x86.
> 
> https://lore.kernel.org/lkml/alpine.DEB.2.21.1907152042110.1767@nanos.tec.linutronix.de/

Was it a clear statement that such change will not make it to x86 or a
request for improving the patch or the description? I'd suggest you
update the x86 patch comment to include the rationale as per your reply
to Thomas and post a new version of the generic series. If Thomas (or
the mm folk) reject it again, we'll revisit the arm64-specific thread.

-- 
Catalin

Re: [PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Hoan Tran]

Hi Catalin,


On 3/11/20 4:27 AM, Catalin Marinas wrote:
> On Thu, Feb 06, 2020 at 12:01:19PM -0800, Hoan Tran wrote:
>> Hi Will,
>>
>> On 2/6/20 2:23 AM, Will Deacon wrote:
>>> On Mon, Feb 03, 2020 at 11:55:14AM -0800, Hoan Tran wrote:
>>>> Some NUMA nodes have memory ranges that span other nodes.
>>>> Even though a pfn is valid and between a node's start and end pfns,
>>>> it may not reside on that node.
>>>>
>>>> This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
>>>> this type of NUMA layout.
>>>>
>>>> Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
>>>> ---
>>>>    arch/arm64/Kconfig | 7 +++++++
>>>>    1 file changed, 7 insertions(+)
>>>>
>>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>>>> index e688dfa..939d28f 100644
>>>> --- a/arch/arm64/Kconfig
>>>> +++ b/arch/arm64/Kconfig
>>>> @@ -959,6 +959,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
>>>>    config HOLES_IN_ZONE
>>>>    	def_bool y
>>>> +# Some NUMA nodes have memory ranges that span other nodes.
>>>> +# Even though a pfn is valid and between a node's start and end pfns,
>>>> +# it may not reside on that node.
>>>> +config NODES_SPAN_OTHER_NODES
>>>> +	def_bool y
>>>> +	depends on ACPI_NUMA
>>>> +
>>>
>>> I thought we agreed to do this in the core code?
>>>
>>> https://lore.kernel.org/lkml/1562887528-5896-1-git-send-email-Hoan@os.amperecomputing.com
>>
>> Yes, but it looks like Thomas didn't agree to apply this patch into
>> x86.
>>
>> https://lore.kernel.org/lkml/alpine.DEB.2.21.1907152042110.1767@nanos.tec.linutronix.de/
> 
> Was it a clear statement that such change will not make it to x86 or a
> request for improving the patch or the description? I'd suggest you
> update the x86 patch comment to include the rationale as per your reply
> to Thomas and post a new version of the generic series. If Thomas (or
> the mm folk) reject it again, we'll revisit the arm64-specific thread.

Yes, I have just sent out a new patch for mm core.

Regards
Hoan

> 

Re: [PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Hoan Tran OS]

Hi Will,

On 6/25/19 3:12 AM, Will Deacon wrote:
> On Mon, Jun 24, 2019 at 02:38:28PM +0000, Hoan Tran OS wrote:
>> Some NUMA nodes have memory ranges that span other nodes.
>> Even though a pfn is valid and between a node's start and end pfns,
>> it may not reside on that node.
>>
>> This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
>> this type of NUMA layout.
>>
>> Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
>> ---
>>   arch/arm64/Kconfig | 7 +++++++
>>   1 file changed, 7 insertions(+)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index 697ea05..21fc168 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -873,6 +873,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
>>   config HOLES_IN_ZONE
>>   	def_bool y
>>   
>> +# Some NUMA nodes have memory ranges that span other nodes.
>> +# Even though a pfn is valid and between a node's start and end pfns,
>> +# it may not reside on that node.
>> +config NODES_SPAN_OTHER_NODES
>> +	def_bool y
>> +	depends on ACPI_NUMA
> 
> How come this is specific to ACPI?

I referred to x86 config. Beside of that, I only can test on the system 
with ACPI supported.

> 
> It would be nice if this was the default, given that only ia64, mips and
> sh appear to be the only NUMA-capable architectures which don't have it.
> In other words, replace the #ifdef CONFIG_NODES_SPAN_OTHER_NODES in
> mm/page_alloc.c with #ifdef CONFIG_NUMA.

Agreed, let me send another patch as your suggestion.

Thanks
Hoan

> Will
> 

Re: [PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Will Deacon]

On Mon, Jun 24, 2019 at 02:38:28PM +0000, Hoan Tran OS wrote:
> Some NUMA nodes have memory ranges that span other nodes.
> Even though a pfn is valid and between a node's start and end pfns,
> it may not reside on that node.
> 
> This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
> this type of NUMA layout.
> 
> Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
> ---
>  arch/arm64/Kconfig | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 697ea05..21fc168 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -873,6 +873,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
>  config HOLES_IN_ZONE
>  	def_bool y
>  
> +# Some NUMA nodes have memory ranges that span other nodes.
> +# Even though a pfn is valid and between a node's start and end pfns,
> +# it may not reside on that node.
> +config NODES_SPAN_OTHER_NODES
> +	def_bool y
> +	depends on ACPI_NUMA

How come this is specific to ACPI?

It would be nice if this was the default, given that only ia64, mips and
sh appear to be the only NUMA-capable architectures which don't have it.
In other words, replace the #ifdef CONFIG_NODES_SPAN_OTHER_NODES in
mm/page_alloc.c with #ifdef CONFIG_NUMA.

Will

[PATCH] arm64: Kconfig: Enable NODES_SPAN_OTHER_NODES config for NUMA

[Hoan Tran OS]

Some NUMA nodes have memory ranges that span other nodes.
Even though a pfn is valid and between a node's start and end pfns,
it may not reside on that node.

This patch enables NODES_SPAN_OTHER_NODES config for NUMA to support
this type of NUMA layout.

Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
---
 arch/arm64/Kconfig | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 697ea05..21fc168 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -873,6 +873,13 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK
 config HOLES_IN_ZONE
 	def_bool y
 
+# Some NUMA nodes have memory ranges that span other nodes.
+# Even though a pfn is valid and between a node's start and end pfns,
+# it may not reside on that node.
+config NODES_SPAN_OTHER_NODES
+	def_bool y
+	depends on ACPI_NUMA
+
 source "kernel/Kconfig.hz"
 
 config ARCH_SUPPORTS_DEBUG_PAGEALLOC
-- 
2.7.4