[PATCH] arch/sh/: fix NUMA build errors

[PATCH] arch/sh/: fix NUMA build errors

[Randy Dunlap]

From: Randy Dunlap <rdunlap@infradead.org>

Fix SUPERH builds that select SYS_SUPPORTS_NUMA but do not select
SYS_SUPPORTS_SMP and SMP.

kernel/sched/topology.c is only built for CONFIG_SMP and then the NUMA
code + data inside topology.c is only built when CONFIG_NUMA is
set/enabled, so these arch/sh/ configs need to select SMP and
SYS_SUPPORTS_SMP to build the NUMA support.

Fixes this build error in 3 different SUPERH configs:

mm/page_alloc.o: In function `get_page_from_freelist':
page_alloc.c:(.text+0x2ca8): undefined reference to `node_reclaim_distance'

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: linux-sh@vger.kernel.org
---
or maybe these should be fixed in the defconfig files?

or alternatively, does it make any sense to support NUMA without SMP?

 arch/sh/Kconfig |    4 ++++
 1 file changed, 4 insertions(+)

--- lnx-54-rc8.orig/arch/sh/Kconfig
+++ lnx-54-rc8/arch/sh/Kconfig
@@ -508,6 +508,8 @@ config CPU_SUBTYPE_SH7722
 	select CPU_SHX2
 	select ARCH_SHMOBILE
 	select ARCH_SPARSEMEM_ENABLE
+	select SYS_SUPPORTS_SMP
+	select SMP
 	select SYS_SUPPORTS_NUMA
 	select SYS_SUPPORTS_SH_CMT
 	select PINCTRL
@@ -518,6 +520,8 @@ config CPU_SUBTYPE_SH7366
 	select CPU_SHX2
 	select ARCH_SHMOBILE
 	select ARCH_SPARSEMEM_ENABLE
+	select SYS_SUPPORTS_SMP
+	select SMP
 	select SYS_SUPPORTS_NUMA
 	select SYS_SUPPORTS_SH_CMT
 

Re: [PATCH] arch/sh/: fix NUMA build errors

[Geert Uytterhoeven]

Hi Randy,

On Tue, Nov 19, 2019 at 1:55 AM Randy Dunlap <rdunlap@infradead.org> wrote:
> From: Randy Dunlap <rdunlap@infradead.org>
> Fix SUPERH builds that select SYS_SUPPORTS_NUMA but do not select
> SYS_SUPPORTS_SMP and SMP.
>
> kernel/sched/topology.c is only built for CONFIG_SMP and then the NUMA
> code + data inside topology.c is only built when CONFIG_NUMA is
> set/enabled, so these arch/sh/ configs need to select SMP and
> SYS_SUPPORTS_SMP to build the NUMA support.
>
> Fixes this build error in 3 different SUPERH configs:
>
> mm/page_alloc.o: In function `get_page_from_freelist':
> page_alloc.c:(.text+0x2ca8): undefined reference to `node_reclaim_distance'
>
> Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
> Cc: Rich Felker <dalias@libc.org>
> Cc: linux-sh@vger.kernel.org
> ---
> or maybe these should be fixed in the defconfig files?
>
> or alternatively, does it make any sense to support NUMA without SMP?

I think it does.  From arch/sh/mm/Kconfig config NUMA help:

        Some SH systems have many various memories scattered around
        the address space, each with varying latencies. This enables
        support for these blocks by binding them to nodes and allowing
        memory policies to be used for prioritizing and controlling
        allocation behaviour.

Probably the NUMA-core is too server/x86-centric, by assuming NUMA is
used only on systems with multiple CPUs, each with their own RAM.

AFAIK, none of the SoCs below are SMP:

> --- lnx-54-rc8.orig/arch/sh/Kconfig
> +++ lnx-54-rc8/arch/sh/Kconfig
> @@ -508,6 +508,8 @@ config CPU_SUBTYPE_SH7722
>         select CPU_SHX2
>         select ARCH_SHMOBILE
>         select ARCH_SPARSEMEM_ENABLE
> +       select SYS_SUPPORTS_SMP
> +       select SMP
>         select SYS_SUPPORTS_NUMA
>         select SYS_SUPPORTS_SH_CMT
>         select PINCTRL
> @@ -518,6 +520,8 @@ config CPU_SUBTYPE_SH7366
>         select CPU_SHX2
>         select ARCH_SHMOBILE
>         select ARCH_SPARSEMEM_ENABLE
> +       select SYS_SUPPORTS_SMP
> +       select SMP
>         select SYS_SUPPORTS_NUMA
>         select SYS_SUPPORTS_SH_CMT

BTW, you didn't have the issue with CPU_SHX3 and CPU_SUBTYPE_SH7785?

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH] arch/sh/: fix NUMA build errors

[Randy Dunlap]

On 11/18/19 11:38 PM, Geert Uytterhoeven wrote:
> Hi Randy,
> 
> On Tue, Nov 19, 2019 at 1:55 AM Randy Dunlap <rdunlap@infradead.org> wrote:
>> From: Randy Dunlap <rdunlap@infradead.org>
>> Fix SUPERH builds that select SYS_SUPPORTS_NUMA but do not select
>> SYS_SUPPORTS_SMP and SMP.
>>
>> kernel/sched/topology.c is only built for CONFIG_SMP and then the NUMA
>> code + data inside topology.c is only built when CONFIG_NUMA is
>> set/enabled, so these arch/sh/ configs need to select SMP and
>> SYS_SUPPORTS_SMP to build the NUMA support.
>>
>> Fixes this build error in 3 different SUPERH configs:
>>
>> mm/page_alloc.o: In function `get_page_from_freelist':
>> page_alloc.c:(.text+0x2ca8): undefined reference to `node_reclaim_distance'
>>
>> Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
>> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
>> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
>> Cc: Rich Felker <dalias@libc.org>
>> Cc: linux-sh@vger.kernel.org
>> ---
>> or maybe these should be fixed in the defconfig files?
>>
>> or alternatively, does it make any sense to support NUMA without SMP?
> 
> I think it does.  From arch/sh/mm/Kconfig config NUMA help:
> 
>         Some SH systems have many various memories scattered around
>         the address space, each with varying latencies. This enables
>         support for these blocks by binding them to nodes and allowing
>         memory policies to be used for prioritizing and controlling
>         allocation behaviour.

Yes, I saw that and suspected it also.

I was (and still am) hoping that a SuperH maintainer comments on
this and on how they are currently building kernels for these
failing configs.  Maybe they have some patches that aren't in-tree yet?


> Probably the NUMA-core is too server/x86-centric, by assuming NUMA is
> used only on systems with multiple CPUs, each with their own RAM.
> 
> AFAIK, none of the SoCs below are SMP:
> 
>> --- lnx-54-rc8.orig/arch/sh/Kconfig
>> +++ lnx-54-rc8/arch/sh/Kconfig
>> @@ -508,6 +508,8 @@ config CPU_SUBTYPE_SH7722
>>         select CPU_SHX2
>>         select ARCH_SHMOBILE
>>         select ARCH_SPARSEMEM_ENABLE
>> +       select SYS_SUPPORTS_SMP
>> +       select SMP
>>         select SYS_SUPPORTS_NUMA
>>         select SYS_SUPPORTS_SH_CMT
>>         select PINCTRL
>> @@ -518,6 +520,8 @@ config CPU_SUBTYPE_SH7366
>>         select CPU_SHX2
>>         select ARCH_SHMOBILE
>>         select ARCH_SPARSEMEM_ENABLE
>> +       select SYS_SUPPORTS_SMP
>> +       select SMP
>>         select SYS_SUPPORTS_NUMA
>>         select SYS_SUPPORTS_SH_CMT
> 
> BTW, you didn't have the issue with CPU_SHX3 and CPU_SUBTYPE_SH7785?

I didn't see any issue there but I will check it again.

thanks.
-- 
~Randy

Re: [PATCH] arch/sh/: fix NUMA build errors

[Randy Dunlap]

On 11/18/19 11:38 PM, Geert Uytterhoeven wrote:
> Hi Randy,
> 
> 
> BTW, you didn't have the issue with CPU_SHX3 and CPU_SUBTYPE_SH7785?

Geert,

I don't see a defconfig with that combination.

Also, http://kisskb.ellerman.id.au/kisskb/branch/linus/head/af42d3466bdc8f39806b26f593604fdc54140bcb/
didn't report any such problem.  I only addressed the 3 build errors that were reported there as
  page_alloc.c:(.text+0x3148): undefined reference to `node_reclaim_distance'

I did just test sdk7786_defconfig successfully, but that's no surprise since it sets CONFIG_SMP.

-- 
~Randy

Re: [PATCH] arch/sh/: fix NUMA build errors

[Randy Dunlap]

On 11/18/19 11:38 PM, Geert Uytterhoeven wrote:
> Hi Randy,
> 
> On Tue, Nov 19, 2019 at 1:55 AM Randy Dunlap <rdunlap@infradead.org> wrote:
>> From: Randy Dunlap <rdunlap@infradead.org>
>> Fix SUPERH builds that select SYS_SUPPORTS_NUMA but do not select
>> SYS_SUPPORTS_SMP and SMP.
>>
>> kernel/sched/topology.c is only built for CONFIG_SMP and then the NUMA
>> code + data inside topology.c is only built when CONFIG_NUMA is
>> set/enabled, so these arch/sh/ configs need to select SMP and
>> SYS_SUPPORTS_SMP to build the NUMA support.
>>
>> Fixes this build error in 3 different SUPERH configs:
>>
>> mm/page_alloc.o: In function `get_page_from_freelist':
>> page_alloc.c:(.text+0x2ca8): undefined reference to `node_reclaim_distance'
>>
>> Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
>> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
>> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
>> Cc: Rich Felker <dalias@libc.org>
>> Cc: linux-sh@vger.kernel.org
>> ---
>> or maybe these should be fixed in the defconfig files?
>>
>> or alternatively, does it make any sense to support NUMA without SMP?
> 
> I think it does.  From arch/sh/mm/Kconfig config NUMA help:
> 
>         Some SH systems have many various memories scattered around
>         the address space, each with varying latencies. This enables
>         support for these blocks by binding them to nodes and allowing
>         memory policies to be used for prioritizing and controlling
>         allocation behaviour.

Note that this help text is under:
config NUMA
	bool "Non-Uniform Memory Access (NUMA) Support"
	depends on MMU && SYS_SUPPORTS_NUMA
	select ARCH_WANT_NUMA_VARIABLE_LOCALITY

but ARCH_WANT_NUMA_VARIABLE_LOCALITY seems to be unimplemented anywhere in
the kernel source tree.  I.e., the Kconfig symbol exists in init/Kconfig,
but there is no code to support its use.


> Probably the NUMA-core is too server/x86-centric, by assuming NUMA is
> used only on systems with multiple CPUs, each with their own RAM.
> 
> AFAIK, none of the SoCs below are SMP:
> 
>> --- lnx-54-rc8.orig/arch/sh/Kconfig
>> +++ lnx-54-rc8/arch/sh/Kconfig
>> @@ -508,6 +508,8 @@ config CPU_SUBTYPE_SH7722
>>         select CPU_SHX2
>>         select ARCH_SHMOBILE
>>         select ARCH_SPARSEMEM_ENABLE
>> +       select SYS_SUPPORTS_SMP
>> +       select SMP
>>         select SYS_SUPPORTS_NUMA
>>         select SYS_SUPPORTS_SH_CMT
>>         select PINCTRL
>> @@ -518,6 +520,8 @@ config CPU_SUBTYPE_SH7366
>>         select CPU_SHX2
>>         select ARCH_SHMOBILE
>>         select ARCH_SPARSEMEM_ENABLE
>> +       select SYS_SUPPORTS_SMP
>> +       select SMP
>>         select SYS_SUPPORTS_NUMA
>>         select SYS_SUPPORTS_SH_CMT
> 
> BTW, you didn't have the issue with CPU_SHX3 and CPU_SUBTYPE_SH7785?
> 
> Gr{oetje,eeting}s,
> 
>                         Geert
> 


-- 
~Randy

Re: [PATCH] arch/sh/: fix NUMA build errors

[Geert Uytterhoeven]

Hi Randy

On Wed, Nov 20, 2019 at 1:41 AM Randy Dunlap <rdunlap@infradead.org> wrote:
> On 11/18/19 11:38 PM, Geert Uytterhoeven wrote:
> > BTW, you didn't have the issue with CPU_SHX3 and CPU_SUBTYPE_SH7785?
>
> I don't see a defconfig with that combination.

OK ;-)

> Also, http://kisskb.ellerman.id.au/kisskb/branch/linus/head/af42d3466bdc8f39806b26f593604fdc54140bcb/
> didn't report any such problem.  I only addressed the 3 build errors that were reported there as
>   page_alloc.c:(.text+0x3148): undefined reference to `node_reclaim_distance'
>
> I did just test sdk7786_defconfig successfully, but that's no surprise since it sets CONFIG_SMP.

Yeah, that's one of the few real SMP SH SoCs.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH] arch/sh/: fix NUMA build errors

[Geert Uytterhoeven]

Hi Randy,

On Wed, Nov 20, 2019 at 5:28 AM Randy Dunlap <rdunlap@infradead.org> wrote:
> On 11/18/19 11:38 PM, Geert Uytterhoeven wrote:
> > On Tue, Nov 19, 2019 at 1:55 AM Randy Dunlap <rdunlap@infradead.org> wrote:
> >> From: Randy Dunlap <rdunlap@infradead.org>
> >> Fix SUPERH builds that select SYS_SUPPORTS_NUMA but do not select
> >> SYS_SUPPORTS_SMP and SMP.
> >>
> >> kernel/sched/topology.c is only built for CONFIG_SMP and then the NUMA
> >> code + data inside topology.c is only built when CONFIG_NUMA is
> >> set/enabled, so these arch/sh/ configs need to select SMP and
> >> SYS_SUPPORTS_SMP to build the NUMA support.
> >>
> >> Fixes this build error in 3 different SUPERH configs:
> >>
> >> mm/page_alloc.o: In function `get_page_from_freelist':
> >> page_alloc.c:(.text+0x2ca8): undefined reference to `node_reclaim_distance'
> >>
> >> Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
> >> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
> >> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
> >> Cc: Rich Felker <dalias@libc.org>
> >> Cc: linux-sh@vger.kernel.org
> >> ---
> >> or maybe these should be fixed in the defconfig files?
> >>
> >> or alternatively, does it make any sense to support NUMA without SMP?
> >
> > I think it does.  From arch/sh/mm/Kconfig config NUMA help:
> >
> >         Some SH systems have many various memories scattered around
> >         the address space, each with varying latencies. This enables
> >         support for these blocks by binding them to nodes and allowing
> >         memory policies to be used for prioritizing and controlling
> >         allocation behaviour.
>
> Note that this help text is under:
> config NUMA
>         bool "Non-Uniform Memory Access (NUMA) Support"
>         depends on MMU && SYS_SUPPORTS_NUMA
>         select ARCH_WANT_NUMA_VARIABLE_LOCALITY
>
> but ARCH_WANT_NUMA_VARIABLE_LOCALITY seems to be unimplemented anywhere in
> the kernel source tree.  I.e., the Kconfig symbol exists in init/Kconfig,
> but there is no code to support its use.

It does control (prevent) another option:

# For architectures that (ab)use NUMA to represent different memory regions
# all cpu-local but of different latencies, such as SuperH.
#
config ARCH_WANT_NUMA_VARIABLE_LOCALITY
        bool

config NUMA_BALANCING
        bool "Memory placement aware NUMA scheduler"
        depends on ARCH_SUPPORTS_NUMA_BALANCING
        depends on !ARCH_WANT_NUMA_VARIABLE_LOCALITY
        depends on SMP && NUMA && MIGRATION
        help
          This option adds support for automatic NUMA aware
memory/task placement.
          The mechanism is quite primitive and is based on migrating memory when
          it has references to the node the task is running on.

          This system will be inactive on UMA systems.

The symbol was set by arch/metag, too (which was removed).

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH] arch/sh/: fix NUMA build errors

[Randy Dunlap]

On 11/20/19 12:30 AM, Geert Uytterhoeven wrote:
> Hi Randy,
> 
> On Wed, Nov 20, 2019 at 5:28 AM Randy Dunlap <rdunlap@infradead.org> wrote:
>> On 11/18/19 11:38 PM, Geert Uytterhoeven wrote:
>>> On Tue, Nov 19, 2019 at 1:55 AM Randy Dunlap <rdunlap@infradead.org> wrote:
>>>> From: Randy Dunlap <rdunlap@infradead.org>
>>>> Fix SUPERH builds that select SYS_SUPPORTS_NUMA but do not select
>>>> SYS_SUPPORTS_SMP and SMP.
>>>>
>>>> kernel/sched/topology.c is only built for CONFIG_SMP and then the NUMA
>>>> code + data inside topology.c is only built when CONFIG_NUMA is
>>>> set/enabled, so these arch/sh/ configs need to select SMP and
>>>> SYS_SUPPORTS_SMP to build the NUMA support.
>>>>
>>>> Fixes this build error in 3 different SUPERH configs:
>>>>
>>>> mm/page_alloc.o: In function `get_page_from_freelist':
>>>> page_alloc.c:(.text+0x2ca8): undefined reference to `node_reclaim_distance'
>>>>
>>>> Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
>>>> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
>>>> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
>>>> Cc: Rich Felker <dalias@libc.org>
>>>> Cc: linux-sh@vger.kernel.org
>>>> ---
>>>> or maybe these should be fixed in the defconfig files?
>>>>
>>>> or alternatively, does it make any sense to support NUMA without SMP?
>>>
>>> I think it does.  From arch/sh/mm/Kconfig config NUMA help:
>>>
>>>         Some SH systems have many various memories scattered around
>>>         the address space, each with varying latencies. This enables
>>>         support for these blocks by binding them to nodes and allowing
>>>         memory policies to be used for prioritizing and controlling
>>>         allocation behaviour.
>>
>> Note that this help text is under:
>> config NUMA
>>         bool "Non-Uniform Memory Access (NUMA) Support"
>>         depends on MMU && SYS_SUPPORTS_NUMA
>>         select ARCH_WANT_NUMA_VARIABLE_LOCALITY
>>
>> but ARCH_WANT_NUMA_VARIABLE_LOCALITY seems to be unimplemented anywhere in
>> the kernel source tree.  I.e., the Kconfig symbol exists in init/Kconfig,
>> but there is no code to support its use.
> 
> It does control (prevent) another option:
> 
> # For architectures that (ab)use NUMA to represent different memory regions
> # all cpu-local but of different latencies, such as SuperH.
> #
> config ARCH_WANT_NUMA_VARIABLE_LOCALITY
>         bool
> 
> config NUMA_BALANCING
>         bool "Memory placement aware NUMA scheduler"
>         depends on ARCH_SUPPORTS_NUMA_BALANCING
>         depends on !ARCH_WANT_NUMA_VARIABLE_LOCALITY
>         depends on SMP && NUMA && MIGRATION
>         help
>           This option adds support for automatic NUMA aware
> memory/task placement.
>           The mechanism is quite primitive and is based on migrating memory when
>           it has references to the node the task is running on.
> 
>           This system will be inactive on UMA systems.
> 
> The symbol was set by arch/metag, too (which was removed).

I see.  thanks.

-- 
~Randy

Re: [PATCH] arch/sh/: fix NUMA build errors

[Randy Dunlap]

On 11/19/19 1:12 PM, Randy Dunlap wrote:
> On 11/18/19 11:38 PM, Geert Uytterhoeven wrote:
>> Hi Randy,
>>
>> On Tue, Nov 19, 2019 at 1:55 AM Randy Dunlap <rdunlap@infradead.org> wrote:
>>> From: Randy Dunlap <rdunlap@infradead.org>
>>> Fix SUPERH builds that select SYS_SUPPORTS_NUMA but do not select
>>> SYS_SUPPORTS_SMP and SMP.
>>>
>>> kernel/sched/topology.c is only built for CONFIG_SMP and then the NUMA
>>> code + data inside topology.c is only built when CONFIG_NUMA is
>>> set/enabled, so these arch/sh/ configs need to select SMP and
>>> SYS_SUPPORTS_SMP to build the NUMA support.
>>>
>>> Fixes this build error in 3 different SUPERH configs:
>>>
>>> mm/page_alloc.o: In function `get_page_from_freelist':
>>> page_alloc.c:(.text+0x2ca8): undefined reference to `node_reclaim_distance'
>>>
>>> Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
>>> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
>>> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
>>> Cc: Rich Felker <dalias@libc.org>
>>> Cc: linux-sh@vger.kernel.org
>>> ---
>>> or maybe these should be fixed in the defconfig files?
>>>
>>> or alternatively, does it make any sense to support NUMA without SMP?
>>
>> I think it does.  From arch/sh/mm/Kconfig config NUMA help:
>>
>>         Some SH systems have many various memories scattered around
>>         the address space, each with varying latencies. This enables
>>         support for these blocks by binding them to nodes and allowing
>>         memory policies to be used for prioritizing and controlling
>>         allocation behaviour.
> 
> Yes, I saw that and suspected it also.
> 
> I was (and still am) hoping that a SuperH maintainer comments on
> this and on how they are currently building kernels for these
> failing configs.  Maybe they have some patches that aren't in-tree yet?
> 

Yoshinori-san,
Can you share with us how you build kernels for SUPERH configs that set
CONFIG_NUMA but do not set CONFIG_SMP?  Do you have any patches for this?


> 
>> Probably the NUMA-core is too server/x86-centric, by assuming NUMA is
>> used only on systems with multiple CPUs, each with their own RAM.

Yes, I looked at all of that code for a couple of days and got nowhere
with trying to separate NUMA from SMP.

thanks.
-- 
~Randy