[PATCH] RISC-V: Increase range and default value of NR_CPUS

[PATCH] RISC-V: Increase range and default value of NR_CPUS

[Anup Patel]

Currently, the range and default value of NR_CPUS is too restrictive
for high-end RISC-V systems with large number of HARTs. The latest
QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
restrictive for QEMU as well. Other major architectures (such as
ARM64, x86_64, MIPS, etc) have a much higher range and default
value of NR_CPUS.

This patch increases NR_CPUS range to 2-256 and default value to
XLEN (i.e. 32 for RV32 and 64 for RV64).

Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
 arch/riscv/Kconfig | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 5adcbd9b5e88..54f125eb10f6 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -274,10 +274,11 @@ config SMP
 	  If you don't know what to do here, say N.
 
 config NR_CPUS
-	int "Maximum number of CPUs (2-32)"
-	range 2 32
+	int "Maximum number of CPUs (2-256)"
+	range 2 256
 	depends on SMP
-	default "8"
+	default "32" if 32BIT
+	default "64" if 64BIT
 
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH] RISC-V: Increase range and default value of NR_CPUS

[Anup Patel]

Currently, the range and default value of NR_CPUS is too restrictive
for high-end RISC-V systems with large number of HARTs. The latest
QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
restrictive for QEMU as well. Other major architectures (such as
ARM64, x86_64, MIPS, etc) have a much higher range and default
value of NR_CPUS.

This patch increases NR_CPUS range to 2-256 and default value to
XLEN (i.e. 32 for RV32 and 64 for RV64).

Signed-off-by: Anup Patel <apatel@ventanamicro.com>
---
 arch/riscv/Kconfig | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 5adcbd9b5e88..54f125eb10f6 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -274,10 +274,11 @@ config SMP
 	  If you don't know what to do here, say N.
 
 config NR_CPUS
-	int "Maximum number of CPUs (2-32)"
-	range 2 32
+	int "Maximum number of CPUs (2-256)"
+	range 2 256
 	depends on SMP
-	default "8"
+	default "32" if 32BIT
+	default "64" if 64BIT
 
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
-- 
2.25.1

Re: [PATCH] RISC-V: Increase range and default value of NR_CPUS

[Ben Dooks]

On 17/03/2022 03:55, Anup Patel wrote:
> Currently, the range and default value of NR_CPUS is too restrictive
> for high-end RISC-V systems with large number of HARTs. The latest
> QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> restrictive for QEMU as well. Other major architectures (such as
> ARM64, x86_64, MIPS, etc) have a much higher range and default
> value of NR_CPUS.

What's the memory overhead for increasing this?

-- 
Ben Dooks				http://www.codethink.co.uk/
Senior Engineer				Codethink - Providing Genius

https://www.codethink.co.uk/privacy.html

Re: [PATCH] RISC-V: Increase range and default value of NR_CPUS

[Ben Dooks]

On 17/03/2022 03:55, Anup Patel wrote:
> Currently, the range and default value of NR_CPUS is too restrictive
> for high-end RISC-V systems with large number of HARTs. The latest
> QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> restrictive for QEMU as well. Other major architectures (such as
> ARM64, x86_64, MIPS, etc) have a much higher range and default
> value of NR_CPUS.

What's the memory overhead for increasing this?

-- 
Ben Dooks				http://www.codethink.co.uk/
Senior Engineer				Codethink - Providing Genius

https://www.codethink.co.uk/privacy.html

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH] RISC-V: Increase range and default value of NR_CPUS

[Arnd Bergmann]

On Fri, Mar 18, 2022 at 3:46 PM Ben Dooks <ben.dooks@codethink.co.uk> wrote:
>
> On 17/03/2022 03:55, Anup Patel wrote:
> > Currently, the range and default value of NR_CPUS is too restrictive
> > for high-end RISC-V systems with large number of HARTs. The latest
> > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > restrictive for QEMU as well.

If qemu allows 512, what is the reason for limiting the kernel to 256?

> > Other major architectures (such as
> > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > value of NR_CPUS.
>
> What's the memory overhead for increasing this?

It's supposed to be very small, I would expect three main sources of overhead:

- cpumask_t variables, those grow once you go beyond the size of
   an unsigned long (32 or 64 bits), so with the default just on the limit, this
   makes no difference.
   Note that you can run out of stack space with NR_CPUS values if
   CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
   a problem for 512 or below.

- percpu variables: these are dynamically allocated based on the number of
  CPUs at boot time, so they should not have any real impact.

- NR_CPUS sized arrays, these are sometimes used in place of
  percpu data. This is only a problem if the array members individually
  are more than a few bytes. There are not too many of these in the kernel,
  as using those is discouraged.

      Arnd

Re: [PATCH] RISC-V: Increase range and default value of NR_CPUS

[Arnd Bergmann]

On Fri, Mar 18, 2022 at 3:46 PM Ben Dooks <ben.dooks@codethink.co.uk> wrote:
>
> On 17/03/2022 03:55, Anup Patel wrote:
> > Currently, the range and default value of NR_CPUS is too restrictive
> > for high-end RISC-V systems with large number of HARTs. The latest
> > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > restrictive for QEMU as well.

If qemu allows 512, what is the reason for limiting the kernel to 256?

> > Other major architectures (such as
> > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > value of NR_CPUS.
>
> What's the memory overhead for increasing this?

It's supposed to be very small, I would expect three main sources of overhead:

- cpumask_t variables, those grow once you go beyond the size of
   an unsigned long (32 or 64 bits), so with the default just on the limit, this
   makes no difference.
   Note that you can run out of stack space with NR_CPUS values if
   CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
   a problem for 512 or below.

- percpu variables: these are dynamically allocated based on the number of
  CPUs at boot time, so they should not have any real impact.

- NR_CPUS sized arrays, these are sometimes used in place of
  percpu data. This is only a problem if the array members individually
  are more than a few bytes. There are not too many of these in the kernel,
  as using those is discouraged.

      Arnd

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH] RISC-V: Increase range and default value of NR_CPUS

[Anup Patel]

On Fri, Mar 18, 2022 at 8:44 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Fri, Mar 18, 2022 at 3:46 PM Ben Dooks <ben.dooks@codethink.co.uk> wrote:
> >
> > On 17/03/2022 03:55, Anup Patel wrote:
> > > Currently, the range and default value of NR_CPUS is too restrictive
> > > for high-end RISC-V systems with large number of HARTs. The latest
> > > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > > restrictive for QEMU as well.
>
> If qemu allows 512, what is the reason for limiting the kernel to 256?

There is no particular reason. I will change this to 2-512 range to match
the maximum number of CPUs supported by the QEMU virt machine.

>
> > > Other major architectures (such as
> > > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > > value of NR_CPUS.
> >
> > What's the memory overhead for increasing this?
>
> It's supposed to be very small, I would expect three main sources of overhead:
>
> - cpumask_t variables, those grow once you go beyond the size of
>    an unsigned long (32 or 64 bits), so with the default just on the limit, this
>    makes no difference.
>    Note that you can run out of stack space with NR_CPUS values if
>    CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
>    a problem for 512 or below.

Yes, the rationale is to have default NR_CPUS such that size of
cpumask_t does not change for RV32 and RV64.

>
> - percpu variables: these are dynamically allocated based on the number of
>   CPUs at boot time, so they should not have any real impact.
>
> - NR_CPUS sized arrays, these are sometimes used in place of
>   percpu data. This is only a problem if the array members individually
>   are more than a few bytes. There are not too many of these in the kernel,
>   as using those is discouraged.

Separately, we are also trying to replace NR_CPUS sized arrays.

Regards,
Anup

Re: [PATCH] RISC-V: Increase range and default value of NR_CPUS

[Anup Patel]

On Fri, Mar 18, 2022 at 8:44 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Fri, Mar 18, 2022 at 3:46 PM Ben Dooks <ben.dooks@codethink.co.uk> wrote:
> >
> > On 17/03/2022 03:55, Anup Patel wrote:
> > > Currently, the range and default value of NR_CPUS is too restrictive
> > > for high-end RISC-V systems with large number of HARTs. The latest
> > > QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is
> > > restrictive for QEMU as well.
>
> If qemu allows 512, what is the reason for limiting the kernel to 256?

There is no particular reason. I will change this to 2-512 range to match
the maximum number of CPUs supported by the QEMU virt machine.

>
> > > Other major architectures (such as
> > > ARM64, x86_64, MIPS, etc) have a much higher range and default
> > > value of NR_CPUS.
> >
> > What's the memory overhead for increasing this?
>
> It's supposed to be very small, I would expect three main sources of overhead:
>
> - cpumask_t variables, those grow once you go beyond the size of
>    an unsigned long (32 or 64 bits), so with the default just on the limit, this
>    makes no difference.
>    Note that you can run out of stack space with NR_CPUS values if
>    CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
>    a problem for 512 or below.

Yes, the rationale is to have default NR_CPUS such that size of
cpumask_t does not change for RV32 and RV64.

>
> - percpu variables: these are dynamically allocated based on the number of
>   CPUs at boot time, so they should not have any real impact.
>
> - NR_CPUS sized arrays, these are sometimes used in place of
>   percpu data. This is only a problem if the array members individually
>   are more than a few bytes. There are not too many of these in the kernel,
>   as using those is discouraged.

Separately, we are also trying to replace NR_CPUS sized arrays.

Regards,
Anup

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv