From: Heinrich Schuchardt <heinrich.schuchardt@canonical.com> To: Palmer Dabbelt <palmer@dabbelt.com> Cc: Paul Walmsley <paul.walmsley@sifive.com>, Arnd Bergmann <arnd@arndb.de>, atishp@atishpatra.org, Alistair Francis <Alistair.Francis@wdc.com>, anup@brainfault.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, apatel@ventanamicro.com Subject: Re: [PATCH v2] RISC-V: Increase range and default value of NR_CPUS Date: Wed, 6 Apr 2022 11:55:43 +0200 [thread overview] Message-ID: <cd8d6641-4a29-7deb-49ae-a80baab0344f@canonical.com> (raw) In-Reply-To: <mhng-9cbb2a46-8bcb-4bfe-b927-0ed469dbd3a6@palmer-mbp2014> On 3/31/22 21:42, Palmer Dabbelt wrote: > On Sat, 19 Mar 2022 05:12:06 PDT (-0700), apatel@ventanamicro.com wrote: >> Currently, the range and default value of NR_CPUS is too restrictive >> for high-end RISC-V systems with large number of HARTs. The latest >> QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is >> restrictive for QEMU as well. Other major architectures (such as >> ARM64, x86_64, MIPS, etc) have a much higher range and default >> value of NR_CPUS. >> >> This patch increases NR_CPUS range to 2-512 and default value to >> XLEN (i.e. 32 for RV32 and 64 for RV64). >> >> Signed-off-by: Anup Patel <apatel@ventanamicro.com> >> --- >> Changes since v1: >> - Updated NR_CPUS range to 2-512 which reflects maximum number of >> CPUs supported by QEMU virt machine. >> --- >> arch/riscv/Kconfig | 7 ++++--- >> 1 file changed, 4 insertions(+), 3 deletions(-) >> >> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >> index 5adcbd9b5e88..423ac17f598c 100644 >> --- a/arch/riscv/Kconfig >> +++ b/arch/riscv/Kconfig >> @@ -274,10 +274,11 @@ config SMP >> If you don't know what to do here, say N. >> >> config NR_CPUS >> - int "Maximum number of CPUs (2-32)" >> - range 2 32 >> + int "Maximum number of CPUs (2-512)" >> + range 2 512 For SBI_V01=y there seems to be a hard constraint to XLEN bits. See __sbi_v01_cpumask_to_hartmask() in rch/riscv/kernel/sbi.c. So shouldn't this be something like: range 2 512 !SBI_V01 range 2 32 SBI_V01 && 32BIT range 2 64 SBI_V01 && 64BIT >> depends on SMP >> - default "8" >> + default "32" if 32BIT >> + default "64" if 64BIT >> >> config HOTPLUG_CPU >> bool "Support for hot-pluggable CPUs" > > I'm getting all sorts of boot issues with more than 32 CPUs, even on the > latest QEMU master. I'm not opposed to increasing the CPU count in > theory, but if we're going to have a setting that goes up to a huge > number it needs to at least boot. I've got 64 host threads, so it > shouldn't just be a scheduling thing. Currently high performing hardware for RISC-V is missing. So it makes sense to build software via QEMU on x86_64 or arm64 with as many hardware threads as available (128 is not uncommon). OpenSBI currently is limited to 128 threads: include/sbi/sbi_hartmask.h:22: #define SBI_HARTMASK_MAX_BITS 128 This is just an arbitrary value we can be modified. U-Boot v2022.04 qemu-riscv64_smode_defconfig has a value of CONFIG_SYS_MALLOC_F_LEN that is to low. This leads to a boot failure for more than 16 harts. A patch to correct this is pending: [PATCH v2 1/1] riscv: alloc space exhausted https://lore.kernel.org/u-boot/CAN5B=eKt=tFLZ2z3aNHJqsnJzpdA0oikcrC2i1_=ZDD=f+M0jA@mail.gmail.com/T/#t With QEMU 7.0 and the U-Boot fix booting into a 5.17 defconfig kernel with 64 virtual cores worked fine for me. Best regards Heinrich > > If there was some hardware that actually boots on these I'd be happy to > take it, but given that it's just QEMU I'd prefer to sort out the bugs > first. It's probably just latent bugs somewhere, but allowing users to > turn on configs we know don't work just seems like the wrong way to go. > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv > _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
WARNING: multiple messages have this Message-ID (diff)
From: Heinrich Schuchardt <heinrich.schuchardt@canonical.com> To: Palmer Dabbelt <palmer@dabbelt.com> Cc: Paul Walmsley <paul.walmsley@sifive.com>, Arnd Bergmann <arnd@arndb.de>, atishp@atishpatra.org, Alistair Francis <Alistair.Francis@wdc.com>, anup@brainfault.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, apatel@ventanamicro.com Subject: Re: [PATCH v2] RISC-V: Increase range and default value of NR_CPUS Date: Wed, 6 Apr 2022 11:55:43 +0200 [thread overview] Message-ID: <cd8d6641-4a29-7deb-49ae-a80baab0344f@canonical.com> (raw) In-Reply-To: <mhng-9cbb2a46-8bcb-4bfe-b927-0ed469dbd3a6@palmer-mbp2014> On 3/31/22 21:42, Palmer Dabbelt wrote: > On Sat, 19 Mar 2022 05:12:06 PDT (-0700), apatel@ventanamicro.com wrote: >> Currently, the range and default value of NR_CPUS is too restrictive >> for high-end RISC-V systems with large number of HARTs. The latest >> QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is >> restrictive for QEMU as well. Other major architectures (such as >> ARM64, x86_64, MIPS, etc) have a much higher range and default >> value of NR_CPUS. >> >> This patch increases NR_CPUS range to 2-512 and default value to >> XLEN (i.e. 32 for RV32 and 64 for RV64). >> >> Signed-off-by: Anup Patel <apatel@ventanamicro.com> >> --- >> Changes since v1: >> - Updated NR_CPUS range to 2-512 which reflects maximum number of >> CPUs supported by QEMU virt machine. >> --- >> arch/riscv/Kconfig | 7 ++++--- >> 1 file changed, 4 insertions(+), 3 deletions(-) >> >> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig >> index 5adcbd9b5e88..423ac17f598c 100644 >> --- a/arch/riscv/Kconfig >> +++ b/arch/riscv/Kconfig >> @@ -274,10 +274,11 @@ config SMP >> If you don't know what to do here, say N. >> >> config NR_CPUS >> - int "Maximum number of CPUs (2-32)" >> - range 2 32 >> + int "Maximum number of CPUs (2-512)" >> + range 2 512 For SBI_V01=y there seems to be a hard constraint to XLEN bits. See __sbi_v01_cpumask_to_hartmask() in rch/riscv/kernel/sbi.c. So shouldn't this be something like: range 2 512 !SBI_V01 range 2 32 SBI_V01 && 32BIT range 2 64 SBI_V01 && 64BIT >> depends on SMP >> - default "8" >> + default "32" if 32BIT >> + default "64" if 64BIT >> >> config HOTPLUG_CPU >> bool "Support for hot-pluggable CPUs" > > I'm getting all sorts of boot issues with more than 32 CPUs, even on the > latest QEMU master. I'm not opposed to increasing the CPU count in > theory, but if we're going to have a setting that goes up to a huge > number it needs to at least boot. I've got 64 host threads, so it > shouldn't just be a scheduling thing. Currently high performing hardware for RISC-V is missing. So it makes sense to build software via QEMU on x86_64 or arm64 with as many hardware threads as available (128 is not uncommon). OpenSBI currently is limited to 128 threads: include/sbi/sbi_hartmask.h:22: #define SBI_HARTMASK_MAX_BITS 128 This is just an arbitrary value we can be modified. U-Boot v2022.04 qemu-riscv64_smode_defconfig has a value of CONFIG_SYS_MALLOC_F_LEN that is to low. This leads to a boot failure for more than 16 harts. A patch to correct this is pending: [PATCH v2 1/1] riscv: alloc space exhausted https://lore.kernel.org/u-boot/CAN5B=eKt=tFLZ2z3aNHJqsnJzpdA0oikcrC2i1_=ZDD=f+M0jA@mail.gmail.com/T/#t With QEMU 7.0 and the U-Boot fix booting into a 5.17 defconfig kernel with 64 virtual cores worked fine for me. Best regards Heinrich > > If there was some hardware that actually boots on these I'd be happy to > take it, but given that it's just QEMU I'd prefer to sort out the bugs > first. It's probably just latent bugs somewhere, but allowing users to > turn on configs we know don't work just seems like the wrong way to go. > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv >
next prev parent reply other threads:[~2022-04-06 10:10 UTC|newest] Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-03-19 12:12 [PATCH v2] RISC-V: Increase range and default value of NR_CPUS Anup Patel 2022-03-19 12:12 ` Anup Patel 2022-03-31 19:42 ` Palmer Dabbelt 2022-03-31 19:42 ` Palmer Dabbelt 2022-04-06 9:55 ` Heinrich Schuchardt [this message] 2022-04-06 9:55 ` Heinrich Schuchardt 2022-04-06 10:10 ` Anup Patel 2022-04-06 10:10 ` Anup Patel 2022-04-08 16:38 ` Heinrich Schuchardt 2022-04-08 16:38 ` Heinrich Schuchardt 2022-04-08 16:45 ` Anup Patel 2022-04-08 16:45 ` Anup Patel 2022-04-11 22:18 ` Atish Patra 2022-04-11 22:18 ` Atish Patra 2022-04-06 23:00 ` Atish Patra 2022-04-06 23:00 ` Atish Patra
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=cd8d6641-4a29-7deb-49ae-a80baab0344f@canonical.com \ --to=heinrich.schuchardt@canonical.com \ --cc=Alistair.Francis@wdc.com \ --cc=anup@brainfault.org \ --cc=apatel@ventanamicro.com \ --cc=arnd@arndb.de \ --cc=atishp@atishpatra.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=palmer@dabbelt.com \ --cc=paul.walmsley@sifive.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.