From: Libo Chen <libo.chen@oracle.com> To: gregkh@linuxfoundation.org, masahiroy@kernel.org, tglx@linutronix.de, peterz@infradead.org, mingo@kernel.org, vbabka@suse.cz, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org Subject: [PATCH RESEND 0/1] lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK Date: Tue, 12 Apr 2022 16:15:07 -0700 [thread overview] Message-ID: <20220412231508.32629-1-libo.chen@oracle.com> (raw) We encountered an issue related to NR_CPUMASK_BITS on ARM64. It turned out CPUMASK_OFFSTACK was not enabled and NR_CPUS is set to 4096 in our aarch64 config, so each cpumask operation is unnecessarily expensive esp. on small VMs. There are decent number of cpumask operations in the scheduler. Many of them are on the hot paths. The cumulative effect is quite siginificant. With CONFIG_CPUMASK_OFFSTACKT set, we saw ~10% gain on ApacheBench and a few percentage on redis in a 32-core ARM64 VM with 5.16 kernel. One may argue why not just set NR_CPUS to smaller values, the thing is, for example, we support VMs with flexible shapes. The customer can specify the number of CPUs ranging from 1 to 160. You cannot just prepare 160 kernel images for each release with one for each shape. And kernel with NR_CPUS=160 doesn't perform well on a 1-CPU VM. It's important that we can set CPUMASK_OFFSTACK=y so that the kernel can dynamically allocate cpumasks based on the actual number of CPUs in the system. The problem I am trying to address in this patch is currently CPUMASK_OFFSTACK depends on DEBUG_PER_CPU_MAPS except for x86 and it doesn't even show up in kconfig menu as well as kconfig without DEBUG_PER_CPU_MAPS=y. We should remove such outdated, unnecessary dependency. Of course, I am open to other ideas. And people who are more familar with this matter can shed a light on why this dependency has been kept for so long. My goal here is to determine if DEBUG_PER_CPU_MAPS is absoultely necessary for CPUMASK_OFFSTACK. RESEND: - Correct the subject line of cover letter - Send to more people, esp. gkh for more attention since no one seems to be responsible for this part of code Libo Chen (1): lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.27.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
WARNING: multiple messages have this Message-ID (diff)
From: Libo Chen <libo.chen@oracle.com> To: gregkh@linuxfoundation.org, masahiroy@kernel.org, tglx@linutronix.de, peterz@infradead.org, mingo@kernel.org, vbabka@suse.cz, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org Subject: [PATCH RESEND 0/1] lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK Date: Tue, 12 Apr 2022 16:15:07 -0700 [thread overview] Message-ID: <20220412231508.32629-1-libo.chen@oracle.com> (raw) We encountered an issue related to NR_CPUMASK_BITS on ARM64. It turned out CPUMASK_OFFSTACK was not enabled and NR_CPUS is set to 4096 in our aarch64 config, so each cpumask operation is unnecessarily expensive esp. on small VMs. There are decent number of cpumask operations in the scheduler. Many of them are on the hot paths. The cumulative effect is quite siginificant. With CONFIG_CPUMASK_OFFSTACKT set, we saw ~10% gain on ApacheBench and a few percentage on redis in a 32-core ARM64 VM with 5.16 kernel. One may argue why not just set NR_CPUS to smaller values, the thing is, for example, we support VMs with flexible shapes. The customer can specify the number of CPUs ranging from 1 to 160. You cannot just prepare 160 kernel images for each release with one for each shape. And kernel with NR_CPUS=160 doesn't perform well on a 1-CPU VM. It's important that we can set CPUMASK_OFFSTACK=y so that the kernel can dynamically allocate cpumasks based on the actual number of CPUs in the system. The problem I am trying to address in this patch is currently CPUMASK_OFFSTACK depends on DEBUG_PER_CPU_MAPS except for x86 and it doesn't even show up in kconfig menu as well as kconfig without DEBUG_PER_CPU_MAPS=y. We should remove such outdated, unnecessary dependency. Of course, I am open to other ideas. And people who are more familar with this matter can shed a light on why this dependency has been kept for so long. My goal here is to determine if DEBUG_PER_CPU_MAPS is absoultely necessary for CPUMASK_OFFSTACK. RESEND: - Correct the subject line of cover letter - Send to more people, esp. gkh for more attention since no one seems to be responsible for this part of code Libo Chen (1): lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.27.0
next reply other threads:[~2022-04-12 23:17 UTC|newest] Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-04-12 23:15 Libo Chen [this message] 2022-04-12 23:15 ` [PATCH RESEND 0/1] lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK Libo Chen 2022-04-12 23:15 ` [PATCH RESEND 1/1] " Libo Chen 2022-04-12 23:15 ` Libo Chen 2022-04-13 0:18 ` Randy Dunlap 2022-04-13 0:18 ` Randy Dunlap 2022-04-13 1:35 ` Libo Chen 2022-04-13 1:35 ` Libo Chen 2022-04-13 2:13 ` Randy Dunlap 2022-04-13 2:13 ` Randy Dunlap 2022-04-13 2:34 ` Libo Chen 2022-04-13 2:34 ` Libo Chen 2022-04-13 5:54 ` Randy Dunlap 2022-04-13 5:54 ` Randy Dunlap 2022-04-13 6:56 ` Libo Chen 2022-04-13 6:56 ` Libo Chen 2022-04-13 8:37 ` Masahiro Yamada 2022-04-13 8:37 ` Masahiro Yamada 2022-04-13 15:41 ` Randy Dunlap 2022-04-13 15:41 ` Randy Dunlap 2022-04-13 19:28 ` Libo Chen 2022-04-13 19:28 ` Libo Chen 2022-04-13 20:52 ` Arnd Bergmann 2022-04-13 20:52 ` Arnd Bergmann 2022-04-13 21:50 ` Libo Chen 2022-04-13 21:50 ` Libo Chen 2022-04-14 1:20 ` Randy Dunlap 2022-04-14 1:20 ` Randy Dunlap 2022-04-14 11:41 ` Arnd Bergmann 2022-04-14 11:41 ` Arnd Bergmann 2022-04-14 18:01 ` Libo Chen 2022-04-14 18:01 ` Libo Chen 2022-04-13 13:11 ` kernel test robot 2022-04-13 13:11 ` kernel test robot 2022-04-13 14:33 ` kernel test robot 2022-04-13 14:33 ` kernel test robot
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220412231508.32629-1-libo.chen@oracle.com \ --to=libo.chen@oracle.com \ --cc=akpm@linux-foundation.org \ --cc=gregkh@linuxfoundation.org \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kbuild@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=masahiroy@kernel.org \ --cc=mingo@kernel.org \ --cc=peterz@infradead.org \ --cc=tglx@linutronix.de \ --cc=vbabka@suse.cz \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.