From: Srikar Dronamraju <srikar@linux.vnet.ibm.com> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Vishal Chourasia <vishalc@linux.vnet.ibm.com>, Peter Zijlstra <peterz@infradead.org>, linux-kernel@vger.kernel.org, mingo@redhat.com, vincent.guittot@linaro.org, vschneid@redhat.com, sshegde@linux.ibm.com, linuxppc-dev@lists.ozlabs.org, ritesh.list@gmail.com, aneesh.kumar@linux.ibm.com Subject: Re: sched/debug: CPU hotplug operation suffers in a large cpu systems Date: Tue, 8 Nov 2022 20:21:00 +0530 [thread overview] Message-ID: <20221108145100.GG145013@linux.vnet.ibm.com> (raw) In-Reply-To: <Y2pKh3H0Ukvmfuco@kroah.com> * Greg Kroah-Hartman <gregkh@linuxfoundation.org> [2022-11-08 13:24:39]: > On Tue, Nov 08, 2022 at 03:30:46PM +0530, Vishal Chourasia wrote: Hi Greg, > > > > Thanks Greg & Peter for your direction. > > > > While we pursue the idea of having debugfs based on kernfs, we thought about > > having a boot time parameter which would disable creating and updating of the > > sched_domain debugfs files and this would also be useful even when the kernfs > > solution kicks in, as users who may not care about these debugfs files would > > benefit from a faster CPU hotplug operation. > > Ick, no, you would be adding a new user/kernel api that you will be > required to support for the next 20+ years. Just to get over a > short-term issue before you solve the problem properly. > > If you really do not want these debugfs files, just disable debugfs from > your system. That should be a better short-term solution, right? > > Or better yet, disable SCHED_DEBUG, why can't you do that? Thanks a lot for your quick inputs. CONFIG_SCHED_DEBUG disables a lot more stuff than just updation of debugfs files. Information like /sys/kernel/debug/sched/debug and system-wide and per process wide information would be lost when that config is disabled. Most users would still be using distribution kernels and most distribution kernels that I know of seem to have CONFIG_SCHED_DEBUG enabled. In a large system, lets say close to 2000 CPUs and we are offlining around 1750 CPUs. For example ppc64_cpu --smt=1 on a powerpc. Even if we move to a lesser overhead kernfs based implementation, we would still be creating files and deleting files for every CPU offline. Most users may not even be aware of these files. However for a few users who may be using these files once a while, we end up creating and deleting these files for all users. The overhead increases exponentially with the number of CPUs. I would assume the max number of CPUs are going to increase in future further. Hence our approach was to reduce the overhead for those users who are sure they don't depend on these files. We still keep the creating of the files as the default approach so that others who depend on it are not going to be impacted. > > thanks, > > greg k-h -- Thanks and Regards Srikar Dronamraju
WARNING: multiple messages have this Message-ID (diff)
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: ritesh.list@gmail.com, vschneid@redhat.com, vincent.guittot@linaro.org, Peter Zijlstra <peterz@infradead.org>, aneesh.kumar@linux.ibm.com, Vishal Chourasia <vishalc@linux.vnet.ibm.com>, linux-kernel@vger.kernel.org, sshegde@linux.ibm.com, mingo@redhat.com, linuxppc-dev@lists.ozlabs.org Subject: Re: sched/debug: CPU hotplug operation suffers in a large cpu systems Date: Tue, 8 Nov 2022 20:21:00 +0530 [thread overview] Message-ID: <20221108145100.GG145013@linux.vnet.ibm.com> (raw) In-Reply-To: <Y2pKh3H0Ukvmfuco@kroah.com> * Greg Kroah-Hartman <gregkh@linuxfoundation.org> [2022-11-08 13:24:39]: > On Tue, Nov 08, 2022 at 03:30:46PM +0530, Vishal Chourasia wrote: Hi Greg, > > > > Thanks Greg & Peter for your direction. > > > > While we pursue the idea of having debugfs based on kernfs, we thought about > > having a boot time parameter which would disable creating and updating of the > > sched_domain debugfs files and this would also be useful even when the kernfs > > solution kicks in, as users who may not care about these debugfs files would > > benefit from a faster CPU hotplug operation. > > Ick, no, you would be adding a new user/kernel api that you will be > required to support for the next 20+ years. Just to get over a > short-term issue before you solve the problem properly. > > If you really do not want these debugfs files, just disable debugfs from > your system. That should be a better short-term solution, right? > > Or better yet, disable SCHED_DEBUG, why can't you do that? Thanks a lot for your quick inputs. CONFIG_SCHED_DEBUG disables a lot more stuff than just updation of debugfs files. Information like /sys/kernel/debug/sched/debug and system-wide and per process wide information would be lost when that config is disabled. Most users would still be using distribution kernels and most distribution kernels that I know of seem to have CONFIG_SCHED_DEBUG enabled. In a large system, lets say close to 2000 CPUs and we are offlining around 1750 CPUs. For example ppc64_cpu --smt=1 on a powerpc. Even if we move to a lesser overhead kernfs based implementation, we would still be creating files and deleting files for every CPU offline. Most users may not even be aware of these files. However for a few users who may be using these files once a while, we end up creating and deleting these files for all users. The overhead increases exponentially with the number of CPUs. I would assume the max number of CPUs are going to increase in future further. Hence our approach was to reduce the overhead for those users who are sure they don't depend on these files. We still keep the creating of the files as the default approach so that others who depend on it are not going to be impacted. > > thanks, > > greg k-h -- Thanks and Regards Srikar Dronamraju
next prev parent reply other threads:[~2022-11-08 14:53 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-10-17 13:10 sched/debug: CPU hotplug operation suffers in a large cpu systems Vishal Chourasia 2022-10-17 14:19 ` Peter Zijlstra 2022-10-17 14:54 ` Greg Kroah-Hartman 2022-10-18 10:37 ` Vishal Chourasia 2022-10-18 11:04 ` Greg Kroah-Hartman 2022-10-18 11:04 ` Greg Kroah-Hartman 2022-10-26 6:37 ` Vishal Chourasia 2022-10-26 6:37 ` Vishal Chourasia 2022-10-26 7:02 ` Greg Kroah-Hartman 2022-10-26 7:02 ` Greg Kroah-Hartman 2022-10-26 9:10 ` Peter Zijlstra 2022-10-26 9:10 ` Peter Zijlstra 2022-11-08 10:00 ` Vishal Chourasia 2022-11-08 10:00 ` Vishal Chourasia 2022-11-08 12:24 ` Greg Kroah-Hartman 2022-11-08 12:24 ` Greg Kroah-Hartman 2022-11-08 14:51 ` Srikar Dronamraju [this message] 2022-11-08 14:51 ` Srikar Dronamraju 2022-11-08 15:38 ` Greg Kroah-Hartman 2022-11-08 15:38 ` Greg Kroah-Hartman 2022-12-12 19:17 ` Phil Auld 2022-12-12 19:17 ` Phil Auld 2022-12-13 2:17 ` kernel test robot 2022-12-13 2:17 ` kernel test robot 2022-12-13 6:23 ` Greg Kroah-Hartman 2022-12-13 6:23 ` Greg Kroah-Hartman 2022-12-13 13:22 ` Phil Auld 2022-12-13 13:22 ` Phil Auld 2022-12-13 14:31 ` Greg Kroah-Hartman 2022-12-13 14:31 ` Greg Kroah-Hartman 2022-12-13 14:45 ` Phil Auld 2022-12-13 14:45 ` Phil Auld 2023-01-19 15:31 ` Phil Auld 2023-01-19 15:31 ` Phil Auld 2022-12-13 23:41 ` Michael Ellerman 2022-12-13 23:41 ` Michael Ellerman 2022-12-14 2:26 ` Phil Auld 2022-12-14 2:26 ` Phil Auld
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20221108145100.GG145013@linux.vnet.ibm.com \ --to=srikar@linux.vnet.ibm.com \ --cc=aneesh.kumar@linux.ibm.com \ --cc=gregkh@linuxfoundation.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mingo@redhat.com \ --cc=peterz@infradead.org \ --cc=ritesh.list@gmail.com \ --cc=sshegde@linux.ibm.com \ --cc=vincent.guittot@linaro.org \ --cc=vishalc@linux.vnet.ibm.com \ --cc=vschneid@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.