From: Qais Yousef <qyousef@layalina.io> To: Juri Lelli <juri.lelli@redhat.com> Cc: peterz@infradead.org, mingo@redhat.com, rostedt@goodmis.org, tj@kernel.org, linux-kernel@vger.kernel.org, luca.abeni@santannapisa.it, claudio@evidence.eu.com, tommaso.cucinotta@santannapisa.it, bristot@redhat.com, mathieu.poirier@linaro.org, lizefan@huawei.com, longman@redhat.com, dietmar.eggemann@arm.com, cgroups@vger.kernel.org, Vincent Guittot <vincent.guittot@linaro.org>, Wei Wang <wvw@google.com>, Rick Yiu <rickyiu@google.com>, Quentin Perret <qperret@google.com> Subject: Re: [PATCH v9 3/8] cpuset: Rebuild root domain deadline accounting information Date: Fri, 16 Dec 2022 23:35:01 +0000 [thread overview] Message-ID: <20221216233501.gh6m75e7s66dmjgo@airbuntu> (raw) In-Reply-To: <20190719140000.31694-4-juri.lelli@redhat.com> Hi On 07/19/19 15:59, Juri Lelli wrote: > When the topology of root domains is modified by CPUset or CPUhotplug > operations information about the current deadline bandwidth held in the > root domain is lost. > > This patch addresses the issue by recalculating the lost deadline > bandwidth information by circling through the deadline tasks held in > CPUsets and adding their current load to the root domain they are > associated with. > > Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org> > Signed-off-by: Juri Lelli <juri.lelli@redhat.com> > --- We see that rebuild_root_domain() can take 10+ ms (I get a max of 20ms quite consistently) on suspend/resume. Do we actually need to rebuild_root_domain() if we're going through a suspend/resume cycle? ie: would something like the below make sense? We'd skip this logic if cpuhp_tasks_frozen is set which indicates it's not a real hotplug operation but we're suspending/resuming. Cheers -- Qais Yousef --->8--- From 4cfd50960ad872c5eb810ad3038eaf840bab5182 Mon Sep 17 00:00:00 2001 From: Qais Yousef <qyousef@layalina.io> Date: Tue, 29 Nov 2022 19:01:52 +0000 Subject: [PATCH] sched: cpuset: Don't rebuild sched domains on suspend-resume Commit f9a25f776d78 ("cpusets: Rebuild root domain deadline accounting information") enabled rebuilding sched domain on cpuset and hotplug operations to correct deadline accounting. Rebuilding sched domain is a slow operation and we see 10+ ms delays on suspend-resume because of that. Since nothing is expected to change on suspend-resume operation; skip rebuilding the sched domains to regain some of the time lost. Debugged-by: Rick Yiu <rickyiu@google.com> Signed-off-by: Qais Yousef (Google) <qyousef@layalina.io> --- kernel/cgroup/cpuset.c | 6 ++++++ kernel/sched/deadline.c | 3 +++ 2 files changed, 9 insertions(+) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index b474289c15b8..2ff68d625b7b 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -1067,6 +1067,9 @@ static void update_tasks_root_domain(struct cpuset *cs) struct css_task_iter it; struct task_struct *task; + if (cpuhp_tasks_frozen) + return; + css_task_iter_start(&cs->css, 0, &it); while ((task = css_task_iter_next(&it))) @@ -1084,6 +1087,9 @@ static void rebuild_root_domains(void) lockdep_assert_cpus_held(); lockdep_assert_held(&sched_domains_mutex); + if (cpuhp_tasks_frozen) + return; + rcu_read_lock(); /* diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 0d97d54276cc..42c1143a3956 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2575,6 +2575,9 @@ void dl_clear_root_domain(struct root_domain *rd) { unsigned long flags; + if (cpuhp_tasks_frozen) + return; + raw_spin_lock_irqsave(&rd->dl_bw.lock, flags); rd->dl_bw.total_bw = 0; raw_spin_unlock_irqrestore(&rd->dl_bw.lock, flags); -- 2.25.1
WARNING: multiple messages have this Message-ID (diff)
From: Qais Yousef <qyousef-wp2msK0BRk8tq7phqP6ubQ@public.gmane.org> To: Juri Lelli <juri.lelli-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> Cc: peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org, mingo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, rostedt-nx8X9YLhiw1AfugRpC6u6w@public.gmane.org, tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, luca.abeni-5rdYK369eBLQB0XuIGIEkQ@public.gmane.org, claudio-YOzL5CV4y4YG1A2ADO40+w@public.gmane.org, tommaso.cucinotta-5rdYK369eBLQB0XuIGIEkQ@public.gmane.org, bristot-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, mathieu.poirier-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org, lizefan-hv44wF8Li93QT0dZR+AlfA@public.gmane.org, longman-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, dietmar.eggemann-5wv7dgnIgG8@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Vincent Guittot <vincent.guittot-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>, Wei Wang <wvw-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Rick Yiu <rickyiu-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Quentin Perret <qperret-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> Subject: Re: [PATCH v9 3/8] cpuset: Rebuild root domain deadline accounting information Date: Fri, 16 Dec 2022 23:35:01 +0000 [thread overview] Message-ID: <20221216233501.gh6m75e7s66dmjgo@airbuntu> (raw) In-Reply-To: <20190719140000.31694-4-juri.lelli-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> Hi On 07/19/19 15:59, Juri Lelli wrote: > When the topology of root domains is modified by CPUset or CPUhotplug > operations information about the current deadline bandwidth held in the > root domain is lost. > > This patch addresses the issue by recalculating the lost deadline > bandwidth information by circling through the deadline tasks held in > CPUsets and adding their current load to the root domain they are > associated with. > > Signed-off-by: Mathieu Poirier <mathieu.poirier-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> > Signed-off-by: Juri Lelli <juri.lelli-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> > --- We see that rebuild_root_domain() can take 10+ ms (I get a max of 20ms quite consistently) on suspend/resume. Do we actually need to rebuild_root_domain() if we're going through a suspend/resume cycle? ie: would something like the below make sense? We'd skip this logic if cpuhp_tasks_frozen is set which indicates it's not a real hotplug operation but we're suspending/resuming. Cheers -- Qais Yousef --->8--- From 4cfd50960ad872c5eb810ad3038eaf840bab5182 Mon Sep 17 00:00:00 2001 From: Qais Yousef <qyousef-wp2msK0BRk8tq7phqP6ubQ@public.gmane.org> Date: Tue, 29 Nov 2022 19:01:52 +0000 Subject: [PATCH] sched: cpuset: Don't rebuild sched domains on suspend-resume Commit f9a25f776d78 ("cpusets: Rebuild root domain deadline accounting information") enabled rebuilding sched domain on cpuset and hotplug operations to correct deadline accounting. Rebuilding sched domain is a slow operation and we see 10+ ms delays on suspend-resume because of that. Since nothing is expected to change on suspend-resume operation; skip rebuilding the sched domains to regain some of the time lost. Debugged-by: Rick Yiu <rickyiu-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> Signed-off-by: Qais Yousef (Google) <qyousef-wp2msK0BRk8tq7phqP6ubQ@public.gmane.org> --- kernel/cgroup/cpuset.c | 6 ++++++ kernel/sched/deadline.c | 3 +++ 2 files changed, 9 insertions(+) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index b474289c15b8..2ff68d625b7b 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -1067,6 +1067,9 @@ static void update_tasks_root_domain(struct cpuset *cs) struct css_task_iter it; struct task_struct *task; + if (cpuhp_tasks_frozen) + return; + css_task_iter_start(&cs->css, 0, &it); while ((task = css_task_iter_next(&it))) @@ -1084,6 +1087,9 @@ static void rebuild_root_domains(void) lockdep_assert_cpus_held(); lockdep_assert_held(&sched_domains_mutex); + if (cpuhp_tasks_frozen) + return; + rcu_read_lock(); /* diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 0d97d54276cc..42c1143a3956 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2575,6 +2575,9 @@ void dl_clear_root_domain(struct root_domain *rd) { unsigned long flags; + if (cpuhp_tasks_frozen) + return; + raw_spin_lock_irqsave(&rd->dl_bw.lock, flags); rd->dl_bw.total_bw = 0; raw_spin_unlock_irqrestore(&rd->dl_bw.lock, flags); -- 2.25.1
next prev parent reply other threads:[~2022-12-16 23:35 UTC|newest] Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-07-19 13:59 [PATCH v9 0/8] sched/deadline: fix cpusets bandwidth accounting Juri Lelli 2019-07-19 13:59 ` [PATCH v9 1/8] sched/topology: Adding function partition_sched_domains_locked() Juri Lelli 2019-07-25 16:19 ` [tip:sched/core] sched/topology: Add partition_sched_domains_locked() tip-bot for Mathieu Poirier 2019-07-19 13:59 ` [PATCH v9 2/8] sched/core: Streamlining calls to task_rq_unlock() Juri Lelli 2019-07-22 8:21 ` Dietmar Eggemann 2019-07-22 8:32 ` Juri Lelli 2019-07-22 9:08 ` Dietmar Eggemann 2019-07-23 10:32 ` Peter Zijlstra 2019-07-23 10:31 ` Peter Zijlstra 2019-07-23 13:11 ` Tejun Heo 2019-07-23 14:58 ` Juri Lelli 2019-07-25 16:20 ` [tip:sched/core] sched/core: Streamle " tip-bot for Mathieu Poirier 2019-07-19 13:59 ` [PATCH v9 3/8] cpuset: Rebuild root domain deadline accounting information Juri Lelli 2019-07-25 13:53 ` Ingo Molnar 2019-07-25 13:56 ` Ingo Molnar 2019-07-25 14:07 ` Juri Lelli 2019-07-25 16:00 ` Mathieu Poirier 2019-07-25 16:21 ` [tip:sched/core] cpusets: " tip-bot for Mathieu Poirier 2019-11-04 18:57 ` [PATCH v9 3/8] cpuset: " Michal Koutný 2019-11-05 12:03 ` Michal Koutný 2022-12-16 23:35 ` Qais Yousef [this message] 2022-12-16 23:35 ` Qais Yousef 2022-12-17 2:26 ` Waiman Long 2022-12-17 2:26 ` Waiman Long 2022-12-17 22:10 ` Qais Yousef 2022-12-17 22:10 ` Qais Yousef 2022-12-19 8:07 ` Vincent Guittot 2022-12-19 8:07 ` Vincent Guittot 2022-12-20 11:43 ` Qais Yousef 2022-12-20 11:43 ` Qais Yousef 2023-01-15 16:41 ` Qais Yousef 2023-01-15 16:41 ` Qais Yousef 2019-07-19 13:59 ` [PATCH v9 4/8] sched/deadline: Fix bandwidth accounting at all levels after offline migration Juri Lelli 2019-07-22 11:07 ` Dietmar Eggemann 2019-07-22 12:28 ` Juri Lelli 2019-07-22 13:21 ` Dietmar Eggemann 2019-07-22 13:35 ` Juri Lelli 2019-07-22 14:29 ` Dietmar Eggemann 2019-07-25 16:21 ` [tip:sched/core] " tip-bot for Juri Lelli 2019-07-19 13:59 ` [PATCH v9 5/8] cgroup/cpuset: convert cpuset_mutex to percpu_rwsem Juri Lelli 2019-07-25 16:22 ` [tip:sched/core] cgroup/cpuset: Convert " tip-bot for Juri Lelli 2019-07-19 13:59 ` [PATCH v9 6/8] cgroup/cpuset: Change cpuset_rwsem and hotplug lock order Juri Lelli 2019-07-25 16:23 ` [tip:sched/core] " tip-bot for Juri Lelli 2019-07-19 13:59 ` [PATCH v9 7/8] rcu/tree: Setschedule gp ktread to SCHED_FIFO outside of atomic region Juri Lelli 2019-07-25 16:24 ` [tip:sched/core] rcu/tree: Call setschedule() " tip-bot for Juri Lelli 2019-07-19 14:00 ` [PATCH v9 8/8] sched/core: Prevent race condition between cpuset and __sched_setscheduler() Juri Lelli 2019-07-25 16:24 ` [tip:sched/core] " tip-bot for Juri Lelli 2019-07-24 8:45 ` [PATCH v9 0/8] sched/deadline: fix cpusets bandwidth accounting Dietmar Eggemann
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20221216233501.gh6m75e7s66dmjgo@airbuntu \ --to=qyousef@layalina.io \ --cc=bristot@redhat.com \ --cc=cgroups@vger.kernel.org \ --cc=claudio@evidence.eu.com \ --cc=dietmar.eggemann@arm.com \ --cc=juri.lelli@redhat.com \ --cc=linux-kernel@vger.kernel.org \ --cc=lizefan@huawei.com \ --cc=longman@redhat.com \ --cc=luca.abeni@santannapisa.it \ --cc=mathieu.poirier@linaro.org \ --cc=mingo@redhat.com \ --cc=peterz@infradead.org \ --cc=qperret@google.com \ --cc=rickyiu@google.com \ --cc=rostedt@goodmis.org \ --cc=tj@kernel.org \ --cc=tommaso.cucinotta@santannapisa.it \ --cc=vincent.guittot@linaro.org \ --cc=wvw@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.