From: Scott Wood <swood@redhat.com>
To: Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Vincent Guittot <vincent.guittot@linaro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Mel Gorman <mgorman@suse.de>,
Valentin Schneider <valentin.schneider@arm.com>,
linux-kernel@vger.kernel.org, linux-rt-users@vger.kernel.org,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Thomas Gleixner <tglx@linutronix.de>,
Scott Wood <swood@redhat.com>
Subject: [PATCH v2 2/3] sched/fair: Enable interrupts when dropping lock in newidle_balance()
Date: Wed, 28 Apr 2021 18:28:20 -0500 [thread overview]
Message-ID: <20210428232821.2506201-3-swood@redhat.com> (raw)
In-Reply-To: <20210428232821.2506201-1-swood@redhat.com>
When combined with the next patch, which breaks out of rebalancing
when an RT task is runnable, significant latency reductions are seen
on systems with many CPUs.
Signed-off-by: Scott Wood <swood@redhat.com>
---
kernel/sched/fair.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index ff369c38a5b5..aa8c87b6aff8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -10521,6 +10521,8 @@ static void nohz_newidle_balance(struct rq *this_rq)
return;
raw_spin_unlock(&this_rq->lock);
+ if (newidle_balance_in_callback)
+ local_irq_enable();
/*
* This CPU is going to be idle and blocked load of idle CPUs
* need to be updated. Run the ilb locally as it is a good
@@ -10529,6 +10531,8 @@ static void nohz_newidle_balance(struct rq *this_rq)
*/
if (!_nohz_idle_balance(this_rq, NOHZ_STATS_KICK, CPU_NEWLY_IDLE))
kick_ilb(NOHZ_STATS_KICK);
+ if (newidle_balance_in_callback)
+ local_irq_disable();
raw_spin_lock(&this_rq->lock);
}
@@ -10599,6 +10603,8 @@ static int do_newidle_balance(struct rq *this_rq, struct rq_flags *rf)
}
raw_spin_unlock(&this_rq->lock);
+ if (newidle_balance_in_callback)
+ local_irq_enable();
update_blocked_averages(this_cpu);
rcu_read_lock();
@@ -10636,6 +10642,8 @@ static int do_newidle_balance(struct rq *this_rq, struct rq_flags *rf)
}
rcu_read_unlock();
+ if (newidle_balance_in_callback)
+ local_irq_disable();
raw_spin_lock(&this_rq->lock);
if (curr_cost > this_rq->max_idle_balance_cost)
--
2.27.0
next prev parent reply other threads:[~2021-04-28 23:28 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-28 23:28 [PATCH v2 0/3] newidle_balance() PREEMPT_RT latency mitigations Scott Wood
2021-04-28 23:28 ` [PATCH v2 1/3] sched/fair: Call newidle_balance() from balance_callback on PREEMPT_RT Scott Wood
2021-05-05 12:13 ` Vincent Guittot
2021-05-07 15:19 ` Valentin Schneider
2021-04-28 23:28 ` Scott Wood [this message]
2021-04-28 23:28 ` [PATCH v2 3/3] sched/fair: break out of newidle balancing if an RT task appears Scott Wood
2021-04-29 4:11 ` kernel test robot
2021-04-29 6:37 ` kernel test robot
2021-05-07 11:03 ` Peter Zijlstra
2021-05-15 7:29 ` Mike Galbraith
2021-05-15 8:36 ` Mike Galbraith
2021-04-29 7:12 ` [PATCH v2 0/3] newidle_balance() PREEMPT_RT latency mitigations Vincent Guittot
2021-05-01 22:03 ` Scott Wood
2021-05-02 3:25 ` Mike Galbraith
2021-05-03 16:33 ` Scott Wood
2021-05-03 18:52 ` Mike Galbraith
2021-05-03 21:57 ` Scott Wood
2021-05-04 4:07 ` Mike Galbraith
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210428232821.2506201-3-swood@redhat.com \
--to=swood@redhat.com \
--cc=bigeasy@linutronix.de \
--cc=dietmar.eggemann@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).