From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932355AbcFTQhj (ORCPT ); Mon, 20 Jun 2016 12:37:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36952 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752091AbcFTQhc (ORCPT ); Mon, 20 Jun 2016 12:37:32 -0400 Date: Mon, 20 Jun 2016 18:27:41 +0200 From: Jiri Olsa To: Peter Zijlstra Cc: Jiri Olsa , Ingo Molnar , lkml , James Hartsock , Rik van Riel , Srivatsa Vaddagiri , Kirill Tkhai , Frederic Weisbecker Subject: Re: [PATCH 2/4] sched/fair: Introduce idle enter/exit balance callbacks Message-ID: <20160620162741.GB10167@krava> References: <1466424914-8981-1-git-send-email-jolsa@kernel.org> <1466424914-8981-3-git-send-email-jolsa@kernel.org> <20160620143003.GI30154@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160620143003.GI30154@twins.programming.kicks-ass.net> User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Mon, 20 Jun 2016 16:27:46 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Jun 20, 2016 at 04:30:03PM +0200, Peter Zijlstra wrote: > On Mon, Jun 20, 2016 at 02:15:12PM +0200, Jiri Olsa wrote: > > Introducing idle enter/exit balance callbacks to keep > > balance.idle_cpus_mask cpumask of current idle cpus > > in system. > > > > It's used only when REBALANCE_AFFINITY feature is > > switched on. The code functionality of this feature > > is introduced in following patch. > > > > Signed-off-by: Jiri Olsa > > --- > > kernel/sched/fair.c | 32 ++++++++++++++++++++++++++++++++ > > kernel/sched/idle.c | 2 ++ > > kernel/sched/sched.h | 3 +++ > > 3 files changed, 37 insertions(+) > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index f19c9435c64d..78c4127f2f3a 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -7802,6 +7802,37 @@ static inline int on_null_domain(struct rq *rq) > > return unlikely(!rcu_dereference_sched(rq->sd)); > > } > > > > +static struct { > > + cpumask_var_t idle_cpus_mask; > > + atomic_t nr_cpus; > > +} balance ____cacheline_aligned; > > How is this different from the nohz idle cpu mask? well, the nohz idle cpu mask is deep in the nohz code: tick_irq_exit if ((idle_cpu(cpu) && !need_resched()) || tick_nohz_full_cpu(cpu)) { if (!in_interrupt()) tick_nohz_irq_exit if (ts->inidle) { __tick_nohz_idle_enter(ts) tick_nohz_idle_enter __tick_nohz_idle_enter if (can_stop_idle_tick(cpu, ts)) { tick_nohz_stop_sched_tick if (ts->tick_stopped) nohz_balance_enter_idle(cpu) { set idle mask for cpu } } else { tick_nohz_full_update_tick(ts); if (can_stop_full_tick(ts)) tick_nohz_stop_sched_tick if (ts->tick_stopped) nohz_balance_enter_idle(cpu) { set idle mask for cpu } } cpu_idle_loop tick_nohz_idle_enter __tick_nohz_idle_enter(ts) tick_nohz_idle_enter __tick_nohz_idle_enter if (can_stop_idle_tick(cpu, ts)) { tick_nohz_stop_sched_tick if (ts->tick_stopped) nohz_balance_enter_idle(cpu) { set idle mask for cpu } sched_cpu_dying nohz_balance_exit_idle(cpu) { unset idle mask for cpu } trigger_load_balance nohz_kick_needed nohz_balance_exit_idle(cpu) { unset idle mask for cpu } ... I might have missed some of the paths so it might not be that easy to switch it to use the easy change I added as part of this RFC, but AFAIU it should be the same idle mask, but this approach might be too naive and miss some idle enter/exit paths.. CC-ing Frederic thanks, jirka