From: Tim Chen <tim.c.chen@linux.intel.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>,
rjw@rjwysocki.net, tglx@linutronix.de, mingo@redhat.com,
bp@suse.de, x86@kernel.org, linux-pm@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org
Subject: Re: [PATCH v2 2/8] sched: Extend scheduler's asym packing
Date: Thu, 8 Sep 2016 11:03:26 -0700 [thread overview]
Message-ID: <20160908180325.GA23509@linux.intel.com> (raw)
In-Reply-To: <20160908080111.GT10153@twins.programming.kicks-ass.net>
On Thu, Sep 08, 2016 at 10:01:11AM +0200, Peter Zijlstra wrote:
> On Thu, Sep 01, 2016 at 01:33:38PM -0700, Srinivas Pandruvada wrote:
> > +++ b/kernel/sched/sched.h
> > @@ -532,6 +532,17 @@ struct dl_rq {
> >
> > #ifdef CONFIG_SMP
> >
> > +static inline bool sched_asym_prefer(int a, int b)
> > +{
> > + return arch_asym_cpu_priority(a) > arch_asym_cpu_priority(b);
> > +}
> > +
> > +/*
> > + * Return lowest numbered cpu in the group as the most preferred cpu
> > + * for ASYM_PACKING for default case.
> > + */
> > +#define group_priority_cpu(group) group->asym_prefer_cpu
> > +
>
> Both these are no longer used outside of sched/fair.c, and should
> probably be moved there. Also the later has no reason for being a macro.
Peter,
The sched_asym_prefer is used in sched/core.c to set asym_prefer_cpu
in a group. So we will still need the definition in sched.h. I've removed
the group_priority_cpu macro.
Updated patch below.
Thanks.
Tim
--->8---
From: Tim Chen <tim.c.chen@linux.intel.com>
Subject: [PATCH 2/8] sched: Extend scheduler's asym packing
We generalize the scheduler's asym packing to provide an ordering
of the cpu beyond just the cpu number. This allows the use of the
ASYM_PACKING scheduler machinery to move loads to preferred CPU in a
sched domain. The preference is defined with the cpu priority
given by arch_asym_cpu_priority(cpu).
We also record the most preferred cpu in a sched group when
we build the cpu's capacity for fast lookup of preferred cpu
during load balancing.
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
---
include/linux/sched.h | 2 ++
kernel/sched/core.c | 18 ++++++++++++++++++
kernel/sched/fair.c | 35 ++++++++++++++++++++++++-----------
kernel/sched/sched.h | 6 ++++++
4 files changed, 50 insertions(+), 11 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 62c68e5..aeea288 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1052,6 +1052,8 @@ static inline int cpu_numa_flags(void)
}
#endif
+int arch_asym_cpu_priority(int cpu);
+
struct sched_domain_attr {
int relax_domain_level;
};
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index e86c4a5..08135ca 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6237,7 +6237,25 @@ static void init_sched_groups_capacity(int cpu, struct sched_domain *sd)
WARN_ON(!sg);
do {
+ int cpu, max_cpu = -1, prev_cpu = -1;
+
sg->group_weight = cpumask_weight(sched_group_cpus(sg));
+
+ if (!(sd->flags & SD_ASYM_PACKING))
+ goto next;
+
+ for_each_cpu(cpu, sched_group_cpus(sg)) {
+ if (prev_cpu < 0) {
+ prev_cpu = cpu;
+ max_cpu = cpu;
+ } else {
+ if (sched_asym_prefer(cpu, max_cpu))
+ max_cpu = cpu;
+ }
+ }
+ sg->asym_prefer_cpu = max_cpu;
+
+next:
sg = sg->next;
} while (sg != sd->groups);
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 039de34..8e2a078 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -100,6 +100,16 @@ const_debug unsigned int sysctl_sched_migration_cost = 500000UL;
*/
unsigned int __read_mostly sysctl_sched_shares_window = 10000000UL;
+#ifdef CONFIG_SMP
+/*
+ * For asym packing, by default the lower numbered cpu has higher priority.
+ */
+int __weak arch_asym_cpu_priority(int cpu)
+{
+ return -cpu;
+}
+#endif
+
#ifdef CONFIG_CFS_BANDWIDTH
/*
* Amount of runtime to allocate from global (tg) to local (per-cfs_rq) pool
@@ -6862,16 +6872,18 @@ static bool update_sd_pick_busiest(struct lb_env *env,
if (env->idle == CPU_NOT_IDLE)
return true;
/*
- * ASYM_PACKING needs to move all the work to the lowest
- * numbered CPUs in the group, therefore mark all groups
- * higher than ourself as busy.
+ * ASYM_PACKING needs to move all the work to the highest
+ * prority CPUs in the group, therefore mark all groups
+ * of lower priority than ourself as busy.
*/
- if (sgs->sum_nr_running && env->dst_cpu < group_first_cpu(sg)) {
+ if (sgs->sum_nr_running &&
+ sched_asym_prefer(env->dst_cpu, sg->asym_prefer_cpu)) {
if (!sds->busiest)
return true;
- /* Prefer to move from highest possible cpu's work */
- if (group_first_cpu(sds->busiest) < group_first_cpu(sg))
+ /* Prefer to move from lowest priority cpu's work */
+ if (sched_asym_prefer(sds->busiest->asym_prefer_cpu,
+ sg->asym_prefer_cpu))
return true;
}
@@ -7023,8 +7035,8 @@ static int check_asym_packing(struct lb_env *env, struct sd_lb_stats *sds)
if (!sds->busiest)
return 0;
- busiest_cpu = group_first_cpu(sds->busiest);
- if (env->dst_cpu > busiest_cpu)
+ busiest_cpu = sds->busiest->asym_prefer_cpu;
+ if (sched_asym_prefer(busiest_cpu, env->dst_cpu))
return 0;
env->imbalance = DIV_ROUND_CLOSEST(
@@ -7365,10 +7377,11 @@ static int need_active_balance(struct lb_env *env)
/*
* ASYM_PACKING needs to force migrate tasks from busy but
- * higher numbered CPUs in order to pack all tasks in the
- * lowest numbered CPUs.
+ * lower priority CPUs in order to pack all tasks in the
+ * highest priority CPUs.
*/
- if ((sd->flags & SD_ASYM_PACKING) && env->src_cpu > env->dst_cpu)
+ if ((sd->flags & SD_ASYM_PACKING) &&
+ sched_asym_prefer(env->dst_cpu, env->src_cpu))
return 1;
}
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index c64fc51..b6f449d 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -532,6 +532,11 @@ struct dl_rq {
#ifdef CONFIG_SMP
+static inline bool sched_asym_prefer(int a, int b)
+{
+ return arch_asym_cpu_priority(a) > arch_asym_cpu_priority(b);
+}
+
/*
* We add the notion of a root-domain which will be used to define per-domain
* variables. Each exclusive cpuset essentially defines an island domain by
@@ -884,6 +889,7 @@ struct sched_group {
unsigned int group_weight;
struct sched_group_capacity *sgc;
+ int asym_prefer_cpu; /* cpu of highest priority in group */
/*
* The CPUs this group covers.
--
2.5.5
next prev parent reply other threads:[~2016-09-08 18:03 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-01 20:33 [PATCH v2 0/8] Support Intel® Turbo Boost Max Technology 3.0 Srinivas Pandruvada
2016-09-01 20:33 ` [PATCH v2 1/8] sched, x86: Add SD_ASYM_PACKING flags to x86 cpu topology for ITMT Srinivas Pandruvada
2016-09-01 20:33 ` [PATCH v2 2/8] sched: Extend scheduler's asym packing Srinivas Pandruvada
2016-09-08 8:01 ` Peter Zijlstra
2016-09-08 18:03 ` Tim Chen [this message]
2016-09-08 18:21 ` Peter Zijlstra
2016-09-01 20:33 ` [PATCH v2 3/8] x86, cpu: provide a function topology_num_packages to enumerate #packages Srinivas Pandruvada
2016-09-01 20:33 ` [PATCH v2 4/8] sched, x86: use arch_update_cpu_topology to indicate x86 need sched domain rebuild Srinivas Pandruvada
2016-09-01 20:33 ` [PATCH v2 5/8] sched,x86: Enable Turbo Boost Max Technology Srinivas Pandruvada
2016-09-08 7:59 ` Peter Zijlstra
2016-09-08 18:09 ` Tim Chen
2016-09-08 18:22 ` Peter Zijlstra
2016-09-08 18:28 ` Srinivas Pandruvada
2016-09-08 18:33 ` Peter Zijlstra
2016-09-08 18:41 ` Srinivas Pandruvada
2016-09-08 21:22 ` Rafael J. Wysocki
2016-09-01 20:33 ` [PATCH v2 6/8] acpi: bus: Enable HWP CPPC objects Srinivas Pandruvada
2016-09-01 23:15 ` kbuild test robot
2016-09-01 23:46 ` Srinivas Pandruvada
2016-09-01 20:33 ` [PATCH v2 7/8] acpi: bus: Set _OSC for diverse core support Srinivas Pandruvada
2016-09-01 20:33 ` [PATCH v2 8/8] cpufreq: intel_pstate: Use CPPC to get max performance Srinivas Pandruvada
2016-09-01 22:31 ` kbuild test robot
2016-09-08 8:02 ` [PATCH v2 0/8] Support Intel® Turbo Boost Max Technology 3.0 Peter Zijlstra
2016-09-15 12:14 ` Pavel Machek
2016-09-15 12:46 ` One Thousand Gnomes
2016-09-16 16:03 ` Tim Chen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160908180325.GA23509@linux.intel.com \
--to=tim.c.chen@linux.intel.com \
--cc=bp@suse.de \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rjw@rjwysocki.net \
--cc=srinivas.pandruvada@linux.intel.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).