[PATCH 00/13] High performance balancing logic for big.LITTLE

[PATCH 00/13] High performance balancing logic for big.LITTLE

[Arseniy Krasnov]

				Prologue

The patch set introduces an extension to the default Linux CPU scheduler (CFS).
The main purpose of the extension is utilization of a big.LITTLE CPU for maximum
performance. Such solution may be useful for users of OdroidXU-3 board
(supporting 8 cores) who doesn't care about power efficiency.

Maximum utilization was reached using the following policies:

1) A15 cores must be utilized as much as possible e.g. idle A15 cores always
   pull some task from A7 core.

2) After execution of a task on A7 core for some period of time it should be
   swapped with an appropriate task from A15 cluster in order to achieve
   fairness.

3) Load of big and little clusters is balanced according to frequency and
   A15/A7 slowdown coefficient.

			Approach Description

The scheduler creates a hierarchy of two domains: MC and HMP. The MC domain is a
default domain for SCHED_MC config. The HMP domain contains two clusters: A15
and A7 CPUs. Balancing between HMP domains is performed by the new logic, in MC
domains, in turn, balancing is done by the default logic of the 'load_balance()'
function.

To perform balancing between HMP domains, the load of each cluster is calculated
in scheduler's softirq handler. Then, this value is scaled according to each
cluster's frequency and slowdown coefficient which is a ratio of busy-loop
performance on A15 and A7. There are three ways of migration between two
clusters: from A15 cluster to A7 cluster (if load on A15 cluster is too high),
from A7 cluster to A15 cluster (otherwise) and task swapping when load on both
clusters is the same. To migrate some task from one cluster to another firstly
this task should be selected. To find a task suitable for migration the
scheduler uses a special per-task metric called 'druntime'. It is based on CFS's
vruntime metric but its grow direction depends on a core where the task is
executed: for A15 core it grows up, for A7 core, in turn, it goes down. So,
being the druntime value close to zero means that the task is executed on both
clusters for the same amount of time. As a result, to get a task for migration
it scans each runqueue to find a task with highest/lowest druntime depending on
which cluster is scanned; after, when the task is found, it is moved to another
cluster. These balancing steps are performed in each scheduler balancing
operation executed by softirq.

To get maximum performance A15 cores must be fully utilized; this means that
idle A15 cores are always able to pull tasks from A7 cores while A7 cores cannot
do that from A15 cores.

An finally, let's look to fairness - it is provided by swapping of tasks during
every softirq balancing: when balance is broken it tries to repair the balance
moving tasks from one cluster to another, then when the clusters are balanced,
the tasks are swapped during each softirq balancing. In addition to this logic,
'select_task_rq_fair' was modified in order to place woken tasks to least loaded
CPU, because it won't break the balance between A15 and A7 cores.

				Test results

Several test kits were used for performance measurement of the solution.
All comparision is done against the Linaro MP scheduler.

The first test case is a parsec benchmark suite. It contains different types of
tasks like cluster searching or pattern recognition in order to test scheduler
performance. Results of some benchmarks are listed in the text below (in
seconds):

Streamcluster:

Developed by Princeton University and solves the online clustering problem.
Streamcluster was included in the PARSEC benchmark suite because of the
importance of data mining algorithms and the prevalence of problems with
streaming characteristics.

	Threads		HPERF_HMP	Linaro MP
	1		27,333		27,422
	2		14,162		14,197
	3		10,099		10,168
	4		8,227		8,332
	5		10,922		23,349
	6		10,85		22,507
	7		11,39		22,041
	8		12,307		21,181
	9		20,339		22,115
	10		21,33		23,746
	11		23,289		24,831
	12		25,363		26,699
	13		34,091		34,84
	14		34,758		38,661
	15		35,743		38,688
	16		38,1		44,735
	17		41,165		77,098
	18		44,223		102,633
	19		46,177		113,748
	20		48,22		119,146
	21		52,372		135,499
	22		54,319		136,454
	23		56,218		141,924
	24		57,843		145,727
	25		61,759		158,754
	26		63,179		163,915
	27		64,987		167,559
	28		67,329		171,203
	29		70,489		185,171
	30		73,084		189,303
	31		75,264		192,487
	32		77,015		197,27
	avg		40,373		87,543

Bodytrack:

This computer vision application is an Intel RMS workload which tracks a human
body with multiple cameras through an image sequence. This benchmark was
included due to the increasing significance of computer vision algorithms in
areas such as video surveillance, character animation and computer interfaces.

	Threads		HPERF_HMP	Linaro MP
	1		15,884		16,632
	2		8,536		9,42
	3		6,037		7,257
	4		4,84		6,076
	5		8,835		5,739
	6		4,437		5,513
	7		4,119		5,474
	8		3,992		5,115
	9		3,854		5,164
	10		3,92		4,911
	11		3,854		4,932
	12		3,83		4,816
	13		3,839		5,643
	14		3,861		4,816
	15		3,889		4,896
	16		3,845		4,854
	17		3,872		4,837
	18		3,852		4,876
	19		4,304		4,868
	20		3,915		4,928
	21		3,87		4,841
	22		3,858		4,995
	23		3,881		4,97
	24		3,876		4,899
	25		3,854		4,96
	26		3,869		4,902
	27		3,874		4,979
	28		3,88		4,928
	29		3,914		5,008
	30		3,889		5,216
	31		3,898		5,242
	32		3,894		5,199
	avg		4,689		5,653

Blackscholes:

This application is an Intel RMS benchmark. It calculates the prices for a
portfolio of European options analytically with the Black-Scholes partial
differential equation. There is no closed-form expression for the blackscholes
equation and as such it must be computed numerically.

	Threads		HPERF_HMP	Linaro MP
	1		7,293		6,807
	2		3,886		4,044
	3		2,906		2,911
	4		2,429		2,427
	5		2,58		2,985
	6		2,401		2,672
	7		2,205		2,411
	8		2,132		2,293
	9		2,074		2,41
	10		2,067		2,264
	11		2,054		2,205
	12		2,091		2,222
	13		2,042		2,28
	14		2,035		2,222
	15		2,026		2,25
	16		2,024		2,177
	17		2,021		2,173
	18		2,033		2,09
	19		2,03		2,05
	20		2,024		2,158
	21		2,002		2,175
	22		2,026		2,179
	23		2,017		2,134
	24		2,01		2,156
	25		2,009		2,155
	26		2,013		2,179
	27		2,017		2,177
	28		2,019		2,189
	29		2,013		2,158
	30		2,002		2,162
	31		2,016		2,16
	32		2,012		2,159
	avg		2,328		2,469

Also, well known Antutu benchmark was executed on Exynos 5433 board:

					HPERF_HMP	Linaro MP
	Integral benchmark result  	42400		36860 
	Result: hperf_hmp is 15% better.


Arseniy Krasnov (13):
  hperf_hmp: add new config for arm and arm64.
  hperf_hmp: introduce hew domain flag.
  hperf_hmp: add sched domains initialization.
  hperf_hmp: scheduler initialization routines.
  hperf_hmp: introduce druntime metric.
  hperf_hmp: is_hmp_imbalance introduced.
  hperf_hmp: migration auxiliary functions.
  hperf_hmp: swap tasks function.
  hperf_hmp: one way balancing function.
  hperf_hmp: idle pull function.
  hperf_hmp: task CPU selection logic.
  hperf_hmp: rest of logic.
  hperf_hmp: cpufreq routines.

 arch/arm/Kconfig           |   21 +
 arch/arm/kernel/topology.c |    6 +-
 arch/arm64/Kconfig         |   21 +
 include/linux/sched.h      |   17 +
 kernel/sched/core.c        |   65 +-
 kernel/sched/fair.c        | 1553 ++++++++++++++++++++++++++++++++++++++++----
 kernel/sched/sched.h       |   16 +
 7 files changed, 1586 insertions(+), 113 deletions(-)

-- 
1.9.1

[PATCH 01/13] hperf_hmp: add new config for arm and arm64.

[Arseniy Krasnov]

	New config option which enables new scheduler logic: HPERF_HMP. Also
adds the following options:
'HPERF_HMP_DEBUG': enables extra runtime checks of balancing parameteres.
'HMP_FAST_CPU_MASK': CPU mask of A15 cluster(in hex string).
'HMP_SLOW_CPU_MASK': CPU mask of A7 cluster(in hex string).

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 arch/arm/Kconfig   | 21 +++++++++++++++++++++
 arch/arm64/Kconfig | 21 +++++++++++++++++++++
 2 files changed, 42 insertions(+)

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 72ad724..0581914 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -1387,6 +1387,27 @@ config SCHED_MC
 	  making when dealing with multi-core CPU chips at a cost of slightly
 	  increased overhead in some places. If unsure say N here.
 
+config HPERF_HMP
+	bool "HPERF_HMP load balancing enhancements for ARM big.LITTLE"
+	select SCHED_MC
+	help
+	  Uses HPERF_HMP load balancing algorithm between A7 and A15 CPU domains.
+
+config HPERF_HMP_DEBUG
+	bool "Additional HPERF_HMP runtime debug checks"
+	depends on HPERF_HMP
+	default n
+
+config HMP_FAST_CPU_MASK
+	string "Fast (Cortex-A15) CPU mask for HPERF_HMP"
+	default ""
+	depends on HPERF_HMP
+
+config HMP_SLOW_CPU_MASK
+	string "Slow (Cortex-A7) CPU mask for HPERF_HMP"
+	default ""
+	depends on HPERF_HMP
+
 config SCHED_SMT
 	bool "SMT scheduler support"
 	depends on ARM_CPU_TOPOLOGY
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 07d1811..71a8983 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -412,6 +412,27 @@ config SCHED_MC
 	  making when dealing with multi-core CPU chips at a cost of slightly
 	  increased overhead in some places. If unsure say N here.
 
+config HPERF_HMP
+	bool "HPERF_HMP load balancing enhancements for ARM big.LITTLE"
+	select SCHED_MC
+	help
+	  Uses HPERF_HMP load balancing algorithm between A7 and A15 CPU domains.
+
+config HPERF_HMP_DEBUG
+	bool "Additional HPERF_HMP runtime debug checks"
+	depends on HPERF_HMP
+	default n
+
+config HMP_FAST_CPU_MASK
+	string "Fast (Cortex-A15) CPU mask for HPERF_HMP"
+	default ""
+	depends on HPERF_HMP
+
+config HMP_SLOW_CPU_MASK
+	string "Slow (Cortex-A7) CPU mask for HPERF_HMP"
+	default ""
+	depends on HPERF_HMP
+
 config SCHED_SMT
 	bool "SMT scheduler support"
 	help
-- 
1.9.1

[PATCH 02/13] hperf_hmp: introduce hew domain flag.

[Arseniy Krasnov]

	New scheduler domain type: HMP. Each big.LITTLE cluster is detected by
scheduler as HMP domain. HPERF_HMP logic works between two HMP domains, the
default CFS logic, in turn, works inside the HMP domain.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 arch/arm/kernel/topology.c | 6 +++++-
 include/linux/sched.h      | 4 ++++
 kernel/sched/core.c        | 9 ++++++++-
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index 08b7847..7fcc5fe 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -285,7 +285,11 @@ static struct sched_domain_topology_level arm_topology[] = {
 	{ cpu_corepower_mask, cpu_corepower_flags, SD_INIT_NAME(GMC) },
 	{ cpu_coregroup_mask, cpu_core_flags, SD_INIT_NAME(MC) },
 #endif
-	{ cpu_cpu_mask, SD_INIT_NAME(DIE) },
+	{ cpu_cpu_mask,
+#ifdef CONFIG_HPERF_HMP
+	 .flags = SD_HMP_BALANCE,
+#endif
+	 SD_INIT_NAME(DIE)},
 	{ NULL, },
 };
 
diff --git a/include/linux/sched.h b/include/linux/sched.h
index b7b9501..eb084df 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -990,6 +990,10 @@ extern void wake_up_q(struct wake_q_head *head);
 #define SD_OVERLAP		0x2000	/* sched_domains of this level overlap */
 #define SD_NUMA			0x4000	/* cross-node balancing */
 
+#ifdef CONFIG_HPERF_HMP
+#define SD_HMP_BALANCE		0x8000	/* Use HMP load balancing algorithm */
+#endif
+
 #ifdef CONFIG_SCHED_SMT
 static inline int cpu_smt_flags(void)
 {
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index bcd214e..16092e0 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6410,6 +6410,9 @@ sd_init(struct sched_domain_topology_level *tl, int cpu)
 					| 0*SD_PREFER_SIBLING
 					| 0*SD_NUMA
 					| sd_flags
+#ifdef CONFIG_HPERF_HMP
+					| (tl->flags & SD_HMP_BALANCE)
+#endif
 					,
 
 		.last_balance		= jiffies,
@@ -6472,7 +6475,11 @@ static struct sched_domain_topology_level default_topology[] = {
 #ifdef CONFIG_SCHED_MC
 	{ cpu_coregroup_mask, cpu_core_flags, SD_INIT_NAME(MC) },
 #endif
-	{ cpu_cpu_mask, SD_INIT_NAME(DIE) },
+	{ cpu_cpu_mask,
+#ifdef CONFIG_HPERF_HMP
+	 .flags = SD_HMP_BALANCE,
+#endif
+	 SD_INIT_NAME(DIE)},
 	{ NULL, },
 };
 
-- 
1.9.1

[PATCH 03/13] hperf_hmp: add sched domains initialization.

[Arseniy Krasnov]

	Attaching CPU clusters as 'sched_group' to HMP domains. Each HMP domain
has two pointers to A15 and A7 scheduling groups(struct sched_group).

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 include/linux/sched.h |  4 ++++
 kernel/sched/core.c   | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index eb084df..aa72125 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1057,6 +1057,10 @@ struct sched_domain {
 	u64 max_newidle_lb_cost;
 	unsigned long next_decay_max_lb_cost;
 
+#ifdef CONFIG_HPERF_HMP
+	struct sched_group *a15_group;
+	struct sched_group *a7_group;
+#endif
 #ifdef CONFIG_SCHEDSTATS
 	/* load_balance() stats */
 	unsigned int lb_count[CPU_MAX_IDLE_TYPES];
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 16092e0..e3a632f 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -90,6 +90,16 @@
 #define CREATE_TRACE_POINTS
 #include <trace/events/sched.h>
 
+#ifdef CONFIG_HPERF_HMP
+/* cpumask for A15 cpus */
+static DECLARE_BITMAP(cpu_fastest_bits, CONFIG_NR_CPUS);
+struct cpumask *cpu_fastest_mask = to_cpumask(cpu_fastest_bits);
+
+/* cpumask for A7 cpus */
+static DECLARE_BITMAP(cpu_slowest_bits, CONFIG_NR_CPUS);
+struct cpumask *cpu_slowest_mask = to_cpumask(cpu_slowest_bits);
+#endif
+
 DEFINE_MUTEX(sched_domains_mutex);
 DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
 
@@ -6971,6 +6981,45 @@ static int build_sched_domains(const struct cpumask *cpu_map,
 		sd = *per_cpu_ptr(d.sd, i);
 		cpu_attach_domain(sd, d.rd, i);
 	}
+
+#ifdef CONFIG_HPERF_HMP
+	for (i = nr_cpumask_bits - 1; i >= 0; i--) {
+		if (!cpumask_test_cpu(i, cpu_map))
+			continue;
+
+		for (sd = *per_cpu_ptr(d.sd, i); sd; sd = sd->parent) {
+			struct sched_group *sg;
+			sd->a7_group = NULL;
+			sd->a15_group = NULL;
+
+			/* Process only HMP domains */
+			if (!(sd->flags & SD_HMP_BALANCE))
+				continue;
+
+			/*
+			 * Process sched groups of this domain.
+			 * Attach sg to hmp domains.
+			 */
+			sg = sd->groups;
+			do {
+				if (!sg->sgc)
+					goto next_sg;
+#ifdef CONFIG_SCHED_DEBUG
+				printk(KERN_EMERG "Attaching CPUs 0x%08lX to domain %s\n",
+				       sched_group_cpus(sg)->bits[0], sd->name);
+#endif
+				if (cpumask_intersects(sched_group_cpus(sg),
+							cpu_fastest_mask))
+					sd->a15_group = sg;
+				else
+					sd->a7_group = sg;
+next_sg:
+				sg = sg->next;
+			} while (sg != sd->groups);
+		}
+	}
+#endif /* CONFIG_HPERF_HMP */
+
 	rcu_read_unlock();
 
 	ret = 0;
-- 
1.9.1

[PATCH 04/13] hperf_hmp: scheduler initialization routines.

[Arseniy Krasnov]

	Adds new fields to 'rq' structure and routine called during fair class
setup, which initializes some HMP scheduler variables: big and little cluster
masks. They are read from kernel config(if set), else default values are used.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 kernel/sched/core.c  |  4 ++++
 kernel/sched/fair.c  | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 kernel/sched/sched.h | 15 +++++++++++++++
 3 files changed, 65 insertions(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index e3a632f..8747e06 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7488,6 +7488,10 @@ void __init sched_init(void)
 #endif
 		init_rq_hrtick(rq);
 		atomic_set(&rq->nr_iowait, 0);
+#ifdef CONFIG_HPERF_HMP
+		rq->druntime_sum = 0;
+		rq->nr_hmp_tasks = 0;
+#endif
 	}
 
 	set_load_weight(&init_task);
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 9a5e60f..c57007f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -100,6 +100,11 @@ const_debug unsigned int sysctl_sched_migration_cost = 500000UL;
  */
 unsigned int __read_mostly sysctl_sched_shares_window = 10000000UL;
 
+#ifdef CONFIG_HPERF_HMP
+extern void hmp_set_cpu_masks(struct cpumask *, struct cpumask *);
+static unsigned int freq_scale_cpu_power[CONFIG_NR_CPUS];
+#endif /* CONFIG_HPERF_HMP */
+
 #ifdef CONFIG_CFS_BANDWIDTH
 /*
  * Amount of runtime to allocate from global (tg) to local (per-cfs_rq) pool
@@ -8305,8 +8310,38 @@ void show_numa_stats(struct task_struct *p, struct seq_file *m)
 #endif /* CONFIG_NUMA_BALANCING */
 #endif /* CONFIG_SCHED_DEBUG */
 
+#ifdef CONFIG_HPERF_HMP
+static unsigned long default_fast_mask = 0x0F;
+static unsigned long default_slow_mask = 0xF0;
+
+void hmp_set_cpu_masks(struct cpumask *fast_mask, struct cpumask *slow_mask)
+{
+	cpumask_clear(fast_mask);
+	cpumask_clear(slow_mask);
+
+	/* try to parse CPU masks from config */
+	if (strlen(CONFIG_HMP_FAST_CPU_MASK) &&
+	    strlen(CONFIG_HMP_SLOW_CPU_MASK)) {
+		if (cpumask_parse(CONFIG_HMP_FAST_CPU_MASK, fast_mask) ||
+		    cpumask_parse(CONFIG_HMP_SLOW_CPU_MASK, slow_mask))
+			pr_err("hperf_hmp: Failed to get CPU masks from config!\n");
+		else
+			return;
+	}
+
+	pr_err("hperf_hmp: Fast mask will be: %08lX, slow mask: %08lX\n",
+	       default_fast_mask, default_slow_mask);
+
+	fast_mask->bits[0] = default_fast_mask;
+	slow_mask->bits[0] = default_slow_mask;
+}
+#endif
+
 __init void init_sched_fair_class(void)
 {
+#ifdef CONFIG_HPERF_HMP
+	int cpu;
+#endif
 #ifdef CONFIG_SMP
 	open_softirq(SCHED_SOFTIRQ, run_rebalance_domains);
 
@@ -8315,6 +8350,17 @@ __init void init_sched_fair_class(void)
 	zalloc_cpumask_var(&nohz.idle_cpus_mask, GFP_NOWAIT);
 	cpu_notifier(sched_ilb_notifier, 0);
 #endif
+
+#ifdef CONFIG_HPERF_HMP
+	for_each_possible_cpu(cpu)
+		freq_scale_cpu_power[cpu] = SCHED_CAPACITY_SCALE;
+	hmp_set_cpu_masks(cpu_fastest_mask, cpu_slowest_mask);
+	pr_info("hperf_hmp: fast CPUs mask: %08X\n",
+		(unsigned int)cpumask_bits(cpu_fastest_mask)[0]);
+	pr_info("hperf_hmp: slow CPUs mask: %08X\n",
+		(unsigned int)cpumask_bits(cpu_slowest_mask)[0]);
+#endif
+
 #endif /* SMP */
 
 }
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 6d2a119..94828dc 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -597,6 +597,11 @@ struct rq {
 	 */
 	unsigned long nr_uninterruptible;
 
+#ifdef CONFIG_HPERF_HMP
+	/* shows the amount of accumulated unfairness by tasks of this rq */
+	long druntime_sum;
+	unsigned int nr_hmp_tasks;
+#endif
 	struct task_struct *curr, *idle, *stop;
 	unsigned long next_balance;
 	struct mm_struct *prev_mm;
@@ -892,6 +897,16 @@ static inline unsigned int group_first_cpu(struct sched_group *group)
 
 extern int group_balance_cpu(struct sched_group *sg);
 
+#ifdef CONFIG_HPERF_HMP
+extern struct cpumask *cpu_fastest_mask;
+extern struct cpumask *cpu_slowest_mask;
+
+static inline bool cpu_is_fastest(int cpu)
+{
+	return cpumask_test_cpu(cpu, cpu_fastest_mask);
+}
+#endif
+
 #else
 
 static inline void sched_ttwu_pending(void) { }
-- 
1.9.1

[PATCH 05/13] hperf_hmp: introduce druntime metric.

[Arseniy Krasnov]

	This patch adds special per-task metric to look for candidate for
migration between HMP domains(clusters). 'druntime' grows up when task runs on
A7 cluster, and goes down on A15 cluster. Also druntime is scaled according load
on little cluster in order to align its value with big cluster's total druntime.
For migration from big/little to little/big cluster task with lowest/highest
'druntime' chosen. 'druntime' is used to execute each task on each cluster
approximately same amount of time. 'druntime' is calculated each call of default
'update_curr' function.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 include/linux/sched.h |   3 ++
 kernel/sched/core.c   |   3 ++
 kernel/sched/fair.c   | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 121 insertions(+)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index aa72125..89c1bf3 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1257,6 +1257,9 @@ struct sched_entity {
 	struct list_head	group_node;
 	unsigned int		on_rq;
 
+#ifdef CONFIG_HPERF_HMP
+	long			druntime;
+#endif
 	u64			exec_start;
 	u64			sum_exec_runtime;
 	u64			vruntime;
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 8747e06..6883a00 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2085,6 +2085,9 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p)
 	p->se.prev_sum_exec_runtime	= 0;
 	p->se.nr_migrations		= 0;
 	p->se.vruntime			= 0;
+#ifdef CONFIG_HPERF_HMP
+	p->se.druntime			= 0;
+#endif
 	INIT_LIST_HEAD(&p->se.group_node);
 
 #ifdef CONFIG_SCHEDSTATS
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index c57007f..e94fab4 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -102,6 +102,10 @@ unsigned int __read_mostly sysctl_sched_shares_window = 10000000UL;
 
 #ifdef CONFIG_HPERF_HMP
 extern void hmp_set_cpu_masks(struct cpumask *, struct cpumask *);
+static atomic_t a15_nr_hmp_busy = ATOMIC_INIT(0);
+static atomic_t a7_nr_hmp_busy = ATOMIC_INIT(0);
+static atomic_t hmp_imbalance = ATOMIC_INIT(0);
+
 static unsigned int freq_scale_cpu_power[CONFIG_NR_CPUS];
 #endif /* CONFIG_HPERF_HMP */
 
@@ -660,6 +664,115 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se)
 	return calc_delta_fair(sched_slice(cfs_rq, se), se);
 }
 
+#ifdef CONFIG_HPERF_HMP
+static bool
+is_task_hmp(struct task_struct *task, const struct cpumask *task_cpus)
+{
+	if (!task_cpus)
+		task_cpus = tsk_cpus_allowed(task);
+
+	/*
+	 * Check if a task has cpus_allowed only for one CPU domain (A15 or A7)
+	 */
+	return !(cpumask_intersects(task_cpus, cpu_fastest_mask) ^
+		 cpumask_intersects(task_cpus, cpu_slowest_mask));
+}
+
+#ifdef CONFIG_HPERF_HMP_DEBUG
+static inline void check_druntime_sum(struct rq *rq, long druntime_sum)
+{
+	BUG_ON(rq->cfs.h_nr_running == 0 && druntime_sum != 0);
+
+	if (cpu_is_fastest(rq->cpu))
+		BUG_ON(druntime_sum > 0);
+	else
+		BUG_ON(druntime_sum < 0);
+}
+#else
+static inline void check_druntime_sum(struct rq *rq, long druntime_sum)
+{
+}
+#endif
+
+static inline void add_druntime_sum(struct rq *rq, long delta)
+{
+	rq->druntime_sum += delta;
+	check_druntime_sum(rq, rq->druntime_sum);
+}
+/* Updates druntime for a task */
+static inline void
+update_hmp_stat(struct cfs_rq *cfs_rq, struct sched_entity *curr,
+		unsigned long delta_exec)
+{
+	long to_add;
+	unsigned int hmp_fairness_threshold = 240;
+	struct rq *rq = rq_of(cfs_rq);
+	int a7_nr_hmp_busy_tmp;
+
+	if (atomic_read(&hmp_imbalance) == 0)
+		return;
+
+	if (!curr->on_rq)
+		return;
+
+	if (!entity_is_task(curr))
+		return;
+
+	if (!task_of(curr)->on_rq)
+		return;
+
+	if (!cfs_rq->h_nr_running)
+		return;
+
+	if (!is_task_hmp(task_of(curr), NULL))
+		return;
+
+	delta_exec = delta_exec >> 10;
+
+	if (cpu_is_fastest(rq->cpu))
+		to_add = -delta_exec;
+	else
+		to_add = delta_exec;
+
+	to_add -= curr->druntime;
+
+	/* Avoid values with the different sign */
+	if ((cpu_is_fastest(rq->cpu) && to_add >= 0) ||
+	    (!cpu_is_fastest(rq->cpu) && to_add <= 0))
+		return;
+
+	to_add /= (long)(2 + 4 * hmp_fairness_threshold /
+			(cfs_rq->h_nr_running + 1));
+
+	a7_nr_hmp_busy_tmp = atomic_read(&a7_nr_hmp_busy);
+	/* druntime balancing between the domains */
+	if (!cpu_is_fastest(rq->cpu) && a7_nr_hmp_busy_tmp) {
+		to_add *= atomic_read(&a15_nr_hmp_busy);
+		to_add /= a7_nr_hmp_busy_tmp;
+	}
+
+	if (cpu_is_fastest(rq->cpu)) {
+		if (curr->druntime < 0)
+			add_druntime_sum(rq, to_add);
+		else if ((curr->druntime + to_add) < 0)
+			add_druntime_sum(rq, curr->druntime + to_add);
+	} else {
+		if (curr->druntime > 0)
+			add_druntime_sum(rq, to_add);
+		else if ((curr->druntime + to_add) > 0)
+			add_druntime_sum(rq, curr->druntime + to_add);
+	}
+
+	curr->druntime += to_add;
+}
+#else
+static inline void
+update_hmp_stat(struct cfs_rq *cfs_rq, struct sched_entity *curr,
+	      unsigned long delta_exec)
+{
+}
+#endif /* CONFIG_HPERF_HMP */
+
 #ifdef CONFIG_SMP
 static int select_idle_sibling(struct task_struct *p, int cpu);
 static unsigned long task_h_load(struct task_struct *p);
@@ -735,6 +848,8 @@ static void update_curr(struct cfs_rq *cfs_rq)
 	}
 
 	account_cfs_rq_runtime(cfs_rq, delta_exec);
+
+	update_hmp_stat(cfs_rq, curr, delta_exec);
 }
 
 static void update_curr_fair(struct rq *rq)
-- 
1.9.1

[PATCH 06/13] hperf_hmp: is_hmp_imbalance introduced.

[Arseniy Krasnov]

	'is_hmp_imbalance' function calculates imbalance between clusters, four
cases are possible: balancing from/to one of clusters, task swap(when clusters
are balanced) or skip rebalance. Function calculates load difference between two
cluster(cluster load / cluster power) and threshold when balancing is needed.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 kernel/sched/fair.c | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index e94fab4..3ab39b6 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -104,9 +104,21 @@ unsigned int __read_mostly sysctl_sched_shares_window = 10000000UL;
 extern void hmp_set_cpu_masks(struct cpumask *, struct cpumask *);
 static atomic_t a15_nr_hmp_busy = ATOMIC_INIT(0);
 static atomic_t a7_nr_hmp_busy = ATOMIC_INIT(0);
+
+/* Total weight of all running tasks on A15 and A7 CPU domains */
+static atomic_long_t a15_total_weight = ATOMIC_LONG_INIT(0);
+static atomic_long_t a7_total_weight = ATOMIC_LONG_INIT(0);
+
 static atomic_t hmp_imbalance = ATOMIC_INIT(0);
 
 static unsigned int freq_scale_cpu_power[CONFIG_NR_CPUS];
+
+enum hmp_balance_actions {
+	SWAP_TASKS,
+	A15_TO_A7,
+	A7_TO_A15,
+	SKIP_REBALANCE,
+};
 #endif /* CONFIG_HPERF_HMP */
 
 #ifdef CONFIG_CFS_BANDWIDTH
@@ -7016,6 +7028,97 @@ static int should_we_balance(struct lb_env *env)
 	 */
 	return balance_cpu == env->dst_cpu;
 }
+#ifdef CONFIG_HPERF_HMP
+/**
+ * is_hmp_imbalance(): Calculates imbalance between HMP domains.
+ * @sd: Current sched domain.
+ *
+ * Returns migration direction(see SWAP_TASKS, A15_TO_A7, A7_TO_A15,
+ * SKIP_REBALANCE).
+ *
+ * Imbalance depends on load of tasks on A15 cores and A7 cores,
+ * current CPU's frequencies, and A7 slowdown coefficient which is about 2.4.
+ */
+static int is_hmp_imbalance(struct sched_domain *sd)
+{
+	int imbalance, cpu;
+	int a15_group_power = 0, a7_group_power = 0,
+				hmp_imbalance_min_threshold;
+	int a15_group_load, a7_group_load, a15_a7_group_power;
+	unsigned int a7_balanced_num;
+	int reminder, divisor;
+	unsigned int a15_balanced_num;
+	long long int hmp_imbalance_threshold;
+
+	if (!sd->a15_group) {
+		return SKIP_REBALANCE;
+	}
+
+	if (!sd->a7_group) {
+		return SKIP_REBALANCE;
+	}
+	for_each_online_cpu(cpu) {
+		if (cpu_is_fastest(cpu))
+			a15_group_power += freq_scale_cpu_power[cpu];
+		else
+			a7_group_power += freq_scale_cpu_power[cpu];
+	}
+
+	if (a15_group_power == 0 || a7_group_power == 0) {
+		return SKIP_REBALANCE;
+	}
+
+	a15_balanced_num = 0;
+	a7_balanced_num = 0;
+
+	for_each_online_cpu(cpu) {
+		if (cpu_rq(cpu)->cfs.h_nr_running <= 1) {
+			if (cpu_is_fastest(cpu))
+				a15_balanced_num++;
+			else
+				a7_balanced_num++;
+		}
+	}
+
+	a7_group_load = atomic_long_read(&a7_total_weight);
+
+	if (atomic_long_read(&a7_total_weight) == 0 &&
+	    (a15_balanced_num == sd->a15_group->group_weight)) {
+		return SKIP_REBALANCE;
+	}
+
+	a15_group_load = atomic_long_read(&a15_total_weight);
+	a15_a7_group_power = a15_group_power + a7_group_power;
+
+	imbalance = (a15_group_load * 1024) / (a15_group_power) -
+		    (a7_group_load * 1024) / (a7_group_power);
+	hmp_imbalance_threshold = ((long long int)NICE_0_LOAD *
+				   1024 * a15_a7_group_power);
+	divisor = 2 * a15_group_power * a7_group_power;
+	hmp_imbalance_threshold = div_s64_rem(hmp_imbalance_threshold,
+						divisor, &reminder);
+	hmp_imbalance_min_threshold = hmp_imbalance_threshold >> 3;
+
+	if (imbalance < hmp_imbalance_min_threshold &&
+	    imbalance > -hmp_imbalance_min_threshold) {
+		atomic_set(&hmp_imbalance, 0);
+		return SKIP_REBALANCE;
+	}
+
+	if (imbalance > hmp_imbalance_threshold) {
+		return A15_TO_A7;
+	} else {
+		if (imbalance < -hmp_imbalance_threshold) {
+			if (a7_balanced_num == sd->a7_group->group_weight)
+				return SWAP_TASKS;
+			else
+				return A7_TO_A15;
+		} else {
+			return SWAP_TASKS;
+		}
+	}
+}
+#endif /* CONFIG_HPERF_HMP */
 
 /*
  * Check this_cpu to ensure it is balanced within domain. Attempt to move
-- 
1.9.1

[PATCH 07/13] hperf_hmp: migration auxiliary functions.

[Arseniy Krasnov]

	Adds functions used for migration: scanning every runqueue from another
cluster for migration process, searching task to migrate from runqueue mentioned
above and function to move task from one CPU to another.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 include/linux/sched.h |   6 +
 kernel/sched/fair.c   | 301 ++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 307 insertions(+)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 89c1bf3..dafda4b 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1259,6 +1259,12 @@ struct sched_entity {
 
 #ifdef CONFIG_HPERF_HMP
 	long			druntime;
+
+	/* Time of last migration between HMP domains (in jiffies)*/
+	unsigned long		last_migration;
+
+	/* If set, don't touch for migration */
+	int			migrate_candidate;
 #endif
 	u64			exec_start;
 	u64			sum_exec_runtime;
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 3ab39b6..ff05364 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7118,6 +7118,307 @@ static int is_hmp_imbalance(struct sched_domain *sd)
 		}
 	}
 }
+
+/**
+ * hmp_can_migrate_task(): Checks whether specified task could be migrated.
+ * @p: task to check.
+ * @env: migration parameters.
+ *
+ * Returns 1 if migration possible, else 0.
+ */
+static int hmp_can_migrate_task(struct task_struct *p, struct lb_env *env)
+{
+	if (!cpumask_test_cpu(env->dst_cpu, tsk_cpus_allowed(p))) {
+		schedstat_inc(p, se.statistics.nr_failed_migrations_affine);
+		return 0;
+	}
+	env->flags &= ~LBF_ALL_PINNED;
+
+	if (task_running(env->src_rq, p)) {
+		schedstat_inc(p, se.statistics.nr_failed_migrations_running);
+		return 0;
+	}
+	return 1;
+}
+
+/**
+ * detach_specified_task(): Detaches specified task.
+ * @pm: Task to move.
+ * @env: Migration parameters.
+ *
+ * Returns moved task.
+ */
+static struct task_struct *
+detach_specified_task(struct task_struct *p, struct lb_env *env)
+{
+	lockdep_assert_held(&env->src_rq->lock);
+
+	/* If task to move falls asleep, so don't scan runqueue and return */
+	if (p->se.migrate_candidate == 0)
+		return 0;
+
+	if (throttled_lb_pair(task_group(p), env->src_rq->cpu, env->dst_cpu))
+		goto exit;
+
+	if (!hmp_can_migrate_task(p, env))
+		goto exit;
+
+	detach_task(p, env);
+	/*
+	 * Right now, this is only the third place move_task()
+	 * is called, so we can safely collect move_task()
+	 * stats here rather than inside move_task().
+	 */
+	schedstat_inc(env->sd, lb_gained[env->idle]);
+	return p;
+exit:
+	p->se.migrate_candidate = 0;
+
+	return NULL;
+}
+
+/**
+ * migrate_runnable_task(): Moves task that isn't running to destination CPU.
+ * @migrate_task: Task to migrate.
+ * @destination_cpu: Destination CPU.
+ *
+ * Returns moved weight.
+ *
+ * Runqueue's of @migrate_task and @destination_cpu must be locked.
+ */
+static unsigned migrate_runnable_task(struct task_struct *migrate_task,
+				      int destination_cpu)
+{
+	struct sched_domain *sd = NULL;
+	int src_cpu = task_cpu(migrate_task);
+	struct rq *src_rq = task_rq(migrate_task);
+	int dst_cpu = destination_cpu;
+	struct rq *dst_rq = cpu_rq(dst_cpu);
+	unsigned int ld_moved = 0;
+	struct task_struct *p = NULL;
+
+#ifdef CONFIG_HPERF_HMP_DEBUG
+	BUG_ON(src_rq == dst_rq);
+#else
+	if (WARN_ON(src_rq == dst_rq))
+		return 0;
+#endif
+
+	rcu_read_lock();
+	for_each_domain(dst_cpu, sd) {
+		if (cpumask_test_cpu(src_cpu, sched_domain_span(sd)))
+			break;
+	}
+	if (likely(sd)) {
+		struct lb_env env = {
+			.sd		= sd,
+			.dst_cpu	= dst_cpu,
+			.dst_rq		= dst_rq,
+			.src_cpu	= src_cpu,
+			.src_rq		= src_rq,
+			.idle		= CPU_NOT_IDLE,
+		};
+
+		schedstat_inc(sd, alb_count);
+		p = detach_specified_task(migrate_task, &env);
+		if (p) {
+			migrate_task->se.last_migration = jiffies;
+			schedstat_inc(sd, alb_pushed);
+			ld_moved = migrate_task->se.load.weight;
+		} else
+			schedstat_inc(sd, alb_failed);
+	}
+	rcu_read_unlock();
+
+	if (p)
+		attach_task(dst_rq, p);
+
+	if (migrate_task->se.migrate_candidate)
+		migrate_task->se.migrate_candidate = 0;
+	return ld_moved;
+}
+
+/* A task can't be migrated more often than 4 ms between A7 and A15 CPUs */
+static int se_is_old(struct sched_entity *se)
+{
+	const unsigned int migration_delay = 4; /* ms */
+
+	return time_after(jiffies,
+			se->last_migration + msecs_to_jiffies(migration_delay));
+}
+
+/**
+ * get_opposite_group(): Gets A15 of A7 group of domain.
+ * @sd: Current sched domain.
+ * @domain: Flag, which group is needed.
+ *
+ * Returns pointer to sched group.
+ */
+static struct sched_group *get_opposite_group(struct sched_domain *sd,
+					      int domain)
+{
+	if (!domain)
+		return sd->a15_group;
+	else
+		return sd->a7_group;
+}
+
+/**
+ * get_unfair_rq(): Returns runqueue which most fits for HMP migration.
+ * @sd: Current sched_domain.
+ * @this_cpu: without NO_HZ same as smp_processor_id().
+ *
+ * Returns struct rq*.
+ *
+ * Returned runqueue will be locked.
+ */
+static struct rq *get_unfair_rq(struct sched_domain *sd, int this_cpu)
+{
+	struct rq *unfair_rq = NULL;
+	struct sched_group *opposite_sg;
+	struct cpumask *opposite_mask;
+	int druntime;
+	int cpu;
+
+	opposite_sg = get_opposite_group(sd, cpu_is_fastest(this_cpu));
+
+	if (!opposite_sg)
+		return NULL;
+
+	opposite_mask = sched_group_cpus(opposite_sg);
+	druntime = cpu_is_fastest(this_cpu) ? INT_MIN : INT_MAX;
+
+	/* Check rq's of opposite domain */
+	for_each_cpu_and(cpu, opposite_mask, cpu_online_mask) {
+		struct rq *rq = cpu_rq(cpu);
+		long tmp_druntime;
+
+		/*
+		 * Note: the value is read without a spinlock and can be
+		 *       outdated. But it is fine in the long run.
+		 */
+		tmp_druntime = rq->druntime_sum;
+
+		/* Skip empty rqs or rqs waiting for stopper */
+		if (rq->active_balance || !rq->cfs.h_nr_running)
+			continue;
+
+		if (cpu_is_fastest(cpu)) {
+			if (tmp_druntime < druntime) {
+				druntime = tmp_druntime;
+				unfair_rq = rq;
+			}
+		} else {
+			if (tmp_druntime > druntime) {
+				druntime = tmp_druntime;
+				unfair_rq = rq;
+			}
+		}
+	}
+
+	if (unfair_rq) {
+		raw_spin_lock(&unfair_rq->lock);
+		if (!unfair_rq->cfs.h_nr_running || unfair_rq->active_balance) {
+			raw_spin_unlock(&unfair_rq->lock);
+			return NULL;
+		}
+	}
+
+	return unfair_rq;
+}
+
+/**
+ * get_migration_candidate(): Get task which most fits for HMP migration.
+ * @sd: Current sched domain.
+ * @unfair_rq: Runqueue to scan for migration task.
+ * @idle_flag: Determines unfair_rq is idle for not. If 1, then ignore task's
+ * @destination_cpu: Destination CPU for task from @unfair_rq
+ * druntime and last migration time.
+ *
+ * Returns struct task_struct*.
+ *
+ * @unfair_rq must be locked. @sd must have SD_HMP_BALANCE flag.
+ */
+static struct task_struct *get_migration_candidate(struct sched_domain *sd,
+						   struct rq *unfair_rq,
+						   int idle_flag,
+						   int destination_cpu)
+{
+	long druntime;
+	struct task_struct *p;
+	struct list_head *tasks;
+	struct task_struct *candidate = NULL;
+	unsigned int count = sched_nr_migrate_break;
+
+	if (unfair_rq->cfs.h_nr_running < count)
+		count = unfair_rq->cfs.h_nr_running;
+
+	tasks = &unfair_rq->cfs_tasks;
+	druntime = cpu_is_fastest(unfair_rq->cpu) ? LONG_MAX : LONG_MIN;
+
+	while (!list_empty(tasks)) {
+		p = list_first_entry(tasks, struct task_struct, se.group_node);
+
+		if (!count)
+			break;
+
+		count--;
+		/* this task pinned by someone else for HMP migration */
+		if (p->se.migrate_candidate)
+			goto next;
+
+		/* if task can't run on destination cpu, skip */
+		if (!cpumask_test_cpu(destination_cpu, tsk_cpus_allowed(p)))
+			goto next;
+
+		/* check for 4ms timestamp, if idle_pull then don't care*/
+		if (!se_is_old(&p->se) && !idle_flag)
+			goto next;
+
+		if (cpu_is_fastest(unfair_rq->cpu)) {
+			if (p->se.druntime < druntime &&
+			    (p->se.druntime < 0 || idle_flag)) {
+				candidate = p;
+				druntime = p->se.druntime;
+			}
+		} else {
+			if (p->se.druntime > druntime &&
+			    (p->se.druntime > 0 || idle_flag)) {
+				candidate = p;
+				druntime = p->se.druntime;
+			}
+		}
+
+next:
+		list_move_tail(&p->se.group_node, tasks);
+	}
+
+	if (candidate)
+		candidate->se.migrate_candidate = 1;
+
+	return candidate;
+}
+
+/**
+ * try_to_move_task(): Migrates task if it isn't running.
+ * @migrate_task: Task to migrate.
+ * @destination_cpu: Destination cpu for @migrate_task.
+ * @stopper_needed: Flag which show that stopper thread needed to migrate task.
+ *
+ * Returns moved weight and flag that stopper needed or not.
+ *
+ * Runqueues of @migrate_task and @destination_cpu must be locked.
+ */
+static unsigned int try_to_move_task(struct task_struct *migrate_task,
+				int destination_cpu, int *stopper_needed)
+{
+	if (task_running(task_rq(migrate_task), migrate_task)) {
+		*stopper_needed = 1;
+		return migrate_task->se.load.weight;
+	}
+
+	return migrate_runnable_task(migrate_task, destination_cpu);
+}
 #endif /* CONFIG_HPERF_HMP */
 
 /*
-- 
1.9.1

[PATCH 08/13] hperf_hmp: swap tasks function.

[Arseniy Krasnov]

	'swap_tasks' performs migration between current CPU and CPU from another
cluster. It scans two runqueues looking for tasks using 'druntime' metric. When
both tasks are found it pulls task from another cluster, and push task from the
current CPU.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 kernel/sched/fair.c  | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++
 kernel/sched/sched.h |   1 +
 2 files changed, 101 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index ff05364..028d329 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7419,6 +7419,106 @@ static unsigned int try_to_move_task(struct task_struct *migrate_task,
 
 	return migrate_runnable_task(migrate_task, destination_cpu);
 }
+
+/**
+ * swap_tasks(): swaps two tasks from different HMP domains
+ * @sd: Current sched domain
+ * @this_cpu: without NO_HZ same as smp_processor_id().
+ *
+ * Returns weight of migrated tasks.
+ */
+static unsigned int swap_tasks(struct sched_domain *sd, int this_cpu)
+{
+	unsigned int ld_moved = 0;
+	int local_stopper = 0;
+	int foreign_stopper = 0;
+	struct rq *local_rq = cpu_rq(this_cpu);
+	struct rq *foreign_rq = NULL;
+	struct task_struct *local_task = NULL;
+	struct task_struct *foreign_task = NULL;
+	unsigned long local_flags;
+
+	local_irq_save(local_flags);
+	foreign_rq = get_unfair_rq(sd, this_cpu);
+
+	if (!foreign_rq) {
+		local_irq_restore(local_flags);
+		return 0;
+	}
+
+	double_lock_balance(foreign_rq, local_rq);
+
+	/* rq's waiting for stopper execution, return */
+	if (foreign_rq->active_balance)
+		goto unlock;
+
+	if (local_rq->active_balance)
+		goto unlock;
+
+	foreign_task = get_migration_candidate(sd, foreign_rq, 0, this_cpu);
+
+	if (!foreign_task)
+		goto unlock;
+
+	/* Get local task for migration */
+	local_task = get_migration_candidate(sd, local_rq, 0, foreign_rq->cpu);
+
+	if (!local_task) {
+		foreign_task->se.migrate_candidate = 0;
+		goto unlock;
+	}
+	/* First try to push local task */
+	ld_moved = try_to_move_task(local_task, foreign_rq->cpu,
+					&local_stopper);
+
+	/* If failed to move, then return, don't try to move foreign task */
+	if (!ld_moved) {
+		local_task->se.migrate_candidate = 0;
+		foreign_task->se.migrate_candidate = 0;
+		goto unlock;
+	}
+
+	/*
+	 * Migration is possible, but task is running,
+	 * so mark rq to run stopper.
+	 */
+	if (local_stopper) {
+		local_rq->push_cpu = foreign_rq->cpu;
+		local_rq->migrate_task = local_task;
+		local_rq->active_balance = 1;
+	}
+
+	/* Now try to pull task from another cpu */
+	ld_moved = try_to_move_task(foreign_task, this_cpu,
+					&foreign_stopper);
+
+	/* Failed to move foreign_task */
+	if (!ld_moved)
+		foreign_task->se.migrate_candidate = 0;
+
+	/* Migration is possible, mark rq to run stopper */
+	if (foreign_stopper) {
+		foreign_rq->push_cpu = this_cpu;
+		foreign_rq->migrate_task = foreign_task;
+		foreign_rq->active_balance = 1;
+	}
+
+unlock:
+	double_rq_unlock(local_rq, foreign_rq);
+	local_irq_restore(local_flags);
+
+	if (local_stopper)
+		stop_one_cpu_nowait(local_rq->cpu,
+				    active_load_balance_cpu_stop, local_rq,
+				    &local_rq->active_balance_work);
+
+	if (foreign_stopper)
+		stop_one_cpu_nowait(foreign_rq->cpu,
+				    active_load_balance_cpu_stop, foreign_rq,
+				    &foreign_rq->active_balance_work);
+
+	return ld_moved;
+}
 #endif /* CONFIG_HPERF_HMP */
 
 /*
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 94828dc..47e9605 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -598,6 +598,7 @@ struct rq {
 	unsigned long nr_uninterruptible;
 
 #ifdef CONFIG_HPERF_HMP
+	struct task_struct *migrate_task; /* task from this rq for migration */
 	/* shows the amount of accumulated unfairness by tasks of this rq */
 	long druntime_sum;
 	unsigned int nr_hmp_tasks;
-- 
1.9.1

[PATCH 09/13] hperf_hmp: one way balancing function.

[Arseniy Krasnov]

	Almost identical functions which push/pull task from/to current CPU
to/from another cluster. Called when balancing between clusters is broken and we
need to fix it.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 kernel/sched/fair.c | 254 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 254 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 028d329..4fda1ec 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7519,6 +7519,260 @@ unlock:
 
 	return ld_moved;
 }
+
+/* Get idlest cpu from opposite domain of this_cpu */
+static int get_idlest_cpu(struct sched_domain *sd, int this_cpu)
+{
+	struct sched_group *opposite_sg;
+	struct cpumask *opposite_mask;
+	unsigned long load = ULONG_MAX;
+	int idlest_cpu = -1;
+	int cpu;
+
+	opposite_sg = get_opposite_group(sd, cpu_is_fastest(this_cpu));
+	opposite_mask = sched_group_cpus(opposite_sg);
+
+	for_each_cpu_and(cpu, opposite_mask, cpu_online_mask) {
+		if (cpu_rq(cpu)->load.weight < load) {
+			load = cpu_rq(cpu)->load.weight;
+			idlest_cpu = cpu;
+		}
+	}
+	return idlest_cpu;
+}
+
+/**
+ * move_a15_to_a7(): Moves one task from A15 to A7.
+ * @sd: Current sched domain.
+ * @this_cpu: without NO_HZ same as smp_processor_id().
+ *
+ * Returns moved weight.
+ *
+ * Chooses task to migrate by druntime.
+ */
+static unsigned int move_a15_to_a7(struct sched_domain *sd, int this_cpu)
+{
+	struct task_struct *task_to_move;
+	struct rq *local_rq = NULL;
+	struct rq *foreign_rq = NULL;
+	int local_stopper_flag = 0;
+	int foreign_stopper_flag = 0;
+	unsigned long local_flags;
+	unsigned int ld_moved = 0;
+
+	local_rq = cpu_rq(this_cpu);
+	local_irq_save(local_flags);
+
+	if (!cpu_is_fastest(this_cpu)) {
+		/* this A7 pulls task from A15 */
+		foreign_rq = get_unfair_rq(sd, this_cpu);
+
+		if (!foreign_rq) {
+			local_irq_restore(local_flags);
+			return 0;
+		}
+
+		double_lock_balance(foreign_rq, local_rq);
+
+		if (foreign_rq->active_balance)
+			goto unlock;
+
+		if (local_rq->active_balance)
+			goto unlock;
+
+		if (foreign_rq->cfs.h_nr_running <= 1)
+			goto unlock;
+
+		task_to_move = get_migration_candidate(sd, foreign_rq, 0,
+						       this_cpu);
+
+		if (!task_to_move)
+			goto unlock;
+
+		ld_moved = try_to_move_task(task_to_move, this_cpu,
+						&foreign_stopper_flag);
+
+		if (!ld_moved) {
+			task_to_move->se.migrate_candidate = 0;
+			goto unlock;
+		}
+
+		if (foreign_stopper_flag) {
+			foreign_rq->active_balance = 1;
+			foreign_rq->push_cpu = this_cpu;
+			foreign_rq->migrate_task = task_to_move;
+		}
+	} else {
+		/* this A15 push task to A7 */
+		int dst_cpu = get_idlest_cpu(sd, this_cpu);
+
+		if (dst_cpu == -1) {
+			local_irq_restore(local_flags);
+			return 0;
+		}
+
+		foreign_rq = cpu_rq(dst_cpu);
+		raw_spin_lock(&foreign_rq->lock);
+		double_lock_balance(foreign_rq, local_rq);
+
+		if (local_rq->cfs.h_nr_running <= 1)
+			goto unlock;
+
+		if (foreign_rq->active_balance)
+			goto unlock;
+
+		if (local_rq->active_balance)
+			goto unlock;
+
+		task_to_move = get_migration_candidate(sd, local_rq, 0,
+						       foreign_rq->cpu);
+
+		if (!task_to_move)
+			goto unlock;
+
+		ld_moved = try_to_move_task(task_to_move, dst_cpu,
+						&local_stopper_flag);
+
+		if (!ld_moved) {
+			task_to_move->se.migrate_candidate = 0;
+			goto unlock;
+		}
+
+		if (local_stopper_flag) {
+			local_rq->active_balance = 1;
+			local_rq->push_cpu = dst_cpu;
+			local_rq->migrate_task = task_to_move;
+		}
+	}
+unlock:
+	double_rq_unlock(local_rq, foreign_rq);
+	local_irq_restore(local_flags);
+
+	if (foreign_stopper_flag)
+		stop_one_cpu_nowait(foreign_rq->cpu,
+				    active_load_balance_cpu_stop, foreign_rq,
+				    &foreign_rq->active_balance_work);
+
+	if (local_stopper_flag)
+		stop_one_cpu_nowait(local_rq->cpu,
+				    active_load_balance_cpu_stop, local_rq,
+				    &local_rq->active_balance_work);
+
+	return ld_moved;
+}
+
+/**
+ * move_a7_to_a15(): Moves one task from A7 to A15.
+ * @sd: Current sched domain.
+ * @this_cpu: without NO_HZ same as smp_processor_id().
+ *
+ * Returns moved weight.
+ *
+ * Chooses task to migrate by druntime.
+ */
+static unsigned int move_a7_to_a15(struct sched_domain *sd, int this_cpu)
+{
+	struct task_struct *task_to_move;
+	struct rq *local_rq = NULL;
+	struct rq *foreign_rq = NULL;
+	int local_stopper_flag = 0;
+	int foreign_stopper_flag = 0;
+	unsigned long local_flags;
+	unsigned int ld_moved = 0;
+
+	local_rq = cpu_rq(this_cpu);
+	local_irq_save(local_flags);
+
+	if (cpu_is_fastest(this_cpu)) {
+		/* this A15 pulls task from A7 */
+		foreign_rq = get_unfair_rq(sd, this_cpu);
+
+		if (!foreign_rq) {
+			local_irq_restore(local_flags);
+			return 0;
+		}
+		double_lock_balance(foreign_rq, local_rq);
+
+		if (local_rq->active_balance)
+			goto unlock;
+
+		if (foreign_rq->active_balance)
+			goto unlock;
+
+		task_to_move = get_migration_candidate(sd, foreign_rq, 0,
+						       this_cpu);
+
+		if (!task_to_move)
+			goto unlock;
+
+		ld_moved = try_to_move_task(task_to_move, this_cpu,
+						&foreign_stopper_flag);
+
+		if (!ld_moved) {
+			task_to_move->se.migrate_candidate = 0;
+			goto unlock;
+		}
+
+		if (foreign_stopper_flag) {
+			foreign_rq->active_balance = 1;
+			foreign_rq->push_cpu = this_cpu;
+			foreign_rq->migrate_task = task_to_move;
+		}
+	} else {
+		/* this A7 push task to A15*/
+		int dst_cpu = get_idlest_cpu(sd, this_cpu);
+
+		if (dst_cpu == -1) {
+			local_irq_restore(local_flags);
+			return 0;
+		}
+
+		foreign_rq = cpu_rq(dst_cpu);
+		raw_spin_lock(&foreign_rq->lock);
+		double_lock_balance(foreign_rq, local_rq);
+
+		if (foreign_rq->active_balance)
+			goto unlock;
+
+		if (local_rq->active_balance)
+			goto unlock;
+
+		task_to_move = get_migration_candidate(sd, local_rq, 0,
+						       foreign_rq->cpu);
+
+		if (!task_to_move)
+			goto unlock;
+
+		ld_moved = try_to_move_task(task_to_move, dst_cpu,
+						&local_stopper_flag);
+
+		if (!ld_moved) {
+			task_to_move->se.migrate_candidate = 0;
+			goto unlock;
+		}
+
+		if (local_stopper_flag) {
+			local_rq->active_balance = 1;
+			local_rq->push_cpu = dst_cpu;
+			local_rq->migrate_task = task_to_move;
+		}
+	}
+unlock:
+	double_rq_unlock(local_rq, foreign_rq);
+	local_irq_restore(local_flags);
+
+	if (foreign_stopper_flag)
+		stop_one_cpu_nowait(foreign_rq->cpu,
+				    active_load_balance_cpu_stop, foreign_rq,
+				    &foreign_rq->active_balance_work);
+
+	if (local_stopper_flag)
+		stop_one_cpu_nowait(local_rq->cpu,
+				    active_load_balance_cpu_stop, local_rq,
+				    &local_rq->active_balance_work);
+
+	return ld_moved;
+}
 #endif /* CONFIG_HPERF_HMP */
 
 /*
-- 
1.9.1

[PATCH 10/13] hperf_hmp: idle pull function.

[Arseniy Krasnov]

	HMP idle pull is triggered when CPU becomes idle. It tries to pull task
from another cluster when it is overloaded. Also A7 can't pull alone task from
A15, but A15 can do that with A7 core. Task for migration is chosen in the same
way as for other HMP migration cases - using 'druntime' metric. Only difference
is that migration task doesn't need to run 5ms on its cluster before migration.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 kernel/sched/fair.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 4fda1ec..fd16729 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7421,6 +7421,72 @@ static unsigned int try_to_move_task(struct task_struct *migrate_task,
 }
 
 /**
+ * hmp_idle_pull(): Pulls task from opposite domain of this_cpu to this_cpu.
+ * @sd: Current sched domain.
+ * @this_cpu: without NO_HZ same as smp_processor_id().
+ *
+ * Returns moved weight.
+ *
+ * Chooses task by its druntime. Ignores task's druntime and
+ * time of last HMP migration. Also A7 can't pulls task from A15
+ * if A15 become idle.
+ */
+static unsigned int hmp_idle_pull(struct sched_domain *sd, int this_cpu)
+{
+	unsigned int ld_moved = 0;
+	struct task_struct *task_to_pull;
+	unsigned long local_flags;
+	int idle_stopper = 0;
+	struct rq *local_rq;
+	struct rq *rq;
+
+	local_irq_save(local_flags);
+	local_rq = cpu_rq(this_cpu);
+	rq = get_unfair_rq(sd, this_cpu);
+
+	if (!rq) {
+		local_irq_restore(local_flags);
+		return 0;
+	}
+	double_lock_balance(rq, local_rq);
+
+	if (rq->active_balance)
+		goto unlock;
+
+	if (local_rq->active_balance)
+		goto unlock;
+
+	/* Forbids secondary CPUs to pull alone task from primary CPUs */
+	if (!cpu_is_fastest(this_cpu) && rq->cfs.h_nr_running <= 1)
+		goto unlock;
+
+	/* Get task to pull from opposite domain to this_cpu */
+	task_to_pull = get_migration_candidate(sd, rq, 1, this_cpu);
+
+	if (!task_to_pull)
+		goto unlock;
+
+	ld_moved = try_to_move_task(task_to_pull, this_cpu, &idle_stopper);
+
+	if (idle_stopper) {
+		rq->push_cpu = this_cpu;
+		rq->active_balance = 1;
+		rq->migrate_task = task_to_pull;
+	}
+
+unlock:
+	double_rq_unlock(local_rq, rq);
+	local_irq_restore(local_flags);
+
+	if (idle_stopper)
+		stop_one_cpu_nowait(rq->cpu, active_load_balance_cpu_stop,
+				    rq, &rq->active_balance_work);
+
+	return ld_moved;
+}
+
+
+/**
  * swap_tasks(): swaps two tasks from different HMP domains
  * @sd: Current sched domain
  * @this_cpu: without NO_HZ same as smp_processor_id().
-- 
1.9.1

[PATCH 11/13] hperf_hmp: task CPU selection logic.

[Arseniy Krasnov]

	Adds new runqueue selection logic. If task is newly woken(fork or exec)
or it is not WF_SYNC wakeup, idlest CPU from both clusters is selected. Else,
default wake up logic is used('want_affine'). If it fails, idlest CPU from both
clusters is selected.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 kernel/sched/fair.c | 132 ++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 101 insertions(+), 31 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index fd16729..79be023 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4798,6 +4798,62 @@ static int wake_affine(struct sched_domain *sd, struct task_struct *p, int sync)
 	return 1;
 }
 
+#ifdef CONFIG_HPERF_HMP
+/**
+ * hmp_select_task_rq_fair(): selects cpu for task.
+ * @p: task which needs cpu
+ *
+ * Returns cpu for task.
+ *
+ * Selects idlest cpu for task @p.
+ */
+static int
+hmp_select_task_rq_fair(struct task_struct *p)
+{
+	int cpu;
+	int new_cpu;
+	unsigned long load;
+	unsigned long scaled_load;
+
+	new_cpu = task_cpu(p);
+
+	load = ULONG_MAX;
+	/* First check primary cpus */
+	for_each_cpu_and(cpu, cpu_online_mask, cpu_fastest_mask) {
+		if (cpumask_test_cpu(cpu, tsk_cpus_allowed(p))) {
+			/* Select idle cpu if it exists */
+			if (idle_cpu(cpu))
+				return cpu;
+			/* Otherwise select the least loaded cpu */
+			scaled_load = (weighted_cpuload(cpu) *
+				       SCHED_CAPACITY_SCALE) /
+				       freq_scale_cpu_power[cpu];
+			if (scaled_load < load) {
+				new_cpu = cpu;
+				load = scaled_load;
+			}
+		}
+	}
+
+	/* Then check secondary cpus */
+	for_each_cpu_and(cpu, cpu_online_mask, cpu_slowest_mask) {
+		if (cpumask_test_cpu(cpu, tsk_cpus_allowed(p))) {
+			if (idle_cpu(cpu))
+				return cpu;
+			scaled_load = (weighted_cpuload(cpu) *
+				       SCHED_CAPACITY_SCALE) /
+				       freq_scale_cpu_power[cpu];
+			if (scaled_load < load) {
+				new_cpu = cpu;
+				load = scaled_load;
+			}
+		}
+	}
+
+	return new_cpu;
+}
+
+#else /* CONFIG_HPERF_HMP */
 /*
  * find_idlest_group finds and returns the least busy CPU group within the
  * domain.
@@ -4905,6 +4961,7 @@ find_idlest_cpu(struct sched_group *group, struct task_struct *p, int this_cpu)
 	return shallowest_idle_cpu != -1 ? shallowest_idle_cpu : least_loaded_cpu;
 }
 
+#endif /* CONFIG_HPERF_HMP */
 /*
  * Try and locate an idle CPU in the sched_domain.
  */
@@ -4998,6 +5055,11 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 	int want_affine = 0;
 	int sync = wake_flags & WF_SYNC;
 
+#ifdef CONFIG_HPERF_HMP
+	if (!(sd_flag & SD_BALANCE_WAKE) || !sync)
+		return hmp_select_task_rq_fair(p);
+#endif
+
 	if (sd_flag & SD_BALANCE_WAKE)
 		want_affine = !wake_wide(p) && cpumask_test_cpu(cpu, tsk_cpus_allowed(p));
 
@@ -5030,41 +5092,49 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
 
 	if (!sd) {
 		if (sd_flag & SD_BALANCE_WAKE) /* XXX always ? */
-			new_cpu = select_idle_sibling(p, new_cpu);
-
-	} else while (sd) {
-		struct sched_group *group;
-		int weight;
+			if (IS_ENABLED(CONFIG_HPERF_HMP) && sync)
+				new_cpu = prev_cpu;
+			else
+				new_cpu = select_idle_sibling(p, prev_cpu);
+	} else {
+#ifdef CONFIG_HPERF_HMP
+		new_cpu = hmp_select_task_rq_fair(p);
+#else
+		while (sd) {
+			struct sched_group *group;
+			int weight;
 
-		if (!(sd->flags & sd_flag)) {
-			sd = sd->child;
-			continue;
-		}
+			if (!(sd->flags & sd_flag)) {
+				sd = sd->child;
+				continue;
+			}
 
-		group = find_idlest_group(sd, p, cpu, sd_flag);
-		if (!group) {
-			sd = sd->child;
-			continue;
-		}
+			group = find_idlest_group(sd, p, cpu, sd_flag);
+			if (!group) {
+				sd = sd->child;
+				continue;
+			}
 
-		new_cpu = find_idlest_cpu(group, p, cpu);
-		if (new_cpu == -1 || new_cpu == cpu) {
-			/* Now try balancing at a lower domain level of cpu */
-			sd = sd->child;
-			continue;
-		}
+			new_cpu = find_idlest_cpu(group, p, cpu);
+			if (new_cpu == -1 || new_cpu == cpu) {
+				/* Now try balancing at a lower domain level of cpu */
+				sd = sd->child;
+				continue;
+			}
 
-		/* Now try balancing at a lower domain level of new_cpu */
-		cpu = new_cpu;
-		weight = sd->span_weight;
-		sd = NULL;
-		for_each_domain(cpu, tmp) {
-			if (weight <= tmp->span_weight)
-				break;
-			if (tmp->flags & sd_flag)
-				sd = tmp;
-		}
-		/* while loop will break here if sd == NULL */
+			/* Now try balancing at a lower domain level of new_cpu */
+			cpu = new_cpu;
+			weight = sd->span_weight;
+			sd = NULL;
+			for_each_domain(cpu, tmp) {
+				if (weight <= tmp->span_weight)
+					break;
+				if (tmp->flags & sd_flag)
+					sd = tmp;
+			}
+			/* while loop will break here if sd == NULL */
+	}
+#endif
 	}
 	rcu_read_unlock();
 
-- 
1.9.1

[PATCH 12/13] hperf_hmp: rest of logic.

[Arseniy Krasnov]

	Inserts call to main logic from 'load_balance', balance parameters
calculation during enqueue/dequeue task from runqueue and affinity mask
change callback for fair scheduling class.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 kernel/sched/fair.c | 204 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 202 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 79be023..06f6518 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -677,6 +677,16 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq, struct sched_entity *se)
 }
 
 #ifdef CONFIG_HPERF_HMP
+static void hmp_calculate_imbalance(void)
+{
+	if (atomic_long_read(&a7_total_weight) == 0) {
+		atomic_set(&hmp_imbalance, 0);
+		return;
+	}
+
+	atomic_set(&hmp_imbalance, 1);
+}
+
 static bool
 is_task_hmp(struct task_struct *task, const struct cpumask *task_cpus)
 {
@@ -711,6 +721,13 @@ static inline void add_druntime_sum(struct rq *rq, long delta)
 	rq->druntime_sum += delta;
 	check_druntime_sum(rq, rq->druntime_sum);
 }
+
+static inline void sub_druntime_sum(struct rq *rq, long delta)
+{
+	rq->druntime_sum -= delta;
+	check_druntime_sum(rq, rq->druntime_sum);
+}
+
 /* Updates druntime for a task */
 static inline void
 update_hmp_stat(struct cfs_rq *cfs_rq, struct sched_entity *curr,
@@ -861,7 +878,9 @@ static void update_curr(struct cfs_rq *cfs_rq)
 
 	account_cfs_rq_runtime(cfs_rq, delta_exec);
 
+#ifdef	CONFIG_HPERF_HMP
 	update_hmp_stat(cfs_rq, curr, delta_exec);
+#endif
 }
 
 static void update_curr_fair(struct rq *rq)
@@ -4200,6 +4219,66 @@ static inline void hrtick_update(struct rq *rq)
 }
 #endif
 
+#ifdef CONFIG_HPERF_HMP
+#ifdef CONFIG_HPERF_HMP_DEBUG
+static void check_nr_hmp_tasks(struct rq *rq)
+{
+	if (rq->nr_hmp_tasks > rq->cfs.h_nr_running) {
+		pr_emerg("HMP BUG: rq->nr_hmp_tasks = %u, "
+			 "rq->cfs.h_nr_running = %u\n", rq->nr_hmp_tasks,
+			 rq->cfs.h_nr_running);
+		BUG();
+	}
+}
+#else
+static void check_nr_hmp_tasks(struct rq *rq) { }
+#endif
+
+static void nr_hmp_tasks_inc(struct rq *rq)
+{
+	if (!rq->nr_hmp_tasks) {
+		if (cpu_is_fastest(rq->cpu))
+			atomic_inc(&a15_nr_hmp_busy);
+		else
+			atomic_inc(&a7_nr_hmp_busy);
+	}
+
+	rq->nr_hmp_tasks++;
+	check_nr_hmp_tasks(rq);
+}
+
+static void nr_hmp_tasks_dec(struct rq *rq)
+{
+	rq->nr_hmp_tasks--;
+
+	if (!rq->nr_hmp_tasks) {
+		if (cpu_is_fastest(rq->cpu))
+			atomic_dec(&a15_nr_hmp_busy);
+		else
+			atomic_dec(&a7_nr_hmp_busy);
+	}
+	check_nr_hmp_tasks(rq);
+}
+
+static void
+set_cpus_allowed_hmp(struct task_struct *p, const struct cpumask *new_mask)
+{
+	bool is_hmp_before, is_hmp_after;
+
+	cpumask_copy(&p->cpus_allowed, new_mask);
+	p->nr_cpus_allowed = cpumask_weight(new_mask);
+	is_hmp_before = is_task_hmp(p, NULL);
+	is_hmp_after  = is_task_hmp(p, new_mask);
+
+	if (!p->on_cpu && p->se.on_rq && (is_hmp_before != is_hmp_after)) {
+		if (is_hmp_after)
+			nr_hmp_tasks_inc(rq_of(cfs_rq_of(&p->se)));
+		else
+			nr_hmp_tasks_dec(rq_of(cfs_rq_of(&p->se)));
+	}
+}
+#endif
+
 /*
  * The enqueue_task method is called before nr_running is
  * increased. Here we update the fair scheduling stats and
@@ -4241,8 +4320,24 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 		update_cfs_shares(cfs_rq);
 	}
 
-	if (!se)
+	if (!se) {
 		add_nr_running(rq, 1);
+#ifdef CONFIG_HPERF_HMP
+		if (is_task_hmp(p, NULL))
+			nr_hmp_tasks_inc(rq);
+
+		if (cpu_is_fastest(rq->cpu)) {
+			atomic_long_add(p->se.load.weight, &a15_total_weight);
+			if (p->se.druntime < 0)
+				add_druntime_sum(rq, p->se.druntime);
+		} else {
+			atomic_long_add(p->se.load.weight, &a7_total_weight);
+			if (p->se.druntime > 0)
+				add_druntime_sum(rq, p->se.druntime);
+		}
+		hmp_calculate_imbalance();
+#endif
+	}
 
 	hrtick_update(rq);
 }
@@ -4301,8 +4396,30 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 		update_cfs_shares(cfs_rq);
 	}
 
-	if (!se)
+	if (!se) {
 		sub_nr_running(rq, 1);
+#ifdef CONFIG_HPERF_HMP
+		if (is_task_hmp(p, NULL))
+			nr_hmp_tasks_dec(rq);
+
+		/*
+		 * Set this field to 0 because if task selected for migration
+		 * fall asleep it will never be selected again for migration.
+		 */
+		p->se.migrate_candidate = 0;
+
+		if (cpu_is_fastest(rq->cpu)) {
+			atomic_long_sub(p->se.load.weight, &a15_total_weight);
+			if (p->se.druntime < 0)
+				sub_druntime_sum(rq, p->se.druntime);
+		} else {
+			atomic_long_sub(p->se.load.weight, &a7_total_weight);
+			if (p->se.druntime > 0)
+				sub_druntime_sum(rq, p->se.druntime);
+		}
+		hmp_calculate_imbalance();
+#endif
+	}
 
 	hrtick_update(rq);
 }
@@ -5848,6 +5965,11 @@ int can_migrate_task(struct task_struct *p, struct lb_env *env)
 	}
 
 	schedstat_inc(p, se.statistics.nr_failed_migrations_hot);
+
+#ifdef CONFIG_HPERF_HMP
+	if (env->src_rq->migrate_task) /*idle pull*/
+		return 1;
+#endif
 	return 0;
 }
 
@@ -5879,6 +6001,10 @@ static struct task_struct *detach_one_task(struct lb_env *env)
 		if (!can_migrate_task(p, env))
 			continue;
 
+#ifdef CONFIG_HPERF_HMP
+		if (p->se.migrate_candidate)
+			continue;
+#endif
 		detach_task(p, env);
 
 		/*
@@ -5946,6 +6072,10 @@ static int detach_tasks(struct lb_env *env)
 		if ((load / 2) > env->imbalance)
 			goto next;
 
+#ifdef CONFIG_HPERF_HMP
+		if (p->se.migrate_candidate)
+			goto next;
+#endif
 		detach_task(p, env);
 		list_add(&p->se.group_node, &env->tasks);
 
@@ -7909,6 +8039,44 @@ unlock:
 
 	return ld_moved;
 }
+
+/**
+ * hmp_do_rebalance(): Checks imbalance in HMP domain and performs balancing.
+ *
+ * @sd: Current sched domain.
+ * @this_cpu: without NO_HZ same as smp_processor_id().
+ *
+ * Returns moved weight.
+ */
+static unsigned int hmp_do_rebalance(struct sched_domain *sd, int this_cpu)
+{
+	unsigned int ld_moved = 0;
+	switch (is_hmp_imbalance(sd)) {
+	case SWAP_TASKS:
+		ld_moved = swap_tasks(sd, this_cpu);
+		break;
+	case A15_TO_A7:
+		ld_moved = move_a15_to_a7(sd, this_cpu);
+		break;
+	case A7_TO_A15:
+		ld_moved = move_a7_to_a15(sd, this_cpu);
+		break;
+	case SKIP_REBALANCE:
+	default:
+		break;
+	}
+	return ld_moved;
+}
+
+/* HMP balancing entry point */
+static unsigned int hmp_load_balance(struct sched_domain *sd,
+				     enum cpu_idle_type idle, int this_cpu)
+{
+	if (idle == CPU_NEWLY_IDLE || idle == CPU_IDLE)
+		return hmp_idle_pull(sd, this_cpu);
+	else
+		return hmp_do_rebalance(sd, this_cpu);
+}
 #endif /* CONFIG_HPERF_HMP */
 
 /*
@@ -7938,6 +8106,11 @@ static int load_balance(int this_cpu, struct rq *this_rq,
 		.tasks		= LIST_HEAD_INIT(env.tasks),
 	};
 
+#ifdef CONFIG_HPERF_HMP
+	/* It is HMP domain, so branch to HPERF_HMP logic */
+	if (sd->flags & SD_HMP_BALANCE)
+		return hmp_load_balance(sd, idle, this_cpu);
+#endif
 	/*
 	 * For NEWLY_IDLE load_balancing, we don't need to consider
 	 * other cpus in our group
@@ -8322,6 +8495,14 @@ static int active_load_balance_cpu_stop(void *data)
 	struct sched_domain *sd;
 	struct task_struct *p = NULL;
 
+#ifdef CONFIG_HPERF_HMP_DEBUG
+	if (!cpumask_test_cpu(target_cpu, cpu_fastest_mask) &&
+	    !cpumask_test_cpu(target_cpu, cpu_slowest_mask)) {
+		pr_emerg("hperf_hmp: %s: for CPU#%i target_cpu is invalid: %i!\n",
+		       __func__, busiest_cpu, target_cpu);
+		BUG();
+	}
+#endif
 	raw_spin_lock_irq(&busiest_rq->lock);
 
 	/* make sure the requested cpu hasn't gone down in the meantime */
@@ -8349,6 +8530,9 @@ static int active_load_balance_cpu_stop(void *data)
 	}
 
 	if (likely(sd)) {
+#ifdef CONFIG_HPERF_HMP
+		struct task_struct *migrate_task;
+#endif
 		struct lb_env env = {
 			.sd		= sd,
 			.dst_cpu	= target_cpu,
@@ -8360,7 +8544,19 @@ static int active_load_balance_cpu_stop(void *data)
 
 		schedstat_inc(sd, alb_count);
 
+#ifdef CONFIG_HPERF_HMP
+		if (env.src_rq->migrate_task) {
+			migrate_task = env.src_rq->migrate_task;
+			p = detach_specified_task(migrate_task, &env);
+			if (p)
+				migrate_task->se.last_migration = jiffies;
+			env.src_rq->migrate_task = NULL;
+		} else {
+			p = detach_one_task(&env);
+		}
+#else
 		p = detach_one_task(&env);
+#endif
 		if (p)
 			schedstat_inc(sd, alb_pushed);
 		else
@@ -9267,8 +9463,12 @@ const struct sched_class fair_sched_class = {
 
 	.task_waking		= task_waking_fair,
 	.task_dead		= task_dead_fair,
+#ifdef CONFIG_HPERF_HMP
+	.set_cpus_allowed	= set_cpus_allowed_hmp,
+#else
 	.set_cpus_allowed	= set_cpus_allowed_common,
 #endif
+#endif
 
 	.set_curr_task          = set_curr_task_fair,
 	.task_tick		= task_tick_fair,
-- 
1.9.1

[PATCH 13/13] hperf_hmp: cpufreq routines.

[Arseniy Krasnov]

	Adds CPU frequency change notifier in fair scheduling class. Every time
when governor changes frequency, it calls callback from this patch. Frequency of
each CPU is used for imbalance calculation.

Signed-off-by: Tarek Dakhran <t.dakhran@samsung.com>
Signed-off-by: Sergey Dyasly <s.dyasly@samsung.com>
Signed-off-by: Dmitriy Safonov <d.safonov@partner.samsung.com>
Signed-off-by: Arseniy Krasnov <a.krasnov@samsung.com>
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
 kernel/sched/fair.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 06f6518..87dc0db 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -33,6 +33,10 @@
 
 #include <trace/events/sched.h>
 
+#ifdef CONFIG_HPERF_HMP
+#include <linux/cpufreq.h>
+#endif
+
 #include "sched.h"
 
 /*
@@ -101,6 +105,11 @@ const_debug unsigned int sysctl_sched_migration_cost = 500000UL;
 unsigned int __read_mostly sysctl_sched_shares_window = 10000000UL;
 
 #ifdef CONFIG_HPERF_HMP
+/*
+ * Log level of hperf_hmp messages. Bigger means more messages.
+ * Maximum level is 3.
+ */
+unsigned int sysctl_sched_hperf_hmp_log_level;
 extern void hmp_set_cpu_masks(struct cpumask *, struct cpumask *);
 static atomic_t a15_nr_hmp_busy = ATOMIC_INIT(0);
 static atomic_t a7_nr_hmp_busy = ATOMIC_INIT(0);
@@ -7229,6 +7238,73 @@ static int should_we_balance(struct lb_env *env)
 	return balance_cpu == env->dst_cpu;
 }
 #ifdef CONFIG_HPERF_HMP
+static void hperf_hmp_vprint(unsigned int log_level, const char *format,
+			  va_list ap)
+{
+	if (sysctl_sched_hperf_hmp_log_level < log_level)
+		return;
+	vprintk(format, ap);
+}
+
+static void hperf_hmp_print(unsigned int log_level, const char *format, ...)
+{
+	va_list ap;
+
+	va_start(ap, format);
+	hperf_hmp_vprint(log_level, format, ap);
+	va_end(ap);
+}
+
+/* Called when frequency is changed */
+static int hmp_cpufreq_callback(struct notifier_block *nb,
+				unsigned long event, void *data)
+{
+	struct cpufreq_freqs *new_freq = data;
+
+	/* recount power only after change of frequency */
+	if (event != CPUFREQ_POSTCHANGE)
+		return NOTIFY_DONE;
+
+	if (!new_freq)
+		return NOTIFY_DONE;
+
+	freq_scale_cpu_power[new_freq->cpu] = (new_freq->new >> 10);
+
+	/* Apply slowdown coefficient of 1.9 for A7 CPUs */
+	if (!cpu_is_fastest(new_freq->cpu)) {
+		freq_scale_cpu_power[new_freq->cpu] *= 10;
+		freq_scale_cpu_power[new_freq->cpu] /= 19;
+	}
+
+	hperf_hmp_print(2, KERN_INFO "hperf_hmp: CPU#%i new frequency is: %u MHz\n",
+		     new_freq->cpu, new_freq->new / 1000);
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block cpufreq_notifier = {
+	.notifier_call = hmp_cpufreq_callback
+};
+
+static int __init register_sched_cpufreq_notifier(void)
+{
+	int err = 0;
+	int cpu;
+
+	for_each_online_cpu(cpu)
+		freq_scale_cpu_power[cpu] = capacity_of(cpu);
+
+	err = cpufreq_register_notifier(&cpufreq_notifier,
+					CPUFREQ_TRANSITION_NOTIFIER);
+	if (!err)
+		pr_info("hperf_hmp: registered cpufreq transition notifier\n");
+	else
+		pr_info("hperf_hmp: failed to register cpufreq notifier!\n");
+
+	return err;
+}
+core_initcall(register_sched_cpufreq_notifier);
+
 /**
  * is_hmp_imbalance(): Calculates imbalance between HMP domains.
  * @sd: Current sched domain.
-- 
1.9.1

Re: [PATCH 11/13] hperf_hmp: task CPU selection logic.

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 3195 bytes --]

Hi Arseniy,

[auto build test WARNING on tip/sched/core]
[also build test WARNING on v4.3 next-20151106]

url:    https://github.com/0day-ci/linux/commits/Arseniy-Krasnov/High-performance-balancing-logic-for-big-LITTLE/20151106-200901
config: x86_64-randconfig-x018-11051832 (attached as .config)
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All warnings (new ones prefixed by >>):

   kernel/sched/fair.c: In function 'select_task_rq_fair':
>> kernel/sched/fair.c:5159:6: warning: suggest explicit braces to avoid ambiguous 'else' [-Wparentheses]
      if (sd_flag & SD_BALANCE_WAKE) /* XXX always ? */
         ^

vim +/else +5159 kernel/sched/fair.c

29cd8bae kernel/sched_fair.c Peter Zijlstra  2009-09-17  5143  			break;
f03542a7 kernel/sched/fair.c Alex Shi        2012-07-26  5144  		}
29cd8bae kernel/sched_fair.c Peter Zijlstra  2009-09-17  5145  
f03542a7 kernel/sched/fair.c Alex Shi        2012-07-26  5146  		if (tmp->flags & sd_flag)
c88d5910 kernel/sched_fair.c Peter Zijlstra  2009-09-10  5147  			sd = tmp;
63b0e9ed kernel/sched/fair.c Mike Galbraith  2015-07-14  5148  		else if (!want_affine)
63b0e9ed kernel/sched/fair.c Mike Galbraith  2015-07-14  5149  			break;
c88d5910 kernel/sched_fair.c Peter Zijlstra  2009-09-10  5150  	}
4ae7d5ce kernel/sched_fair.c Ingo Molnar     2008-03-19  5151  
63b0e9ed kernel/sched/fair.c Mike Galbraith  2015-07-14  5152  	if (affine_sd) {
63b0e9ed kernel/sched/fair.c Mike Galbraith  2015-07-14  5153  		sd = NULL; /* Prefer wake_affine over balance flags */
63b0e9ed kernel/sched/fair.c Mike Galbraith  2015-07-14  5154  		if (cpu != prev_cpu && wake_affine(affine_sd, p, sync))
63b0e9ed kernel/sched/fair.c Mike Galbraith  2015-07-14  5155  			new_cpu = cpu;
8b911acd kernel/sched_fair.c Mike Galbraith  2010-03-11  5156  	}
3b640894 kernel/sched_fair.c Peter Zijlstra  2009-09-16  5157  
63b0e9ed kernel/sched/fair.c Mike Galbraith  2015-07-14  5158  	if (!sd) {
63b0e9ed kernel/sched/fair.c Mike Galbraith  2015-07-14 @5159  		if (sd_flag & SD_BALANCE_WAKE) /* XXX always ? */
9b7aaf11 kernel/sched/fair.c Arseniy Krasnov 2015-11-06  5160  			if (IS_ENABLED(CONFIG_HPERF_HMP) && sync)
9b7aaf11 kernel/sched/fair.c Arseniy Krasnov 2015-11-06  5161  				new_cpu = prev_cpu;
9b7aaf11 kernel/sched/fair.c Arseniy Krasnov 2015-11-06  5162  			else
9b7aaf11 kernel/sched/fair.c Arseniy Krasnov 2015-11-06  5163  				new_cpu = select_idle_sibling(p, prev_cpu);
9b7aaf11 kernel/sched/fair.c Arseniy Krasnov 2015-11-06  5164  	} else {
9b7aaf11 kernel/sched/fair.c Arseniy Krasnov 2015-11-06  5165  #ifdef CONFIG_HPERF_HMP
9b7aaf11 kernel/sched/fair.c Arseniy Krasnov 2015-11-06  5166  		new_cpu = hmp_select_task_rq_fair(p);
9b7aaf11 kernel/sched/fair.c Arseniy Krasnov 2015-11-06  5167  #else

:::::: The code at line 5159 was first introduced by commit
:::::: 63b0e9edceec10fa41ec33393a1515a5ff444277 sched/fair: Beef up wake_wide()

:::::: TO: Mike Galbraith <umgwanakikbuti@gmail.com>
:::::: CC: Ingo Molnar <mingo@kernel.org>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 21230 bytes --]

Re: [PATCH 07/13] hperf_hmp: migration auxiliary functions.

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 6314 bytes --]

Hi Arseniy,

[auto build test WARNING on tip/sched/core]
[also build test WARNING on v4.3 next-20151106]

url:    https://github.com/0day-ci/linux/commits/Arseniy-Krasnov/High-performance-balancing-logic-for-big-LITTLE/20151106-200901
reproduce: make htmldocs

All warnings (new ones prefixed by >>):

   include/linux/init.h:1: warning: no structured comments found
>> kernel/sched/fair.c:7198: warning: No description found for parameter 'p'
>> kernel/sched/fair.c:7198: warning: Excess function parameter 'pm' description in 'detach_specified_task'
   kernel/sys.c:1: warning: no structured comments found
   drivers/dma-buf/seqno-fence.c:1: warning: no structured comments found
   drivers/dma-buf/reservation.c:1: warning: no structured comments found
   include/linux/reservation.h:1: warning: no structured comments found
   include/media/v4l2-dv-timings.h:29: warning: cannot understand function prototype: 'const struct v4l2_dv_timings v4l2_dv_timings_presets[]; '
   include/media/v4l2-dv-timings.h:147: warning: No description found for parameter 'frame_height'
   include/media/v4l2-dv-timings.h:147: warning: No description found for parameter 'hfreq'
   include/media/v4l2-dv-timings.h:147: warning: No description found for parameter 'vsync'
   include/media/v4l2-dv-timings.h:147: warning: No description found for parameter 'active_width'
   include/media/v4l2-dv-timings.h:147: warning: No description found for parameter 'polarities'
   include/media/v4l2-dv-timings.h:147: warning: No description found for parameter 'interlaced'
   include/media/v4l2-dv-timings.h:147: warning: No description found for parameter 'fmt'
   include/media/v4l2-dv-timings.h:171: warning: No description found for parameter 'frame_height'
   include/media/v4l2-dv-timings.h:171: warning: No description found for parameter 'hfreq'
   include/media/v4l2-dv-timings.h:171: warning: No description found for parameter 'vsync'
   include/media/v4l2-dv-timings.h:171: warning: No description found for parameter 'polarities'
   include/media/v4l2-dv-timings.h:171: warning: No description found for parameter 'interlaced'
   include/media/v4l2-dv-timings.h:171: warning: No description found for parameter 'aspect'
   include/media/v4l2-dv-timings.h:171: warning: No description found for parameter 'fmt'
   include/media/v4l2-dv-timings.h:184: warning: No description found for parameter 'hor_landscape'
   include/media/v4l2-dv-timings.h:184: warning: No description found for parameter 'vert_portrait'
   include/media/videobuf2-core.h:112: warning: No description found for parameter 'get_dmabuf'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_alloc'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_put'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_get_dmabuf'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_get_userptr'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_put_userptr'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_prepare'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_finish'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_attach_dmabuf'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_detach_dmabuf'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_map_dmabuf'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_unmap_dmabuf'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_vaddr'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_cookie'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_num_users'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_mem_mmap'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_buf_init'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_buf_prepare'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_buf_finish'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_buf_cleanup'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_buf_queue'
   include/media/videobuf2-core.h:233: warning: No description found for parameter 'cnt_buf_done'
   drivers/media/dvb-core/dvbdev.h:199: warning: Excess function parameter 'device' description in 'dvb_register_device'
   drivers/media/dvb-core/dvbdev.h:199: warning: Excess function parameter 'adapter_nums' description in 'dvb_register_device'
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef member 'e_handler' description in 'hsi_client'
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef member 'pclaimed' description in 'hsi_client'
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef member 'nb' description in 'hsi_client'

vim +/p +7198 kernel/sched/fair.c

  7182		if (task_running(env->src_rq, p)) {
  7183			schedstat_inc(p, se.statistics.nr_failed_migrations_running);
  7184			return 0;
  7185		}
  7186		return 1;
  7187	}
  7188	
  7189	/**
  7190	 * detach_specified_task(): Detaches specified task.
  7191	 * @pm: Task to move.
  7192	 * @env: Migration parameters.
  7193	 *
  7194	 * Returns moved task.
  7195	 */
  7196	static struct task_struct *
  7197	detach_specified_task(struct task_struct *p, struct lb_env *env)
> 7198	{
  7199		lockdep_assert_held(&env->src_rq->lock);
  7200	
  7201		/* If task to move falls asleep, so don't scan runqueue and return */
  7202		if (p->se.migrate_candidate == 0)
  7203			return 0;
  7204	
  7205		if (throttled_lb_pair(task_group(p), env->src_rq->cpu, env->dst_cpu))
  7206			goto exit;

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 6062 bytes --]

Re: [PATCH 00/13] High performance balancing logic for big.LITTLE

[Peter Zijlstra]

No, no, no, no.

This is horrible and exactly what I've been telling people I do not want
to see.

This is very arch specific scheduler code, and very badly done. It
doesn't even call the groups big and little, it goes so far as to put a7
and a15 in sched domain member names.

It doesn't get topology information from device tree but from hard coded
CONFIG strings.

It introduces a swap_task function while we already have one.

It doesn't integrate with any of the other bits that make up and
influence energy consumption such as cpuidle and cpufreq (very minor one
way).

It doesn't mention the existing energy-aware-scheduling effort, nor how
that approach cannot be made to work for this.

It has a completely broken SoB chain.

It introduces new metrics (like druntime) without first defining them;
and in general very poor Changelogs.

In general, this makes me very sad. Please start by participating in the
existing discussion.