[RFC PATCH 0/1] sched: Add a new API to find the prefer idlest cpu

[RFC PATCH 0/1] sched: Add a new API to find the prefer idlest cpu

[Shirley Ma]

Introduce a new API to choose per-cpu thread from cgroup control cpuset
(allowed) and preferred cpuset (local numa-node).

The receiving cpus of a networking device are not under cgroup controls.
When such a networking device uses per-cpu thread model, the cpu which
is chose to process the packets might not be part of cgroup cpusets
without this API. On numa system, the preferred cpusets would help to
reduce expensive cross memory access to/from the other node.

Signed-off-by: Shirley Ma <xma@us.ibm.com>
---

 include/linux/sched.h |    2 ++
 kernel/sched/fair.c   |   30 ++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 0 deletions(-)

Thanks
Shirley

[RFC PATCH 1/1] sched: Add a new API to find the prefer idlest cpu

[Shirley Ma]

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 64d9df5..46cc4a7 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2806,4 +2806,6 @@ static inline unsigned long rlimit_max(unsigned int limit)
 
 #endif /* __KERNEL__ */
 
+extern int find_idlest_prefer_cpu(struct cpumask *prefer,
+				 struct cpumask *allowed, int prev_cpu);
 #endif
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index c099cc6..7240868 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -26,6 +26,7 @@
 #include <linux/slab.h>
 #include <linux/profile.h>
 #include <linux/interrupt.h>
+#include <linux/export.h>
 
 #include <trace/events/sched.h>
 
@@ -2809,6 +2810,35 @@ unlock:
 
 	return new_cpu;
 }
+
+/*
+ * This API is used to find the most idle cpu from both preferred and
+ * allowed cpuset (such as cgroup controls cpuset). It helps per-cpu thread
+ * model to pick up the allowed local cpu to be scheduled.
+ * If these two cpusets have intersects, the cpu is chose from the intersects,
+ * if there is no intersects, then the cpu is chose from the allowed cpuset.
+ * prev_cpu helps to better local cache when prev_cpu is not busy.
+ */
+int find_idlest_prefer_cpu(struct cpumask *prefer, struct cpumask *allowed,
+			  int prev_cpu)
+{
+	unsigned long load, min_load = ULONG_MAX;
+	int check, i, idlest = -1;
+
+	check = cpumask_intersects(prefer, allowed);
+	/* Traverse only the allowed CPUs */
+	if (check == 0)
+		prefer = allowed;
+	for_each_cpu_and(i, prefer, allowed) {
+		load = weighted_cpuload(i);
+		if (load < min_load || (load == min_load && i == prev_cpu)) {
+			min_load = load;
+			idlest = i;
+		}
+	}
+	return idlest;
+}
+EXPORT_SYMBOL(find_idlest_prefer_cpu);
 #endif /* CONFIG_SMP */
 
 static unsigned long

Shirley

Re: [RFC PATCH 0/1] sched: Add a new API to find the prefer idlest cpu

[Shirley Ma]

Hello Ingo, Peter,

	Have you had chance to review below patch?

Thanks
Shirley

On Sun, 2012-07-22 at 23:57 -0700, Shirley Ma wrote:
> Introduce a new API to choose per-cpu thread from cgroup control cpuset
> (allowed) and preferred cpuset (local numa-node).
> 
> The receiving cpus of a networking device are not under cgroup controls.
> When such a networking device uses per-cpu thread model, the cpu which
> is chose to process the packets might not be part of cgroup cpusets
> without this API. On numa system, the preferred cpusets would help to
> reduce expensive cross memory access to/from the other node.
> 
> Signed-off-by: Shirley Ma <xma@us.ibm.com>
> ---
> 
>  include/linux/sched.h |    2 ++
>  kernel/sched/fair.c   |   30 ++++++++++++++++++++++++++++++
>  2 files changed, 32 insertions(+), 0 deletions(-)
> 
> Thanks
> Shirley
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [RFC PATCH 0/1] sched: Add a new API to find the prefer idlest cpu

[Peter Zijlstra]

On Fri, 2012-08-17 at 08:39 -0700, Shirley Ma wrote:

> Hello Ingo, Peter,
>         Have you had chance to review below patch?

Well no of course not, nobody CC'ed us..

Your patch submission sucks, you split the changelog and the actual
patch into two different emails.

It doesn't mention who would be calling this and how often. Nor does it
seem to explain where the cpumask arguments come from.

Please try again.

Re: [RFC PATCH 0/1] sched: Add a new API to find the prefer idlest cpu

[Shirley Ma]

On Fri, 2012-08-17 at 18:48 +0200, Peter Zijlstra wrote:
> On Fri, 2012-08-17 at 08:39 -0700, Shirley Ma wrote:
> 
> > Hello Ingo, Peter,
> >         Have you had chance to review below patch?
> 
> Well no of course not, nobody CC'ed us..
> 
> Your patch submission sucks, you split the changelog and the actual
> patch into two different emails.
> 
> It doesn't mention who would be calling this and how often. Nor does
> it
> seem to explain where the cpumask arguments come from.
> 
> Please try again.

Thanks for your comments. I must be spoiled by net-dev maintainers. I
used to submit the patch this way thought my linux email in the past
many years. So I need to reply on the changelog for the actual patch? I
hope I won't make any more suck submissions again.

I will try to explain more details why this API needs to be exported.

Thanks again.
Shirley