* [PATCH] sched/core: Reduce cost of sched_move_task when config autogroup
@ 2023-03-19 7:56 wuchi
2023-03-20 8:28 ` Peter Zijlstra
0 siblings, 1 reply; 4+ messages in thread
From: wuchi @ 2023-03-19 7:56 UTC (permalink / raw)
To: mingo, peterz, juri.lelli, vincent.guittot, dietmar.eggemann,
rostedt, bsegall, mgorman, bristot, vschneid
Cc: linux-kernel
Some sched_move_task calls of autogroup is useless when the
task_struct->sched_task_group isn't changed because of task_group
of cpu_cgroup overlay task_group of autogroup. The overlay key codes
are as follows:
sched_cgroup_fork->autogroup_task_group->task_wants_autogroup
sched_change_group->autogroup_task_group->autogroup_task_group
sched_move_task eg:
task A belongs to cpu_cgroup0 and autogroup0, it will always to
cpu_cgroup0 when doing exit. So there is no need to do {de|en}queue.
The call graph is as follow.
do_exit
sched_autogroup_exit_task
sched_move_task
dequeue_task
sched_change_group
A.sched_task_group = sched_get_task_group
enqueue_task
So do some check before dequeue task in sched_move_task.
Signed-off-by: wuchi <wuchi.zero@gmail.com>
---
kernel/sched/core.c | 29 +++++++++++++++++++++++++++--
1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index a380f34789a2..acc9a0e391f4 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -10330,7 +10330,7 @@ void sched_release_group(struct task_group *tg)
spin_unlock_irqrestore(&task_group_lock, flags);
}
-static void sched_change_group(struct task_struct *tsk)
+static struct task_group *sched_get_task_group(struct task_struct *tsk)
{
struct task_group *tg;
@@ -10342,7 +10342,28 @@ static void sched_change_group(struct task_struct *tsk)
tg = container_of(task_css_check(tsk, cpu_cgrp_id, true),
struct task_group, css);
tg = autogroup_task_group(tsk, tg);
- tsk->sched_task_group = tg;
+
+ return tg;
+}
+
+static bool sched_task_group_changed(struct task_struct *tsk)
+{
+ /*
+ * Some sched_move_task calls of autogroup is useless when the
+ * task_struct->sched_task_group isn't changed because of task_group
+ * of cpu_cgroup overlay task_group of autogroup. so do some check
+ * before dequeue task in sched_move_task.
+ */
+#ifdef CONFIG_SCHED_AUTOGROUP
+ return sched_get_task_group(tsk) != tsk->sched_task_group;
+#else
+ return true;
+#endif /* CONFIG_SCHED_AUTOGROUP */
+}
+
+static void sched_change_group(struct task_struct *tsk)
+{
+ tsk->sched_task_group = sched_get_task_group(tsk);
#ifdef CONFIG_FAIR_GROUP_SCHED
if (tsk->sched_class->task_change_group)
@@ -10369,6 +10390,9 @@ void sched_move_task(struct task_struct *tsk)
rq = task_rq_lock(tsk, &rf);
update_rq_clock(rq);
+ if (!sched_task_group_changed(tsk))
+ goto unlock;
+
running = task_current(rq, tsk);
queued = task_on_rq_queued(tsk);
@@ -10391,6 +10415,7 @@ void sched_move_task(struct task_struct *tsk)
resched_curr(rq);
}
+unlock:
task_rq_unlock(rq, tsk, &rf);
}
--
2.20.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] sched/core: Reduce cost of sched_move_task when config autogroup
2023-03-19 7:56 [PATCH] sched/core: Reduce cost of sched_move_task when config autogroup wuchi
@ 2023-03-20 8:28 ` Peter Zijlstra
2023-03-20 12:40 ` chi wu
0 siblings, 1 reply; 4+ messages in thread
From: Peter Zijlstra @ 2023-03-20 8:28 UTC (permalink / raw)
To: wuchi
Cc: mingo, juri.lelli, vincent.guittot, dietmar.eggemann, rostedt,
bsegall, mgorman, bristot, vschneid, linux-kernel
On Sun, Mar 19, 2023 at 03:56:43PM +0800, wuchi wrote:
> Some sched_move_task calls of autogroup is useless when the
> task_struct->sched_task_group isn't changed because of task_group
> of cpu_cgroup overlay task_group of autogroup. The overlay key codes
> are as follows:
>
> sched_cgroup_fork->autogroup_task_group->task_wants_autogroup
> sched_change_group->autogroup_task_group->autogroup_task_group
>
> sched_move_task eg:
> task A belongs to cpu_cgroup0 and autogroup0, it will always to
> cpu_cgroup0 when doing exit. So there is no need to do {de|en}queue.
> The call graph is as follow.
>
> do_exit
> sched_autogroup_exit_task
> sched_move_task
> dequeue_task
> sched_change_group
> A.sched_task_group = sched_get_task_group
> enqueue_task
>
> So do some check before dequeue task in sched_move_task.
No immediate objection; but the $subject seems to suggest you did this
for performance reasons, yet there are no performance results in this
Changelog. How much does this save under what load?
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] sched/core: Reduce cost of sched_move_task when config autogroup
2023-03-20 8:28 ` Peter Zijlstra
@ 2023-03-20 12:40 ` chi wu
2023-03-20 15:13 ` Peter Zijlstra
0 siblings, 1 reply; 4+ messages in thread
From: chi wu @ 2023-03-20 12:40 UTC (permalink / raw)
To: Peter Zijlstra
Cc: mingo, juri.lelli, vincent.guittot, dietmar.eggemann, rostedt,
bsegall, mgorman, bristot, vschneid, linux-kernel
Peter Zijlstra <peterz@infradead.org> 于2023年3月20日周一 16:28写道:
>
> On Sun, Mar 19, 2023 at 03:56:43PM +0800, wuchi wrote:
> > Some sched_move_task calls of autogroup is useless when the
> > task_struct->sched_task_group isn't changed because of task_group
> > of cpu_cgroup overlay task_group of autogroup. The overlay key codes
> > are as follows:
> >
> > sched_cgroup_fork->autogroup_task_group->task_wants_autogroup
> > sched_change_group->autogroup_task_group->autogroup_task_group
> >
> > sched_move_task eg:
> > task A belongs to cpu_cgroup0 and autogroup0, it will always to
> > cpu_cgroup0 when doing exit. So there is no need to do {de|en}queue.
> > The call graph is as follow.
> >
> > do_exit
> > sched_autogroup_exit_task
> > sched_move_task
> > dequeue_task
> > sched_change_group
> > A.sched_task_group = sched_get_task_group
> > enqueue_task
> >
> > So do some check before dequeue task in sched_move_task.
>
> No immediate objection; but the $subject seems to suggest you did this
> for performance reasons, yet there are no performance results in this
> Changelog. How much does this save under what load?
cpu:bogomips=4600.00
kernel:6.3.0-rc3
cpu cgroup: 6:cpu,cpuacct:/user.slice
run cmds:
#!/bin/bash
for i in {0..10000}; do
sleep 0 &
done
wait
bpftrace -e 'k:sched_move_task { @ts[tid] = nsecs; }
kr:sched_move_task /@ts[tid]/ { @ns += nsecs - @ts[tid];
delete(@ts[tid]); }'
cost time(ns):
without patch: 43528033
with patch: 18934496
diff: -24593537 -56.5%
About the change, move sched_task_group_changed before
update_rq_clock as following and add the performance results in
the Changelog?
@@ -10369,6 +10390,9 @@ void sched_move_task(struct task_struct *tsk)
rq = task_rq_lock(tsk, &rf);
+ if (!sched_task_group_changed(tsk))
+ goto unlock;
+
update_rq_clock(rq);
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] sched/core: Reduce cost of sched_move_task when config autogroup
2023-03-20 12:40 ` chi wu
@ 2023-03-20 15:13 ` Peter Zijlstra
0 siblings, 0 replies; 4+ messages in thread
From: Peter Zijlstra @ 2023-03-20 15:13 UTC (permalink / raw)
To: chi wu
Cc: mingo, juri.lelli, vincent.guittot, dietmar.eggemann, rostedt,
bsegall, mgorman, bristot, vschneid, linux-kernel
On Mon, Mar 20, 2023 at 08:40:51PM +0800, chi wu wrote:
> Peter Zijlstra <peterz@infradead.org> 于2023年3月20日周一 16:28写道:
> >
> > On Sun, Mar 19, 2023 at 03:56:43PM +0800, wuchi wrote:
> > > Some sched_move_task calls of autogroup is useless when the
> > > task_struct->sched_task_group isn't changed because of task_group
> > > of cpu_cgroup overlay task_group of autogroup. The overlay key codes
> > > are as follows:
> > >
> > > sched_cgroup_fork->autogroup_task_group->task_wants_autogroup
> > > sched_change_group->autogroup_task_group->autogroup_task_group
> > >
> > > sched_move_task eg:
> > > task A belongs to cpu_cgroup0 and autogroup0, it will always to
> > > cpu_cgroup0 when doing exit. So there is no need to do {de|en}queue.
> > > The call graph is as follow.
> > >
> > > do_exit
> > > sched_autogroup_exit_task
> > > sched_move_task
> > > dequeue_task
> > > sched_change_group
> > > A.sched_task_group = sched_get_task_group
> > > enqueue_task
> > >
> > > So do some check before dequeue task in sched_move_task.
> >
> > No immediate objection; but the $subject seems to suggest you did this
> > for performance reasons, yet there are no performance results in this
> > Changelog. How much does this save under what load?
>
> cpu:bogomips=4600.00
> kernel:6.3.0-rc3
> cpu cgroup: 6:cpu,cpuacct:/user.slice
>
> run cmds:
>
> #!/bin/bash
> for i in {0..10000}; do
> sleep 0 &
> done
> wait
>
> bpftrace -e 'k:sched_move_task { @ts[tid] = nsecs; }
> kr:sched_move_task /@ts[tid]/ { @ns += nsecs - @ts[tid];
> delete(@ts[tid]); }'
>
> cost time(ns):
> without patch: 43528033
> with patch: 18934496
> diff: -24593537 -56.5%
>
> About the change, move sched_task_group_changed before
> update_rq_clock as following and add the performance results in
> the Changelog?
Yes please, that gives good motivation for the change.
Thanks!
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-03-20 15:19 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-19 7:56 [PATCH] sched/core: Reduce cost of sched_move_task when config autogroup wuchi
2023-03-20 8:28 ` Peter Zijlstra
2023-03-20 12:40 ` chi wu
2023-03-20 15:13 ` Peter Zijlstra
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.