From: David Vernet <void@manifault.com>
To: Abel Wu <wuyun.abel@bytedance.com>
Cc: linux-kernel@vger.kernel.org, mingo@redhat.com,
peterz@infradead.org, juri.lelli@redhat.com,
vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
bristot@redhat.com, vschneid@redhat.com, gautham.shenoy@amd.com,
kprateek.nayak@amd.com, aaron.lu@intel.com, clm@meta.com,
tj@kernel.org, roman.gushchin@linux.dev, kernel-team@meta.com
Subject: Re: [PATCH v2 4/7] sched/fair: Add SHARED_RUNQ sched feature and skeleton calls
Date: Wed, 12 Jul 2023 16:34:30 -0500 [thread overview]
Message-ID: <20230712213430.GE12207@maniforge> (raw)
In-Reply-To: <6111b87c-b68c-2ba9-ac60-333af67082fd@bytedance.com>
On Wed, Jul 12, 2023 at 04:39:03PM +0800, Abel Wu wrote:
> On 7/11/23 4:03 AM, David Vernet wrote:
> > @@ -6467,6 +6489,9 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
> > dequeue_throttle:
> > util_est_update(&rq->cfs, p, task_sleep);
> > hrtick_update(rq);
> > +
> > + if (sched_feat(SHARED_RUNQ))
> > + shared_runq_dequeue_task(p);
>
> Would disabling SHARED_RUNQ causing task list nodes left
> in the shared stateful runqueue?
Hi Abel,
Yes, good call, there will be some stale tasks. The obvious way to get
around that would of course be to always call
shared_runq_dequeue_task(p) on the __dequeue_entity() path, but it would
be silly to tax a hot path in the scheduler in support of a feature
that's disabled by default.
At first I was thinking that the only issue would be some overhead in
clearing stale tasks once it was re-enabled, but that we'd be OK because
of this check in shared_runq_pick_next_task():
298 if (task_on_rq_queued(p) && !task_on_cpu(rq, p)) {
299 update_rq_clock(src_rq);
300 src_rq = move_queued_task(src_rq, &src_rf, p, cpu_of(rq));
301 }
So we wouldn't migrate tasks that weren't actually suitable. But that's
obviously wrong. It's not safe to keep stale tasks in that list for (at
least) two reasons.
- A task could exit (which would be easy to fix by just adding a dequeue
call in task_dead_fair())
- We could have a double-add if a task is re-enqueued in the list after
having been previously enqueued, but then never dequeued due to the
timing of disabling SHARED_RUNQ.
Not sure what the best solution is here. We could always address this by
draining the list when the feature is disabled, but there's not yet a
mechanism to hook in a callback to be invoked when a scheduler feature
is enabled/disabled. It shouldn't be too hard to add that, assuming
other sched folks are amenable to it. It should just be a matter of
adding another __SCHED_FEAT_NR-sized table of NULL-able callbacks that
are invoked on enable / disable state change, and which can be specified
in a new SCHED_FEAT_CALLBACK or something macro.
Peter -- what do you think?
Thanks,
David
next prev parent reply other threads:[~2023-07-12 21:34 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-10 20:03 [PATCH v2 0/7] sched: Implement shared runqueue in CFS David Vernet
2023-07-10 20:03 ` [PATCH v2 1/7] sched: Expose move_queued_task() from core.c David Vernet
2023-07-10 20:03 ` [PATCH v2 2/7] sched: Move is_cpu_allowed() into sched.h David Vernet
2023-07-10 20:03 ` [PATCH v2 3/7] sched: Check cpu_active() earlier in newidle_balance() David Vernet
2023-07-10 20:03 ` [PATCH v2 4/7] sched/fair: Add SHARED_RUNQ sched feature and skeleton calls David Vernet
2023-07-11 9:45 ` Peter Zijlstra
2023-07-11 16:19 ` David Vernet
2023-07-12 8:39 ` Abel Wu
2023-07-12 21:34 ` David Vernet [this message]
2023-07-10 20:03 ` [PATCH v2 5/7] sched: Implement shared runqueue in CFS David Vernet
2023-07-11 10:18 ` Peter Zijlstra
2023-07-11 16:26 ` David Vernet
2023-07-12 6:00 ` Gautham R. Shenoy
2023-07-12 19:13 ` David Vernet
2023-07-12 10:47 ` Abel Wu
2023-07-12 22:16 ` David Vernet
2023-07-13 3:43 ` Abel Wu
2023-07-13 4:05 ` David Vernet
2023-07-13 7:58 ` Aaron Lu
2023-07-13 8:29 ` Peter Zijlstra
2023-07-10 20:03 ` [PATCH v2 6/7] sched: Shard per-LLC shared runqueues David Vernet
2023-07-11 10:49 ` Peter Zijlstra
2023-07-11 19:57 ` David Vernet
2023-07-12 10:06 ` Gautham R. Shenoy
2023-07-12 12:22 ` Peter Zijlstra
2023-07-10 20:03 ` [PATCH v2 7/7] sched: Move shared_runq to __{enqueue,dequeue}_entity() David Vernet
2023-07-11 10:51 ` Peter Zijlstra
2023-07-11 16:30 ` David Vernet
2023-07-11 11:42 ` [PATCH v2 0/7] sched: Implement shared runqueue in CFS Peter Zijlstra
2023-07-11 21:33 ` David Vernet
2023-07-21 9:12 ` Gautham R. Shenoy
2023-07-25 20:22 ` David Vernet
2023-08-02 6:32 ` Gautham R. Shenoy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230712213430.GE12207@maniforge \
--to=void@manifault.com \
--cc=aaron.lu@intel.com \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=clm@meta.com \
--cc=dietmar.eggemann@arm.com \
--cc=gautham.shenoy@amd.com \
--cc=juri.lelli@redhat.com \
--cc=kernel-team@meta.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=roman.gushchin@linux.dev \
--cc=rostedt@goodmis.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=wuyun.abel@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).