From: Peter Zijlstra <peterz@infradead.org>
To: Youssef Esmat <youssefesmat@chromium.org>
Cc: Daniel Jordan <daniel.m.jordan@oracle.com>,
mingo@kernel.org, vincent.guittot@linaro.org,
linux-kernel@vger.kernel.org, juri.lelli@redhat.com,
dietmar.eggemann@arm.com, rostedt@goodmis.org,
bsegall@google.com, mgorman@suse.de, bristot@redhat.com,
corbet@lwn.net, qyousef@layalina.io, chris.hyser@oracle.com,
patrick.bellasi@matbug.net, pjt@google.com, pavel@ucw.cz,
qperret@google.com, tim.c.chen@linux.intel.com,
joshdon@google.com, timj@gnu.org, kprateek.nayak@amd.com,
yu.c.chen@intel.com, joel@joelfernandes.org, efault@gmx.de,
tglx@linutronix.de
Subject: Re: [PATCH 00/15] sched: EEVDF and latency-nice and/or slice-attr
Date: Thu, 5 Oct 2023 14:05:57 +0200 [thread overview]
Message-ID: <20231005120557.GA743@noisy.programming.kicks-ass.net> (raw)
In-Reply-To: <20231002184136.GA1539@noisy.programming.kicks-ass.net>
[-- Attachment #1: Type: text/plain, Size: 10061 bytes --]
On Mon, Oct 02, 2023 at 08:41:36PM +0200, Peter Zijlstra wrote:
> When mixing request sizes things become a little more interesting.
>
> Let me ponder this a little bit more.
Using the attached program (I got *REALLY* fed up trying to draw these
diagrams by hand), let us illustrate the difference between Earliest
*Eligible* Virtual Deadline First and the one with the Eligible test
taken out: EVDF.
Specifically, the program was used with the following argument for
EEVDF:
./eevdf -e "0,1024,6" -e "1,1024,2" -e "2,1024,18" -v 19
and with an additional '-n' for the EVDF column.
EEVDF EVDF
d = 6 d = 6
d = 2 d = 2
d = 18 d = 18
q = 2 q = 2
t=0 V=1 t=0 V=1
A |----< A |----<
>B |< >B |<
C |----------------< C |----------------<
|*--------|---------|---------|---------|---- |*--------|---------|---------|---------|----
t=2 V=1 t=2 V=1
>A |----< A |----<
B |< >B |<
C |----------------< C |----------------<
|*--------|---------|---------|---------|---- |*--------|---------|---------|---------|----
t=8 V=3 t=4 V=2
A |----< >A |----<
>B |< B |<
C |----------------< C |----------------<
|--*------|---------|---------|---------|---- |-*-------|---------|---------|---------|----
t=10 V=4 t=10 V=4
A |----< A |----<
B |< >B |<
>C |----------------< C |----------------<
|---*-----|---------|---------|---------|---- |---*-----|---------|---------|---------|----
t=28 V=10 t=12 V=5
A |----< A |----<
>B |< >B |<
C |----------------< C |----------------<
|---------*---------|---------|---------|---- |----*----|---------|---------|---------|----
t=30 V=11 t=14 V=5
A |----< A |----<
>B |< >B |<
C |----------------< C |----------------<
|---------|*--------|---------|---------|---- |----*----|---------|---------|---------|----
t=32 V=11 t=16 V=6
A |----< >A |----<
>B |< B |<
C |----------------< C |----------------<
|---------|*--------|---------|---------|---- |-----*---|---------|---------|---------|----
t=34 V=12 t=22 V=8
>A |----< A |----<
B |< >B |<
C |----------------< C |----------------<
|---------|-*-------|---------|---------|---- |-------*-|---------|---------|---------|----
t=40 V=14 t=24 V=9
A |----< A |----<
>B |< >B |<
C |----------------< C |----------------<
|---------|---*-----|---------|---------|---- |--------*|---------|---------|---------|----
t=42 V=15 t=26 V=9
A |----< A |----<
>B |< >B |<
C |----------------< C |----------------<
|---------|----*----|---------|---------|---- |--------*|---------|---------|---------|----
t=44 V=15 t=28 V=10
A |----< >A |----<
>B |< B |<
C |----------------< C |----------------<
|---------|----*----|---------|---------|---- |---------*---------|---------|---------|----
t=46 V=16 t=34 V=12
>A |----< A |----<
B |< >B |<
C |----------------< C |----------------<
|---------|-----*---|---------|---------|---- |---------|-*-------|---------|---------|----
t=52 V=18 t=36 V=13
A |----< A |----<
>B |< B |<
C |----------------< >C |----------------<
|---------|-------*-|---------|---------|---- |---------|--*------|---------|---------|----
t=54 V=19 t=54 V=19
A |----< A |----<
>B |< >B |<
C |----------------< C |----------------<
|---------|--------*|---------|---------|---- |---------|--------*|---------|---------|----
lags: -10, 6 lags: -7, 11
BAaaBCccccccccBBBAaaBBBAaaBB BBAaaBBBAaaBBBAaaBCccccccccB
As I wrote before; EVDF has worse lag bounds, but this is not
insurmountable. The biggest problem that I can see is that of wakeup
preemption. Currently we allow to preempt when 'current' has reached V
(RUN_TO_PARITY in pick_eevdf()).
With these rules, when EEVDF schedules C (our large slice task) at t=10
above, it is only a little behind C and can be reaily preempted after
about 2 time units.
However, EVDF will delay scheduling C until much later, see how A and B
walk far ahead of V until t=36. Only when will we pick C. But this means
that we're firmly stuck with C for at least 11 time units. A newly
placed task will be around V and will have no chance to preempt.
That said, I do have me a patch to cure some of that:
https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/commit/?h=sched/eevdf&id=d7edbe431f31762e516f2730196f41322edcc621
That would allow a task with a shorter request time to preempt in spite
of RUN_TO_PARITY.
However, in this example V is only 2/3 of the way to C's deadline, but
it we were to have many more tasks, you'll see V gets closer and closer
to C's deadline and it will become harder and harder to place such that
preemption becomes viable.
Adding 4 more tasks:
./eevdf -e "0,1024,6" -e "1,1024,2" -e "2,1024,18" -v 19 -n -e "3,1024,2" -e "4,1024,2" -e "5,1024,2" -e "6,1024,2"
t=92 V=16
A |----<
B |<
>C |----------------<
D |<
E |<
F |<
G |<
|---------|-----*---|---------|---------|----
And I worry this will create very real latency spikes.
That said; I do see not having the eligibility check can help. So I'm
not opposed to having a sched_feat for this, but I would not want to
default to EVDF.
[-- Attachment #2: eevdf.c --]
[-- Type: text/x-csrc, Size: 4009 bytes --]
/* GPL-2.0 */
#include <stdio.h>
#include <limits.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdbool.h>
#include <sys/param.h>
bool eligible = true;
unsigned long V_lim = 20;
struct entity {
unsigned long vruntime;
unsigned long weight;
unsigned long request;
unsigned long vdeadline;
int idx;
};
unsigned int gcd(unsigned int a, unsigned int b)
{
int gcd, m = MIN(a, b);
for (int i = 1; i <= m; i++) {
if (a%i == 0 && b%i == 0)
gcd = i;
}
return gcd;
}
int init_entities(int nr, struct entity *es)
{
unsigned int q = 0;
for (int i = 0; i < nr; i++) {
unsigned long d = (1024 * es[i].request) / es[i].weight;
printf("d = %d\n", d);
if (!q)
q = d;
else
q = gcd(q, d);
es[i].vdeadline = es[i].vruntime + d;
es[i].idx = i;
}
printf("q = %d\n\n", q);
return q;
}
int run_entity(struct entity *e)
{
unsigned long d = e->vdeadline - e->vruntime;
d *= e->weight;
d /= 1024;
e->vruntime = e->vdeadline;
e->vdeadline += (1024 * e->request) / e->weight;
return d;
}
unsigned long avg_vruntime(int nr, struct entity *es)
{
unsigned long W = 0, V = 0;
for (int i = 0; i < nr; i++) {
V += es[i].weight * es[i].vruntime;
W += es[i].weight;
}
V /= W;
return V;
}
struct entity *pick_entity(int nr, struct entity *es)
{
unsigned long W = 0, V = 0;
struct entity *e = NULL;
for (int i = 0; i < nr; i++) {
V += es[i].weight * es[i].vruntime;
W += es[i].weight;
}
for (int i = 0; i < nr; i++) {
if (eligible && W*es[i].vruntime > V)
continue;
if (!e || es[i].vdeadline < e->vdeadline)
e = &es[i];
}
return e;
}
void __print_space(int n)
{
for (int i = 0; i < n; i++)
putchar(' ');
}
void __print_arrow(int n)
{
putchar('|');
for (int i = 1; i < (n-1); i++)
putchar('-');
putchar('<');
}
void print_entity(struct entity *e)
{
__print_space(e->vruntime);
__print_arrow(e->vdeadline - e->vruntime);
}
void print_entities(int nr, struct entity *es, struct entity *p)
{
for (int i = 0; i < nr; i++) {
if (&es[i] == p)
putchar('>');
else
putchar(' ');
putchar('A' + i);
putchar(' ');
print_entity(&es[i]);
putchar('\n');
}
}
void print_timeline(unsigned long V)
{
char timeline[] = "|---------|---------|---------|---------|----";
if (V > sizeof(timeline)-1) {
printf("Whoopsie! out of time\n");
exit(0);
}
timeline[V] = '*';
__print_space(3);
puts(timeline);
putchar('\n');
}
void update_lags(int nr, struct entity *es, unsigned long V, long *min, long *max)
{
for (int i = 0; i < nr; i++) {
long lag = V - es[i].vruntime;
if (lag < *min)
*min = lag;
if (lag > *max)
*max = lag;
}
}
int main(int argc, char *argv[])
{
unsigned int s = 0, t = 0, n = 0, q = 1;
long min_lag = 0, max_lag = 0;
struct entity *e, es[8];
unsigned long V;
char S[1024];
int opt;
const int N = sizeof(es) / sizeof(es[0]);
while ((opt = getopt(argc, argv, "nv:e:")) != -1) {
unsigned int v,w,r;
switch (opt) {
case 'n':
eligible = false;
break;
case 'v':
V_lim = atol(optarg);
break;
case 'e':
if (n >= N) {
printf("Whoopsie! too many entities\n");
exit(0);
}
if (sscanf(optarg, "%u,%u,%u", &v,&w,&r) == 3) {
es[n++] = (struct entity) {
.vruntime = v,
.weight = w,
.request = r,
};
}
break;
default:
printf("Whoopsie!, bad arguments\n");
exit(0);
}
}
if (!n) {
printf("Whoopsie!, no entities\n");
exit(0);
}
q = init_entities(n, es);
do {
int d;
V = avg_vruntime(n, es);
printf("t=%d V=%ld\n", t, V);
update_lags(n, es, V, &min_lag, &max_lag);
e = pick_entity(n, es);
if (!e) {
printf("Whoopsie, no pick\n");
exit(0);
}
print_entities(n, es, e);
print_timeline(V);
d = run_entity(e);
t += d;
for (int i = 0; i < d; i += q) {
char c = 'A' + e->idx;
if (i)
c = 'a' + e->idx;
S[s++] = c;
S[s] = '\0';
}
putchar('\n');
} while (V < V_lim);
printf("lags: %ld, %ld\n\n", min_lag, max_lag);
puts(S);
putchar('\n');
return 0;
}
next prev parent reply other threads:[~2023-10-05 14:32 UTC|newest]
Thread overview: 104+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-31 11:58 [PATCH 00/15] sched: EEVDF and latency-nice and/or slice-attr Peter Zijlstra
2023-05-31 11:58 ` [PATCH 01/15] sched/fair: Add avg_vruntime Peter Zijlstra
2023-06-02 13:51 ` Vincent Guittot
2023-06-02 14:27 ` Peter Zijlstra
2023-06-05 7:18 ` Vincent Guittot
2023-08-10 7:10 ` [tip: sched/core] sched/fair: Add cfs_rq::avg_vruntime tip-bot2 for Peter Zijlstra
2023-10-11 4:15 ` [PATCH 01/15] sched/fair: Add avg_vruntime Abel Wu
2023-10-11 7:30 ` Peter Zijlstra
2023-10-11 8:30 ` Abel Wu
2023-10-11 9:45 ` Peter Zijlstra
2023-10-11 10:05 ` Peter Zijlstra
2023-10-11 13:08 ` Peter Zijlstra
2023-05-31 11:58 ` [PATCH 02/15] sched/fair: Remove START_DEBIT Peter Zijlstra
2023-08-10 7:10 ` [tip: sched/core] sched/fair: Remove sched_feat(START_DEBIT) tip-bot2 for Peter Zijlstra
2023-05-31 11:58 ` [PATCH 03/15] sched/fair: Add lag based placement Peter Zijlstra
2023-08-10 7:10 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2023-10-11 12:00 ` [PATCH 03/15] " Abel Wu
2023-10-11 13:24 ` Peter Zijlstra
2023-10-12 7:04 ` Abel Wu
2023-10-13 7:37 ` Peter Zijlstra
2023-10-13 8:14 ` Abel Wu
2023-10-12 19:15 ` Benjamin Segall
2023-10-12 22:34 ` Peter Zijlstra
2023-10-13 16:35 ` Peter Zijlstra
2023-10-14 8:08 ` Mike Galbraith
2023-10-13 14:34 ` Peter Zijlstra
2023-05-31 11:58 ` [PATCH 04/15] rbtree: Add rb_add_augmented_cached() helper Peter Zijlstra
2023-08-10 7:10 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2023-05-31 11:58 ` [PATCH 05/15] sched/fair: Implement an EEVDF like policy Peter Zijlstra
2023-08-10 7:10 ` [tip: sched/core] sched/fair: Implement an EEVDF-like scheduling policy tip-bot2 for Peter Zijlstra
2023-09-29 21:40 ` [PATCH 05/15] sched/fair: Implement an EEVDF like policy Benjamin Segall
2023-10-02 17:39 ` Peter Zijlstra
2023-10-11 4:14 ` Abel Wu
2023-10-11 7:33 ` Peter Zijlstra
2023-10-11 11:49 ` Abel Wu
2023-09-30 0:09 ` [PATCH] sched/fair: fix pick_eevdf to always find the correct se Benjamin Segall
2023-10-03 10:42 ` [tip: sched/urgent] sched/fair: Fix pick_eevdf() tip-bot2 for Benjamin Segall
[not found] ` <CGME20231004203940eucas1p2f73b017497d1f4239a6e236fdb6019e2@eucas1p2.samsung.com>
2023-10-04 20:39 ` [PATCH] sched/fair: fix pick_eevdf to always find the correct se Marek Szyprowski
2023-10-09 7:53 ` [tip: sched/urgent] sched/eevdf: Fix pick_eevdf() tip-bot2 for Benjamin Segall
2023-10-11 12:12 ` [PATCH] sched/fair: fix pick_eevdf to always find the correct se Abel Wu
2023-10-11 13:14 ` Peter Zijlstra
2023-10-12 10:04 ` Abel Wu
2023-10-11 21:01 ` Benjamin Segall
2023-10-12 10:25 ` Abel Wu
2023-10-12 17:51 ` Benjamin Segall
2023-10-13 3:46 ` Abel Wu
2023-10-13 16:51 ` Benjamin Segall
2023-05-31 11:58 ` [PATCH 06/15] sched: Commit to lag based placement Peter Zijlstra
2023-08-10 7:10 ` [tip: sched/core] sched/fair: " tip-bot2 for Peter Zijlstra
2023-05-31 11:58 ` [PATCH 07/15] sched/smp: Use lag to simplify cross-runqueue placement Peter Zijlstra
2023-08-10 7:10 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2023-09-12 15:32 ` [PATCH 07/15] " Sebastian Andrzej Siewior
2023-09-13 9:03 ` Peter Zijlstra
2023-10-04 1:17 ` [PATCH] sched/fair: Preserve PLACE_DEADLINE_INITIAL deadline Daniel Jordan
2023-10-04 13:09 ` [PATCH v2] " Daniel Jordan
2023-10-04 15:46 ` Chen Yu
2023-10-06 16:31 ` Daniel Jordan
2023-10-12 4:48 ` K Prateek Nayak
2023-10-05 5:56 ` [PATCH] " K Prateek Nayak
2023-10-06 16:35 ` Daniel Jordan
2023-10-06 16:48 ` [PATCH] sched/fair: Always update_curr() before placing at enqueue Daniel Jordan
2023-10-06 19:58 ` Peter Zijlstra
2023-10-18 0:43 ` Daniel Jordan
2023-10-16 5:39 ` K Prateek Nayak
2023-05-31 11:58 ` [PATCH 08/15] sched: Commit to EEVDF Peter Zijlstra
2023-06-16 21:23 ` Joel Fernandes
2023-06-22 12:01 ` Ingo Molnar
2023-06-22 13:11 ` Joel Fernandes
2023-08-10 7:10 ` [tip: sched/core] sched/fair: " tip-bot2 for Peter Zijlstra
2023-05-31 11:58 ` [PATCH 09/15] sched/debug: Rename min_granularity to base_slice Peter Zijlstra
2023-08-10 7:10 ` [tip: sched/core] sched/debug: Rename sysctl_sched_min_granularity to sysctl_sched_base_slice tip-bot2 for Peter Zijlstra
2023-05-31 11:58 ` [PATCH 10/15] sched/fair: Propagate enqueue flags into place_entity() Peter Zijlstra
2023-08-10 7:10 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2023-05-31 11:58 ` [PATCH 11/15] sched/eevdf: Better handle mixed slice length Peter Zijlstra
2023-06-02 13:45 ` Vincent Guittot
2023-06-02 15:06 ` Peter Zijlstra
2023-06-10 6:34 ` Chen Yu
2023-06-10 11:22 ` Peter Zijlstra
2023-05-31 11:58 ` [RFC][PATCH 12/15] sched: Introduce latency-nice as a per-task attribute Peter Zijlstra
2023-05-31 11:58 ` [RFC][PATCH 13/15] sched/fair: Implement latency-nice Peter Zijlstra
2023-06-06 14:54 ` Vincent Guittot
2023-06-08 10:34 ` Peter Zijlstra
2023-06-08 12:44 ` Peter Zijlstra
2023-10-11 23:24 ` Benjamin Segall
2023-05-31 11:58 ` [RFC][PATCH 14/15] sched/fair: Add sched group latency support Peter Zijlstra
2023-05-31 11:58 ` [RFC][PATCH 15/15] sched/eevdf: Use sched_attr::sched_runtime to set request/slice Peter Zijlstra
2023-06-01 13:55 ` Vincent Guittot
2023-06-08 11:52 ` Peter Zijlstra
2023-08-24 0:52 ` [PATCH 00/15] sched: EEVDF and latency-nice and/or slice-attr Daniel Jordan
2023-09-06 13:13 ` Peter Zijlstra
2023-09-29 16:54 ` Youssef Esmat
2023-10-02 15:55 ` Youssef Esmat
2023-10-02 18:41 ` Peter Zijlstra
2023-10-05 12:05 ` Peter Zijlstra [this message]
2023-10-05 14:14 ` Peter Zijlstra
2023-10-05 14:42 ` Peter Zijlstra
2023-10-05 18:23 ` Youssef Esmat
2023-10-06 0:36 ` Youssef Esmat
2023-10-10 8:08 ` Peter Zijlstra
2023-10-07 22:04 ` Peter Zijlstra
2023-10-09 14:41 ` Peter Zijlstra
2023-10-10 0:51 ` Youssef Esmat
2023-10-10 8:01 ` Peter Zijlstra
2023-10-16 16:50 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231005120557.GA743@noisy.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=chris.hyser@oracle.com \
--cc=corbet@lwn.net \
--cc=daniel.m.jordan@oracle.com \
--cc=dietmar.eggemann@arm.com \
--cc=efault@gmx.de \
--cc=joel@joelfernandes.org \
--cc=joshdon@google.com \
--cc=juri.lelli@redhat.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=patrick.bellasi@matbug.net \
--cc=pavel@ucw.cz \
--cc=pjt@google.com \
--cc=qperret@google.com \
--cc=qyousef@layalina.io \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=timj@gnu.org \
--cc=vincent.guittot@linaro.org \
--cc=youssefesmat@chromium.org \
--cc=yu.c.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).