From: Juri Lelli <juri.lelli@redhat.com>
To: Daniel Bristot de Oliveira <bristot@redhat.com>
Cc: Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Mark Simmons <msimmons@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH] sched/deadline: Unthrottle PI boosted threads while enqueuing
Date: Fri, 18 Sep 2020 08:00:26 +0200 [thread overview]
Message-ID: <20200918060026.GC261845@localhost.localdomain> (raw)
In-Reply-To: <5076e003450835ec74e6fa5917d02c4fa41687e6.1600170294.git.bristot@redhat.com>
Hi Daniel,
On 16/09/20 09:06, Daniel Bristot de Oliveira wrote:
> stress-ng has a test (stress-ng --cyclic) that creates a set of threads
> under SCHED_DEADLINE with the following parameters:
>
> dl_runtime = 10000 (10 us)
> dl_deadline = 100000 (100 us)
> dl_period = 100000 (100 us)
>
> These parameters are very aggressive. When using a system without HRTICK
> set, these threads can easily execute longer than the dl_runtime because
> the throttling happens with 1/HZ resolution.
>
> During the main part of the test, the system works just fine because
> the workload does not try to run over the 10 us. The problem happens at
> the end of the test, on the exit() path. During exit(), the threads need
> to do some cleanups that require real-time mutex locks, mainly those
> related to memory management, resulting in this scenario:
>
> Note: locks are rt_mutexes...
> ------------------------------------------------------------------------
> TASK A: TASK B: TASK C:
> activation
> activation
> activation
>
> lock(a): OK! lock(b): OK!
> <overrun runtime>
> lock(a)
> -> block (task A owns it)
> -> self notice/set throttled
> +--< -> arm replenished timer
> | switch-out
> | lock(b)
> | -> <C prio > B prio>
> | -> boost TASK B
> | unlock(a) switch-out
> | -> handle lock a to B
> | -> wakeup(B)
> | -> B is throttled:
> | -> do not enqueue
> | switch-out
> |
> |
> +---------------------> replenishment timer
> -> TASK B is boosted:
> -> do not enqueue
> ------------------------------------------------------------------------
>
> BOOM: TASK B is runnable but !enqueued, holding TASK C: the system
> crashes with hung task C.
>
> This problem is avoided by removing the throttle state from the boosted
> thread while boosting it (by TASK A in the example above), allowing it to
> be queued and run boosted.
>
> The next replenishment will take care of the runtime overrun, pushing
> the deadline further away. See the "while (dl_se->runtime <= 0)" on
> replenish_dl_entity() for more information.
>
> Signed-off-by: Daniel Bristot de Oliveira <bristot@redhat.com>
> Reported-by: Mark Simmons <msimmons@redhat.com>
> Reviewed-by: Juri Lelli <juri.lelli@redhat.com>
> Tested-by: Mark Simmons <msimmons@redhat.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@redhat.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Ben Segall <bsegall@google.com>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
> Cc: linux-kernel@vger.kernel.org
>
> ---
Thanks for this fix.
Acked-by: Juri Lelli <juri.lelli@redhat.com>
Best,
Juri
next prev parent reply other threads:[~2020-09-18 6:00 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-16 7:06 [PATCH] sched/deadline: Unthrottle PI boosted threads while enqueuing Daniel Bristot de Oliveira
2020-09-18 6:00 ` Juri Lelli [this message]
2020-10-02 15:57 ` Daniel Bristot de Oliveira
2020-10-02 16:01 ` Peter Zijlstra
2020-10-05 7:43 ` [tip: sched/core] " tip-bot2 for Daniel Bristot de Oliveira
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200918060026.GC261845@localhost.localdomain \
--to=juri.lelli@redhat.com \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=msimmons@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).