From: Dietmar Eggemann <dietmar.eggemann@arm.com>
To: Peter Zijlstra <peterz@infradead.org>,
luca abeni <luca.abeni@santannapisa.it>
Cc: linux-kernel@vger.kernel.org,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
"Rafael J . Wysocki" <rafael@kernel.org>,
Ingo Molnar <mingo@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
"Paul E . McKenney" <paulmck@linux.ibm.com>,
Joel Fernandes <joel@joelfernandes.org>,
Quentin Perret <quentin.perret@arm.com>,
Luc Van Oostenryck <luc.vanoostenryck@gmail.com>,
Morten Rasmussen <morten.rasmussen@arm.com>,
Juri Lelli <juri.lelli@redhat.com>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Patrick Bellasi <patrick.bellasi@arm.com>,
Tommaso Cucinotta <tommaso.cucinotta@santannapisa.it>
Subject: Re: [RFC PATCH 3/6] sched/dl: Try better placement even for deadline tasks that do not block
Date: Thu, 11 Jul 2019 13:17:17 +0200 [thread overview]
Message-ID: <dac4462d-7384-cf8a-6619-2781c16caa89@arm.com> (raw)
In-Reply-To: <20190709134200.GD3402@hirez.programming.kicks-ass.net>
On 7/9/19 3:42 PM, Peter Zijlstra wrote:
> On Tue, Jul 09, 2019 at 03:24:36PM +0200, luca abeni wrote:
>> Hi Peter,
>>
>> On Mon, 8 Jul 2019 15:55:36 +0200
>> Peter Zijlstra <peterz@infradead.org> wrote:
>>
>>> On Mon, May 06, 2019 at 06:48:33AM +0200, Luca Abeni wrote:
>>>> @@ -1223,8 +1250,17 @@ static void update_curr_dl(struct rq *rq)
>>>> dl_se->dl_overrun = 1;
>>>>
>>>> __dequeue_task_dl(rq, curr, 0);
>>>> - if (unlikely(dl_se->dl_boosted
>>>> || !start_dl_timer(curr)))
>>>> + if (unlikely(dl_se->dl_boosted
>>>> || !start_dl_timer(curr))) { enqueue_task_dl(rq, curr,
>>>> ENQUEUE_REPLENISH); +#ifdef CONFIG_SMP
>>>> + } else if (dl_se->dl_adjust) {
>>>> + if (rq->migrating_task == NULL) {
>>>> + queue_balance_callback(rq,
>>>> &per_cpu(dl_migrate_head, rq->cpu), migrate_dl_task);
>>>
>>> I'm not entirely sure about this one.
>>>
>>> That is, we only do those callbacks from:
>>>
>>> schedule_tail()
>>> __schedule()
>>> rt_mutex_setprio()
>>> __sched_setscheduler()
>>>
>>> and the above looks like it can happen outside of those.
>>
>> Sorry, I did not know the constraints or requirements for using
>> queue_balance_callback()...
>>
>> I used it because I wanted to trigger a migration from
>> update_curr_dl(), but invoking double_lock_balance() from this function
>> obviously resulted in a warning. So, I probably misunderstood the
>> purpose of the balance callback API, and I misused it.
>>
>> What would have been the "right way" to trigger a migration for a task
>> when it is throttled?
>
> I'm thinking we'll end up in schedule() pretty soon after a throttle to
> make 'current' go away, right? We could put the queue_balance_callback()
> in dequeue_task_dl() or something.
Is this what you are concerned about?
(2 Cpus (CPU1, CPU2), 4 deadline task (thread0-X)) with
@@ -1137,6 +1137,13 @@ static inline void rq_pin_lock(struct rq *rq, struct rq_flags *rf)
rf->cookie = lockdep_pin_lock(&rq->lock);
#ifdef CONFIG_SCHED_DEBUG
+#ifdef CONFIG_SMP
+ /*
+ * There should not be pending callbacks at the start of rq_lock();
+ * all sites that handle them flush them at the end.
+ */
+ WARN_ON_ONCE(rq->balance_callback);
+#endif
[ 87.251233] *** <--- queue_balance_callback(migrate_dl_task) p=[thread0-2 3626] on CPU1
[ 87.251237] *** <--- queue_balance_callback(migrate_dl_task) p=[thread0-3 3627] on CPU2
[ 87.251261] WARNING: CPU: 2 PID: 3627 at kernel/sched/sched.h:1145 __schedule+0x56c/0x690
[ 87.259173] WARNING: CPU: 1 PID: 3626 at kernel/sched/sched.h:1145 try_to_wake_up+0x68c/0x778
[ 87.615871] WARNING: CPU: 1 PID: 3626 at kernel/sched/sched.h:1145 scheduler_tick+0x110/0x118
[ 87.615882] WARNING: CPU: 2 PID: 3616 at kernel/sched/sched.h:1145 task_rq_lock+0xe8/0xf0
[ 88.012571] WARNING: CPU: 1 PID: 3626 at kernel/sched/sched.h:1145 update_blocked_averages+0x924/0x998
[ 88.176844] WARNING: CPU: 2 PID: 3616 at kernel/sched/sched.h:1145 load_balance+0x4d0/0xbc0
[ 88.381905] WARNING: CPU: 2 PID: 3616 at kernel/sched/sched.h:1145 load_balance+0x7d8/0xbc0
[ 88.586991] *** ---> migrate_dl_task() p=[thread0-3 3627] to CPU1
[ 88.587008] *** ---> migrate_dl_task() p=[thread0-2 3626] to CPU1
[ 89.335307] *** <--- queue_balance_callback(migrate_dl_task) p=[thread0-0 3624] on CPU2
[ 89.343252] *** ---> migrate_dl_task() p=[thread0-0 3624] to CPU1
[ 89.379309] *** <--- queue_balance_callback(migrate_dl_task) p=[thread0-1 3625] on CPU2
[ 89.387249] *** ---> migrate_dl_task() p=[thread0-1 3625] to CPU1
[ 89.591323] *** <--- queue_balance_callback(migrate_dl_task) p=[thread0-3 3627] on CPU2
[ 89.599263] *** ---> migrate_dl_task() p=[thread0-3 3627] to CPU2
next prev parent reply other threads:[~2019-07-11 11:17 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-06 4:48 [RFC PATCH 0/6] Capacity awareness for SCHED_DEADLINE Luca Abeni
2019-05-06 4:48 ` [RFC PATCH 1/6] sched/dl: Improve deadline admission control for asymmetric CPU capacities Luca Abeni
2019-05-07 13:48 ` Quentin Perret
2019-05-07 13:55 ` Vincent Guittot
2019-05-07 14:02 ` Quentin Perret
2019-05-07 14:25 ` luca abeni
2019-05-07 14:31 ` Quentin Perret
2019-05-07 14:43 ` luca abeni
2019-07-08 11:22 ` Dietmar Eggemann
2019-07-08 15:05 ` Quentin Perret
2019-06-18 16:41 ` Alessio Balsini
2019-05-06 4:48 ` [RFC PATCH 2/6] sched/dl: Capacity-aware migrations Luca Abeni
2019-05-07 13:35 ` Quentin Perret
2019-05-07 14:17 ` luca abeni
2019-05-07 15:04 ` Quentin Perret
2019-05-07 14:10 ` Quentin Perret
2019-05-07 14:41 ` luca abeni
2019-05-07 15:02 ` Quentin Perret
2019-05-08 8:04 ` Juri Lelli
2019-05-08 8:17 ` luca abeni
2019-07-04 12:05 ` Dietmar Eggemann
2019-07-08 7:41 ` luca abeni
2019-07-08 10:41 ` Dietmar Eggemann
2019-05-06 4:48 ` [RFC PATCH 3/6] sched/dl: Try better placement even for deadline tasks that do not block Luca Abeni
2019-05-07 14:13 ` Quentin Perret
2019-05-07 16:00 ` Morten Rasmussen
2019-05-08 8:01 ` Juri Lelli
2019-05-08 8:14 ` luca abeni
2019-05-08 9:22 ` Juri Lelli
2019-07-08 13:55 ` Peter Zijlstra
2019-07-09 13:24 ` luca abeni
2019-07-09 13:42 ` Peter Zijlstra
2019-07-11 11:17 ` Dietmar Eggemann [this message]
2019-07-11 12:00 ` Peter Zijlstra
2019-07-11 15:33 ` Dietmar Eggemann
2019-07-09 14:44 ` Dietmar Eggemann
2019-05-06 4:48 ` [RFC PATCH 4/6] sched/dl: Improve capacity-aware wakeup Luca Abeni
2019-05-08 9:08 ` Juri Lelli
2019-05-08 9:24 ` luca abeni
2019-05-08 12:05 ` Juri Lelli
2019-05-08 12:47 ` luca abeni
2019-05-08 13:10 ` Juri Lelli
2019-05-08 14:12 ` luca abeni
2019-05-06 4:48 ` [RFC PATCH 5/6] sched/dl: If the task does not fit anywhere, select the fastest core Luca Abeni
2019-05-06 4:48 ` [RFC PATCH 6/6] sched/dl: Try not to select a too fast core Luca Abeni
2019-05-07 15:57 ` Quentin Perret
2019-05-08 6:26 ` luca abeni
2019-05-09 13:46 ` Quentin Perret
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=dac4462d-7384-cf8a-6619-2781c16caa89@arm.com \
--to=dietmar.eggemann@arm.com \
--cc=bristot@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=joel@joelfernandes.org \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luc.vanoostenryck@gmail.com \
--cc=luca.abeni@santannapisa.it \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=patrick.bellasi@arm.com \
--cc=paulmck@linux.ibm.com \
--cc=peterz@infradead.org \
--cc=quentin.perret@arm.com \
--cc=rafael@kernel.org \
--cc=tommaso.cucinotta@santannapisa.it \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).