Re: [PATCH v2 5/6] sched/rt: Better manage pushing unfit tasks on wakeup

From: Pavan Kondeti <pkondeti@codeaurora.org>
To: Qais Yousef <qais.yousef@arm.com>
Cc: Pavan Kondeti <pkondeti@codeaurora.org>,
	Ingo Molnar <mingo@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 5/6] sched/rt: Better manage pushing unfit tasks on wakeup
Date: Mon, 24 Feb 2020 21:34:41 +0530	[thread overview]
Message-ID: <CAEU1=PncyV=-vqjkDHSJ4hUhhTfYUgVN-HAe4zXMHtFx1oc5XA@mail.gmail.com> (raw)
In-Reply-To: <20200224121139.cbz2dt5heiouknif@e107158-lin.cambridge.arm.com>

Hi Qais,

On Mon, Feb 24, 2020 at 5:42 PM Qais Yousef <qais.yousef@arm.com> wrote:
[...]
> We could do, temporarily, to get these fixes into 5.6. But I do think
> select_task_rq_rt() doesn't do a good enough job into pushing unfit tasks to
> the right CPUs.
>
> I don't understand the reasons behind your objection. It seems you think that
> select_task_rq_rt() should be enough, but not AFAICS. Can you be a bit more
> detailed please?
>
> FWIW, here's a screenshot of what I see
>
>         https://imgur.com/a/peV27nE
>
> After the first activation, select_task_rq_rt() fails to find the right CPU
> (due to the same move all tasks to the cpumask_fist()) - but when the task
> wakes up on 4, the logic I put causes it to migrate to CPU2, which is the 2nd
> big core. CPU1 and CPU2 are the big cores on Juno.
>
> Now maybe we should fix select_task_rq_rt() to better balance tasks, but not
> sure how easy is that.
>

Thanks for the trace. Now things are clear to me. Two RT tasks woke up
simultaneously and the first task got its previous CPU i.e CPU#1. The next task
goes through find_lowest_rq() and got the same CPU#1. Since this task priority
is not more than the just queued task (already queued on CPU#1), it is sent
to its previous CPU i.e CPU#4 in your case.

From task_woken_rt() path, CPU#4 attempts push_rt_tasks(). CPU#4 is
not overloaded,
but we have rt_task_fits_capacity() check which forces the push. Since the CPU
is not overloaded, your has_unfit_tasks() comes to rescue and push the
task. Since
the task has not scheduled in yet, it is eligible for push. You added checks
to skip resched_curr() in push_rt_tasks() otherwise the push won't happen.

Finally, I understood your patch. Obviously this is not clear to me
before. I am not
sure if this patch is the right approach to solve this race. I will
think a bit more.

Thanks,
Pavan
-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project