Re: [PATCH 2/4] sched/deadline: Improve admission control for asymmetric CPU capacities

From: Juri Lelli <juri.lelli@redhat.com>
To: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Valentin Schneider <valentin.schneider@arm.com>,
	luca abeni <luca.abeni@santannapisa.it>,
	Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Daniel Bristot de Oliveira <bristot@redhat.com>,
	Wei Wang <wvw@google.com>, Quentin Perret <qperret@google.com>,
	Alessio Balsini <balsini@google.com>,
	Pavan Kondeti <pkondeti@codeaurora.org>,
	Patrick Bellasi <patrick.bellasi@matbug.net>,
	Morten Rasmussen <morten.rasmussen@arm.com>,
	Qais Yousef <qais.yousef@arm.com>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/4] sched/deadline: Improve admission control for asymmetric CPU capacities
Date: Fri, 17 Apr 2020 17:08:28 +0200	[thread overview]
Message-ID: <20200417150828.GS9767@localhost.localdomain> (raw)
In-Reply-To: <8734b37e-5602-79dc-c7a8-c41fd9eb86b5@arm.com>

On 17/04/20 16:55, Dietmar Eggemann wrote:
> On 17.04.20 14:19, Juri Lelli wrote:
> > On 09/04/20 19:29, Dietmar Eggemann wrote:
> 
> [...]
> 
> >> Maybe we can do a hybrid. We have rd->span and rd->sum_cpu_capacity and
> >> with the help of an extra per-cpu cpumask we could just
> > 
> > Hummm, I like the idea, but
> > 
> >> DEFINE_PER_CPU(cpumask_var_t, dl_bw_mask);
> >>
> >> dl_bw_cpus(int i) {
> > 
> > This works if calls are always local to the rd we are interested into
> > (argument 'i' isn't used). Are we always doing that?
> 
> I thought so. The existing dl_bw_cpus(int i) implementation already
> assumes this by using:
> 
>     struct root_domain *rd = cpu_rq(i)->rd;

Hummm, can't dl_task_can_attach() call it with a dest_cpu different from
this_cpu?

Current implementation uses 'i' argument to get to the right root_domain
(e.g., when moving tasks between execlusive set).

>     ...
> 
>     for_each_cpu_and(i, rd->span, cpu_active_mask)
> 
> Or did you refer to something else here?
> 
> And the patch would not introduce new places in which we call
> dl_bw_cpus(). It will just replace some with a dl_bw_capacity() call.
> 
> >>     struct cpumask *cpus = this_cpu_cpumask_var_ptr(dl_bw_mask);
> >>     ...
> >>     cpumask_and(cpus, rd->span, cpu_active_mask);
> >>
> >>     return cpumask_weight(cpus);
> >> }
> >>
> >> and
> >>
> >> dl_bw_capacity(int i) {
> >>
> >>     struct cpumask *cpus = this_cpu_cpumask_var_ptr(dl_bw_mask);
> >>     ...
> >>     cpumask_and(cpus, rd->span, cpu_active_mask);
> >>     if (cpumask_equal(cpus, rd->span))
> >>         return rd->sum_cpu_capacity;
> > 
> > What if capacities change between invocations (with the same span)?
> > Can that happen?
> 
> The CPU capacity should only change during initial bring-up. On
> asymmetric CPU capacity systems we have to re-create the Sched Domain
> (SD) topology after CPUfreq becomes available.
> 
> After the initial build and this first rebuild of the SD topology, the
> CPU capacity should be stable.
> 
> Everything which might follow afterwards (starting EAS, exclusive
> cpusets or CPU hp) will not change the CPU capacity.
> 
> Obviously, if you defer loading CPUfreq driver after you started DL
> scheduling you can break things. But this is not considered a valid
> environment here.

OK. Makes sense.