Re: [PATCH v3 10/14] sched: Introduce arch_cpu_allowed_mask() to limit fallback rq selection

From: Will Deacon <will@kernel.org>
To: Quentin Perret <qperret@google.com>
Cc: linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Morten Rasmussen <morten.rasmussen@arm.com>,
	Qais Yousef <qais.yousef@arm.com>,
	Suren Baghdasaryan <surenb@google.com>, Tejun Heo <tj@kernel.org>,
	Li Zefan <lizefan@huawei.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Ingo Molnar <mingo@redhat.com>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	kernel-team@android.com
Subject: Re: [PATCH v3 10/14] sched: Introduce arch_cpu_allowed_mask() to limit fallback rq selection
Date: Thu, 19 Nov 2020 11:07:09 +0000	[thread overview]
Message-ID: <20201119110709.GD3946@willie-the-truck> (raw)
In-Reply-To: <20201119093850.GD2416649@google.com>

On Thu, Nov 19, 2020 at 09:38:50AM +0000, Quentin Perret wrote:
> On Friday 13 Nov 2020 at 09:37:15 (+0000), Will Deacon wrote:
> > Asymmetric systems may not offer the same level of userspace ISA support
> > across all CPUs, meaning that some applications cannot be executed by
> > some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do
> > not feature support for 32-bit applications on both clusters.
> > 
> > On such a system, we must take care not to migrate a task to an
> > unsupported CPU when forcefully moving tasks in select_fallback_rq()
> > in response to a CPU hot-unplug operation.
> > 
> > Introduce an arch_cpu_allowed_mask() hook which, given a task argument,
> > allows an architecture to return a cpumask of CPUs that are capable of
> > executing that task. The default implementation returns the
> > cpu_possible_mask, since sane machines do not suffer from per-cpu ISA
> > limitations that affect scheduling. The new mask is used when selecting
> > the fallback runqueue as a last resort before forcing a migration to the
> > first active CPU.
> > 
> > Signed-off-by: Will Deacon <will@kernel.org>
> > ---
> >  kernel/sched/core.c | 13 ++++++++++---
> >  1 file changed, 10 insertions(+), 3 deletions(-)
> > 
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index 818c8f7bdf2a..8df38ebfe769 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -1696,6 +1696,11 @@ void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags)
> >  
> >  #ifdef CONFIG_SMP
> >  
> > +/* Must contain at least one active CPU */
> > +#ifndef arch_cpu_allowed_mask
> > +#define  arch_cpu_allowed_mask(p)	cpu_possible_mask
> > +#endif
> > +
> >  /*
> >   * Per-CPU kthreads are allowed to run on !active && online CPUs, see
> >   * __set_cpus_allowed_ptr() and select_fallback_rq().
> > @@ -1708,7 +1713,10 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu)
> >  	if (is_per_cpu_kthread(p))
> >  		return cpu_online(cpu);
> >  
> > -	return cpu_active(cpu);
> > +	if (!cpu_active(cpu))
> > +		return false;
> > +
> > +	return cpumask_test_cpu(cpu, arch_cpu_allowed_mask(p));
> >  }
> >  
> >  /*
> > @@ -2361,10 +2369,9 @@ static int select_fallback_rq(int cpu, struct task_struct *p)
> >  			}
> >  			fallthrough;
> >  		case possible:
> > -			do_set_cpus_allowed(p, cpu_possible_mask);
> > +			do_set_cpus_allowed(p, arch_cpu_allowed_mask(p));
> 
> Nit: I'm wondering if this should be called arch_cpu_possible_mask()
> instead?

I'm open to renaming it, so if nobody else has any better ideas then I'll
go with this.

> In any case:
> 
> Reviewed-by: Quentin Perret <qperret@google.com?

Ta!

Will