From: Qais Yousef <qais.yousef@arm.com>
To: Will Deacon <will@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org,
linux-kernel@vger.kernel.org,
Catalin Marinas <catalin.marinas@arm.com>,
Marc Zyngier <maz@kernel.org>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Peter Zijlstra <peterz@infradead.org>,
Morten Rasmussen <morten.rasmussen@arm.com>,
Suren Baghdasaryan <surenb@google.com>,
Quentin Perret <qperret@google.com>, Tejun Heo <tj@kernel.org>,
Li Zefan <lizefan@huawei.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Ingo Molnar <mingo@redhat.com>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
"Rafael J. Wysocki" <rjw@rjwysocki.net>,
kernel-team@android.com
Subject: Re: [PATCH v6 12/21] sched: Allow task CPU affinity to be restricted on asymmetric systems
Date: Fri, 21 May 2021 18:11:32 +0100 [thread overview]
Message-ID: <20210521171132.ev56j4isuxtf2zqa@e107158-lin.cambridge.arm.com> (raw)
In-Reply-To: <20210518094725.7701-13-will@kernel.org>
On 05/18/21 10:47, Will Deacon wrote:
> Asymmetric systems may not offer the same level of userspace ISA support
> across all CPUs, meaning that some applications cannot be executed by
> some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do
> not feature support for 32-bit applications on both clusters.
>
> Although userspace can carefully manage the affinity masks for such
> tasks, one place where it is particularly problematic is execve()
> because the CPU on which the execve() is occurring may be incompatible
> with the new application image. In such a situation, it is desirable to
> restrict the affinity mask of the task and ensure that the new image is
> entered on a compatible CPU. From userspace's point of view, this looks
> the same as if the incompatible CPUs have been hotplugged off in the
> task's affinity mask. Similarly, if a subsequent execve() reverts to
> a compatible image, then the old affinity is restored if it is still
> valid.
>
> In preparation for restricting the affinity mask for compat tasks on
> arm64 systems without uniform support for 32-bit applications, introduce
> {force,relax}_compatible_cpus_allowed_ptr(), which respectively restrict
> and restore the affinity mask for a task based on the compatible CPUs.
>
> Reviewed-by: Quentin Perret <qperret@google.com>
> Signed-off-by: Will Deacon <will@kernel.org>
> ---
> include/linux/sched.h | 2 +
> kernel/sched/core.c | 165 ++++++++++++++++++++++++++++++++++++++----
> kernel/sched/sched.h | 1 +
> 3 files changed, 152 insertions(+), 16 deletions(-)
>
> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index db32d4f7e5b3..91a6cfeae242 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -1691,6 +1691,8 @@ extern void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new
> extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask);
> extern int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, int node);
> extern void release_user_cpus_ptr(struct task_struct *p);
> +extern void force_compatible_cpus_allowed_ptr(struct task_struct *p);
> +extern void relax_compatible_cpus_allowed_ptr(struct task_struct *p);
> #else
> static inline void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask)
> {
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 808bbe669a6d..ba66bcf8e812 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -2357,26 +2357,21 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag
> }
>
> /*
> - * Change a given task's CPU affinity. Migrate the thread to a
> - * proper CPU and schedule it away if the CPU it's executing on
> - * is removed from the allowed bitmask.
> - *
> - * NOTE: the caller must have a valid reference to the task, the
> - * task must not exit() & deallocate itself prematurely. The
> - * call is not atomic; no spinlocks may be held.
> + * Called with both p->pi_lock and rq->lock held; drops both before returning.
> */
> -static int __set_cpus_allowed_ptr(struct task_struct *p,
> - const struct cpumask *new_mask,
> - u32 flags)
> +static int __set_cpus_allowed_ptr_locked(struct task_struct *p,
> + const struct cpumask *new_mask,
> + u32 flags,
> + struct rq *rq,
> + struct rq_flags *rf)
> + __releases(rq->lock)
> + __releases(p->pi_lock)
> {
> const struct cpumask *cpu_valid_mask = cpu_active_mask;
> const struct cpumask *cpu_allowed_mask = task_cpu_possible_mask(p);
> unsigned int dest_cpu;
> - struct rq_flags rf;
> - struct rq *rq;
> int ret = 0;
>
> - rq = task_rq_lock(p, &rf);
> update_rq_clock(rq);
>
> if (p->flags & PF_KTHREAD || is_migration_disabled(p)) {
> @@ -2430,20 +2425,158 @@ static int __set_cpus_allowed_ptr(struct task_struct *p,
>
> __do_set_cpus_allowed(p, new_mask, flags);
>
> - return affine_move_task(rq, p, &rf, dest_cpu, flags);
> + if (flags & SCA_USER)
> + release_user_cpus_ptr(p);
Why do we need to release the pointer here?
Doesn't this mean if a 32bit task requests to change its affinity, then we'll
lose this info and a subsequent execve() to a 64bit application means we won't
be able to restore the original mask?
ie:
p0-64bit
execve(32bit_app)
// p1-32bit created
p1-32bit.change_affinity()
relase_user_cpus_ptr()
execve(64bit_app) // lost info about p0 affinity?
Hmm I think this helped me to get the answer. p1 changed its affinity, then
there's nothing to be inherited by a new execve(), so yes we no longer need
this info.
> +
> + return affine_move_task(rq, p, rf, dest_cpu, flags);
>
> out:
> - task_rq_unlock(rq, p, &rf);
> + task_rq_unlock(rq, p, rf);
>
> return ret;
> }
[...]
> +/*
> + * Change a given task's CPU affinity to the intersection of its current
> + * affinity mask and @subset_mask, writing the resulting mask to @new_mask
> + * and pointing @p->user_cpus_ptr to a copy of the old mask.
> + * If the resulting mask is empty, leave the affinity unchanged and return
> + * -EINVAL.
> + */
> +static int restrict_cpus_allowed_ptr(struct task_struct *p,
> + struct cpumask *new_mask,
> + const struct cpumask *subset_mask)
> +{
> + struct rq_flags rf;
> + struct rq *rq;
> + int err;
> + struct cpumask *user_mask = NULL;
> +
> + if (!p->user_cpus_ptr)
> + user_mask = kmalloc(cpumask_size(), GFP_KERNEL);
> +
> + rq = task_rq_lock(p, &rf);
> +
> + /*
> + * We're about to butcher the task affinity, so keep track of what
> + * the user asked for in case we're able to restore it later on.
> + */
> + if (user_mask) {
> + cpumask_copy(user_mask, p->cpus_ptr);
> + p->user_cpus_ptr = user_mask;
> + }
> +
> + /*
> + * Forcefully restricting the affinity of a deadline task is
> + * likely to cause problems, so fail and noisily override the
> + * mask entirely.
> + */
> + if (task_has_dl_policy(p) && dl_bandwidth_enabled()) {
> + err = -EPERM;
> + goto err_unlock;
free(user_mark) first?
> + }
> +
> + if (!cpumask_and(new_mask, &p->cpus_mask, subset_mask)) {
> + err = -EINVAL;
> + goto err_unlock;
ditto
> + }
> +
> + return __set_cpus_allowed_ptr_locked(p, new_mask, false, rq, &rf);
> +
> +err_unlock:
> + task_rq_unlock(rq, p, &rf);
> + return err;
> +}
Thanks
--
Qais Yousef
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-05-21 17:13 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-18 9:47 [PATCH v6 00/21] Add support for 32-bit tasks on asymmetric AArch32 systems Will Deacon
2021-05-18 9:47 ` [PATCH v6 01/21] arm64: cpuinfo: Split AArch32 registers out into a separate struct Will Deacon
2021-05-21 10:47 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 02/21] arm64: Allow mismatched 32-bit EL0 support Will Deacon
2021-05-21 10:25 ` Catalin Marinas
2021-05-24 12:05 ` Will Deacon
2021-05-24 13:49 ` Catalin Marinas
2021-05-21 10:41 ` Catalin Marinas
2021-05-24 12:09 ` Will Deacon
2021-05-24 13:46 ` Catalin Marinas
2021-05-21 15:22 ` Qais Yousef
2021-05-24 20:21 ` Will Deacon
2021-05-18 9:47 ` [PATCH v6 03/21] KVM: arm64: Kill 32-bit vCPUs on systems with mismatched " Will Deacon
2021-05-21 10:47 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 04/21] arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs Will Deacon
2021-05-21 10:55 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 05/21] arm64: Advertise CPUs capable of running 32-bit applications in sysfs Will Deacon
2021-05-21 11:00 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 06/21] sched: Introduce task_cpu_possible_mask() to limit fallback rq selection Will Deacon
2021-05-21 16:03 ` Peter Zijlstra
2021-05-24 12:17 ` Will Deacon
2021-05-18 9:47 ` [PATCH v6 07/21] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Will Deacon
2021-05-21 17:39 ` Qais Yousef
2021-05-24 20:21 ` Will Deacon
2021-05-18 9:47 ` [PATCH v6 08/21] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Will Deacon
2021-05-21 16:25 ` Qais Yousef
2021-05-24 21:09 ` Will Deacon
2021-05-18 9:47 ` [PATCH v6 09/21] sched: Reject CPU affinity changes based on task_cpu_possible_mask() Will Deacon
2021-05-18 9:47 ` [PATCH v6 10/21] sched: Introduce task_struct::user_cpus_ptr to track requested affinity Will Deacon
2021-05-18 9:47 ` [PATCH v6 11/21] sched: Split the guts of sched_setaffinity() into a helper function Will Deacon
2021-05-21 16:41 ` Qais Yousef
2021-05-24 21:16 ` Will Deacon
2021-05-18 9:47 ` [PATCH v6 12/21] sched: Allow task CPU affinity to be restricted on asymmetric systems Will Deacon
2021-05-21 17:11 ` Qais Yousef [this message]
2021-05-24 21:43 ` Will Deacon
2021-05-18 9:47 ` [PATCH v6 13/21] sched: Admit forcefully-affined tasks into SCHED_DEADLINE Will Deacon
2021-05-18 10:20 ` Quentin Perret
2021-05-18 10:28 ` Will Deacon
2021-05-18 10:48 ` Quentin Perret
2021-05-18 10:59 ` Will Deacon
2021-05-18 13:19 ` Quentin Perret
2021-05-20 9:13 ` Juri Lelli
2021-05-20 10:16 ` Will Deacon
2021-05-20 10:33 ` Quentin Perret
2021-05-20 12:38 ` Juri Lelli
2021-05-20 12:38 ` Daniel Bristot de Oliveira
2021-05-20 15:06 ` Dietmar Eggemann
2021-05-20 16:00 ` Daniel Bristot de Oliveira
2021-05-20 17:55 ` Dietmar Eggemann
2021-05-20 18:03 ` Will Deacon
2021-05-21 11:26 ` Dietmar Eggemann
2021-05-20 18:01 ` Will Deacon
2021-05-21 5:25 ` Juri Lelli
2021-05-21 8:15 ` Quentin Perret
2021-05-21 8:39 ` Juri Lelli
2021-05-21 10:37 ` Will Deacon
2021-05-21 11:23 ` Dietmar Eggemann
2021-05-21 13:02 ` Quentin Perret
2021-05-21 14:04 ` Juri Lelli
2021-05-21 17:47 ` Dietmar Eggemann
2021-05-21 13:00 ` Daniel Bristot de Oliveira
2021-05-21 13:12 ` Quentin Perret
2021-05-24 20:47 ` Will Deacon
2021-05-18 9:47 ` [PATCH v6 14/21] freezer: Add frozen_or_skipped() helper function Will Deacon
2021-05-18 9:47 ` [PATCH v6 15/21] sched: Defer wakeup in ttwu() for unschedulable frozen tasks Will Deacon
2021-05-18 9:47 ` [PATCH v6 16/21] arm64: Implement task_cpu_possible_mask() Will Deacon
2021-05-24 14:57 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 17/21] arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit EL0 Will Deacon
2021-05-24 15:02 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 18/21] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Will Deacon
2021-05-24 15:46 ` Catalin Marinas
2021-05-24 20:32 ` Will Deacon
2021-05-25 9:43 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 19/21] arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0 Will Deacon
2021-05-24 15:47 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 20/21] arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores Will Deacon
2021-05-24 15:47 ` Catalin Marinas
2021-05-18 9:47 ` [PATCH v6 21/21] Documentation: arm64: describe asymmetric 32-bit support Will Deacon
2021-05-21 17:37 ` Qais Yousef
2021-05-24 21:46 ` Will Deacon
2021-05-24 16:22 ` Catalin Marinas
2021-05-21 17:45 ` [PATCH v6 00/21] Add support for 32-bit tasks on asymmetric AArch32 systems Qais Yousef
2021-05-24 22:08 ` Will Deacon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210521171132.ev56j4isuxtf2zqa@e107158-lin.cambridge.arm.com \
--to=qais.yousef@arm.com \
--cc=catalin.marinas@arm.com \
--cc=gregkh@linuxfoundation.org \
--cc=hannes@cmpxchg.org \
--cc=juri.lelli@redhat.com \
--cc=kernel-team@android.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=maz@kernel.org \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=peterz@infradead.org \
--cc=qperret@google.com \
--cc=rjw@rjwysocki.net \
--cc=surenb@google.com \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).