From: Will Deacon <will@kernel.org> To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon <will@kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Marc Zyngier <maz@kernel.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Peter Zijlstra <peterz@infradead.org>, Morten Rasmussen <morten.rasmussen@arm.com>, Qais Yousef <qais.yousef@arm.com>, Suren Baghdasaryan <surenb@google.com>, Quentin Perret <qperret@google.com>, Tejun Heo <tj@kernel.org>, Johannes Weiner <hannes@cmpxchg.org>, Ingo Molnar <mingo@redhat.com>, Juri Lelli <juri.lelli@redhat.com>, Vincent Guittot <vincent.guittot@linaro.org>, "Rafael J. Wysocki" <rjw@rjwysocki.net>, Dietmar Eggemann <dietmar.eggemann@arm.com>, Daniel Bristot de Oliveira <bristot@redhat.com>, Valentin Schneider <valentin.schneider@arm.com>, Mark Rutland <mark.rutland@arm.com>, kernel-team@android.com, Valentin Schneider <Valentin.Schneider@arm.com> Subject: [PATCH v9 07/20] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Date: Tue, 8 Jun 2021 19:03:00 +0100 [thread overview] Message-ID: <20210608180313.11502-8-will@kernel.org> (raw) In-Reply-To: <20210608180313.11502-1-will@kernel.org> Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. Modify guarantee_online_cpus() to take task_cpu_possible_mask() into account when trying to find a suitable set of online CPUs for a given task. This will avoid passing an invalid mask to set_cpus_allowed_ptr() during ->attach() and will subsequently allow the cpuset hierarchy to be taken into account when forcefully overriding the affinity mask for a task which requires migration to a compatible CPU. Cc: Tejun Heo <tj@kernel.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Reviewed-by: Valentin Schneider <Valentin.Schneider@arm.com> Signed-off-by: Will Deacon <will@kernel.org> --- include/linux/cpuset.h | 2 +- kernel/cgroup/cpuset.c | 43 +++++++++++++++++++++++++----------------- 2 files changed, 27 insertions(+), 18 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index ed6ec677dd6b..414a8e694413 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -185,7 +185,7 @@ static inline void cpuset_read_unlock(void) { } static inline void cpuset_cpus_allowed(struct task_struct *p, struct cpumask *mask) { - cpumask_copy(mask, cpu_possible_mask); + cpumask_copy(mask, task_cpu_possible_mask(p)); } static inline void cpuset_cpus_allowed_fallback(struct task_struct *p) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 6ec7303d5b1f..4e7c271e3800 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -372,18 +372,29 @@ static inline bool is_in_v2_mode(void) } /* - * Return in pmask the portion of a cpusets's cpus_allowed that - * are online. If none are online, walk up the cpuset hierarchy - * until we find one that does have some online cpus. + * Return in pmask the portion of a task's cpusets's cpus_allowed that + * are online and are capable of running the task. If none are found, + * walk up the cpuset hierarchy until we find one that does have some + * appropriate cpus. * * One way or another, we guarantee to return some non-empty subset * of cpu_online_mask. * * Call with callback_lock or cpuset_mutex held. */ -static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) +static void guarantee_online_cpus(struct task_struct *tsk, + struct cpumask *pmask) { - while (!cpumask_intersects(cs->effective_cpus, cpu_online_mask)) { + const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); + struct cpuset *cs; + + if (WARN_ON(!cpumask_and(pmask, possible_mask, cpu_online_mask))) + cpumask_copy(pmask, cpu_online_mask); + + rcu_read_lock(); + cs = task_cs(tsk); + + while (!cpumask_intersects(cs->effective_cpus, pmask)) { cs = parent_cs(cs); if (unlikely(!cs)) { /* @@ -393,11 +404,13 @@ static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) * cpuset's effective_cpus is on its way to be * identical to cpu_online_mask. */ - cpumask_copy(pmask, cpu_online_mask); - return; + goto out_unlock; } } - cpumask_and(pmask, cs->effective_cpus, cpu_online_mask); + cpumask_and(pmask, pmask, cs->effective_cpus); + +out_unlock: + rcu_read_unlock(); } /* @@ -2199,15 +2212,13 @@ static void cpuset_attach(struct cgroup_taskset *tset) percpu_down_write(&cpuset_rwsem); - /* prepare for attach */ - if (cs == &top_cpuset) - cpumask_copy(cpus_attach, cpu_possible_mask); - else - guarantee_online_cpus(cs, cpus_attach); - guarantee_online_mems(cs, &cpuset_attach_nodemask_to); cgroup_taskset_for_each(task, css, tset) { + if (cs != &top_cpuset) + guarantee_online_cpus(task, cpus_attach); + else + cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); /* * can_attach beforehand should guarantee that this doesn't * fail. TODO: have a better way to handle failure here @@ -3302,9 +3313,7 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) unsigned long flags; spin_lock_irqsave(&callback_lock, flags); - rcu_read_lock(); - guarantee_online_cpus(task_cs(tsk), pmask); - rcu_read_unlock(); + guarantee_online_cpus(tsk, pmask); spin_unlock_irqrestore(&callback_lock, flags); } -- 2.32.0.rc1.229.g3e70b5a671-goog
WARNING: multiple messages have this Message-ID (diff)
From: Will Deacon <will@kernel.org> To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon <will@kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Marc Zyngier <maz@kernel.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Peter Zijlstra <peterz@infradead.org>, Morten Rasmussen <morten.rasmussen@arm.com>, Qais Yousef <qais.yousef@arm.com>, Suren Baghdasaryan <surenb@google.com>, Quentin Perret <qperret@google.com>, Tejun Heo <tj@kernel.org>, Johannes Weiner <hannes@cmpxchg.org>, Ingo Molnar <mingo@redhat.com>, Juri Lelli <juri.lelli@redhat.com>, Vincent Guittot <vincent.guittot@linaro.org>, "Rafael J. Wysocki" <rjw@rjwysocki.net>, Dietmar Eggemann <dietmar.eggemann@arm.com>, Daniel Bristot de Oliveira <bristot@redhat.com>, Valentin Schneider <valentin.schneider@arm.com>, Mark Rutland <mark.rutland@arm.com>, kernel-team@android.com, Valentin Schneider <Valentin.Schneider@arm.com> Subject: [PATCH v9 07/20] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Date: Tue, 8 Jun 2021 19:03:00 +0100 [thread overview] Message-ID: <20210608180313.11502-8-will@kernel.org> (raw) In-Reply-To: <20210608180313.11502-1-will@kernel.org> Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. Modify guarantee_online_cpus() to take task_cpu_possible_mask() into account when trying to find a suitable set of online CPUs for a given task. This will avoid passing an invalid mask to set_cpus_allowed_ptr() during ->attach() and will subsequently allow the cpuset hierarchy to be taken into account when forcefully overriding the affinity mask for a task which requires migration to a compatible CPU. Cc: Tejun Heo <tj@kernel.org> Cc: Johannes Weiner <hannes@cmpxchg.org> Reviewed-by: Valentin Schneider <Valentin.Schneider@arm.com> Signed-off-by: Will Deacon <will@kernel.org> --- include/linux/cpuset.h | 2 +- kernel/cgroup/cpuset.c | 43 +++++++++++++++++++++++++----------------- 2 files changed, 27 insertions(+), 18 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index ed6ec677dd6b..414a8e694413 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -185,7 +185,7 @@ static inline void cpuset_read_unlock(void) { } static inline void cpuset_cpus_allowed(struct task_struct *p, struct cpumask *mask) { - cpumask_copy(mask, cpu_possible_mask); + cpumask_copy(mask, task_cpu_possible_mask(p)); } static inline void cpuset_cpus_allowed_fallback(struct task_struct *p) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 6ec7303d5b1f..4e7c271e3800 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -372,18 +372,29 @@ static inline bool is_in_v2_mode(void) } /* - * Return in pmask the portion of a cpusets's cpus_allowed that - * are online. If none are online, walk up the cpuset hierarchy - * until we find one that does have some online cpus. + * Return in pmask the portion of a task's cpusets's cpus_allowed that + * are online and are capable of running the task. If none are found, + * walk up the cpuset hierarchy until we find one that does have some + * appropriate cpus. * * One way or another, we guarantee to return some non-empty subset * of cpu_online_mask. * * Call with callback_lock or cpuset_mutex held. */ -static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) +static void guarantee_online_cpus(struct task_struct *tsk, + struct cpumask *pmask) { - while (!cpumask_intersects(cs->effective_cpus, cpu_online_mask)) { + const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); + struct cpuset *cs; + + if (WARN_ON(!cpumask_and(pmask, possible_mask, cpu_online_mask))) + cpumask_copy(pmask, cpu_online_mask); + + rcu_read_lock(); + cs = task_cs(tsk); + + while (!cpumask_intersects(cs->effective_cpus, pmask)) { cs = parent_cs(cs); if (unlikely(!cs)) { /* @@ -393,11 +404,13 @@ static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) * cpuset's effective_cpus is on its way to be * identical to cpu_online_mask. */ - cpumask_copy(pmask, cpu_online_mask); - return; + goto out_unlock; } } - cpumask_and(pmask, cs->effective_cpus, cpu_online_mask); + cpumask_and(pmask, pmask, cs->effective_cpus); + +out_unlock: + rcu_read_unlock(); } /* @@ -2199,15 +2212,13 @@ static void cpuset_attach(struct cgroup_taskset *tset) percpu_down_write(&cpuset_rwsem); - /* prepare for attach */ - if (cs == &top_cpuset) - cpumask_copy(cpus_attach, cpu_possible_mask); - else - guarantee_online_cpus(cs, cpus_attach); - guarantee_online_mems(cs, &cpuset_attach_nodemask_to); cgroup_taskset_for_each(task, css, tset) { + if (cs != &top_cpuset) + guarantee_online_cpus(task, cpus_attach); + else + cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); /* * can_attach beforehand should guarantee that this doesn't * fail. TODO: have a better way to handle failure here @@ -3302,9 +3313,7 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) unsigned long flags; spin_lock_irqsave(&callback_lock, flags); - rcu_read_lock(); - guarantee_online_cpus(task_cs(tsk), pmask); - rcu_read_unlock(); + guarantee_online_cpus(tsk, pmask); spin_unlock_irqrestore(&callback_lock, flags); } -- 2.32.0.rc1.229.g3e70b5a671-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-06-08 18:03 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-06-08 18:02 [PATCH v9 00/20] Add support for 32-bit tasks on asymmetric AArch32 systems Will Deacon 2021-06-08 18:02 ` Will Deacon 2021-06-08 18:02 ` [PATCH v9 01/20] arm64: cpuinfo: Split AArch32 registers out into a separate struct Will Deacon 2021-06-08 18:02 ` Will Deacon 2021-06-08 18:02 ` [PATCH v9 02/20] arm64: Allow mismatched 32-bit EL0 support Will Deacon 2021-06-08 18:02 ` Will Deacon 2021-06-08 18:02 ` [PATCH v9 03/20] KVM: arm64: Kill 32-bit vCPUs on systems with mismatched " Will Deacon 2021-06-08 18:02 ` Will Deacon 2021-06-08 18:02 ` [PATCH v9 04/20] arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs Will Deacon 2021-06-08 18:02 ` Will Deacon 2021-06-08 18:02 ` [PATCH v9 05/20] sched: Introduce task_cpu_possible_mask() to limit fallback rq selection Will Deacon 2021-06-08 18:02 ` Will Deacon 2021-06-08 18:02 ` [PATCH v9 06/20] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Will Deacon 2021-06-08 18:02 ` Will Deacon 2021-06-10 12:59 ` Valentin Schneider 2021-06-10 12:59 ` Valentin Schneider 2021-06-08 18:03 ` Will Deacon [this message] 2021-06-08 18:03 ` [PATCH v9 07/20] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Will Deacon 2021-06-08 18:03 ` [PATCH v9 08/20] cpuset: Cleanup cpuset_cpus_allowed_fallback() use in select_fallback_rq() Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-10 12:59 ` Valentin Schneider 2021-06-10 12:59 ` Valentin Schneider 2021-06-08 18:03 ` [PATCH v9 09/20] sched: Reject CPU affinity changes based on task_cpu_possible_mask() Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 10/20] sched: Introduce task_struct::user_cpus_ptr to track requested affinity Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 11/20] sched: Split the guts of sched_setaffinity() into a helper function Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 12/20] sched: Allow task CPU affinity to be restricted on asymmetric systems Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-10 13:00 ` Valentin Schneider 2021-06-10 13:00 ` Valentin Schneider 2021-06-08 18:03 ` [PATCH v9 13/20] sched: Introduce dl_task_check_affinity() to check proposed affinity Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 14/20] arm64: Implement task_cpu_possible_mask() Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 15/20] arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit EL0 Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 16/20] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 17/20] arm64: Advertise CPUs capable of running 32-bit applications in sysfs Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 18/20] arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0 Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 19/20] arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-08 18:03 ` [PATCH v9 20/20] Documentation: arm64: describe asymmetric 32-bit support Will Deacon 2021-06-08 18:03 ` Will Deacon 2021-06-11 16:15 ` [PATCH v9 00/20] Add support for 32-bit tasks on asymmetric AArch32 systems Will Deacon 2021-06-11 16:15 ` Will Deacon
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210608180313.11502-8-will@kernel.org \ --to=will@kernel.org \ --cc=bristot@redhat.com \ --cc=catalin.marinas@arm.com \ --cc=dietmar.eggemann@arm.com \ --cc=gregkh@linuxfoundation.org \ --cc=hannes@cmpxchg.org \ --cc=juri.lelli@redhat.com \ --cc=kernel-team@android.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mark.rutland@arm.com \ --cc=maz@kernel.org \ --cc=mingo@redhat.com \ --cc=morten.rasmussen@arm.com \ --cc=peterz@infradead.org \ --cc=qais.yousef@arm.com \ --cc=qperret@google.com \ --cc=rjw@rjwysocki.net \ --cc=surenb@google.com \ --cc=tj@kernel.org \ --cc=valentin.schneider@arm.com \ --cc=vincent.guittot@linaro.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.