From: Suren Baghdasaryan <surenb@google.com> To: Will Deacon <will@kernel.org> Cc: linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Marc Zyngier <maz@kernel.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Peter Zijlstra <peterz@infradead.org>, Morten Rasmussen <morten.rasmussen@arm.com>, Qais Yousef <qais.yousef@arm.com>, Quentin Perret <qperret@google.com>, Tejun Heo <tj@kernel.org>, Li Zefan <lizefan@huawei.com>, Johannes Weiner <hannes@cmpxchg.org>, Ingo Molnar <mingo@redhat.com>, Juri Lelli <juri.lelli@redhat.com>, Vincent Guittot <vincent.guittot@linaro.org>, kernel-team <kernel-team@android.com> Subject: Re: [PATCH v5 08/15] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Date: Sun, 27 Dec 2020 19:54:04 -0800 [thread overview] Message-ID: <CAJuCfpGdG4Wmrb-pVPTFuqWkfMKTkhL0f-_Dv6M-0M7VDnW5Bw@mail.gmail.com> (raw) In-Reply-To: <20201208132835.6151-9-will@kernel.org> On Tue, Dec 8, 2020 at 5:29 AM Will Deacon <will@kernel.org> wrote: > > Asymmetric systems may not offer the same level of userspace ISA support > across all CPUs, meaning that some applications cannot be executed by > some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do > not feature support for 32-bit applications on both clusters. > > Modify guarantee_online_cpus() to take task_cpu_possible_mask() into > account when trying to find a suitable set of online CPUs for a given > task. This will avoid passing an invalid mask to set_cpus_allowed_ptr() > during ->attach() and will subsequently allow the cpuset hierarchy to be > taken into account when forcefully overriding the affinity mask for a > task which requires migration to a compatible CPU. > > Cc: Li Zefan <lizefan@huawei.com> > Cc: Tejun Heo <tj@kernel.org> > Cc: Johannes Weiner <hannes@cmpxchg.org> > Signed-off-by: Will Deacon <will@kernel.org> > --- > include/linux/cpuset.h | 3 ++- > kernel/cgroup/cpuset.c | 33 +++++++++++++++++++-------------- > 2 files changed, 21 insertions(+), 15 deletions(-) > > diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h > index 04c20de66afc..414a8e694413 100644 > --- a/include/linux/cpuset.h > +++ b/include/linux/cpuset.h > @@ -15,6 +15,7 @@ > #include <linux/cpumask.h> > #include <linux/nodemask.h> > #include <linux/mm.h> > +#include <linux/mmu_context.h> > #include <linux/jump_label.h> > > #ifdef CONFIG_CPUSETS > @@ -184,7 +185,7 @@ static inline void cpuset_read_unlock(void) { } > static inline void cpuset_cpus_allowed(struct task_struct *p, > struct cpumask *mask) > { > - cpumask_copy(mask, cpu_possible_mask); > + cpumask_copy(mask, task_cpu_possible_mask(p)); > } > > static inline void cpuset_cpus_allowed_fallback(struct task_struct *p) > diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c > index e970737c3ed2..d30febf1f69f 100644 > --- a/kernel/cgroup/cpuset.c > +++ b/kernel/cgroup/cpuset.c > @@ -372,18 +372,26 @@ static inline bool is_in_v2_mode(void) > } > > /* > - * Return in pmask the portion of a cpusets's cpus_allowed that > - * are online. If none are online, walk up the cpuset hierarchy > - * until we find one that does have some online cpus. > + * Return in pmask the portion of a task's cpusets's cpus_allowed that > + * are online and are capable of running the task. If none are found, > + * walk up the cpuset hierarchy until we find one that does have some > + * appropriate cpus. > * > * One way or another, we guarantee to return some non-empty subset > * of cpu_online_mask. > * > * Call with callback_lock or cpuset_mutex held. > */ > -static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) > +static void guarantee_online_cpus(struct task_struct *tsk, > + struct cpumask *pmask) > { > - while (!cpumask_intersects(cs->effective_cpus, cpu_online_mask)) { > + struct cpuset *cs = task_cs(tsk); > + const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); > + > + if (WARN_ON(!cpumask_and(pmask, possible_mask, cpu_online_mask))) IIUC, this represents the case when there is no online CPU that can run this task. In this situation guarantee_online_cpus() will return an online CPU which can't run the task (because we ignore possible_mask). I don't think this can be considered a valid fallback path. However I think patch [13/15] ensures that we never end up in this situation by disallowing to offline the last 32-bit capable CPU. If that's true then maybe the patches can be reordered so that [13/15] comes before this one and this condition can be treated as a bug here? > + cpumask_copy(pmask, cpu_online_mask); > + > + while (!cpumask_intersects(cs->effective_cpus, pmask)) { > cs = parent_cs(cs); > if (unlikely(!cs)) { > /* > @@ -393,11 +401,10 @@ static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) > * cpuset's effective_cpus is on its way to be > * identical to cpu_online_mask. > */ > - cpumask_copy(pmask, cpu_online_mask); > return; > } > } > - cpumask_and(pmask, cs->effective_cpus, cpu_online_mask); > + cpumask_and(pmask, pmask, cs->effective_cpus); > } > > /* > @@ -2176,15 +2183,13 @@ static void cpuset_attach(struct cgroup_taskset *tset) > > percpu_down_write(&cpuset_rwsem); > > - /* prepare for attach */ > - if (cs == &top_cpuset) > - cpumask_copy(cpus_attach, cpu_possible_mask); > - else > - guarantee_online_cpus(cs, cpus_attach); > - > guarantee_online_mems(cs, &cpuset_attach_nodemask_to); > > cgroup_taskset_for_each(task, css, tset) { > + if (cs != &top_cpuset) > + guarantee_online_cpus(task, cpus_attach); > + else > + cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); > /* > * can_attach beforehand should guarantee that this doesn't > * fail. TODO: have a better way to handle failure here > @@ -3280,7 +3285,7 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) > > spin_lock_irqsave(&callback_lock, flags); > rcu_read_lock(); > - guarantee_online_cpus(task_cs(tsk), pmask); > + guarantee_online_cpus(tsk, pmask); > rcu_read_unlock(); > spin_unlock_irqrestore(&callback_lock, flags); > } > -- > 2.29.2.576.ga3fc446d84-goog >
WARNING: multiple messages have this Message-ID (diff)
From: Suren Baghdasaryan <surenb@google.com> To: Will Deacon <will@kernel.org> Cc: linux-arch@vger.kernel.org, Marc Zyngier <maz@kernel.org>, kernel-team <kernel-team@android.com>, Vincent Guittot <vincent.guittot@linaro.org>, Juri Lelli <juri.lelli@redhat.com>, Quentin Perret <qperret@google.com>, Peter Zijlstra <peterz@infradead.org>, Catalin Marinas <catalin.marinas@arm.com>, Johannes Weiner <hannes@cmpxchg.org>, LKML <linux-kernel@vger.kernel.org>, Qais Yousef <qais.yousef@arm.com>, Li Zefan <lizefan@huawei.com>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Tejun Heo <tj@kernel.org>, Ingo Molnar <mingo@redhat.com>, Morten Rasmussen <morten.rasmussen@arm.com>, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v5 08/15] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Date: Sun, 27 Dec 2020 19:54:04 -0800 [thread overview] Message-ID: <CAJuCfpGdG4Wmrb-pVPTFuqWkfMKTkhL0f-_Dv6M-0M7VDnW5Bw@mail.gmail.com> (raw) In-Reply-To: <20201208132835.6151-9-will@kernel.org> On Tue, Dec 8, 2020 at 5:29 AM Will Deacon <will@kernel.org> wrote: > > Asymmetric systems may not offer the same level of userspace ISA support > across all CPUs, meaning that some applications cannot be executed by > some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do > not feature support for 32-bit applications on both clusters. > > Modify guarantee_online_cpus() to take task_cpu_possible_mask() into > account when trying to find a suitable set of online CPUs for a given > task. This will avoid passing an invalid mask to set_cpus_allowed_ptr() > during ->attach() and will subsequently allow the cpuset hierarchy to be > taken into account when forcefully overriding the affinity mask for a > task which requires migration to a compatible CPU. > > Cc: Li Zefan <lizefan@huawei.com> > Cc: Tejun Heo <tj@kernel.org> > Cc: Johannes Weiner <hannes@cmpxchg.org> > Signed-off-by: Will Deacon <will@kernel.org> > --- > include/linux/cpuset.h | 3 ++- > kernel/cgroup/cpuset.c | 33 +++++++++++++++++++-------------- > 2 files changed, 21 insertions(+), 15 deletions(-) > > diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h > index 04c20de66afc..414a8e694413 100644 > --- a/include/linux/cpuset.h > +++ b/include/linux/cpuset.h > @@ -15,6 +15,7 @@ > #include <linux/cpumask.h> > #include <linux/nodemask.h> > #include <linux/mm.h> > +#include <linux/mmu_context.h> > #include <linux/jump_label.h> > > #ifdef CONFIG_CPUSETS > @@ -184,7 +185,7 @@ static inline void cpuset_read_unlock(void) { } > static inline void cpuset_cpus_allowed(struct task_struct *p, > struct cpumask *mask) > { > - cpumask_copy(mask, cpu_possible_mask); > + cpumask_copy(mask, task_cpu_possible_mask(p)); > } > > static inline void cpuset_cpus_allowed_fallback(struct task_struct *p) > diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c > index e970737c3ed2..d30febf1f69f 100644 > --- a/kernel/cgroup/cpuset.c > +++ b/kernel/cgroup/cpuset.c > @@ -372,18 +372,26 @@ static inline bool is_in_v2_mode(void) > } > > /* > - * Return in pmask the portion of a cpusets's cpus_allowed that > - * are online. If none are online, walk up the cpuset hierarchy > - * until we find one that does have some online cpus. > + * Return in pmask the portion of a task's cpusets's cpus_allowed that > + * are online and are capable of running the task. If none are found, > + * walk up the cpuset hierarchy until we find one that does have some > + * appropriate cpus. > * > * One way or another, we guarantee to return some non-empty subset > * of cpu_online_mask. > * > * Call with callback_lock or cpuset_mutex held. > */ > -static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) > +static void guarantee_online_cpus(struct task_struct *tsk, > + struct cpumask *pmask) > { > - while (!cpumask_intersects(cs->effective_cpus, cpu_online_mask)) { > + struct cpuset *cs = task_cs(tsk); > + const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); > + > + if (WARN_ON(!cpumask_and(pmask, possible_mask, cpu_online_mask))) IIUC, this represents the case when there is no online CPU that can run this task. In this situation guarantee_online_cpus() will return an online CPU which can't run the task (because we ignore possible_mask). I don't think this can be considered a valid fallback path. However I think patch [13/15] ensures that we never end up in this situation by disallowing to offline the last 32-bit capable CPU. If that's true then maybe the patches can be reordered so that [13/15] comes before this one and this condition can be treated as a bug here? > + cpumask_copy(pmask, cpu_online_mask); > + > + while (!cpumask_intersects(cs->effective_cpus, pmask)) { > cs = parent_cs(cs); > if (unlikely(!cs)) { > /* > @@ -393,11 +401,10 @@ static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) > * cpuset's effective_cpus is on its way to be > * identical to cpu_online_mask. > */ > - cpumask_copy(pmask, cpu_online_mask); > return; > } > } > - cpumask_and(pmask, cs->effective_cpus, cpu_online_mask); > + cpumask_and(pmask, pmask, cs->effective_cpus); > } > > /* > @@ -2176,15 +2183,13 @@ static void cpuset_attach(struct cgroup_taskset *tset) > > percpu_down_write(&cpuset_rwsem); > > - /* prepare for attach */ > - if (cs == &top_cpuset) > - cpumask_copy(cpus_attach, cpu_possible_mask); > - else > - guarantee_online_cpus(cs, cpus_attach); > - > guarantee_online_mems(cs, &cpuset_attach_nodemask_to); > > cgroup_taskset_for_each(task, css, tset) { > + if (cs != &top_cpuset) > + guarantee_online_cpus(task, cpus_attach); > + else > + cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); > /* > * can_attach beforehand should guarantee that this doesn't > * fail. TODO: have a better way to handle failure here > @@ -3280,7 +3285,7 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) > > spin_lock_irqsave(&callback_lock, flags); > rcu_read_lock(); > - guarantee_online_cpus(task_cs(tsk), pmask); > + guarantee_online_cpus(tsk, pmask); > rcu_read_unlock(); > spin_unlock_irqrestore(&callback_lock, flags); > } > -- > 2.29.2.576.ga3fc446d84-goog > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-12-28 3:55 UTC|newest] Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-12-08 13:28 [PATCH v5 00/15] An alternative series for asymmetric AArch32 systems Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 01/15] arm64: cpuinfo: Split AArch32 registers out into a separate struct Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 02/15] arm64: Allow mismatched 32-bit EL0 support Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 03/15] KVM: arm64: Kill 32-bit vCPUs on systems with mismatched " Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 04/15] arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 05/15] arm64: Advertise CPUs capable of running 32-bit applications in sysfs Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 06/15] sched: Introduce task_cpu_possible_mask() to limit fallback rq selection Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 07/15] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-17 12:15 ` Qais Yousef 2020-12-17 12:15 ` Qais Yousef 2020-12-17 13:44 ` Peter Zijlstra 2020-12-17 13:44 ` Peter Zijlstra 2020-12-17 14:59 ` Will Deacon 2020-12-17 14:59 ` Will Deacon 2020-12-17 15:00 ` Qais Yousef 2020-12-17 15:00 ` Qais Yousef 2020-12-08 13:28 ` [PATCH v5 08/15] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-28 3:54 ` Suren Baghdasaryan [this message] 2020-12-28 3:54 ` Suren Baghdasaryan 2020-12-08 13:28 ` [PATCH v5 09/15] sched: Reject CPU affinity changes based on task_cpu_possible_mask() Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 10/15] sched: Introduce force_compatible_cpus_allowed_ptr() to limit CPU affinity Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-28 4:29 ` Suren Baghdasaryan 2020-12-28 4:29 ` Suren Baghdasaryan 2020-12-08 13:28 ` [PATCH v5 11/15] arm64: Implement task_cpu_possible_mask() Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 12/15] arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit EL0 Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 13/15] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 14/15] arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0 Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-08 13:28 ` [PATCH v5 15/15] arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores Will Deacon 2020-12-08 13:28 ` Will Deacon 2020-12-15 17:36 ` [PATCH v5 00/15] An alternative series for asymmetric AArch32 systems Peter Zijlstra 2020-12-15 17:36 ` Peter Zijlstra 2020-12-15 18:50 ` Will Deacon 2020-12-15 18:50 ` Will Deacon 2020-12-17 10:55 ` Peter Zijlstra 2020-12-17 10:55 ` Peter Zijlstra 2020-12-16 11:16 ` Qais Yousef 2020-12-16 11:16 ` Qais Yousef 2020-12-16 14:14 ` Will Deacon 2020-12-16 14:14 ` Will Deacon 2020-12-16 16:48 ` Qais Yousef 2020-12-16 16:48 ` Qais Yousef 2020-12-16 18:21 ` Suren Baghdasaryan 2020-12-16 18:21 ` Suren Baghdasaryan
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=CAJuCfpGdG4Wmrb-pVPTFuqWkfMKTkhL0f-_Dv6M-0M7VDnW5Bw@mail.gmail.com \ --to=surenb@google.com \ --cc=catalin.marinas@arm.com \ --cc=gregkh@linuxfoundation.org \ --cc=hannes@cmpxchg.org \ --cc=juri.lelli@redhat.com \ --cc=kernel-team@android.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=lizefan@huawei.com \ --cc=maz@kernel.org \ --cc=mingo@redhat.com \ --cc=morten.rasmussen@arm.com \ --cc=peterz@infradead.org \ --cc=qais.yousef@arm.com \ --cc=qperret@google.com \ --cc=tj@kernel.org \ --cc=vincent.guittot@linaro.org \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.