From: Qais Yousef <qais.yousef@arm.com> To: Quentin Perret <qperret@google.com> Cc: Will Deacon <will@kernel.org>, linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Catalin Marinas <catalin.marinas@arm.com>, Marc Zyngier <maz@kernel.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Peter Zijlstra <peterz@infradead.org>, Morten Rasmussen <morten.rasmussen@arm.com>, Suren Baghdasaryan <surenb@google.com>, Tejun Heo <tj@kernel.org>, Li Zefan <lizefan@huawei.com>, Johannes Weiner <hannes@cmpxchg.org>, Ingo Molnar <mingo@redhat.com>, Juri Lelli <juri.lelli@redhat.com>, Vincent Guittot <vincent.guittot@linaro.org>, kernel-team@android.com Subject: Re: [PATCH v4 09/14] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Date: Tue, 1 Dec 2020 11:58:42 +0000 [thread overview] Message-ID: <20201201115842.t77abecneuesd5ih@e107158-lin.cambridge.arm.com> (raw) In-Reply-To: <20201130173610.GA1715200@google.com> On 11/30/20 17:36, Quentin Perret wrote: > On Monday 30 Nov 2020 at 17:05:31 (+0000), Qais Yousef wrote: > > I create 3 cpusets: 64bit, 32bit and mix. As the name indicates, 64bit contains > > all 64bit-only cpus, 32bit contains 32bit-capable ones and mix has a mixture of > > both. > > > > If I try to move my test binary to 64bit cpuset, it moves there and I see the > > WARN_ON_ONCE() triggered. The task has attached to the new cpuset but > > set_allowed_cpus_ptr() has failed and we end up with whatever affinity we had > > previously. Breaking cpusets effectively. > > Right, and so does exec'ing from a 64 bit task into 32 bit executable > from within a 64 bit-only cpuset :( . And there is nothing we can really True. The kernel can decide to kill the task or force detach it then, no? Sending SIGKILL makes more sense. > do about it, we cannot fail the exec because we need this to work for > existing apps, and there is no way the Android framework can know > upfront. It knows upfront it has enabled asym aarch32. So it needs to make sure not to create 'invalid' cpusets? > > So the only thing we can do really is WARN() and proceed to ignore the > cpuset, which is what this series does :/. It's not exactly pretty but I > don't think we can do much better than that TBH, and it's the same thing > for the example you brought up. Failing cpuset_can_attach() will not > help, we can only WARN and proceed ... I think for cases where we can prevent userspace from doing something wrong, we should. Like trying to attach to a cpuset that will result in an empty mask. FWIW, it does something similar with deadline tasks. See task_can_attach(). Similarly for the case when userspace tries to modify the cpuset.cpus such that a task will end up with empty cpumask. We now have the new case that some tasks can only run on a subset of cpu_possible_mask. So the definition of empty cpumask has gained an extra meaning. > > Now, Android should be fine with that I think. We only need the kernel > to implement a safe fallback mechanism when userspace gives > contradictory commands, because we know there are edge cases userspace > _cannot_ deal with correctly, but this fallback doesn't need to be > highly optimized (at least for Android), but I'm happy to hear what > others think. Why not go with our original patch that fixes affinity then in the arch code if the task wakes up on the wrong cpu? It is much simpler approach IMO to achieve the same thing. I was under the impression that if we go down teaching the scheduler about asym ISA, then we have to deal with these edge cases. I don't think we're far away from getting there. Thanks -- Qais Yousef
WARNING: multiple messages have this Message-ID (diff)
From: Qais Yousef <qais.yousef@arm.com> To: Quentin Perret <qperret@google.com> Cc: linux-arch@vger.kernel.org, Juri Lelli <juri.lelli@redhat.com>, kernel-team@android.com, Vincent Guittot <vincent.guittot@linaro.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Peter Zijlstra <peterz@infradead.org>, Catalin Marinas <catalin.marinas@arm.com>, Johannes Weiner <hannes@cmpxchg.org>, linux-kernel@vger.kernel.org, Suren Baghdasaryan <surenb@google.com>, Ingo Molnar <mingo@redhat.com>, Li Zefan <lizefan@huawei.com>, Marc Zyngier <maz@kernel.org>, Tejun Heo <tj@kernel.org>, Will Deacon <will@kernel.org>, Morten Rasmussen <morten.rasmussen@arm.com>, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v4 09/14] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Date: Tue, 1 Dec 2020 11:58:42 +0000 [thread overview] Message-ID: <20201201115842.t77abecneuesd5ih@e107158-lin.cambridge.arm.com> (raw) In-Reply-To: <20201130173610.GA1715200@google.com> On 11/30/20 17:36, Quentin Perret wrote: > On Monday 30 Nov 2020 at 17:05:31 (+0000), Qais Yousef wrote: > > I create 3 cpusets: 64bit, 32bit and mix. As the name indicates, 64bit contains > > all 64bit-only cpus, 32bit contains 32bit-capable ones and mix has a mixture of > > both. > > > > If I try to move my test binary to 64bit cpuset, it moves there and I see the > > WARN_ON_ONCE() triggered. The task has attached to the new cpuset but > > set_allowed_cpus_ptr() has failed and we end up with whatever affinity we had > > previously. Breaking cpusets effectively. > > Right, and so does exec'ing from a 64 bit task into 32 bit executable > from within a 64 bit-only cpuset :( . And there is nothing we can really True. The kernel can decide to kill the task or force detach it then, no? Sending SIGKILL makes more sense. > do about it, we cannot fail the exec because we need this to work for > existing apps, and there is no way the Android framework can know > upfront. It knows upfront it has enabled asym aarch32. So it needs to make sure not to create 'invalid' cpusets? > > So the only thing we can do really is WARN() and proceed to ignore the > cpuset, which is what this series does :/. It's not exactly pretty but I > don't think we can do much better than that TBH, and it's the same thing > for the example you brought up. Failing cpuset_can_attach() will not > help, we can only WARN and proceed ... I think for cases where we can prevent userspace from doing something wrong, we should. Like trying to attach to a cpuset that will result in an empty mask. FWIW, it does something similar with deadline tasks. See task_can_attach(). Similarly for the case when userspace tries to modify the cpuset.cpus such that a task will end up with empty cpumask. We now have the new case that some tasks can only run on a subset of cpu_possible_mask. So the definition of empty cpumask has gained an extra meaning. > > Now, Android should be fine with that I think. We only need the kernel > to implement a safe fallback mechanism when userspace gives > contradictory commands, because we know there are edge cases userspace > _cannot_ deal with correctly, but this fallback doesn't need to be > highly optimized (at least for Android), but I'm happy to hear what > others think. Why not go with our original patch that fixes affinity then in the arch code if the task wakes up on the wrong cpu? It is much simpler approach IMO to achieve the same thing. I was under the impression that if we go down teaching the scheduler about asym ISA, then we have to deal with these edge cases. I don't think we're far away from getting there. Thanks -- Qais Yousef _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-12-01 11:59 UTC|newest] Thread overview: 122+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-24 15:50 [PATCH v4 00/14] An alternative series for asymmetric AArch32 systems Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-24 15:50 ` [PATCH v4 01/14] arm64: cpuinfo: Split AArch32 registers out into a separate struct Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-24 15:50 ` [PATCH v4 02/14] arm64: Allow mismatched 32-bit EL0 support Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 10:25 ` Marc Zyngier 2020-11-27 10:25 ` Marc Zyngier 2020-11-27 11:50 ` Will Deacon 2020-11-27 11:50 ` Will Deacon 2020-11-27 13:09 ` Qais Yousef 2020-11-27 13:09 ` Qais Yousef 2020-12-01 16:56 ` Will Deacon 2020-12-01 16:56 ` Will Deacon 2020-12-02 13:16 ` Qais Yousef 2020-12-02 13:16 ` Qais Yousef 2020-11-24 15:50 ` [PATCH v4 03/14] KVM: arm64: Kill 32-bit vCPUs on systems with mismatched " Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 10:26 ` Marc Zyngier 2020-11-27 10:26 ` Marc Zyngier 2020-11-27 11:53 ` Will Deacon 2020-11-27 11:53 ` Will Deacon 2020-11-27 17:14 ` Marc Zyngier 2020-11-27 17:14 ` Marc Zyngier 2020-11-27 17:24 ` Quentin Perret 2020-11-27 17:24 ` Quentin Perret 2020-11-27 18:16 ` Marc Zyngier 2020-11-27 18:16 ` Marc Zyngier 2020-12-01 16:57 ` Will Deacon 2020-12-01 16:57 ` Will Deacon 2020-12-02 8:18 ` Marc Zyngier 2020-12-02 8:18 ` Marc Zyngier 2020-12-02 17:27 ` Will Deacon 2020-12-02 17:27 ` Will Deacon 2020-11-24 15:50 ` [PATCH v4 04/14] arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 13:12 ` Qais Yousef 2020-11-27 13:12 ` Qais Yousef 2020-12-01 16:56 ` Will Deacon 2020-12-01 16:56 ` Will Deacon 2020-12-02 13:52 ` Qais Yousef 2020-12-02 13:52 ` Qais Yousef 2020-12-02 17:42 ` Will Deacon 2020-12-02 17:42 ` Will Deacon 2020-11-24 15:50 ` [PATCH v4 05/14] arm64: Advertise CPUs capable of running 32-bit applications in sysfs Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-24 15:50 ` [PATCH v4 06/14] arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0 Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 13:17 ` Qais Yousef 2020-11-27 13:17 ` Qais Yousef 2020-12-01 16:56 ` Will Deacon 2020-12-01 16:56 ` Will Deacon 2020-11-24 15:50 ` [PATCH v4 07/14] sched: Introduce restrict_cpus_allowed_ptr() to limit task CPU affinity Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 9:49 ` Quentin Perret 2020-11-27 9:49 ` Quentin Perret 2020-11-27 13:19 ` Qais Yousef 2020-11-27 13:19 ` Qais Yousef 2020-12-01 16:56 ` Will Deacon 2020-12-01 16:56 ` Will Deacon 2020-12-02 13:06 ` Qais Yousef 2020-12-02 13:06 ` Qais Yousef 2020-11-24 15:50 ` [PATCH v4 08/14] arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit EL0 Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 10:01 ` Quentin Perret 2020-11-27 10:01 ` Quentin Perret 2020-11-27 13:23 ` Qais Yousef 2020-11-27 13:23 ` Qais Yousef 2020-12-01 16:55 ` Will Deacon 2020-12-01 16:55 ` Will Deacon 2020-12-02 14:07 ` Qais Yousef 2020-12-02 14:07 ` Qais Yousef 2020-11-24 15:50 ` [PATCH v4 09/14] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 13:32 ` Qais Yousef 2020-11-27 13:32 ` Qais Yousef 2020-11-30 17:05 ` Qais Yousef 2020-11-30 17:05 ` Qais Yousef 2020-11-30 17:36 ` Quentin Perret 2020-11-30 17:36 ` Quentin Perret 2020-12-01 11:58 ` Qais Yousef [this message] 2020-12-01 11:58 ` Qais Yousef 2020-12-01 12:37 ` Quentin Perret 2020-12-01 12:37 ` Quentin Perret 2020-12-01 14:11 ` Qais Yousef 2020-12-01 14:11 ` Qais Yousef 2020-12-01 15:56 ` Quentin Perret 2020-12-01 15:56 ` Quentin Perret 2020-12-01 22:30 ` Will Deacon 2020-12-01 22:30 ` Will Deacon 2020-12-02 11:34 ` Qais Yousef 2020-12-02 11:34 ` Qais Yousef 2020-12-02 11:33 ` Qais Yousef 2020-12-02 11:33 ` Qais Yousef 2020-11-24 15:50 ` [PATCH v4 10/14] sched: Introduce arch_task_cpu_possible_mask() to limit fallback rq selection Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-24 15:50 ` [PATCH v4 11/14] sched: Reject CPU affinity changes based on arch_task_cpu_possible_mask() Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 9:54 ` Quentin Perret 2020-11-27 9:54 ` Quentin Perret 2020-11-24 15:50 ` [PATCH v4 12/14] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 13:41 ` Qais Yousef 2020-11-27 13:41 ` Qais Yousef 2020-12-01 22:13 ` Will Deacon 2020-12-01 22:13 ` Will Deacon 2020-12-02 12:59 ` Qais Yousef 2020-12-02 12:59 ` Qais Yousef 2020-12-02 17:42 ` Will Deacon 2020-12-02 17:42 ` Will Deacon 2020-12-02 18:08 ` Qais Yousef 2020-12-02 18:08 ` Qais Yousef 2020-11-24 15:50 ` [PATCH v4 13/14] arm64: Implement arch_task_cpu_possible_mask() Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 13:41 ` Qais Yousef 2020-11-27 13:41 ` Qais Yousef 2020-11-24 15:50 ` [PATCH v4 14/14] arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores Will Deacon 2020-11-24 15:50 ` Will Deacon 2020-11-27 13:58 ` [PATCH v4 00/14] An alternative series for asymmetric AArch32 systems Qais Yousef 2020-11-27 13:58 ` Qais Yousef 2020-12-05 20:43 ` Pavel Machek 2020-12-05 20:43 ` Pavel Machek
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20201201115842.t77abecneuesd5ih@e107158-lin.cambridge.arm.com \ --to=qais.yousef@arm.com \ --cc=catalin.marinas@arm.com \ --cc=gregkh@linuxfoundation.org \ --cc=hannes@cmpxchg.org \ --cc=juri.lelli@redhat.com \ --cc=kernel-team@android.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=lizefan@huawei.com \ --cc=maz@kernel.org \ --cc=mingo@redhat.com \ --cc=morten.rasmussen@arm.com \ --cc=peterz@infradead.org \ --cc=qperret@google.com \ --cc=surenb@google.com \ --cc=tj@kernel.org \ --cc=vincent.guittot@linaro.org \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.