All of lore.kernel.org
 help / color / mirror / Atom feed
From: Will Deacon <will@kernel.org>
To: linux-arm-kernel@lists.infradead.org
Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org,
	Will Deacon <will@kernel.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Marc Zyngier <maz@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Morten Rasmussen <morten.rasmussen@arm.com>,
	Qais Yousef <qais.yousef@arm.com>,
	Suren Baghdasaryan <surenb@google.com>,
	Quentin Perret <qperret@google.com>, Tejun Heo <tj@kernel.org>,
	Li Zefan <lizefan@huawei.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Ingo Molnar <mingo@redhat.com>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	kernel-team@android.com
Subject: [PATCH v3 00/14] An alternative series for asymmetric AArch32 systems
Date: Fri, 13 Nov 2020 09:37:05 +0000	[thread overview]
Message-ID: <20201113093720.21106-1-will@kernel.org> (raw)

Hi again everyone,

	I'm not a scheduler hacker,
	I'm a scheduler hacker's mate.
	I'm only hacking the scheduler,
	'cos trying to run 32-bit applications on systems where not all of the CPUs support it is GREAT.

It's Friday 13th, and I'm back with version three of the increasingly
popular patches I previously posted here:

  v1: https://lore.kernel.org/r/20201027215118.27003-1-will@kernel.org
  v2: https://lore.kernel.org/r/20201109213023.15092-1-will@kernel.org

and which started life as a reimplementation of some patches from Qais:

  https://lore.kernel.org/r/20201021104611.2744565-1-qais.yousef@arm.com

The aim of this series is to allow 32-bit ARM applications to run on
arm64 SoCs where not all of the CPUs support the 32-bit instruction set.

There are some major changes in v3:

  * Add some scheduler hooks for restricting a task's affinity mask
  * Implement these hooks for arm64 so that we can avoid 32-bit tasks
    running on 64-bit-only cores
  * Restrict affinity mask of 32-bit tasks on execve()
  * Prevent hot-unplug of all 32-bit CPUs if we have a mismatched system
  * Ensure 32-bit EL0 cpumask is zero-initialised (oops)

It's worth mentioning that this approach goes directly against my
initial proposal for punting the affinity management to userspace,
because it turns out that doesn't really work. There are cases where the
kernel has to muck with the affinity mask explicitly, such as execve(),
CPU hotplug and cpuset balancing. Ensuring that these don't lead to
random SIGKILLs as far as userspace is concerned means avoiding any
64-bit-only CPUs appearing in the affinity mask for a 32-bit task, at
which point it's easier just to handle everything in the kernel anyway.

- Patches 1-6 hack the arm64 CPU feature code to allow 32-bit tasks to
  run on a mismatched system, but forcing SIGKILL if a task ends up on
  the wrong CPU. This is gated on a command-line option; without it, a
  mismatched system will be treated as 64-bit-only.

- Patches 7-11 add scheduler functionality necessary to constrain the
  CPU affinity mask on a per-task basis and hook this up for execve() on
  arm64.

- Patches 12-14 finish off the arm64 plumbing and remove the logic for
  killing misplaced tasks, as it adds overhead to the context-switch and
  ret-to-user paths.

This seems to do the right thing in my contrived QEMU environment, but
as I say, I'm not a scheduler hacker so I'm open to alternative ideas.

Cheers,

Will

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Qais Yousef <qais.yousef@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Quentin Perret <qperret@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Li Zefan <lizefan@huawei.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: kernel-team@android.com

--->8

Will Deacon (14):
  arm64: cpuinfo: Split AArch32 registers out into a separate struct
  arm64: Allow mismatched 32-bit EL0 support
  KVM: arm64: Kill 32-bit vCPUs on systems with mismatched EL0 support
  arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs
  arm64: Advertise CPUs capable of running 32-bit applications in sysfs
  arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0
  sched: Introduce restrict_cpus_allowed_ptr() to limit task CPU
    affinity
  arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit
    EL0
  cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1
  sched: Introduce arch_cpu_allowed_mask() to limit fallback rq
    selection
  sched: Reject CPU affinity changes based on arch_cpu_allowed_mask()
  arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched
    system
  arm64: Implement arch_cpu_allowed_mask()
  arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores

 .../ABI/testing/sysfs-devices-system-cpu      |   9 +
 .../admin-guide/kernel-parameters.txt         |   7 +
 arch/arm64/include/asm/cpu.h                  |  44 ++--
 arch/arm64/include/asm/cpucaps.h              |   2 +-
 arch/arm64/include/asm/cpufeature.h           |   8 +-
 arch/arm64/include/asm/mmu_context.h          |  12 +
 arch/arm64/kernel/cpufeature.c                | 219 ++++++++++++++----
 arch/arm64/kernel/cpuinfo.c                   |  53 +++--
 arch/arm64/kernel/process.c                   |  17 +-
 arch/arm64/kvm/arm.c                          |  11 +-
 include/linux/sched.h                         |   1 +
 kernel/cgroup/cpuset.c                        |   6 +-
 kernel/sched/core.c                           |  90 +++++--
 13 files changed, 370 insertions(+), 109 deletions(-)

-- 
2.29.2.299.gdc1121823c-goog


WARNING: multiple messages have this Message-ID
From: Will Deacon <will@kernel.org>
To: linux-arm-kernel@lists.infradead.org
Cc: linux-arch@vger.kernel.org, Marc Zyngier <maz@kernel.org>,
	kernel-team@android.com,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Juri Lelli <juri.lelli@redhat.com>,
	Quentin Perret <qperret@google.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	linux-kernel@vger.kernel.org, Qais Yousef <qais.yousef@arm.com>,
	Suren Baghdasaryan <surenb@google.com>,
	Ingo Molnar <mingo@redhat.com>, Li Zefan <lizefan@huawei.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Tejun Heo <tj@kernel.org>, Will Deacon <will@kernel.org>,
	Morten Rasmussen <morten.rasmussen@arm.com>
Subject: [PATCH v3 00/14] An alternative series for asymmetric AArch32 systems
Date: Fri, 13 Nov 2020 09:37:05 +0000	[thread overview]
Message-ID: <20201113093720.21106-1-will@kernel.org> (raw)

Hi again everyone,

	I'm not a scheduler hacker,
	I'm a scheduler hacker's mate.
	I'm only hacking the scheduler,
	'cos trying to run 32-bit applications on systems where not all of the CPUs support it is GREAT.

It's Friday 13th, and I'm back with version three of the increasingly
popular patches I previously posted here:

  v1: https://lore.kernel.org/r/20201027215118.27003-1-will@kernel.org
  v2: https://lore.kernel.org/r/20201109213023.15092-1-will@kernel.org

and which started life as a reimplementation of some patches from Qais:

  https://lore.kernel.org/r/20201021104611.2744565-1-qais.yousef@arm.com

The aim of this series is to allow 32-bit ARM applications to run on
arm64 SoCs where not all of the CPUs support the 32-bit instruction set.

There are some major changes in v3:

  * Add some scheduler hooks for restricting a task's affinity mask
  * Implement these hooks for arm64 so that we can avoid 32-bit tasks
    running on 64-bit-only cores
  * Restrict affinity mask of 32-bit tasks on execve()
  * Prevent hot-unplug of all 32-bit CPUs if we have a mismatched system
  * Ensure 32-bit EL0 cpumask is zero-initialised (oops)

It's worth mentioning that this approach goes directly against my
initial proposal for punting the affinity management to userspace,
because it turns out that doesn't really work. There are cases where the
kernel has to muck with the affinity mask explicitly, such as execve(),
CPU hotplug and cpuset balancing. Ensuring that these don't lead to
random SIGKILLs as far as userspace is concerned means avoiding any
64-bit-only CPUs appearing in the affinity mask for a 32-bit task, at
which point it's easier just to handle everything in the kernel anyway.

- Patches 1-6 hack the arm64 CPU feature code to allow 32-bit tasks to
  run on a mismatched system, but forcing SIGKILL if a task ends up on
  the wrong CPU. This is gated on a command-line option; without it, a
  mismatched system will be treated as 64-bit-only.

- Patches 7-11 add scheduler functionality necessary to constrain the
  CPU affinity mask on a per-task basis and hook this up for execve() on
  arm64.

- Patches 12-14 finish off the arm64 plumbing and remove the logic for
  killing misplaced tasks, as it adds overhead to the context-switch and
  ret-to-user paths.

This seems to do the right thing in my contrived QEMU environment, but
as I say, I'm not a scheduler hacker so I'm open to alternative ideas.

Cheers,

Will

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Qais Yousef <qais.yousef@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Quentin Perret <qperret@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Li Zefan <lizefan@huawei.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: kernel-team@android.com

--->8

Will Deacon (14):
  arm64: cpuinfo: Split AArch32 registers out into a separate struct
  arm64: Allow mismatched 32-bit EL0 support
  KVM: arm64: Kill 32-bit vCPUs on systems with mismatched EL0 support
  arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs
  arm64: Advertise CPUs capable of running 32-bit applications in sysfs
  arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0
  sched: Introduce restrict_cpus_allowed_ptr() to limit task CPU
    affinity
  arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit
    EL0
  cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1
  sched: Introduce arch_cpu_allowed_mask() to limit fallback rq
    selection
  sched: Reject CPU affinity changes based on arch_cpu_allowed_mask()
  arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched
    system
  arm64: Implement arch_cpu_allowed_mask()
  arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores

 .../ABI/testing/sysfs-devices-system-cpu      |   9 +
 .../admin-guide/kernel-parameters.txt         |   7 +
 arch/arm64/include/asm/cpu.h                  |  44 ++--
 arch/arm64/include/asm/cpucaps.h              |   2 +-
 arch/arm64/include/asm/cpufeature.h           |   8 +-
 arch/arm64/include/asm/mmu_context.h          |  12 +
 arch/arm64/kernel/cpufeature.c                | 219 ++++++++++++++----
 arch/arm64/kernel/cpuinfo.c                   |  53 +++--
 arch/arm64/kernel/process.c                   |  17 +-
 arch/arm64/kvm/arm.c                          |  11 +-
 include/linux/sched.h                         |   1 +
 kernel/cgroup/cpuset.c                        |   6 +-
 kernel/sched/core.c                           |  90 +++++--
 13 files changed, 370 insertions(+), 109 deletions(-)

-- 
2.29.2.299.gdc1121823c-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

             reply	other threads:[~2020-11-13  9:37 UTC|newest]

Thread overview: 104+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-13  9:37 Will Deacon [this message]
2020-11-13  9:37 ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 01/14] arm64: cpuinfo: Split AArch32 registers out into a separate struct Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 02/14] arm64: Allow mismatched 32-bit EL0 support Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-19 11:27   ` Valentin Schneider
2020-11-19 11:27     ` Valentin Schneider
2020-11-19 13:12     ` Will Deacon
2020-11-19 13:12       ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 03/14] KVM: arm64: Kill 32-bit vCPUs on systems with mismatched " Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 04/14] arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 05/14] arm64: Advertise CPUs capable of running 32-bit applications in sysfs Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 06/14] arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0 Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 07/14] sched: Introduce restrict_cpus_allowed_ptr() to limit task CPU affinity Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-19  9:18   ` Quentin Perret
2020-11-19  9:18     ` Quentin Perret
2020-11-19 11:03     ` Quentin Perret
2020-11-19 11:03       ` Quentin Perret
2020-11-19 11:05     ` Will Deacon
2020-11-19 11:05       ` Will Deacon
2020-11-19 11:27       ` Valentin Schneider
2020-11-19 11:27         ` Valentin Schneider
2020-11-19 13:13         ` Will Deacon
2020-11-19 13:13           ` Will Deacon
2020-11-19 14:54           ` Valentin Schneider
2020-11-19 14:54             ` Valentin Schneider
2020-11-19 16:41             ` Will Deacon
2020-11-19 16:41               ` Will Deacon
2020-11-19 12:47   ` Valentin Schneider
2020-11-19 12:47     ` Valentin Schneider
2020-11-19 13:13     ` Will Deacon
2020-11-19 13:13       ` Will Deacon
2020-11-19 14:54       ` Valentin Schneider
2020-11-19 14:54         ` Valentin Schneider
2020-11-19 16:09       ` Peter Zijlstra
2020-11-19 16:09         ` Peter Zijlstra
2020-11-19 16:57         ` Valentin Schneider
2020-11-19 16:57           ` Valentin Schneider
2020-11-19 19:25           ` Will Deacon
2020-11-19 19:25             ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 08/14] arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit EL0 Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-19  9:24   ` Quentin Perret
2020-11-19  9:24     ` Quentin Perret
2020-11-19 11:06     ` Will Deacon
2020-11-19 11:06       ` Will Deacon
2020-11-19 16:19       ` Peter Zijlstra
2020-11-19 16:19         ` Peter Zijlstra
2020-11-19 16:30         ` Will Deacon
2020-11-19 16:30           ` Will Deacon
2020-11-19 16:44           ` Peter Zijlstra
2020-11-19 16:44             ` Peter Zijlstra
2020-11-19 16:51             ` Will Deacon
2020-11-19 16:51               ` Will Deacon
2020-11-19 16:14   ` Peter Zijlstra
2020-11-19 16:14     ` Peter Zijlstra
2020-11-19 16:28     ` Will Deacon
2020-11-19 16:28       ` Will Deacon
2020-11-19 16:42       ` Peter Zijlstra
2020-11-19 16:42         ` Peter Zijlstra
2020-11-19 16:48         ` Will Deacon
2020-11-19 16:48           ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 09/14] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-19  9:29   ` Quentin Perret
2020-11-19  9:29     ` Quentin Perret
2020-11-19 11:06     ` Will Deacon
2020-11-19 11:06       ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 10/14] sched: Introduce arch_cpu_allowed_mask() to limit fallback rq selection Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-19  9:38   ` Quentin Perret
2020-11-19  9:38     ` Quentin Perret
2020-11-19 11:07     ` Will Deacon
2020-11-19 11:07       ` Will Deacon
2020-11-19 20:39       ` Will Deacon
2020-11-19 20:39         ` Will Deacon
2020-11-23 14:48         ` Quentin Perret
2020-11-23 14:48           ` Quentin Perret
2020-11-13  9:37 ` [PATCH v3 11/14] sched: Reject CPU affinity changes based on arch_cpu_allowed_mask() Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-19  9:47   ` Quentin Perret
2020-11-19  9:47     ` Quentin Perret
2020-11-19 11:07     ` Will Deacon
2020-11-19 11:07       ` Will Deacon
2020-11-19 14:30       ` Quentin Perret
2020-11-19 14:30         ` Quentin Perret
2020-11-19 16:44         ` Will Deacon
2020-11-19 16:44           ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 12/14] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 13/14] arm64: Implement arch_cpu_allowed_mask() Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-13  9:37 ` [PATCH v3 14/14] arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores Will Deacon
2020-11-13  9:37   ` Will Deacon
2020-11-19 16:11 ` [PATCH v3 00/14] An alternative series for asymmetric AArch32 systems Peter Zijlstra
2020-11-19 16:11   ` Peter Zijlstra
2020-11-19 16:39   ` Will Deacon
2020-11-19 16:39     ` Will Deacon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201113093720.21106-1-will@kernel.org \
    --to=will@kernel.org \
    --cc=catalin.marinas@arm.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hannes@cmpxchg.org \
    --cc=juri.lelli@redhat.com \
    --cc=kernel-team@android.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizefan@huawei.com \
    --cc=maz@kernel.org \
    --cc=mingo@redhat.com \
    --cc=morten.rasmussen@arm.com \
    --cc=peterz@infradead.org \
    --cc=qais.yousef@arm.com \
    --cc=qperret@google.com \
    --cc=surenb@google.com \
    --cc=tj@kernel.org \
    --cc=vincent.guittot@linaro.org \
    --subject='Re: [PATCH v3 00/14] An alternative series for asymmetric AArch32 systems' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.