From: Waiman Long <longman@redhat.com>
To: Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Jonathan Corbet <corbet@lwn.net>, Shuah Khan <shuah@kernel.org>
Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org,
"Andrew Morton" <akpm@linux-foundation.org>,
"Roman Gushchin" <guro@fb.com>, "Phil Auld" <pauld@redhat.com>,
"Peter Zijlstra" <peterz@infradead.org>,
"Juri Lelli" <juri.lelli@redhat.com>,
"Frederic Weisbecker" <frederic@kernel.org>,
"Marcelo Tosatti" <mtosatti@redhat.com>,
"Michal Koutný" <mkoutny@suse.com>,
"Waiman Long" <longman@redhat.com>
Subject: [PATCH v3 8/9] cgroup/cpuset: Update description of cpuset.cpus.partition in cgroup-v2.rst
Date: Tue, 20 Jul 2021 10:18:33 -0400 [thread overview]
Message-ID: <20210720141834.10624-9-longman@redhat.com> (raw)
In-Reply-To: <20210720141834.10624-1-longman@redhat.com>
Update Documentation/admin-guide/cgroup-v2.rst on the newly introduced
"isolated" cpuset partition type as well as the ability to create
non-top cpuset partition with no cpu allocated to it.
Signed-off-by: Waiman Long <longman@redhat.com>
---
Documentation/admin-guide/cgroup-v2.rst | 94 +++++++++++++++----------
1 file changed, 58 insertions(+), 36 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 5c7377b5bd3e..2e101a353ab1 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2080,8 +2080,9 @@ Cpuset Interface Files
It accepts only the following input values when written to.
======== ================================
- "root" a partition root
- "member" a non-root member of a partition
+ "member" Non-root member of a partition
+ "root" Partition root
+ "isolated" Partition root without load balancing
======== ================================
When set to be a partition root, the current cgroup is the
@@ -2090,9 +2091,14 @@ Cpuset Interface Files
partition roots themselves and their descendants. The root
cgroup is always a partition root.
- There are constraints on where a partition root can be set.
- It can only be set in a cgroup if all the following conditions
- are true.
+ When set to "isolated", the CPUs in that partition root will
+ be in an isolated state without any load balancing from the
+ scheduler. Tasks in such a partition must be explicitly bound
+ to each individual CPU.
+
+ There are constraints on where a partition root can be set
+ ("root" or "isolated"). It can only be set in a cgroup if all
+ the following conditions are true.
1) The "cpuset.cpus" is not empty and the list of CPUs are
exclusive, i.e. they are not shared by any of its siblings.
@@ -2103,51 +2109,67 @@ Cpuset Interface Files
eliminating corner cases that have to be handled if such a
condition is allowed.
- Setting it to partition root will take the CPUs away from the
- effective CPUs of the parent cgroup. Once it is set, this
+ Setting it to a partition root will take the CPUs away from
+ the effective CPUs of the parent cgroup. Once it is set, this
file cannot be reverted back to "member" if there are any child
cgroups with cpuset enabled.
- A parent partition cannot distribute all its CPUs to its
- child partitions. There must be at least one cpu left in the
- parent partition.
+ A parent partition may distribute all its CPUs to its child
+ partitions as long as it is not the root cgroup and there is no
+ task directly associated with that parent partition. Otherwise,
+ there must be at least one cpu left in the parent partition.
+ A new task cannot be moved to a partition root with no effective
+ cpu.
+
+ Once becoming a partition root, changes to "cpuset.cpus"
+ is generally allowed as long as the first condition above
+ (cpu exclusivity rule) is true. Other constraints for this
+ operation are as follows.
- Once becoming a partition root, changes to "cpuset.cpus" is
- generally allowed as long as the first condition above is true,
- the change will not take away all the CPUs from the parent
- partition and the new "cpuset.cpus" value is a superset of its
- children's "cpuset.cpus" values.
+ 1) Any newly added CPUs must be a subset of the parent's
+ "cpuset.cpus.effective".
+ 2) Taking away all the CPUs from the parent's "cpuset.cpus.effective"
+ is only allowed if there is no task associated with the
+ parent partition.
+ 3) Deletion of CPUs that have been distributed to child partition
+ roots are not allowed.
Sometimes, external factors like changes to ancestors'
"cpuset.cpus" or cpu hotplug can cause the state of the partition
- root to change. On read, the "cpuset.sched.partition" file
- can show the following values.
+ root to change. On read, the "cpuset.cpus.partition" file can
+ show the following values.
============== ==============================
"member" Non-root member of a partition
"root" Partition root
+ "isolated" Partition root without load balancing
"root invalid" Invalid partition root
============== ==============================
- It is a partition root if the first 2 partition root conditions
- above are true and at least one CPU from "cpuset.cpus" is
- granted by the parent cgroup.
-
- A partition root can become invalid if none of CPUs requested
- in "cpuset.cpus" can be granted by the parent cgroup or the
- parent cgroup is no longer a partition root itself. In this
- case, it is not a real partition even though the restriction
- of the first partition root condition above will still apply.
- The cpu affinity of all the tasks in the cgroup will then be
- associated with CPUs in the nearest ancestor partition.
-
- An invalid partition root can be transitioned back to a
- real partition root if at least one of the requested CPUs
- can now be granted by its parent. In this case, the cpu
- affinity of all the tasks in the formerly invalid partition
- will be associated to the CPUs of the newly formed partition.
- Changing the partition state of an invalid partition root to
- "member" is always allowed even if child cpusets are present.
+ A partition root becomes invalid if all the CPUs requested in
+ "cpuset.cpus" become unavailable. This can happen if all the
+ CPUs have been offlined, or the state of an ancestor partition
+ root become invalid. In this case, it is not a real partition
+ even though the restriction of the cpu exclusivity rule will
+ still apply. The cpu affinity of all the tasks in the cgroup
+ will then be associated with CPUs in the nearest ancestor
+ partition.
+
+ In the special case of a parent partition competing with a child
+ partition for the only CPU left, the parent partition wins and
+ the child partition becomes invalid.
+
+ An invalid partition root can be transitioned back to a real
+ partition root if at least one of the requested CPUs become
+ available again. In this case, the cpu affinity of all the tasks
+ in the formerly invalid partition will be associated to the CPUs
+ of the newly formed partition. Changing the partition state of
+ an invalid partition root to "member" is always allowed even if
+ child cpusets are present. However changing a partition root back
+ to member will not be allowed if child partitions are present.
+
+ Poll and inotify events are triggered when transition to or
+ from invalid partition root happens.
Device controller
--
2.18.1
next prev parent reply other threads:[~2021-07-20 14:35 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-20 14:18 [PATCH v3 0/9] cgroup/cpuset: Add new cpuset partition type & empty effecitve cpus Waiman Long
2021-07-20 14:18 ` [PATCH v3 1/9] cgroup/cpuset: Miscellaneous code cleanup Waiman Long
2021-07-26 22:56 ` Tejun Heo
2021-07-20 14:18 ` [PATCH v3 2/9] cgroup/cpuset: Fix a partition bug with hotplug Waiman Long
2021-07-26 22:59 ` Tejun Heo
2021-07-27 20:16 ` Waiman Long
2021-07-20 14:18 ` [PATCH v3 3/9] cgroup/cpuset: Fix violation of cpuset locking rule Waiman Long
2021-07-26 23:10 ` Tejun Heo
2021-07-20 14:18 ` [PATCH v3 4/9] cgroup/cpuset: Enable event notification when partition become invalid Waiman Long
2021-07-26 23:14 ` Tejun Heo
2021-07-27 20:26 ` Waiman Long
2021-07-27 20:46 ` Waiman Long
2021-07-20 14:18 ` [PATCH v3 5/9] cgroup/cpuset: Clarify the use of invalid partition root Waiman Long
2021-07-20 14:18 ` [PATCH v3 6/9] cgroup/cpuset: Add a new isolated cpus.partition type Waiman Long
2021-07-27 11:42 ` Frederic Weisbecker
2021-07-27 15:56 ` Waiman Long
2021-07-29 11:03 ` Frederic Weisbecker
2021-07-28 16:09 ` Michal Koutný
2021-07-28 16:27 ` Waiman Long
2021-07-28 17:25 ` Michal Koutný
2021-07-20 14:18 ` [PATCH v3 7/9] cgroup/cpuset: Allow non-top parent partition root to distribute out all CPUs Waiman Long
2021-07-20 14:18 ` Waiman Long [this message]
2021-07-20 14:18 ` [PATCH v3 9/9] kselftest/cgroup: Add cpuset v2 partition root state test Waiman Long
2021-07-26 23:17 ` [PATCH v3 0/9] cgroup/cpuset: Add new cpuset partition type & empty effecitve cpus Tejun Heo
2021-07-27 21:14 ` Waiman Long
2021-08-09 22:46 ` Tejun Heo
2021-08-10 1:12 ` Waiman Long
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210720141834.10624-9-longman@redhat.com \
--to=longman@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=frederic@kernel.org \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=juri.lelli@redhat.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=lizefan.x@bytedance.com \
--cc=mkoutny@suse.com \
--cc=mtosatti@redhat.com \
--cc=pauld@redhat.com \
--cc=peterz@infradead.org \
--cc=shuah@kernel.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).