linux-kselftest.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Waiman Long <longman@redhat.com>
To: Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Jonathan Corbet <corbet@lwn.net>, Shuah Khan <shuah@kernel.org>
Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org,
	"Andrew Morton" <akpm@linux-foundation.org>,
	"Roman Gushchin" <guro@fb.com>, "Phil Auld" <pauld@redhat.com>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Juri Lelli" <juri.lelli@redhat.com>,
	"Frederic Weisbecker" <frederic@kernel.org>,
	"Marcelo Tosatti" <mtosatti@redhat.com>,
	"Michal Koutný" <mkoutny@suse.com>,
	"Waiman Long" <longman@redhat.com>
Subject: [PATCH v9 6/7] cgroup/cpuset: Update description of cpuset.cpus.partition in cgroup-v2.rst
Date: Sun,  5 Dec 2021 13:32:19 -0500	[thread overview]
Message-ID: <20211205183220.818872-7-longman@redhat.com> (raw)
In-Reply-To: <20211205183220.818872-1-longman@redhat.com>

Update Documentation/admin-guide/cgroup-v2.rst on the newly introduced
"isolated" cpuset partition type as well as other changes made in other
cpuset patches.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 Documentation/admin-guide/cgroup-v2.rst | 168 ++++++++++++++----------
 1 file changed, 102 insertions(+), 66 deletions(-)

diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 2aeb7ae8b393..9612319b353f 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2099,74 +2099,110 @@ Cpuset Interface Files
 	It accepts only the following input values when written to.
 
 	  ========	================================
-	  "root"	a partition root
-	  "member"	a non-root member of a partition
+	  "member"	Non-root member of a partition
+	  "root"	Partition root
+	  "isolated"	Partition root without load balancing
 	  ========	================================
 
-	When set to be a partition root, the current cgroup is the
-	root of a new partition or scheduling domain that comprises
-	itself and all its descendants except those that are separate
-	partition roots themselves and their descendants.  The root
-	cgroup is always a partition root.
-
-	There are constraints on where a partition root can be set.
-	It can only be set in a cgroup if all the following conditions
-	are true.
-
-	1) The "cpuset.cpus" is not empty and the list of CPUs are
-	   exclusive, i.e. they are not shared by any of its siblings.
-	2) The parent cgroup is a partition root.
-	3) The "cpuset.cpus" is also a proper subset of the parent's
-	   "cpuset.cpus.effective".
-	4) There is no child cgroups with cpuset enabled.  This is for
-	   eliminating corner cases that have to be handled if such a
-	   condition is allowed.
-
-	Setting it to partition root will take the CPUs away from the
-	effective CPUs of the parent cgroup.  Once it is set, this
-	file cannot be reverted back to "member" if there are any child
-	cgroups with cpuset enabled.
-
-	A parent partition cannot distribute all its CPUs to its
-	child partitions.  There must be at least one cpu left in the
-	parent partition.
-
-	Once becoming a partition root, changes to "cpuset.cpus" is
-	generally allowed as long as the first condition above is true,
-	the change will not take away all the CPUs from the parent
-	partition and the new "cpuset.cpus" value is a superset of its
-	children's "cpuset.cpus" values.
-
-	Sometimes, external factors like changes to ancestors'
-	"cpuset.cpus" or cpu hotplug can cause the state of the partition
-	root to change.  On read, the "cpuset.sched.partition" file
-	can show the following values.
-
-	  ==============	==============================
-	  "member"		Non-root member of a partition
-	  "root"		Partition root
-	  "root invalid"	Invalid partition root
-	  ==============	==============================
-
-	It is a partition root if the first 2 partition root conditions
-	above are true and at least one CPU from "cpuset.cpus" is
-	granted by the parent cgroup.
-
-	A partition root can become invalid if none of CPUs requested
-	in "cpuset.cpus" can be granted by the parent cgroup or the
-	parent cgroup is no longer a partition root itself.  In this
-	case, it is not a real partition even though the restriction
-	of the first partition root condition above will still apply.
-	The cpu affinity of all the tasks in the cgroup will then be
-	associated with CPUs in the nearest ancestor partition.
-
-	An invalid partition root can be transitioned back to a
-	real partition root if at least one of the requested CPUs
-	can now be granted by its parent.  In this case, the cpu
-	affinity of all the tasks in the formerly invalid partition
-	will be associated to the CPUs of the newly formed partition.
-	Changing the partition state of an invalid partition root to
-	"member" is always allowed even if child cpusets are present.
+	The root cgroup is always a partition root and its state
+	cannot be changed.  All other non-root cgroups start out as
+	"member".
+
+	When set to "root", the current cgroup is the root of a new
+	partition or scheduling domain that comprises itself and
+	all its descendants except those that are separate partition
+	roots themselves and their descendants.
+
+	The value shown in "cpuset.cpus.effective" of a partition root is
+	the CPUs that the parent partition root can dedicate to the new
+	partition root.  They are subtracted from "cpuset.cpus.effective"
+	of the parent and may be different from "cpuset.cpus"
+
+	When set to "isolated", the CPUs in that partition root will
+	be in an isolated state without any load balancing from the
+	scheduler.  Tasks placed in such a partition with multiple
+	CPUs should be carefully distributed and bound to each of the
+	individual CPUs for optimal performance.
+
+	A partition root ("root" or "isolated") can be in one of the
+	two possible states - valid or invalid.  An invalid partition
+	root is in a degraded state where some state information are
+	retained, but behaves more like a "member".
+
+	On read, the "cpuset.cpus.partition" file can show the following
+	values.
+
+	  ======================	==============================
+	  "member"			Non-root member of a partition
+	  "root"			Partition root
+	  "isolated"			Partition root without load balancing
+	  "root invalid (<reason>)"	Invalid partition root
+	  "isolated invalid (<reason>)"	Invalid isolated partition root
+	  ======================	==============================
+
+	In the case of an invalid partition root, a descriptive string on
+	why the partition is invalid is included within parentheses.
+
+	Almost all possible state transitions among "member", valid
+	and invalid partition roots are allowed except from "member"
+	to invalid partition root.
+
+	Before the "member" to partition root transition can happen,
+	the following conditions must be met or the transition will
+	not be allowed.
+
+	1) The "cpuset.cpus" is non-empty and exclusive, i.e. they are
+	   not shared by any of its siblings.
+	2) The parent cgroup is a valid partition root.
+	3) The "cpuset.cpus" must contain at least one of the CPUs from
+	   parent's "cpuset.cpus", i.e. they overlap.
+	4) There is no child cgroups with cpuset enabled.  This avoids
+	   cpu migrations of multiple cgroups simultaneously which can
+	   be problematic.
+
+	Once becoming a partition root, the only rule restricting
+	changes made to "cpuset.cpus" is the exclusivity rule where
+	none of the siblings of a partition root can share CPUs with
+	it.
+
+	External events like hotplug or inappropriate changes to
+	"cpuset.cpus" can cause a valid partition root to become invalid.
+	Besides the exclusivity rule listed above, the other conditions
+	required to maintain the validity of a partition root are
+	as follows:
+
+	1) The parent cgroup is a valid partition root.
+	2) If "cpuset.cpus.effective" is empty, the partition must have
+	   no task associated with it. Otherwise, the partition becomes
+	   invalid and "cpuset.cpus.effective" will fall back to that
+	   of the nearest non-empty ancestor.
+
+	A corollary of a valid partition root is that
+	"cpuset.cpus.effective" is always a subset of "cpuset.cpus".
+	Note that a task cannot be moved to a cgroup with empty
+	"cpuset.cpus.effective".
+
+	A valid non-root parent partition may distribute out all its CPUs
+	to its child partitions when there is no task associated with it.
+
+	An invalid partition root will be reverted back to a valid
+	one if none of the validity constraints of a valid partition
+	root are violated due to hotplug events or proper changes to
+	"cpuset.cpus" files.
+
+	Changing a partition root (valid or invalid) to "member" is
+	always allowed.  If there are child partition roots underneath
+	it, they will become invalid and unrecoverable.  So care must
+	be taken to double check for this condition before disabling
+	a partition root.
+
+	Poll and inotify events are triggered whenever the state of
+	"cpuset.cpus.partition" changes.  That includes changes caused
+	by write to "cpuset.cpus.partition", cpu hotplug or other
+	changes that modify the validity status of the partition.
+	This will allow user space agents to monitor unexpected changes
+	to "cpuset.cpus.partition" without the need to do continuous
+	polling.
 
 
 Device controller
-- 
2.27.0


  parent reply	other threads:[~2021-12-05 18:33 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-05 18:32 [PATCH v9 0/7] cgroup/cpuset: Add new cpuset partition type & empty effecitve cpus Waiman Long
2021-12-05 18:32 ` [PATCH v9 1/7] cgroup/cpuset: Don't let child cpusets restrict parent in default hierarchy Waiman Long
2021-12-13 20:41   ` Tejun Heo
2021-12-15 12:23     ` Michal Koutný
2021-12-15 17:59       ` Waiman Long
2021-12-17 15:48         ` [PATCH] cgroup/cpuset: Make child cpusets restrict parents on v1 hierarchy Michal Koutný
2021-12-17 16:34           ` Waiman Long
2022-01-12 21:25           ` Tejun Heo
2021-12-05 18:32 ` [PATCH v9 2/7] cgroup/cpuset: Allow no-task partition to have empty cpuset.cpus.effective Waiman Long
2021-12-13 20:45   ` Tejun Heo
2021-12-15  3:24     ` Waiman Long
2021-12-15 10:36       ` Michal Koutný
2021-12-05 18:32 ` [PATCH v9 3/7] cgroup/cpuset: Refining features and constraints of a partition Waiman Long
2021-12-15 14:49   ` Michal Koutný
2021-12-15 16:29     ` Waiman Long
2021-12-16  9:28       ` Michal Koutný
2021-12-05 18:32 ` [PATCH v9 4/7] cgroup/cpuset: Add a new isolated cpus.partition type Waiman Long
2022-01-12 15:21   ` Peter Zijlstra
2022-01-12 15:40     ` Waiman Long
2022-01-12 21:23       ` Tejun Heo
2021-12-05 18:32 ` [PATCH v9 5/7] cgroup/cpuset: Show invalid partition reason string Waiman Long
2021-12-05 18:32 ` Waiman Long [this message]
2021-12-13 21:00   ` [PATCH v9 6/7] cgroup/cpuset: Update description of cpuset.cpus.partition in cgroup-v2.rst Tejun Heo
2021-12-15 14:44     ` Michal Koutný
2021-12-15 18:16       ` Waiman Long
2021-12-15 18:35         ` Tejun Heo
2021-12-15 18:55           ` Waiman Long
2022-01-12 21:21             ` Tejun Heo
2021-12-05 18:32 ` [PATCH v9 7/7] kselftest/cgroup: Add cpuset v2 partition root state test Waiman Long
2021-12-09 15:39 ` [PATCH v9 0/7] cgroup/cpuset: Add new cpuset partition type & empty effecitve cpus Waiman Long

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211205183220.818872-7-longman@redhat.com \
    --to=longman@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=cgroups@vger.kernel.org \
    --cc=corbet@lwn.net \
    --cc=frederic@kernel.org \
    --cc=guro@fb.com \
    --cc=hannes@cmpxchg.org \
    --cc=juri.lelli@redhat.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=lizefan.x@bytedance.com \
    --cc=mkoutny@suse.com \
    --cc=mtosatti@redhat.com \
    --cc=pauld@redhat.com \
    --cc=peterz@infradead.org \
    --cc=shuah@kernel.org \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).