From: Waiman Long <longman@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Tejun Heo <tj@kernel.org>, Li Zefan <lizefan@huawei.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Ingo Molnar <mingo@redhat.com>,
cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-doc@vger.kernel.org, kernel-team@fb.com, pjt@google.com,
luto@amacapital.net, Mike Galbraith <efault@gmx.de>,
torvalds@linux-foundation.org, Roman Gushchin <guro@fb.com>,
Juri Lelli <juri.lelli@redhat.com>,
Patrick Bellasi <patrick.bellasi@arm.com>,
Tom Hromatka <tom.hromatka@oracle.com>
Subject: Re: [PATCH v14 10/12] cpuset: Add documentation about the new "cpuset.sched.partition" flag
Date: Wed, 7 Nov 2018 17:58:58 -0500 [thread overview]
Message-ID: <7f71f746-582b-42d0-11f1-e690ab4d94b0@redhat.com> (raw)
In-Reply-To: <20181106115045.GO22431@hirez.programming.kicks-ass.net>
On 11/06/2018 06:50 AM, Peter Zijlstra wrote:
> On Mon, Oct 15, 2018 at 04:29:35PM -0400, Waiman Long wrote:
>> The cgroup-v2.rst file is updated to document the purpose of the new
>> "cpuset.sched.partition" flag and how its usage.
>>
>> Signed-off-by: Waiman Long <longman@redhat.com>
>> ---
>> Documentation/admin-guide/cgroup-v2.rst | 66 +++++++++++++++++++++++++
>> 1 file changed, 66 insertions(+)
>>
>> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
>> index 533e85cb851b..178cda473a26 100644
>> --- a/Documentation/admin-guide/cgroup-v2.rst
>> +++ b/Documentation/admin-guide/cgroup-v2.rst
>> @@ -1686,6 +1686,72 @@ Cpuset Interface Files
>>
>> Its value will be affected by memory nodes hotplug events.
>>
>> + cpuset.sched.partition
>> + A read-write single value file which exists on non-root
>> + cpuset-enabled cgroups. It accepts either "0" (off) or "1"
>> + (on) when written to.
>> + This flag is set and owned by the
>> + parent cgroup.
> What does that mean? The parent cgroup doesn't 'set' anything at all.
> The user will.
>
>> +
>> + If set, it indicates that the current cgroup is the root of a
>> + new partition or scheduling domain that comprises itself and
>> + all its descendants except those that are separate partition
>> + roots themselves and their descendants. The root cgroup is
>> + always a partition root.
>> +
>> + There are constraints on where this flag can be set. It can
>> + only be set in a cgroup if all the following conditions are true.
>> +
>> + 1) The "cpuset.cpus" is not empty and the list of CPUs are
>> + exclusive, i.e. they are not shared by any of its siblings.
>> + 2) The parent cgroup is a partition root.
>> + 3) The "cpuset.cpus" is also a proper subset of the parent's
>> + "cpuset.cpus.effective".
>> + 4) There is no child cgroups with cpuset enabled. This is for
>> + eliminating corner cases that have to be handled if such a
>> + condition is allowed.
>> +
>> + Setting this flag will take the CPUs away from the effective
>> + CPUs of the parent cgroup. Once it is set, this flag cannot
>> + be cleared if there are any child cgroups with cpuset enabled.
>> +
>> + A parent partition cannot distribute all its CPUs to its
>> + child partitions. There must be at least one cpu left in the
>> + parent partition.
>> +
>> + Once becoming a partition root, changes to "cpuset.cpus" is
>> + generally allowed as long as the first condition above is true,
>> + the change will not take away all the CPUs from the parent
>> + partition and the new "cpuset.cpus" value is a superset of its
>> + children's "cpuset.cpus" values.
>> + Sometimes, external factors like changes to ancestors'
>> + "cpuset.cpus" or cpu hotplug can cause the state of the partition
>> + root to change. On read, the "cpuset.sched.partition" file
>> + can show the following values.
> Are those the only conditions under which that -1 can happen? Parent
> taking away CPUs it previously granted and hotplug?
Yes, if none of the cpus are available. It will become invalid. It still
remains a partition (a partial one) if at least one cpu is allocated to
that partition.
>> +
>> + "0" Not a partition root
>> + "1" Partition root
>> + "-1" Erroneous partition root
>> +
>> + It is a partition root if the first 2 partition root conditions
>> + above are true and at least one CPU from "cpuset.cpus" is
>> + granted by the parent cgroup.
>> +
>> + A partition root can become an erroneous partition root if none
>> + of CPUs requested in "cpuset.cpus" can be granted by the parent
>> + cgroup or the parent cgroup is no longer a partition root.
>> + In this case, it is not a real partition even though the
>> + restriction of the first partition root condition above will
>> + still apply. All the tasks in the cgroup will be migrated to
>> + the nearest ancestor partition.
> Effectively or actual? Actual migrating tasks out of the cgroup is
> irreversible.
I am not talking about actual migration to a different cgroup. I am
talking moving to a different partition. When a partition become
invalid, it will inherent the partition parent's effective cpumask.
>> + An erroneous partition root can be transitioned back to a real
>> + partition root if at least one of the requested CPUs can now be
>> + granted by its parent. In this case, the tasks will be migrated
>> + back to the newly created partition. Clearing the partition
>> + flag of an erroneous partition root is always allowed even if
>> + child cpusets are present.
> So you need to clarify the above point (I think it is effectively),
> because otherwise you don't know which tasks to put back.
I will clarify that.
Thanks,
Longman
next prev parent reply other threads:[~2018-11-07 22:59 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-15 20:29 [PATCH v14 00/12] Enable cpuset controller in default hierarchy Waiman Long
2018-10-15 20:29 ` [PATCH v14 01/12] cpuset: " Waiman Long
2018-10-15 20:29 ` [PATCH v14 02/12] cpuset: Define data structures to support scheduling partition Waiman Long
2018-10-15 20:29 ` [PATCH v14 03/12] cpuset: Simply allocation and freeing of cpumasks Waiman Long
2018-10-19 15:28 ` Tom Hromatka
2018-10-15 20:29 ` [PATCH v14 04/12] cpuset: Add new v2 cpuset.sched.partition flag Waiman Long
2018-11-06 11:35 ` Peter Zijlstra
2018-10-15 20:29 ` [PATCH v14 05/12] cpuset: Add an error state to cpuset.sched.partition Waiman Long
2018-11-06 11:37 ` Peter Zijlstra
2018-11-06 14:17 ` Waiman Long
2018-11-06 11:40 ` Peter Zijlstra
2018-11-07 23:13 ` Waiman Long
2018-11-06 11:40 ` Peter Zijlstra
2018-10-15 20:29 ` [PATCH v14 06/12] cpuset: Track cpusets that use parent's effective_cpus Waiman Long
2018-10-15 20:29 ` [PATCH v14 07/12] cpuset: Make CPU hotplug work with partition Waiman Long
2018-10-15 20:29 ` [PATCH v14 08/12] cpuset: Make generate_sched_domains() " Waiman Long
2018-10-15 20:29 ` [PATCH v14 09/12] cpuset: Expose cpus.effective and mems.effective on cgroup v2 root Waiman Long
2018-10-15 20:29 ` [PATCH v14 10/12] cpuset: Add documentation about the new "cpuset.sched.partition" flag Waiman Long
2018-11-06 11:50 ` Peter Zijlstra
2018-11-06 14:09 ` Waiman Long
2018-11-07 22:58 ` Waiman Long [this message]
2018-10-15 20:29 ` [PATCH v14 11/12] cpuset: Expose cpuset.cpus.subpartitions with cgroup_debug Waiman Long
2018-10-15 20:29 ` [PATCH v14 12/12] cpuset: Show descriptive text when reading cpuset.sched.partition Waiman Long
2018-10-17 15:08 ` Tejun Heo
2018-10-17 15:20 ` Waiman Long
2018-10-19 18:56 ` Waiman Long
2018-10-19 19:24 ` Tejun Heo
2018-10-19 19:32 ` Waiman Long
2018-11-02 14:34 ` Waiman Long
2018-11-06 11:52 ` Peter Zijlstra
2018-11-05 16:36 ` [PATCH v14 00/12] Enable cpuset controller in default hierarchy Tejun Heo
2018-11-05 16:57 ` Peter Zijlstra
2018-11-06 11:53 ` Peter Zijlstra
2018-11-06 11:55 ` Peter Zijlstra
2018-11-06 14:06 ` Waiman Long
[not found] ` <CAOS58YPye=7Ga+y-ujFsgHqo6vdVnjykmON1z+UjNQLvvM_g4w@mail.gmail.com>
2018-11-06 14:11 ` Tejun Heo
2018-11-07 21:32 ` Tejun Heo
2018-11-07 21:52 ` Waiman Long
2018-11-08 9:41 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7f71f746-582b-42d0-11f1-e690ab4d94b0@redhat.com \
--to=longman@redhat.com \
--cc=cgroups@vger.kernel.org \
--cc=efault@gmx.de \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=juri.lelli@redhat.com \
--cc=kernel-team@fb.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=luto@amacapital.net \
--cc=mingo@redhat.com \
--cc=patrick.bellasi@arm.com \
--cc=peterz@infradead.org \
--cc=pjt@google.com \
--cc=tj@kernel.org \
--cc=tom.hromatka@oracle.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).