linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: Tejun Heo <tj@kernel.org>
Cc: Li Zefan <lizefan@huawei.com>, cgroups <cgroups@vger.kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@kernel.org>,
	Valentin Schneider <valentin.schneider@arm.com>,
	Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [regression] cpuset: offlined CPUs removed from affinity masks
Date: Sat, 7 Mar 2020 11:06:47 -0500 (EST)	[thread overview]
Message-ID: <316507033.21078.1583597207356.JavaMail.zimbra@efficios.com> (raw)
In-Reply-To: <20200219161222.GF698990@mtj.thefacebook.com>

----- On Feb 19, 2020, at 11:12 AM, Tejun Heo tj@kernel.org wrote:

> On Wed, Feb 19, 2020 at 11:08:39AM -0500, Mathieu Desnoyers wrote:
>> I wonder if applying the online cpu masks to the per-thread affinity mask
>> is the correct approach ? I suspect what we may be looking for here is to keep
> 
> Oh, the whole thing is wrong.
> 
>> the affinity mask independent of cpu hotplug, and look-up both the per-thread
>> affinity mask and the online cpu mask whenever the scheduler needs to perform
>> "is_cpu_allowed()" to check task placement.
> 
> Yes, that's what it should have done from the get-go. The way it's
> implemented now, maybe we can avoid some specific cases like cpuset
> not being used at all but it'll constantly get in the way if you're
> expecting thread affinity to retain its value across offlines.

Looking into solving this, one key issue seems to get in the way: cpuset
appear to care about not allowing to create a cpuset which has no currently
active CPU where to run, e.g.:

# it is forbidden to create an empty cpuset if the cpu is offlined first:

mkdir /sys/fs/cgroup/cpuset/test

echo 2 > /sys/fs/cgroup/cpuset/test/cpusets.cpus

cat /sys/fs/cgroup/cpuset/test/cpusets.cpu
2

echo 0 > /sys/devices/system/cpu/cpu1/online

echo 1 > /sys/fs/cgroup/cpuset/test/cpuset.cpus
bash: echo: write error: Invalid argument

cat /sys/fs/cgroup/cpuset/test/cpusets.cpu
2


# but it's perfectly fine to generate this empty cpuset by offlining
# a cpu _after_ creating the cpuset:

echo 0 > /sys/devices/system/cpu/cpu2/online

cat /sys/fs/cgroup/cpuset/test/cpusets.cpu
  <----- empty (nothing)

Some further testing seems to show that tasks belonging to that empty
cpuset are placed anywhere on active cpus.

Clearly, there is an intent that cpusets take the active mask into
account to prohibit creating an empty cpuset, but nothing prevents
cpu hotplug from creating an empty cpuset.

I wonder how to solve this inconsistency ?

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

  reply	other threads:[~2020-03-07 16:06 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-16 17:41 [regression] cpuset: offlined CPUs removed from affinity masks Mathieu Desnoyers
2020-01-16 18:27 ` Valentin Schneider
2020-02-17 16:03 ` Mathieu Desnoyers
2020-02-19 15:19   ` Tejun Heo
2020-02-19 15:43     ` Mathieu Desnoyers
2020-02-19 15:47       ` Tejun Heo
2020-02-19 15:50         ` Mathieu Desnoyers
2020-02-19 15:52           ` Tejun Heo
2020-02-19 16:08             ` Mathieu Desnoyers
2020-02-19 16:12               ` Tejun Heo
2020-03-07 16:06                 ` Mathieu Desnoyers [this message]
2020-03-12 18:26                   ` Tejun Heo
2020-03-12 19:47                     ` Mathieu Desnoyers
2020-03-24 18:01                       ` Tejun Heo
2020-03-24 19:30                         ` Mathieu Desnoyers
2020-03-30 19:53                           ` Mathieu Desnoyers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=316507033.21078.1583597207356.JavaMail.zimbra@efficios.com \
    --to=mathieu.desnoyers@efficios.com \
    --cc=cgroups@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizefan@huawei.com \
    --cc=mingo@kernel.org \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    --cc=valentin.schneider@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).