From: Frederic Weisbecker <fweisbec@gmail.com>
To: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Gilad Ben Yossef <giladb@mellanox.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ingo Molnar <mingo@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Andrew Morton <akpm@linux-foundation.org>,
Rik van Riel <riel@redhat.com>, Tejun Heo <tj@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Christoph Lameter <cl@linux.com>,
Viresh Kumar <viresh.kumar@linaro.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will.deacon@arm.com>,
Andy Lutomirski <luto@amacapital.net>,
Daniel Lezcano <daniel.lezcano@linaro.org>,
Francis Giraldeau <francis.giraldeau@gmail.com>,
Andi Kleen <andi@firstfloor.org>, Arnd Bergmann <arnd@arndb.de>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: task isolation discussion at Linux Plumbers
Date: Wed, 9 Nov 2016 11:07:56 +0000 [thread overview]
Message-ID: <CAFTL4hz2RjpWFJRMXu46ESKHXzQn1EN0bhjNs_svYgV5fFX2rQ@mail.gmail.com> (raw)
In-Reply-To: <1605b087-2b3b-77c1-01ac-084e378f5f28@mellanox.com>
2016-11-05 4:04 GMT+00:00 Chris Metcalf <cmetcalf@mellanox.com>:
> A bunch of people got together this week at the Linux Plumbers
> Conference to discuss nohz_full, task isolation, and related stuff.
> (Thanks to Thomas for getting everyone gathered at one place and time!)
>
> Here are the notes I took; I welcome any corrections and follow-up.
>
Thanks for that report Chris!
> == rcu_nocbs ==
>
> We started out by discussing this option. It is automatically enabled
> by nohz_full, but we spent a little while side-tracking on the
> implementation of one kthread per rcu flavor per core. The suggestion
> was made (by Peter or Andy; I forget) that each kthread could handle
> all flavors per core by using a dedicated worklist. It certainly
> seems like removing potentially dozens or hundreds of kthreads from
> larger systems will be a win if this works out.
>
> Paul said he would look into this possibility.
Sounds good.
>
>
> == Remote statistics ==
>
> We discussed the possibility of remote statistics gathering, i.e. load
> average etc. The idea would be that we could have housekeeping
> core(s) periodically iterate over the nohz cores to load their rq
> remotely and do update_current etc. Presumably it should be possible
> for a single housekeeping core to handle doing this for all the
> nohz_full cores, as we only need to do it quite infrequently.
>
> Thomas suggested that this might be the last remaining thing that
> needed to be done to allow disabling the current behavior of falling
> back to a 1 Hz clock in nohz_full.
>
> I believe Thomas said he had a patch to do this already.
>
There are also some other details among update_curr to take care of,
but that's certainly a big piece of it.
I had wished we could find a solution that doesn't involve remote
accounting but at least it could be a first step.
I have let that idea rotting for too long, I need to get my hands into
it for good.
> == Disabling the dyn tick ==
>
> One issue that the current task isolation patch series encounters is
> when we request disabling the dyntick, but it doesn't happen. At the
> moment we just wait until the the tick is properly disabled, by
> busy-waiting in the kernel (calling schedule etc as needed). No one
> is particularly fond of this scheme. The consensus seems to be to try
> harder to figure out what is going on, fix whatever problems exist,
> then consider it a regression going forward if something causes the
> dyntick to become difficult to disable again in the future. I will
> take a look at this and try to gather more data on if and when this is
> happening in 4.9.
>
We could enhance dynticks tracing, expand the tick stop failure codes
for example in order to report more details about what's going on.
> == Missing oneshot_stopped callbacks ==
>
> I raised the issue that various clock_event_device sources don't
> always support oneshot_stopped, which can cause an additional
> final interrupt to occur after the timer infrastructure believes the
> interrupt has been stopped. I have patches to fix this for tile and
> arm64 in my patch series; Thomas volunteered to look at adding
> equivalent support for x86.
>
>
> Many thanks to all those who participated in the discussion.
> Frederic, we wished you had been there!
I wish I had too!
next prev parent reply other threads:[~2016-11-09 11:08 UTC|newest]
Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-08-16 21:19 [PATCH v15 00/13] support "task_isolation" mode Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 01/13] vmstat: add quiet_vmstat_sync function Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 02/13] vmstat: add vmstat_idle function Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 03/13] lru_add_drain_all: factor out lru_add_drain_needed Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 04/13] task_isolation: add initial support Chris Metcalf
2016-08-29 16:33 ` Peter Zijlstra
2016-08-29 16:40 ` Chris Metcalf
2016-08-29 16:48 ` Peter Zijlstra
2016-08-29 16:53 ` Chris Metcalf
2016-08-30 7:59 ` Peter Zijlstra
2016-08-30 7:58 ` Peter Zijlstra
2016-08-30 15:32 ` Chris Metcalf
2016-08-30 16:30 ` Andy Lutomirski
2016-08-30 17:02 ` Chris Metcalf
2016-08-30 18:43 ` Andy Lutomirski
2016-08-30 19:37 ` Chris Metcalf
2016-08-30 19:50 ` Andy Lutomirski
2016-09-02 14:04 ` Chris Metcalf
2016-09-02 17:28 ` Andy Lutomirski
2016-09-09 17:40 ` Chris Metcalf
2016-09-12 17:41 ` Andy Lutomirski
2016-09-12 19:25 ` Chris Metcalf
2016-09-27 14:22 ` Frederic Weisbecker
2016-09-27 14:39 ` Peter Zijlstra
2016-09-27 14:51 ` Frederic Weisbecker
2016-09-27 14:48 ` Paul E. McKenney
2016-09-30 16:59 ` Chris Metcalf
2016-09-01 10:06 ` Peter Zijlstra
2016-09-02 14:03 ` Chris Metcalf
2016-09-02 16:40 ` Peter Zijlstra
2017-02-02 16:13 ` Eugene Syromiatnikov
2017-02-02 18:12 ` Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 05/13] task_isolation: track asynchronous interrupts Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 06/13] arch/x86: enable task isolation functionality Chris Metcalf
2016-08-30 21:46 ` Andy Lutomirski
2016-08-16 21:19 ` [PATCH v15 07/13] arm64: factor work_pending state machine to C Chris Metcalf
2016-08-17 8:05 ` Will Deacon
2016-08-16 21:19 ` [PATCH v15 08/13] arch/arm64: enable task isolation functionality Chris Metcalf
2016-08-26 16:25 ` Catalin Marinas
2016-08-16 21:19 ` [PATCH v15 09/13] arch/tile: " Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 10/13] arm, tile: turn off timer tick for oneshot_stopped state Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 11/13] task_isolation: support CONFIG_TASK_ISOLATION_ALL Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 12/13] task_isolation: add user-settable notification signal Chris Metcalf
2016-08-16 21:19 ` [PATCH v15 13/13] task_isolation self test Chris Metcalf
2016-08-17 19:37 ` [PATCH] Fix /proc/stat freezes (was [PATCH v15] "task_isolation" mode) Christoph Lameter
2016-08-20 1:42 ` Chris Metcalf
2016-09-28 13:16 ` Frederic Weisbecker
2016-08-29 16:27 ` Ping: [PATCH v15 00/13] support "task_isolation" mode Chris Metcalf
2016-09-07 21:11 ` Francis Giraldeau
2016-09-07 21:39 ` Francis Giraldeau
2016-09-08 16:21 ` Francis Giraldeau
2016-09-12 16:01 ` Chris Metcalf
2016-09-12 16:14 ` Peter Zijlstra
2016-09-12 21:15 ` Rafael J. Wysocki
2016-09-13 0:05 ` Rafael J. Wysocki
2016-09-13 16:00 ` Francis Giraldeau
2016-09-13 0:20 ` Francis Giraldeau
2016-09-13 16:12 ` Chris Metcalf
2016-09-27 14:49 ` Frederic Weisbecker
2016-09-27 14:35 ` Frederic Weisbecker
2016-09-30 17:07 ` Chris Metcalf
2016-11-05 4:04 ` task isolation discussion at Linux Plumbers Chris Metcalf
2016-11-05 16:05 ` Christoph Lameter
2016-11-07 16:55 ` Thomas Gleixner
2016-11-07 18:36 ` Thomas Gleixner
2016-11-07 19:12 ` Rik van Riel
2016-11-07 19:16 ` Will Deacon
2016-11-07 19:18 ` Rik van Riel
2016-11-11 20:54 ` Luiz Capitulino
2016-11-09 1:40 ` Paul E. McKenney
2016-11-09 11:14 ` Andy Lutomirski
2016-11-09 17:38 ` Paul E. McKenney
2016-11-09 18:57 ` Will Deacon
2016-11-09 19:11 ` Paul E. McKenney
2016-11-10 1:44 ` Andy Lutomirski
2016-11-10 4:52 ` Paul E. McKenney
2016-11-10 5:10 ` Paul E. McKenney
2016-11-11 17:00 ` Andy Lutomirski
2016-11-09 11:07 ` Frederic Weisbecker [this message]
2016-12-19 14:37 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAFTL4hz2RjpWFJRMXu46ESKHXzQn1EN0bhjNs_svYgV5fFX2rQ@mail.gmail.com \
--to=fweisbec@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=andi@firstfloor.org \
--cc=arnd@arndb.de \
--cc=catalin.marinas@arm.com \
--cc=cl@linux.com \
--cc=cmetcalf@mellanox.com \
--cc=daniel.lezcano@linaro.org \
--cc=francis.giraldeau@gmail.com \
--cc=giladb@mellanox.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@amacapital.net \
--cc=mingo@kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
--cc=viresh.kumar@linaro.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).