From: Alex Belits <abelits@marvell.com>
To: "frederic@kernel.org" <frederic@kernel.org>,
"rostedt@goodmis.org" <rostedt@goodmis.org>
Cc: "mingo@kernel.org" <mingo@kernel.org>,
"peterz@infradead.org" <peterz@infradead.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Prasun Kapoor <pkapoor@marvell.com>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"linux-api@vger.kernel.org" <linux-api@vger.kernel.org>,
"linux-mm@vger.kernel.org" <linux-mm@vger.kernel.org>,
"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>
Subject: [PATCH 10/12] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize
Date: Wed, 4 Mar 2020 16:14:28 +0000 [thread overview]
Message-ID: <426ee885b89985f6bf348d64909eeb6a2cd83162.camel@marvell.com> (raw)
In-Reply-To: <4473787e1b6bc3cc226067e8d122092a678b63de.camel@marvell.com>
From: Yuri Norov <ynorov@marvell.com>
CPUs running isolated tasks are in userspace, so they don't have to
perform ring buffer updates immediately. If ring_buffer_resize()
schedules the update on those CPUs, isolation is broken. To prevent
that, updates for CPUs running isolated tasks are performed locally,
like for offline CPUs.
A race condition between this update and isolation breaking is avoided
at the cost of disabling per_cpu buffer writing for the time of update
when it coincides with isolation breaking.
Signed-off-by: Alex Belits <abelits@marvell.com>
---
kernel/trace/ring_buffer.c | 61 ++++++++++++++++++++++++++++++++++----
1 file changed, 55 insertions(+), 6 deletions(-)
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 61f0e92ace99..6c7479b6411d 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -21,6 +21,7 @@
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/init.h>
+#include <linux/isolation.h>
#include <linux/hash.h>
#include <linux/list.h>
#include <linux/cpu.h>
@@ -1701,6 +1702,37 @@ static void update_pages_handler(struct work_struct *work)
complete(&cpu_buffer->update_done);
}
+static bool update_if_isolated(struct ring_buffer_per_cpu *cpu_buffer,
+ int cpu)
+{
+ bool rv = false;
+
+ if (task_isolation_on_cpu(cpu)) {
+ /*
+ * CPU is running isolated task. Since it may lose
+ * isolation and re-enter kernel simultaneously with
+ * this update, disable recording until it's done.
+ */
+ atomic_inc(&cpu_buffer->record_disabled);
+ /* Make sure, update is done, and isolation state is current */
+ smp_mb();
+ if (task_isolation_on_cpu(cpu)) {
+ /*
+ * If CPU is still running isolated task, we
+ * can be sure that breaking isolation will
+ * happen while recording is disabled, and CPU
+ * will not touch this buffer until the update
+ * is done.
+ */
+ rb_update_pages(cpu_buffer);
+ cpu_buffer->nr_pages_to_update = 0;
+ rv = true;
+ }
+ atomic_dec(&cpu_buffer->record_disabled);
+ }
+ return rv;
+}
+
/**
* ring_buffer_resize - resize the ring buffer
* @buffer: the buffer to resize.
@@ -1784,13 +1816,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size,
if (!cpu_buffer->nr_pages_to_update)
continue;
- /* Can't run something on an offline CPU. */
+ /*
+ * Can't run something on an offline CPU.
+ *
+ * CPUs running isolated tasks don't have to
+ * update ring buffers until they exit
+ * isolation because they are in
+ * userspace. Use the procedure that prevents
+ * race condition with isolation breaking.
+ */
if (!cpu_online(cpu)) {
rb_update_pages(cpu_buffer);
cpu_buffer->nr_pages_to_update = 0;
} else {
- schedule_work_on(cpu,
- &cpu_buffer->update_pages_work);
+ if (!update_if_isolated(cpu_buffer, cpu))
+ schedule_work_on(cpu,
+ &cpu_buffer->update_pages_work);
}
}
@@ -1829,13 +1870,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size,
get_online_cpus();
- /* Can't run something on an offline CPU. */
+ /*
+ * Can't run something on an offline CPU.
+ *
+ * CPUs running isolated tasks don't have to update
+ * ring buffers until they exit isolation because they
+ * are in userspace. Use the procedure that prevents
+ * race condition with isolation breaking.
+ */
if (!cpu_online(cpu_id))
rb_update_pages(cpu_buffer);
else {
- schedule_work_on(cpu_id,
+ if (!update_if_isolated(cpu_buffer, cpu_id))
+ schedule_work_on(cpu_id,
&cpu_buffer->update_pages_work);
- wait_for_completion(&cpu_buffer->update_done);
+ wait_for_completion(&cpu_buffer->update_done);
+ }
}
cpu_buffer->nr_pages_to_update = 0;
--
2.20.1
next prev parent reply other threads:[~2020-03-04 16:14 UTC|newest]
Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-04 16:01 [PATCH 00/12] "Task_isolation" mode Alex Belits
2020-03-04 16:03 ` [PATCH 01/12] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits
2020-03-04 16:04 ` [PATCH 02/12] task_isolation: vmstat: add vmstat_idle function Alex Belits
2020-03-04 16:07 ` [PATCH 03/12] task_isolation: userspace hard isolation from kernel Alex Belits
2020-03-05 18:33 ` Frederic Weisbecker
2020-03-08 5:32 ` [EXT] " Alex Belits
2020-04-28 14:12 ` Marcelo Tosatti
2020-03-06 15:26 ` Frederic Weisbecker
2020-03-08 6:06 ` [EXT] " Alex Belits
2020-03-06 16:00 ` Frederic Weisbecker
2020-03-08 7:16 ` [EXT] " Alex Belits
2020-03-04 16:08 ` [PATCH 04/12] task_isolation: Add task isolation hooks to arch-independent code Alex Belits
2020-03-04 16:09 ` [PATCH 05/12] task_isolation: arch/x86: enable task isolation functionality Alex Belits
2020-03-04 16:10 ` [PATCH 06/12] task_isolation: arch/arm64: " Alex Belits
2020-03-04 16:31 ` Mark Rutland
2020-03-08 4:48 ` [EXT] " Alex Belits
2020-03-04 16:11 ` [PATCH 07/12] task_isolation: arch/arm: " Alex Belits
2020-03-04 16:12 ` [PATCH 08/12] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits
2020-03-06 16:03 ` Frederic Weisbecker
2020-03-08 7:28 ` [EXT] " Alex Belits
2020-03-09 2:38 ` Frederic Weisbecker
2020-03-04 16:13 ` [PATCH 09/12] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits
2020-03-04 16:14 ` Alex Belits [this message]
2020-03-04 16:15 ` [PATCH 11/12] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits
2020-03-06 15:34 ` Frederic Weisbecker
2020-03-08 6:48 ` [EXT] " Alex Belits
2020-03-09 2:28 ` Frederic Weisbecker
2020-03-04 16:16 ` [PATCH 12/12] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits
2020-03-08 3:42 ` [PATCH v2 00/12] "Task_isolation" mode Alex Belits
2020-03-08 3:44 ` [PATCH v2 01/12] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits
2020-03-08 3:46 ` [PATCH v2 02/12] task_isolation: vmstat: add vmstat_idle function Alex Belits
2020-03-08 3:47 ` [PATCH v2 03/12] task_isolation: userspace hard isolation from kernel Alex Belits
[not found] ` <20200307214254.7a8f6c22@hermes.lan>
2020-03-08 7:33 ` [EXT] " Alex Belits
2020-03-27 8:42 ` Marta Rybczynska
2020-04-06 4:31 ` Kevyn-Alexandre Paré
2020-04-06 4:43 ` Kevyn-Alexandre Paré
2020-03-08 3:48 ` [PATCH v2 04/12] task_isolation: Add task isolation hooks to arch-independent code Alex Belits
2020-03-08 3:49 ` [PATCH v2 05/12] task_isolation: arch/x86: enable task isolation functionality Alex Belits
2020-03-08 3:50 ` [PATCH v2 06/12] task_isolation: arch/arm64: " Alex Belits
2020-03-09 16:59 ` Mark Rutland
2020-03-08 3:52 ` [PATCH v2 07/12] task_isolation: arch/arm: " Alex Belits
2020-03-08 3:53 ` [PATCH v2 08/12] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits
2020-03-08 3:54 ` [PATCH v2 09/12] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits
2020-03-08 3:55 ` [PATCH v2 10/12] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits
2020-04-06 4:27 ` Kevyn-Alexandre Paré
2020-03-08 3:56 ` [PATCH v2 11/12] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits
2020-03-08 3:57 ` [PATCH v2 12/12] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits
2020-04-09 15:09 ` [PATCH v3 00/13] "Task_isolation" mode Alex Belits
2020-04-09 15:15 ` [PATCH 01/13] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits
2020-04-09 15:16 ` [PATCH 02/13] task_isolation: vmstat: add vmstat_idle function Alex Belits
2020-04-09 15:17 ` [PATCH v3 03/13] task_isolation: add instruction synchronization memory barrier Alex Belits
2020-04-15 12:44 ` Mark Rutland
2020-04-19 5:02 ` [EXT] " Alex Belits
2020-04-20 12:23 ` Will Deacon
2020-04-20 12:36 ` Mark Rutland
2020-04-20 13:55 ` Will Deacon
2020-04-21 7:41 ` Will Deacon
2020-04-20 12:45 ` Mark Rutland
2020-04-09 15:20 ` [PATCH v3 04/13] task_isolation: userspace hard isolation from kernel Alex Belits
2020-04-09 18:00 ` Andy Lutomirski
2020-04-19 5:07 ` Alex Belits
2020-04-09 15:21 ` [PATCH 05/13] task_isolation: Add task isolation hooks to arch-independent code Alex Belits
2020-04-09 15:22 ` [PATCH 06/13] task_isolation: arch/x86: enable task isolation functionality Alex Belits
2020-04-09 15:23 ` [PATCH v3 07/13] task_isolation: arch/arm64: " Alex Belits
2020-04-22 12:08 ` Catalin Marinas
2020-04-09 15:24 ` [PATCH v3 08/13] task_isolation: arch/arm: " Alex Belits
2020-04-09 15:25 ` [PATCH v3 09/13] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits
2020-04-09 15:26 ` [PATCH v3 10/13] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits
2020-04-09 15:27 ` [PATCH v3 11/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits
2020-04-09 15:27 ` [PATCH v3 12/13] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits
2020-04-09 15:28 ` [PATCH v3 13/13] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=426ee885b89985f6bf348d64909eeb6a2cd83162.camel@marvell.com \
--to=abelits@marvell.com \
--cc=frederic@kernel.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=pkapoor@marvell.com \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).