From: Alex Belits <abelits@marvell.com>
To: "frederic@kernel.org" <frederic@kernel.org>,
"rostedt@goodmis.org" <rostedt@goodmis.org>
Cc: Prasun Kapoor <pkapoor@marvell.com>,
"mingo@kernel.org" <mingo@kernel.org>,
"davem@davemloft.net" <davem@davemloft.net>,
"linux-api@vger.kernel.org" <linux-api@vger.kernel.org>,
"peterz@infradead.org" <peterz@infradead.org>,
"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"will@kernel.org" <will@kernel.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: [PATCH v4 12/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize
Date: Wed, 22 Jul 2020 14:59:13 +0000 [thread overview]
Message-ID: <041a4c844b4a0f69aba6df4686e1952dd0f91931.camel@marvell.com> (raw)
Message-ID: <20200722145913.BKu42i9vhgsu97X7TASxRVVPk7SJOFiOeys1Kx2mYd0@z> (raw)
In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com>
From: Yuri Norov <ynorov@marvell.com>
CPUs running isolated tasks are in userspace, so they don't have to
perform ring buffer updates immediately. If ring_buffer_resize()
schedules the update on those CPUs, isolation is broken. To prevent
that, updates for CPUs running isolated tasks are performed locally,
like for offline CPUs.
A race condition between this update and isolation breaking is avoided
at the cost of disabling per_cpu buffer writing for the time of update
when it coincides with isolation breaking.
Signed-off-by: Yuri Norov <ynorov@marvell.com>
[abelits@marvell.com: updated to prevent race with isolation breaking]
Signed-off-by: Alex Belits <abelits@marvell.com>
---
kernel/trace/ring_buffer.c | 63 ++++++++++++++++++++++++++++++++++----
1 file changed, 57 insertions(+), 6 deletions(-)
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 00867ff82412..22d4731f0def 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -21,6 +21,7 @@
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/init.h>
+#include <linux/isolation.h>
#include <linux/hash.h>
#include <linux/list.h>
#include <linux/cpu.h>
@@ -1705,6 +1706,38 @@ static void update_pages_handler(struct work_struct *work)
complete(&cpu_buffer->update_done);
}
+static bool update_if_isolated(struct ring_buffer_per_cpu *cpu_buffer,
+ int cpu)
+{
+ bool rv = false;
+
+ smp_rmb();
+ if (task_isolation_on_cpu(cpu)) {
+ /*
+ * CPU is running isolated task. Since it may lose
+ * isolation and re-enter kernel simultaneously with
+ * this update, disable recording until it's done.
+ */
+ atomic_inc(&cpu_buffer->record_disabled);
+ /* Make sure, update is done, and isolation state is current */
+ smp_mb();
+ if (task_isolation_on_cpu(cpu)) {
+ /*
+ * If CPU is still running isolated task, we
+ * can be sure that breaking isolation will
+ * happen while recording is disabled, and CPU
+ * will not touch this buffer until the update
+ * is done.
+ */
+ rb_update_pages(cpu_buffer);
+ cpu_buffer->nr_pages_to_update = 0;
+ rv = true;
+ }
+ atomic_dec(&cpu_buffer->record_disabled);
+ }
+ return rv;
+}
+
/**
* ring_buffer_resize - resize the ring buffer
* @buffer: the buffer to resize.
@@ -1794,13 +1827,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size,
if (!cpu_buffer->nr_pages_to_update)
continue;
- /* Can't run something on an offline CPU. */
+ /*
+ * Can't run something on an offline CPU.
+ *
+ * CPUs running isolated tasks don't have to
+ * update ring buffers until they exit
+ * isolation because they are in
+ * userspace. Use the procedure that prevents
+ * race condition with isolation breaking.
+ */
if (!cpu_online(cpu)) {
rb_update_pages(cpu_buffer);
cpu_buffer->nr_pages_to_update = 0;
} else {
- schedule_work_on(cpu,
- &cpu_buffer->update_pages_work);
+ if (!update_if_isolated(cpu_buffer, cpu))
+ schedule_work_on(cpu,
+ &cpu_buffer->update_pages_work);
}
}
@@ -1849,13 +1891,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size,
get_online_cpus();
- /* Can't run something on an offline CPU. */
+ /*
+ * Can't run something on an offline CPU.
+ *
+ * CPUs running isolated tasks don't have to update
+ * ring buffers until they exit isolation because they
+ * are in userspace. Use the procedure that prevents
+ * race condition with isolation breaking.
+ */
if (!cpu_online(cpu_id))
rb_update_pages(cpu_buffer);
else {
- schedule_work_on(cpu_id,
+ if (!update_if_isolated(cpu_buffer, cpu_id))
+ schedule_work_on(cpu_id,
&cpu_buffer->update_pages_work);
- wait_for_completion(&cpu_buffer->update_done);
+ wait_for_completion(&cpu_buffer->update_done);
+ }
}
cpu_buffer->nr_pages_to_update = 0;
--
2.26.2
next prev parent reply other threads:[~2020-07-22 14:59 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-22 14:44 [PATCH v4 00/13] "Task_isolation" mode Alex Belits
2020-07-22 14:48 ` [PATCH v4 02/13] task_isolation: vmstat: add vmstat_idle function Alex Belits
2020-07-22 14:49 ` [PATCH v4 03/13] task_isolation: userspace hard isolation from kernel Alex Belits
2020-07-22 14:49 ` Alex Belits
2020-10-01 13:56 ` Frederic Weisbecker
2020-10-04 14:44 ` [EXT] " Alex Belits
2020-10-04 23:14 ` Frederic Weisbecker
2020-10-05 18:52 ` Nitesh Narayan Lal
2020-10-06 10:35 ` Frederic Weisbecker
2020-10-17 1:13 ` Alex Belits
2020-10-17 1:08 ` Alex Belits
2020-10-17 16:08 ` Thomas Gleixner
2020-10-17 16:15 ` Alex Belits
2020-10-17 20:03 ` Thomas Gleixner
2020-10-06 11:01 ` Alex Belits
2020-10-01 14:40 ` Frederic Weisbecker
2020-10-04 15:01 ` [EXT] " Alex Belits
[not found] ` <04be044c1bcd76b7438b7563edc35383417f12c8.camel-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
2020-07-22 14:47 ` [PATCH v4 01/13] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits
2020-07-22 14:47 ` Alex Belits
2020-07-22 14:51 ` [PATCH v4 04/13] task_isolation: Add task isolation hooks to arch-independent code Alex Belits
2020-07-22 14:51 ` Alex Belits
2020-07-22 14:55 ` [PATCH 08/13] task_isolation: arch/arm64: enable task isolation functionality Alex Belits
2020-07-22 14:55 ` Alex Belits
2020-07-22 14:56 ` [PATCH v4 09/13] task_isolation: arch/arm: " Alex Belits
2020-07-22 14:56 ` Alex Belits
2020-07-22 14:58 ` [PATCH v4 11/13] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits
2020-07-22 14:58 ` Alex Belits
2020-10-01 14:47 ` Frederic Weisbecker
2020-10-04 17:12 ` [EXT] " Alex Belits
2021-01-22 14:13 ` Marcelo Tosatti
2021-01-22 16:13 ` Paolo Abeni
2020-07-22 14:59 ` [PATCH 13/13] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits
2020-07-22 14:59 ` Alex Belits
2020-07-22 14:51 ` [PATCH v4 05/13] task_isolation: Add xen-specific hook Alex Belits
2020-07-22 14:51 ` Alex Belits
2020-07-22 14:53 ` [PATCH 06/13] task_isolation: Add driver-specific hooks Alex Belits
2020-07-22 14:53 ` Alex Belits
2020-07-22 14:54 ` [PATCH v4 07/13] task_isolation: arch/x86: enable task isolation functionality Alex Belits
2020-07-22 14:54 ` Alex Belits
2020-07-22 14:57 ` [PATCH v4 10/13] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits
2020-10-01 14:44 ` Frederic Weisbecker
2020-10-04 15:22 ` [EXT] " Alex Belits
2020-10-06 21:41 ` Frederic Weisbecker
2020-10-17 0:17 ` Alex Belits
2020-07-22 14:59 ` Alex Belits [this message]
2020-07-22 14:59 ` [PATCH v4 12/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits
2020-07-23 13:17 ` [PATCH v4 00/13] "Task_isolation" mode Thomas Gleixner
2020-07-23 14:26 ` Peter Zijlstra
2020-07-23 14:53 ` Thomas Gleixner
2020-07-23 14:29 ` Peter Zijlstra
2020-07-23 15:41 ` [EXT] " Alex Belits
2020-07-23 15:48 ` Peter Zijlstra
2020-07-23 15:48 ` Peter Zijlstra
2020-07-23 16:19 ` Alex Belits
2020-07-23 16:19 ` Alex Belits
2020-07-23 15:18 ` Alex Belits
[not found] ` <831e023422aa0e4cb3da37ceef6fdcd5bc854682.camel-eYqpPyKDWXRBDgjK7y7TUQ@public.gmane.org>
2020-07-23 15:49 ` Peter Zijlstra
2020-07-23 15:49 ` Peter Zijlstra
2020-07-23 16:50 ` Alex Belits
2020-07-23 21:44 ` Thomas Gleixner
2020-07-23 21:44 ` Thomas Gleixner
2020-07-24 3:00 ` [EXT] " Alex Belits
2020-07-24 16:08 ` Thomas Gleixner
2020-07-24 16:08 ` Thomas Gleixner
2020-07-23 21:31 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=041a4c844b4a0f69aba6df4686e1952dd0f91931.camel@marvell.com \
--to=abelits@marvell.com \
--cc=catalin.marinas@arm.com \
--cc=davem@davemloft.net \
--cc=frederic@kernel.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=pkapoor@marvell.com \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).