From: Alex Belits <abelits@marvell.com> To: "frederic@kernel.org" <frederic@kernel.org>, "rostedt@goodmis.org" <rostedt@goodmis.org> Cc: "mingo@kernel.org" <mingo@kernel.org>, "peterz@infradead.org" <peterz@infradead.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Prasun Kapoor <pkapoor@marvell.com>, "tglx@linutronix.de" <tglx@linutronix.de>, "linux-api@vger.kernel.org" <linux-api@vger.kernel.org>, "linux-mm@vger.kernel.org" <linux-mm@vger.kernel.org>, "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org> Subject: [PATCH 10/12] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Date: Wed, 4 Mar 2020 16:14:28 +0000 Message-ID: <426ee885b89985f6bf348d64909eeb6a2cd83162.camel@marvell.com> (raw) In-Reply-To: <4473787e1b6bc3cc226067e8d122092a678b63de.camel@marvell.com> From: Yuri Norov <ynorov@marvell.com> CPUs running isolated tasks are in userspace, so they don't have to perform ring buffer updates immediately. If ring_buffer_resize() schedules the update on those CPUs, isolation is broken. To prevent that, updates for CPUs running isolated tasks are performed locally, like for offline CPUs. A race condition between this update and isolation breaking is avoided at the cost of disabling per_cpu buffer writing for the time of update when it coincides with isolation breaking. Signed-off-by: Alex Belits <abelits@marvell.com> --- kernel/trace/ring_buffer.c | 61 ++++++++++++++++++++++++++++++++++---- 1 file changed, 55 insertions(+), 6 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 61f0e92ace99..6c7479b6411d 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -21,6 +21,7 @@ #include <linux/delay.h> #include <linux/slab.h> #include <linux/init.h> +#include <linux/isolation.h> #include <linux/hash.h> #include <linux/list.h> #include <linux/cpu.h> @@ -1701,6 +1702,37 @@ static void update_pages_handler(struct work_struct *work) complete(&cpu_buffer->update_done); } +static bool update_if_isolated(struct ring_buffer_per_cpu *cpu_buffer, + int cpu) +{ + bool rv = false; + + if (task_isolation_on_cpu(cpu)) { + /* + * CPU is running isolated task. Since it may lose + * isolation and re-enter kernel simultaneously with + * this update, disable recording until it's done. + */ + atomic_inc(&cpu_buffer->record_disabled); + /* Make sure, update is done, and isolation state is current */ + smp_mb(); + if (task_isolation_on_cpu(cpu)) { + /* + * If CPU is still running isolated task, we + * can be sure that breaking isolation will + * happen while recording is disabled, and CPU + * will not touch this buffer until the update + * is done. + */ + rb_update_pages(cpu_buffer); + cpu_buffer->nr_pages_to_update = 0; + rv = true; + } + atomic_dec(&cpu_buffer->record_disabled); + } + return rv; +} + /** * ring_buffer_resize - resize the ring buffer * @buffer: the buffer to resize. @@ -1784,13 +1816,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, if (!cpu_buffer->nr_pages_to_update) continue; - /* Can't run something on an offline CPU. */ + /* + * Can't run something on an offline CPU. + * + * CPUs running isolated tasks don't have to + * update ring buffers until they exit + * isolation because they are in + * userspace. Use the procedure that prevents + * race condition with isolation breaking. + */ if (!cpu_online(cpu)) { rb_update_pages(cpu_buffer); cpu_buffer->nr_pages_to_update = 0; } else { - schedule_work_on(cpu, - &cpu_buffer->update_pages_work); + if (!update_if_isolated(cpu_buffer, cpu)) + schedule_work_on(cpu, + &cpu_buffer->update_pages_work); } } @@ -1829,13 +1870,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, get_online_cpus(); - /* Can't run something on an offline CPU. */ + /* + * Can't run something on an offline CPU. + * + * CPUs running isolated tasks don't have to update + * ring buffers until they exit isolation because they + * are in userspace. Use the procedure that prevents + * race condition with isolation breaking. + */ if (!cpu_online(cpu_id)) rb_update_pages(cpu_buffer); else { - schedule_work_on(cpu_id, + if (!update_if_isolated(cpu_buffer, cpu_id)) + schedule_work_on(cpu_id, &cpu_buffer->update_pages_work); - wait_for_completion(&cpu_buffer->update_done); + wait_for_completion(&cpu_buffer->update_done); + } } cpu_buffer->nr_pages_to_update = 0; -- 2.20.1
next prev parent reply index Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-03-04 16:01 [PATCH 00/12] "Task_isolation" mode Alex Belits 2020-03-04 16:03 ` [PATCH 01/12] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits 2020-03-04 16:04 ` [PATCH 02/12] task_isolation: vmstat: add vmstat_idle function Alex Belits 2020-03-04 16:07 ` [PATCH 03/12] task_isolation: userspace hard isolation from kernel Alex Belits 2020-03-05 18:33 ` Frederic Weisbecker 2020-03-08 5:32 ` [EXT] " Alex Belits 2020-04-28 14:12 ` Marcelo Tosatti 2020-03-06 15:26 ` Frederic Weisbecker 2020-03-08 6:06 ` [EXT] " Alex Belits 2020-03-06 16:00 ` Frederic Weisbecker 2020-03-08 7:16 ` [EXT] " Alex Belits 2020-03-04 16:08 ` [PATCH 04/12] task_isolation: Add task isolation hooks to arch-independent code Alex Belits 2020-03-04 16:09 ` [PATCH 05/12] task_isolation: arch/x86: enable task isolation functionality Alex Belits 2020-03-04 16:10 ` [PATCH 06/12] task_isolation: arch/arm64: " Alex Belits 2020-03-04 16:31 ` Mark Rutland 2020-03-08 4:48 ` [EXT] " Alex Belits 2020-03-04 16:11 ` [PATCH 07/12] task_isolation: arch/arm: " Alex Belits 2020-03-04 16:12 ` [PATCH 08/12] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits 2020-03-06 16:03 ` Frederic Weisbecker 2020-03-08 7:28 ` [EXT] " Alex Belits 2020-03-09 2:38 ` Frederic Weisbecker 2020-03-04 16:13 ` [PATCH 09/12] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits 2020-03-04 16:14 ` Alex Belits [this message] 2020-03-04 16:15 ` [PATCH 11/12] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits 2020-03-06 15:34 ` Frederic Weisbecker 2020-03-08 6:48 ` [EXT] " Alex Belits 2020-03-09 2:28 ` Frederic Weisbecker 2020-03-04 16:16 ` [PATCH 12/12] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits 2020-03-08 3:42 ` [PATCH v2 00/12] "Task_isolation" mode Alex Belits 2020-03-08 3:44 ` [PATCH v2 01/12] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits 2020-03-08 3:46 ` [PATCH v2 02/12] task_isolation: vmstat: add vmstat_idle function Alex Belits 2020-03-08 3:47 ` [PATCH v2 03/12] task_isolation: userspace hard isolation from kernel Alex Belits [not found] ` <20200307214254.7a8f6c22@hermes.lan> 2020-03-08 7:33 ` [EXT] " Alex Belits 2020-03-27 8:42 ` Marta Rybczynska 2020-04-06 4:31 ` Kevyn-Alexandre Paré 2020-04-06 4:43 ` Kevyn-Alexandre Paré 2020-03-08 3:48 ` [PATCH v2 04/12] task_isolation: Add task isolation hooks to arch-independent code Alex Belits 2020-03-08 3:49 ` [PATCH v2 05/12] task_isolation: arch/x86: enable task isolation functionality Alex Belits 2020-03-08 3:50 ` [PATCH v2 06/12] task_isolation: arch/arm64: " Alex Belits 2020-03-09 16:59 ` Mark Rutland 2020-03-08 3:52 ` [PATCH v2 07/12] task_isolation: arch/arm: " Alex Belits 2020-03-08 3:53 ` [PATCH v2 08/12] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits 2020-03-08 3:54 ` [PATCH v2 09/12] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits 2020-03-08 3:55 ` [PATCH v2 10/12] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits 2020-04-06 4:27 ` Kevyn-Alexandre Paré 2020-03-08 3:56 ` [PATCH v2 11/12] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits 2020-03-08 3:57 ` [PATCH v2 12/12] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits 2020-04-09 15:09 ` [PATCH v3 00/13] "Task_isolation" mode Alex Belits 2020-04-09 15:15 ` [PATCH 01/13] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits 2020-04-09 15:16 ` [PATCH 02/13] task_isolation: vmstat: add vmstat_idle function Alex Belits 2020-04-09 15:17 ` [PATCH v3 03/13] task_isolation: add instruction synchronization memory barrier Alex Belits 2020-04-15 12:44 ` Mark Rutland 2020-04-19 5:02 ` [EXT] " Alex Belits 2020-04-20 12:23 ` Will Deacon 2020-04-20 12:36 ` Mark Rutland 2020-04-20 13:55 ` Will Deacon 2020-04-21 7:41 ` Will Deacon 2020-04-20 12:45 ` Mark Rutland 2020-04-09 15:20 ` [PATCH v3 04/13] task_isolation: userspace hard isolation from kernel Alex Belits 2020-04-09 18:00 ` Andy Lutomirski 2020-04-19 5:07 ` Alex Belits 2020-04-09 15:21 ` [PATCH 05/13] task_isolation: Add task isolation hooks to arch-independent code Alex Belits 2020-04-09 15:22 ` [PATCH 06/13] task_isolation: arch/x86: enable task isolation functionality Alex Belits 2020-04-09 15:23 ` [PATCH v3 07/13] task_isolation: arch/arm64: " Alex Belits 2020-04-22 12:08 ` Catalin Marinas 2020-04-09 15:24 ` [PATCH v3 08/13] task_isolation: arch/arm: " Alex Belits 2020-04-09 15:25 ` [PATCH v3 09/13] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits 2020-04-09 15:26 ` [PATCH v3 10/13] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits 2020-04-09 15:27 ` [PATCH v3 11/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits 2020-04-09 15:27 ` [PATCH v3 12/13] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits 2020-04-09 15:28 ` [PATCH v3 13/13] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=426ee885b89985f6bf348d64909eeb6a2cd83162.camel@marvell.com \ --to=abelits@marvell.com \ --cc=frederic@kernel.org \ --cc=linux-api@vger.kernel.org \ --cc=linux-arch@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@vger.kernel.org \ --cc=mingo@kernel.org \ --cc=peterz@infradead.org \ --cc=pkapoor@marvell.com \ --cc=rostedt@goodmis.org \ --cc=tglx@linutronix.de \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-api Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-api/0 linux-api/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-api linux-api/ https://lore.kernel.org/linux-api \ linux-api@vger.kernel.org public-inbox-index linux-api Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-api AGPL code for this site: git clone https://public-inbox.org/public-inbox.git