From: Alex Belits <firstname.lastname@example.org> To: "email@example.com" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org> Cc: Prasun Kapoor <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com> Subject: Re: [PATCH v3 00/13] "Task_isolation" mode Date: Thu, 9 Apr 2020 15:09:53 +0000 Message-ID: <firstname.lastname@example.org> (raw) In-Reply-To: <email@example.com> On Sat, 2020-03-07 at 19:42 -0800, Alex Belits wrote: > This is the updated version of task isolation patchset. > > 1. Commit messages updated to match changes. > 2. Sign-off lines restored from original patches, changes listed > wherever applicable. > 3. arm platform -- added missing calls to syscall check and cleanup > procedure after leaving isolation. > 4. x86 platform -- added missing calls to cleanup procedure after > leaving isolation. > Another update, addressing CPU state / race conditions. I believe, I have some usable solution for the problem of both missing the events and race conditions on isolation entry and exit. The idea is to make sure that CPU core remains in userspace and runs userspace code regardless of what is happening in kernel and userspace in the rest of the system, however any events that results in running anything other than userspace code will result in CPU core re-synchronizing with the rest of the system. Then any kernel code, with the exception of small and clearly defined set of routines that only perform kernel entry / exit themselves, will run on CPU after all synchronization is done. This does require an answer to possible races between isolation entry / exit (including isolation breaking on interrupts) and updates that are normally carried by IPIs. So the solution should involve some mechanism that limits what runs on CPU in its "stale" state, and causes inevitable synchronization before the rest of the kernel is called. This should also include any preemption -- if preemtion happens in that "stale" state after entering the kernel but before synchronization is completed, it should still go through synchronization before running the rest of the kernel. Then as long as it can be demonstrated that routines running in "stale" state can safely run in it, and any event that would normally require IPI, will result in entering the rest of kernel after synchronization, race would cease to be a problem. Any sequence of events would result in exactly the same CPU state when hitting the rest of the kernel, as if CPU processed the update event through IPI. I was under impression that this is already the case, however after some closer look it appears that some barriers must be in place to make sure that the sequence of events is enforced. There is obviously a question of performance -- we don't want to cause any additional flushes or add locking in anything time-critical. Fortunately entering and exiting isolation (as opposed to events that _potentially_ can call isolation-breaking routines) is never performance-critical, it's what starts and ends a task that has no performance-critical communication with kernel. So if a CPU that has isolated task on it is running kernel code, it means that either the task is not isolated yet (we are exiting to userspace), or it is no longer running anything performance-critical (intentionally on exit from isolation, or unintentionally on isolation breaking event). Isolation state is read-mostly, and we would prefer RCU for that if we can guarantee that "stale" state remains safe in all code that runs until synchronization happen. I am not sure of that, so I tried to make something more straightforward, however I might be wrong, and RCU-ifying exit from isolation may be a better way do do it. For now I want to make sure that there is some clearly defined small amount of kernel code that runs before the inevitable synchronization, and that code is unaffected by "stale" state. I have tried to track down all call paths from kernel entry points to the call of fast_task_isolation_cpu_cleanup(), and will post those separately. It's possible that all architecture-specific code already follows some clearly defined rules about this for other reasons, however I am not that familiar with all of it, and tried to check if existing implementation is always safe for running in "stale" state before everything that makes task isolation call its cleanup. For now, this is the implementation that assumes that "stale" state is safe for kernel entry.
next prev parent reply index Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-03-04 16:01 [PATCH 00/12] " Alex Belits 2020-03-04 16:03 ` [PATCH 01/12] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits 2020-03-04 16:04 ` [PATCH 02/12] task_isolation: vmstat: add vmstat_idle function Alex Belits 2020-03-04 16:07 ` [PATCH 03/12] task_isolation: userspace hard isolation from kernel Alex Belits 2020-03-05 18:33 ` Frederic Weisbecker 2020-03-08 5:32 ` [EXT] " Alex Belits 2020-04-28 14:12 ` Marcelo Tosatti 2020-03-06 15:26 ` Frederic Weisbecker 2020-03-08 6:06 ` [EXT] " Alex Belits 2020-03-06 16:00 ` Frederic Weisbecker 2020-03-08 7:16 ` [EXT] " Alex Belits 2020-03-04 16:08 ` [PATCH 04/12] task_isolation: Add task isolation hooks to arch-independent code Alex Belits 2020-03-04 16:09 ` [PATCH 05/12] task_isolation: arch/x86: enable task isolation functionality Alex Belits 2020-03-04 16:10 ` [PATCH 06/12] task_isolation: arch/arm64: " Alex Belits 2020-03-04 16:31 ` Mark Rutland 2020-03-08 4:48 ` [EXT] " Alex Belits 2020-03-04 16:11 ` [PATCH 07/12] task_isolation: arch/arm: " Alex Belits 2020-03-04 16:12 ` [PATCH 08/12] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits 2020-03-06 16:03 ` Frederic Weisbecker 2020-03-08 7:28 ` [EXT] " Alex Belits 2020-03-09 2:38 ` Frederic Weisbecker 2020-03-04 16:13 ` [PATCH 09/12] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits 2020-03-04 16:14 ` [PATCH 10/12] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits 2020-03-04 16:15 ` [PATCH 11/12] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits 2020-03-06 15:34 ` Frederic Weisbecker 2020-03-08 6:48 ` [EXT] " Alex Belits 2020-03-09 2:28 ` Frederic Weisbecker 2020-03-04 16:16 ` [PATCH 12/12] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits 2020-03-08 3:42 ` [PATCH v2 00/12] "Task_isolation" mode Alex Belits 2020-03-08 3:44 ` [PATCH v2 01/12] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits 2020-03-08 3:46 ` [PATCH v2 02/12] task_isolation: vmstat: add vmstat_idle function Alex Belits 2020-03-08 3:47 ` [PATCH v2 03/12] task_isolation: userspace hard isolation from kernel Alex Belits [not found] ` <firstname.lastname@example.org> 2020-03-08 7:33 ` [EXT] " Alex Belits 2020-03-27 8:42 ` Marta Rybczynska 2020-04-06 4:31 ` Kevyn-Alexandre Paré 2020-04-06 4:43 ` Kevyn-Alexandre Paré 2020-03-08 3:48 ` [PATCH v2 04/12] task_isolation: Add task isolation hooks to arch-independent code Alex Belits 2020-03-08 3:49 ` [PATCH v2 05/12] task_isolation: arch/x86: enable task isolation functionality Alex Belits 2020-03-08 3:50 ` [PATCH v2 06/12] task_isolation: arch/arm64: " Alex Belits 2020-03-09 16:59 ` Mark Rutland 2020-03-08 3:52 ` [PATCH v2 07/12] task_isolation: arch/arm: " Alex Belits 2020-03-08 3:53 ` [PATCH v2 08/12] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits 2020-03-08 3:54 ` [PATCH v2 09/12] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits 2020-03-08 3:55 ` [PATCH v2 10/12] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits 2020-04-06 4:27 ` Kevyn-Alexandre Paré 2020-03-08 3:56 ` [PATCH v2 11/12] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits 2020-03-08 3:57 ` [PATCH v2 12/12] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits 2020-04-09 15:09 ` Alex Belits [this message] 2020-04-09 15:15 ` [PATCH 01/13] task_isolation: vmstat: add quiet_vmstat_sync function Alex Belits 2020-04-09 15:16 ` [PATCH 02/13] task_isolation: vmstat: add vmstat_idle function Alex Belits 2020-04-09 15:17 ` [PATCH v3 03/13] task_isolation: add instruction synchronization memory barrier Alex Belits 2020-04-15 12:44 ` Mark Rutland 2020-04-19 5:02 ` [EXT] " Alex Belits 2020-04-20 12:23 ` Will Deacon 2020-04-20 12:36 ` Mark Rutland 2020-04-20 13:55 ` Will Deacon 2020-04-21 7:41 ` Will Deacon 2020-04-20 12:45 ` Mark Rutland 2020-04-09 15:20 ` [PATCH v3 04/13] task_isolation: userspace hard isolation from kernel Alex Belits 2020-04-09 18:00 ` Andy Lutomirski 2020-04-19 5:07 ` Alex Belits 2020-04-09 15:21 ` [PATCH 05/13] task_isolation: Add task isolation hooks to arch-independent code Alex Belits 2020-04-09 15:22 ` [PATCH 06/13] task_isolation: arch/x86: enable task isolation functionality Alex Belits 2020-04-09 15:23 ` [PATCH v3 07/13] task_isolation: arch/arm64: " Alex Belits 2020-04-22 12:08 ` Catalin Marinas 2020-04-09 15:24 ` [PATCH v3 08/13] task_isolation: arch/arm: " Alex Belits 2020-04-09 15:25 ` [PATCH v3 09/13] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Alex Belits 2020-04-09 15:26 ` [PATCH v3 10/13] task_isolation: net: don't flush backlog on CPUs running isolated tasks Alex Belits 2020-04-09 15:27 ` [PATCH v3 11/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Alex Belits 2020-04-09 15:27 ` [PATCH v3 12/13] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Alex Belits 2020-04-09 15:28 ` [PATCH v3 13/13] task_isolation: CONFIG_TASK_ISOLATION prevents distribution of jobs to non-housekeeping CPUs Alex Belits
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-api Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-api/0 linux-api/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-api linux-api/ https://lore.kernel.org/linux-api \ email@example.com public-inbox-index linux-api Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-api AGPL code for this site: git clone https://public-inbox.org/public-inbox.git