From: Peter Zijlstra <peterz@infradead.org> To: Andy Lutomirski <luto@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com>, "Russell King (Oracle)" <linux@armlinux.org.uk>, the arch/x86 maintainers <x86@kernel.org>, Dave Hansen <dave.hansen@intel.com>, Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, Nicholas Piggin <npiggin@gmail.com>, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH 7/8] membarrier: Remove arm (32) support for SYNC_CORE Date: Thu, 17 Jun 2021 17:01:53 +0200 [thread overview] Message-ID: <YMtj4Xoo5lh4fGSx@hirez.programming.kicks-ass.net> (raw) In-Reply-To: <33241b25-4d45-4278-a4e6-ec9c12b0e1f3@www.fastmail.com> On Thu, Jun 17, 2021 at 07:00:26AM -0700, Andy Lutomirski wrote: > On Thu, Jun 17, 2021, at 6:51 AM, Mark Rutland wrote: > > It's not clear to me what "the right thing" would mean specifically, and > > on architectures with userspace cache maintenance JITs can usually do > > the most optimal maintenance, and only need help for the context > > synchronization. > > > > This I simply don't believe -- I doubt that any sane architecture > really works like this. I wrote an email about it to Intel that > apparently generated internal discussion but no results. Consider: > > mmap(some shared library, some previously unmapped address); > > this does no heavyweight synchronization, at least on x86. There is > no "serializing" instruction in the fast path, and it *works* despite > anything the SDM may or may not say. I'm confused; why do you think that is relevant? The only way to get into a memory address space is CR3 write, which is serializing and will flush everything. Since there wasn't anything mapped, nothing could be 'cached' from that location. So that has to work... > We can and, IMO, should develop a sane way for user programs to > install instructions into VMAs, for security-conscious software to > verify them (by splitting the read and write sides?), and for their > consumers to execute them, without knowing any arch details. And I > think this can be done with no IPIs except for possible TLB flushing > when needed, at least on most architectures. It would require a > nontrivial amount of design work, and it would not resemble > sys_cacheflush() or SYNC_CORE. The interesting use-case is where we modify code that is under active execution in a multi-threaded process; where CPU0 runs code and doesn't make any syscalls at all, while CPU1 modifies code that is visible to CPU0. In that case CPU0 can have various internal state still reflecting the old instructions that no longer exist in memory -- presumably. We also need to inject at least a full memory barrier and pipeline flush, to create a proper before and after modified. To reason about when the *other* threads will be able to observe the new code. Now, the SDM documents that prefetch and trace buffers are not flushed on i$ invalidate (actual implementations might of course differ) and doing this requires the SERIALIZE instruction or one of the many instructions that implies this, one of which is IRET. For the cross-modifying case, I really don't see how you can not send an IPI and expect behavour one can reason about, irrespective of any non-coherent behaviour. Now, the SDM documents non-coherent behaviour and requires SERIALIZE, while at the same time any IPI already implies IRET which implies SERIALIZE -- except some Luto guy was having plans to optimize the IRET paths so we couldn't rely on that.
WARNING: multiple messages have this Message-ID (diff)
From: Peter Zijlstra <peterz@infradead.org> To: Andy Lutomirski <luto@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com>, "Russell King (Oracle)" <linux@armlinux.org.uk>, the arch/x86 maintainers <x86@kernel.org>, Dave Hansen <dave.hansen@intel.com>, Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, Nicholas Piggin <npiggin@gmail.com>, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH 7/8] membarrier: Remove arm (32) support for SYNC_CORE Date: Thu, 17 Jun 2021 17:01:53 +0200 [thread overview] Message-ID: <YMtj4Xoo5lh4fGSx@hirez.programming.kicks-ass.net> (raw) In-Reply-To: <33241b25-4d45-4278-a4e6-ec9c12b0e1f3@www.fastmail.com> On Thu, Jun 17, 2021 at 07:00:26AM -0700, Andy Lutomirski wrote: > On Thu, Jun 17, 2021, at 6:51 AM, Mark Rutland wrote: > > It's not clear to me what "the right thing" would mean specifically, and > > on architectures with userspace cache maintenance JITs can usually do > > the most optimal maintenance, and only need help for the context > > synchronization. > > > > This I simply don't believe -- I doubt that any sane architecture > really works like this. I wrote an email about it to Intel that > apparently generated internal discussion but no results. Consider: > > mmap(some shared library, some previously unmapped address); > > this does no heavyweight synchronization, at least on x86. There is > no "serializing" instruction in the fast path, and it *works* despite > anything the SDM may or may not say. I'm confused; why do you think that is relevant? The only way to get into a memory address space is CR3 write, which is serializing and will flush everything. Since there wasn't anything mapped, nothing could be 'cached' from that location. So that has to work... > We can and, IMO, should develop a sane way for user programs to > install instructions into VMAs, for security-conscious software to > verify them (by splitting the read and write sides?), and for their > consumers to execute them, without knowing any arch details. And I > think this can be done with no IPIs except for possible TLB flushing > when needed, at least on most architectures. It would require a > nontrivial amount of design work, and it would not resemble > sys_cacheflush() or SYNC_CORE. The interesting use-case is where we modify code that is under active execution in a multi-threaded process; where CPU0 runs code and doesn't make any syscalls at all, while CPU1 modifies code that is visible to CPU0. In that case CPU0 can have various internal state still reflecting the old instructions that no longer exist in memory -- presumably. We also need to inject at least a full memory barrier and pipeline flush, to create a proper before and after modified. To reason about when the *other* threads will be able to observe the new code. Now, the SDM documents that prefetch and trace buffers are not flushed on i$ invalidate (actual implementations might of course differ) and doing this requires the SERIALIZE instruction or one of the many instructions that implies this, one of which is IRET. For the cross-modifying case, I really don't see how you can not send an IPI and expect behavour one can reason about, irrespective of any non-coherent behaviour. Now, the SDM documents non-coherent behaviour and requires SERIALIZE, while at the same time any IPI already implies IRET which implies SERIALIZE -- except some Luto guy was having plans to optimize the IRET paths so we couldn't rely on that. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-06-17 15:03 UTC|newest] Thread overview: 165+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-06-16 3:21 [PATCH 0/8] membarrier cleanups Andy Lutomirski 2021-06-16 3:21 ` [PATCH 1/8] membarrier: Document why membarrier() works Andy Lutomirski 2021-06-16 4:00 ` Nicholas Piggin 2021-06-16 7:30 ` Peter Zijlstra 2021-06-17 23:45 ` Andy Lutomirski 2021-06-16 3:21 ` [PATCH 2/8] x86/mm: Handle unlazying membarrier core sync in the arch code Andy Lutomirski 2021-06-16 4:25 ` Nicholas Piggin 2021-06-16 18:31 ` Andy Lutomirski 2021-06-16 17:49 ` Mathieu Desnoyers 2021-06-16 17:49 ` Mathieu Desnoyers 2021-06-16 18:31 ` Andy Lutomirski 2021-06-16 3:21 ` [PATCH 3/8] membarrier: Remove membarrier_arch_switch_mm() prototype in core code Andy Lutomirski 2021-06-16 4:26 ` Nicholas Piggin 2021-06-16 17:52 ` Mathieu Desnoyers 2021-06-16 17:52 ` Mathieu Desnoyers 2021-06-16 3:21 ` [PATCH 4/8] membarrier: Make the post-switch-mm barrier explicit Andy Lutomirski 2021-06-16 4:19 ` Nicholas Piggin 2021-06-16 7:35 ` Peter Zijlstra 2021-06-16 18:41 ` Andy Lutomirski 2021-06-17 1:37 ` Nicholas Piggin 2021-06-17 2:57 ` Andy Lutomirski 2021-06-17 5:32 ` Andy Lutomirski 2021-06-17 6:51 ` Nicholas Piggin 2021-06-17 23:49 ` Andy Lutomirski 2021-06-19 2:53 ` Nicholas Piggin 2021-06-19 3:20 ` Andy Lutomirski 2021-06-19 4:27 ` Nicholas Piggin 2021-06-17 9:08 ` [RFC][PATCH] sched: Use lightweight hazard pointers to grab lazy mms Peter Zijlstra 2021-06-17 9:10 ` Peter Zijlstra 2021-06-17 10:00 ` Nicholas Piggin 2021-06-17 9:13 ` Peter Zijlstra 2021-06-17 14:06 ` Andy Lutomirski 2021-06-17 9:28 ` Peter Zijlstra 2021-06-17 14:03 ` Andy Lutomirski 2021-06-17 14:10 ` Andy Lutomirski 2021-06-17 15:45 ` Peter Zijlstra 2021-06-18 3:29 ` Paul E. McKenney 2021-06-18 5:04 ` Andy Lutomirski 2021-06-17 15:02 ` [PATCH 4/8] membarrier: Make the post-switch-mm barrier explicit Paul E. McKenney 2021-06-18 0:06 ` Andy Lutomirski 2021-06-18 3:35 ` Paul E. McKenney 2021-06-17 8:45 ` Peter Zijlstra 2021-06-16 3:21 ` [PATCH 5/8] membarrier, kthread: Use _ONCE accessors for task->mm Andy Lutomirski 2021-06-16 4:28 ` Nicholas Piggin 2021-06-16 18:08 ` Mathieu Desnoyers 2021-06-16 18:08 ` Mathieu Desnoyers 2021-06-16 18:45 ` Andy Lutomirski 2021-06-16 3:21 ` [PATCH 6/8] powerpc/membarrier: Remove special barrier on mm switch Andy Lutomirski 2021-06-16 3:21 ` Andy Lutomirski 2021-06-16 4:36 ` Nicholas Piggin 2021-06-16 4:36 ` Nicholas Piggin 2021-06-16 3:21 ` [PATCH 7/8] membarrier: Remove arm (32) support for SYNC_CORE Andy Lutomirski 2021-06-16 3:21 ` Andy Lutomirski 2021-06-16 9:28 ` Russell King (Oracle) 2021-06-16 9:28 ` Russell King (Oracle) 2021-06-16 10:16 ` Peter Zijlstra 2021-06-16 10:16 ` Peter Zijlstra 2021-06-16 10:20 ` Peter Zijlstra 2021-06-16 10:20 ` Peter Zijlstra 2021-06-16 10:34 ` Russell King (Oracle) 2021-06-16 10:34 ` Russell King (Oracle) 2021-06-16 11:10 ` Peter Zijlstra 2021-06-16 11:10 ` Peter Zijlstra 2021-06-16 13:22 ` Russell King (Oracle) 2021-06-16 13:22 ` Russell King (Oracle) 2021-06-16 15:04 ` Catalin Marinas 2021-06-16 15:04 ` Catalin Marinas 2021-06-16 15:23 ` Russell King (Oracle) 2021-06-16 15:23 ` Russell King (Oracle) 2021-06-16 15:45 ` Catalin Marinas 2021-06-16 15:45 ` Catalin Marinas 2021-06-16 16:00 ` Catalin Marinas 2021-06-16 16:00 ` Catalin Marinas 2021-06-16 16:27 ` Russell King (Oracle) 2021-06-16 16:27 ` Russell King (Oracle) 2021-06-17 8:55 ` Krzysztof Hałasa 2021-06-17 8:55 ` Krzysztof Hałasa 2021-06-17 8:55 ` Krzysztof Hałasa 2021-06-18 12:54 ` Linus Walleij 2021-06-18 12:54 ` Linus Walleij 2021-06-18 12:54 ` Linus Walleij 2021-06-18 13:19 ` Russell King (Oracle) 2021-06-18 13:19 ` Russell King (Oracle) 2021-06-18 13:36 ` Arnd Bergmann 2021-06-18 13:36 ` Arnd Bergmann 2021-06-18 13:36 ` Arnd Bergmann 2021-06-17 10:40 ` Mark Rutland 2021-06-17 10:40 ` Mark Rutland 2021-06-17 11:23 ` Russell King (Oracle) 2021-06-17 11:23 ` Russell King (Oracle) 2021-06-17 11:33 ` Mark Rutland 2021-06-17 11:33 ` Mark Rutland 2021-06-17 13:41 ` Andy Lutomirski 2021-06-17 13:41 ` Andy Lutomirski 2021-06-17 13:51 ` Mark Rutland 2021-06-17 13:51 ` Mark Rutland 2021-06-17 14:00 ` Andy Lutomirski 2021-06-17 14:00 ` Andy Lutomirski 2021-06-17 14:20 ` Mark Rutland 2021-06-17 14:20 ` Mark Rutland 2021-06-17 15:01 ` Peter Zijlstra [this message] 2021-06-17 15:01 ` Peter Zijlstra 2021-06-17 15:13 ` Peter Zijlstra 2021-06-17 15:13 ` Peter Zijlstra 2021-06-17 14:16 ` Mathieu Desnoyers 2021-06-17 14:16 ` Mathieu Desnoyers 2021-06-17 14:05 ` Peter Zijlstra 2021-06-17 14:05 ` Peter Zijlstra 2021-06-18 0:07 ` Andy Lutomirski 2021-06-18 0:07 ` Andy Lutomirski 2021-06-16 3:21 ` [PATCH 8/8] membarrier: Rewrite sync_core_before_usermode() and improve documentation Andy Lutomirski 2021-06-16 3:21 ` Andy Lutomirski 2021-06-16 3:21 ` Andy Lutomirski 2021-06-16 4:45 ` Nicholas Piggin 2021-06-16 4:45 ` Nicholas Piggin 2021-06-16 4:45 ` Nicholas Piggin 2021-06-16 18:52 ` Andy Lutomirski 2021-06-16 18:52 ` Andy Lutomirski 2021-06-16 18:52 ` Andy Lutomirski 2021-06-16 23:48 ` Andy Lutomirski 2021-06-16 23:48 ` Andy Lutomirski 2021-06-16 23:48 ` Andy Lutomirski 2021-06-18 15:27 ` Christophe Leroy 2021-06-18 15:27 ` Christophe Leroy 2021-06-18 15:27 ` Christophe Leroy 2021-06-16 10:20 ` Will Deacon 2021-06-16 10:20 ` Will Deacon 2021-06-16 10:20 ` Will Deacon 2021-06-16 23:58 ` Andy Lutomirski 2021-06-16 23:58 ` Andy Lutomirski 2021-06-16 23:58 ` Andy Lutomirski 2021-06-17 14:47 ` Mathieu Desnoyers 2021-06-17 14:47 ` Mathieu Desnoyers 2021-06-17 14:47 ` Mathieu Desnoyers 2021-06-17 14:47 ` Mathieu Desnoyers 2021-06-18 0:12 ` Andy Lutomirski 2021-06-18 0:12 ` Andy Lutomirski 2021-06-18 0:12 ` Andy Lutomirski 2021-06-18 16:31 ` Mathieu Desnoyers 2021-06-18 16:31 ` Mathieu Desnoyers 2021-06-18 16:31 ` Mathieu Desnoyers 2021-06-18 16:31 ` Mathieu Desnoyers 2021-06-18 19:58 ` Andy Lutomirski 2021-06-18 19:58 ` Andy Lutomirski 2021-06-18 19:58 ` Andy Lutomirski 2021-06-18 20:09 ` Mathieu Desnoyers 2021-06-18 20:09 ` Mathieu Desnoyers 2021-06-18 20:09 ` Mathieu Desnoyers 2021-06-18 20:09 ` Mathieu Desnoyers 2021-06-19 6:02 ` Nicholas Piggin 2021-06-19 6:02 ` Nicholas Piggin 2021-06-19 6:02 ` Nicholas Piggin 2021-06-19 15:50 ` Andy Lutomirski 2021-06-19 15:50 ` Andy Lutomirski 2021-06-19 15:50 ` Andy Lutomirski 2021-06-20 2:10 ` Nicholas Piggin 2021-06-20 2:10 ` Nicholas Piggin 2021-06-20 2:10 ` Nicholas Piggin 2021-06-17 15:16 ` Mathieu Desnoyers 2021-06-17 15:16 ` Mathieu Desnoyers 2021-06-17 15:16 ` Mathieu Desnoyers 2021-06-17 15:16 ` Mathieu Desnoyers 2021-06-18 0:13 ` Andy Lutomirski 2021-06-18 0:13 ` Andy Lutomirski 2021-06-18 0:13 ` Andy Lutomirski
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=YMtj4Xoo5lh4fGSx@hirez.programming.kicks-ass.net \ --to=peterz@infradead.org \ --cc=akpm@linux-foundation.org \ --cc=dave.hansen@intel.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux@armlinux.org.uk \ --cc=luto@kernel.org \ --cc=mark.rutland@arm.com \ --cc=mathieu.desnoyers@efficios.com \ --cc=npiggin@gmail.com \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.