From: Peter Zijlstra <peterz@infradead.org> To: Andy Lutomirski <luto@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com>, "Russell King (Oracle)" <linux@armlinux.org.uk>, the arch/x86 maintainers <x86@kernel.org>, Dave Hansen <dave.hansen@intel.com>, Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, Nicholas Piggin <npiggin@gmail.com>, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH 7/8] membarrier: Remove arm (32) support for SYNC_CORE Date: Thu, 17 Jun 2021 16:05:03 +0200 [thread overview] Message-ID: <YMtWj/y4Hc0CeCux@hirez.programming.kicks-ass.net> (raw) In-Reply-To: <394219d4-36a6-4e7f-a03c-8590551b099a@www.fastmail.com> On Thu, Jun 17, 2021 at 06:41:41AM -0700, Andy Lutomirski wrote: > On Thu, Jun 17, 2021, at 4:33 AM, Mark Rutland wrote: > > Sure, and I agree we should not change cacheflush(). > > > > The point of membarrier(SYNC_CORE) is that you can move the cost of that > > ISB out of the fast-path in the executing thread(s) and into the > > slow-path on the thread which generated the code. > > > > So e.g. rather than an executing thread always having to do: > > > > LDR <reg>, [<funcptr>] > > ISB // in case funcptr was just updated > > BLR <reg> > > > > ... you have the thread generating the code use membarrier(SYNC_CORE) > > prior to plublishing the funcptr, and the fast-path on all the executing > > threads can be: > > > > LDR <reg> [<funcptr>] > > BLR <reg> > > > > ... and thus I think we still want membarrier(SYNC_CORE) so that people > > can do this, even if there are other means to achieve the same > > functionality. > > I had the impression that sys_cacheflush() did that. Am I wrong? Yes, sys_cacheflush() only does what it says on the tin (and only correctly for hardware broadcast -- everything except 11mpcore). It only invalidates the caches, but not the per CPU derived state like prefetch buffers and micro-op buffers, and certainly not instructions already in flight. So anything OoO needs at the very least a complete pipeline stall injected, but probably something stronger to make it flush the buffers. > In any event, I’m even more convinced that no new SYNC_CORE arches > should be added. We need a new API that just does the right thing. I really don't understand why you hate the thing so much; SYNC_CORE is a means of injecting whatever instruction is required to flush all uarch state related to instructions on all theads (not all CPUs) of a process as efficient as possible. The alternative is sending signals to all threads (including the non-running ones) which is known to scale very poorly indeed, or, as Mark suggests above, have very expensive instructions unconditinoally in the instruction stream, which is also undesired.
WARNING: multiple messages have this Message-ID (diff)
From: Peter Zijlstra <peterz@infradead.org> To: Andy Lutomirski <luto@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com>, "Russell King (Oracle)" <linux@armlinux.org.uk>, the arch/x86 maintainers <x86@kernel.org>, Dave Hansen <dave.hansen@intel.com>, Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>, Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, Nicholas Piggin <npiggin@gmail.com>, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH 7/8] membarrier: Remove arm (32) support for SYNC_CORE Date: Thu, 17 Jun 2021 16:05:03 +0200 [thread overview] Message-ID: <YMtWj/y4Hc0CeCux@hirez.programming.kicks-ass.net> (raw) In-Reply-To: <394219d4-36a6-4e7f-a03c-8590551b099a@www.fastmail.com> On Thu, Jun 17, 2021 at 06:41:41AM -0700, Andy Lutomirski wrote: > On Thu, Jun 17, 2021, at 4:33 AM, Mark Rutland wrote: > > Sure, and I agree we should not change cacheflush(). > > > > The point of membarrier(SYNC_CORE) is that you can move the cost of that > > ISB out of the fast-path in the executing thread(s) and into the > > slow-path on the thread which generated the code. > > > > So e.g. rather than an executing thread always having to do: > > > > LDR <reg>, [<funcptr>] > > ISB // in case funcptr was just updated > > BLR <reg> > > > > ... you have the thread generating the code use membarrier(SYNC_CORE) > > prior to plublishing the funcptr, and the fast-path on all the executing > > threads can be: > > > > LDR <reg> [<funcptr>] > > BLR <reg> > > > > ... and thus I think we still want membarrier(SYNC_CORE) so that people > > can do this, even if there are other means to achieve the same > > functionality. > > I had the impression that sys_cacheflush() did that. Am I wrong? Yes, sys_cacheflush() only does what it says on the tin (and only correctly for hardware broadcast -- everything except 11mpcore). It only invalidates the caches, but not the per CPU derived state like prefetch buffers and micro-op buffers, and certainly not instructions already in flight. So anything OoO needs at the very least a complete pipeline stall injected, but probably something stronger to make it flush the buffers. > In any event, I’m even more convinced that no new SYNC_CORE arches > should be added. We need a new API that just does the right thing. I really don't understand why you hate the thing so much; SYNC_CORE is a means of injecting whatever instruction is required to flush all uarch state related to instructions on all theads (not all CPUs) of a process as efficient as possible. The alternative is sending signals to all threads (including the non-running ones) which is known to scale very poorly indeed, or, as Mark suggests above, have very expensive instructions unconditinoally in the instruction stream, which is also undesired. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-06-17 14:06 UTC|newest] Thread overview: 165+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-06-16 3:21 [PATCH 0/8] membarrier cleanups Andy Lutomirski 2021-06-16 3:21 ` [PATCH 1/8] membarrier: Document why membarrier() works Andy Lutomirski 2021-06-16 4:00 ` Nicholas Piggin 2021-06-16 7:30 ` Peter Zijlstra 2021-06-17 23:45 ` Andy Lutomirski 2021-06-16 3:21 ` [PATCH 2/8] x86/mm: Handle unlazying membarrier core sync in the arch code Andy Lutomirski 2021-06-16 4:25 ` Nicholas Piggin 2021-06-16 18:31 ` Andy Lutomirski 2021-06-16 17:49 ` Mathieu Desnoyers 2021-06-16 17:49 ` Mathieu Desnoyers 2021-06-16 18:31 ` Andy Lutomirski 2021-06-16 3:21 ` [PATCH 3/8] membarrier: Remove membarrier_arch_switch_mm() prototype in core code Andy Lutomirski 2021-06-16 4:26 ` Nicholas Piggin 2021-06-16 17:52 ` Mathieu Desnoyers 2021-06-16 17:52 ` Mathieu Desnoyers 2021-06-16 3:21 ` [PATCH 4/8] membarrier: Make the post-switch-mm barrier explicit Andy Lutomirski 2021-06-16 4:19 ` Nicholas Piggin 2021-06-16 7:35 ` Peter Zijlstra 2021-06-16 18:41 ` Andy Lutomirski 2021-06-17 1:37 ` Nicholas Piggin 2021-06-17 2:57 ` Andy Lutomirski 2021-06-17 5:32 ` Andy Lutomirski 2021-06-17 6:51 ` Nicholas Piggin 2021-06-17 23:49 ` Andy Lutomirski 2021-06-19 2:53 ` Nicholas Piggin 2021-06-19 3:20 ` Andy Lutomirski 2021-06-19 4:27 ` Nicholas Piggin 2021-06-17 9:08 ` [RFC][PATCH] sched: Use lightweight hazard pointers to grab lazy mms Peter Zijlstra 2021-06-17 9:10 ` Peter Zijlstra 2021-06-17 10:00 ` Nicholas Piggin 2021-06-17 9:13 ` Peter Zijlstra 2021-06-17 14:06 ` Andy Lutomirski 2021-06-17 9:28 ` Peter Zijlstra 2021-06-17 14:03 ` Andy Lutomirski 2021-06-17 14:10 ` Andy Lutomirski 2021-06-17 15:45 ` Peter Zijlstra 2021-06-18 3:29 ` Paul E. McKenney 2021-06-18 5:04 ` Andy Lutomirski 2021-06-17 15:02 ` [PATCH 4/8] membarrier: Make the post-switch-mm barrier explicit Paul E. McKenney 2021-06-18 0:06 ` Andy Lutomirski 2021-06-18 3:35 ` Paul E. McKenney 2021-06-17 8:45 ` Peter Zijlstra 2021-06-16 3:21 ` [PATCH 5/8] membarrier, kthread: Use _ONCE accessors for task->mm Andy Lutomirski 2021-06-16 4:28 ` Nicholas Piggin 2021-06-16 18:08 ` Mathieu Desnoyers 2021-06-16 18:08 ` Mathieu Desnoyers 2021-06-16 18:45 ` Andy Lutomirski 2021-06-16 3:21 ` [PATCH 6/8] powerpc/membarrier: Remove special barrier on mm switch Andy Lutomirski 2021-06-16 3:21 ` Andy Lutomirski 2021-06-16 4:36 ` Nicholas Piggin 2021-06-16 4:36 ` Nicholas Piggin 2021-06-16 3:21 ` [PATCH 7/8] membarrier: Remove arm (32) support for SYNC_CORE Andy Lutomirski 2021-06-16 3:21 ` Andy Lutomirski 2021-06-16 9:28 ` Russell King (Oracle) 2021-06-16 9:28 ` Russell King (Oracle) 2021-06-16 10:16 ` Peter Zijlstra 2021-06-16 10:16 ` Peter Zijlstra 2021-06-16 10:20 ` Peter Zijlstra 2021-06-16 10:20 ` Peter Zijlstra 2021-06-16 10:34 ` Russell King (Oracle) 2021-06-16 10:34 ` Russell King (Oracle) 2021-06-16 11:10 ` Peter Zijlstra 2021-06-16 11:10 ` Peter Zijlstra 2021-06-16 13:22 ` Russell King (Oracle) 2021-06-16 13:22 ` Russell King (Oracle) 2021-06-16 15:04 ` Catalin Marinas 2021-06-16 15:04 ` Catalin Marinas 2021-06-16 15:23 ` Russell King (Oracle) 2021-06-16 15:23 ` Russell King (Oracle) 2021-06-16 15:45 ` Catalin Marinas 2021-06-16 15:45 ` Catalin Marinas 2021-06-16 16:00 ` Catalin Marinas 2021-06-16 16:00 ` Catalin Marinas 2021-06-16 16:27 ` Russell King (Oracle) 2021-06-16 16:27 ` Russell King (Oracle) 2021-06-17 8:55 ` Krzysztof Hałasa 2021-06-17 8:55 ` Krzysztof Hałasa 2021-06-17 8:55 ` Krzysztof Hałasa 2021-06-18 12:54 ` Linus Walleij 2021-06-18 12:54 ` Linus Walleij 2021-06-18 12:54 ` Linus Walleij 2021-06-18 13:19 ` Russell King (Oracle) 2021-06-18 13:19 ` Russell King (Oracle) 2021-06-18 13:36 ` Arnd Bergmann 2021-06-18 13:36 ` Arnd Bergmann 2021-06-18 13:36 ` Arnd Bergmann 2021-06-17 10:40 ` Mark Rutland 2021-06-17 10:40 ` Mark Rutland 2021-06-17 11:23 ` Russell King (Oracle) 2021-06-17 11:23 ` Russell King (Oracle) 2021-06-17 11:33 ` Mark Rutland 2021-06-17 11:33 ` Mark Rutland 2021-06-17 13:41 ` Andy Lutomirski 2021-06-17 13:41 ` Andy Lutomirski 2021-06-17 13:51 ` Mark Rutland 2021-06-17 13:51 ` Mark Rutland 2021-06-17 14:00 ` Andy Lutomirski 2021-06-17 14:00 ` Andy Lutomirski 2021-06-17 14:20 ` Mark Rutland 2021-06-17 14:20 ` Mark Rutland 2021-06-17 15:01 ` Peter Zijlstra 2021-06-17 15:01 ` Peter Zijlstra 2021-06-17 15:13 ` Peter Zijlstra 2021-06-17 15:13 ` Peter Zijlstra 2021-06-17 14:16 ` Mathieu Desnoyers 2021-06-17 14:16 ` Mathieu Desnoyers 2021-06-17 14:05 ` Peter Zijlstra [this message] 2021-06-17 14:05 ` Peter Zijlstra 2021-06-18 0:07 ` Andy Lutomirski 2021-06-18 0:07 ` Andy Lutomirski 2021-06-16 3:21 ` [PATCH 8/8] membarrier: Rewrite sync_core_before_usermode() and improve documentation Andy Lutomirski 2021-06-16 3:21 ` Andy Lutomirski 2021-06-16 3:21 ` Andy Lutomirski 2021-06-16 4:45 ` Nicholas Piggin 2021-06-16 4:45 ` Nicholas Piggin 2021-06-16 4:45 ` Nicholas Piggin 2021-06-16 18:52 ` Andy Lutomirski 2021-06-16 18:52 ` Andy Lutomirski 2021-06-16 18:52 ` Andy Lutomirski 2021-06-16 23:48 ` Andy Lutomirski 2021-06-16 23:48 ` Andy Lutomirski 2021-06-16 23:48 ` Andy Lutomirski 2021-06-18 15:27 ` Christophe Leroy 2021-06-18 15:27 ` Christophe Leroy 2021-06-18 15:27 ` Christophe Leroy 2021-06-16 10:20 ` Will Deacon 2021-06-16 10:20 ` Will Deacon 2021-06-16 10:20 ` Will Deacon 2021-06-16 23:58 ` Andy Lutomirski 2021-06-16 23:58 ` Andy Lutomirski 2021-06-16 23:58 ` Andy Lutomirski 2021-06-17 14:47 ` Mathieu Desnoyers 2021-06-17 14:47 ` Mathieu Desnoyers 2021-06-17 14:47 ` Mathieu Desnoyers 2021-06-17 14:47 ` Mathieu Desnoyers 2021-06-18 0:12 ` Andy Lutomirski 2021-06-18 0:12 ` Andy Lutomirski 2021-06-18 0:12 ` Andy Lutomirski 2021-06-18 16:31 ` Mathieu Desnoyers 2021-06-18 16:31 ` Mathieu Desnoyers 2021-06-18 16:31 ` Mathieu Desnoyers 2021-06-18 16:31 ` Mathieu Desnoyers 2021-06-18 19:58 ` Andy Lutomirski 2021-06-18 19:58 ` Andy Lutomirski 2021-06-18 19:58 ` Andy Lutomirski 2021-06-18 20:09 ` Mathieu Desnoyers 2021-06-18 20:09 ` Mathieu Desnoyers 2021-06-18 20:09 ` Mathieu Desnoyers 2021-06-18 20:09 ` Mathieu Desnoyers 2021-06-19 6:02 ` Nicholas Piggin 2021-06-19 6:02 ` Nicholas Piggin 2021-06-19 6:02 ` Nicholas Piggin 2021-06-19 15:50 ` Andy Lutomirski 2021-06-19 15:50 ` Andy Lutomirski 2021-06-19 15:50 ` Andy Lutomirski 2021-06-20 2:10 ` Nicholas Piggin 2021-06-20 2:10 ` Nicholas Piggin 2021-06-20 2:10 ` Nicholas Piggin 2021-06-17 15:16 ` Mathieu Desnoyers 2021-06-17 15:16 ` Mathieu Desnoyers 2021-06-17 15:16 ` Mathieu Desnoyers 2021-06-17 15:16 ` Mathieu Desnoyers 2021-06-18 0:13 ` Andy Lutomirski 2021-06-18 0:13 ` Andy Lutomirski 2021-06-18 0:13 ` Andy Lutomirski
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=YMtWj/y4Hc0CeCux@hirez.programming.kicks-ass.net \ --to=peterz@infradead.org \ --cc=akpm@linux-foundation.org \ --cc=dave.hansen@intel.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux@armlinux.org.uk \ --cc=luto@kernel.org \ --cc=mark.rutland@arm.com \ --cc=mathieu.desnoyers@efficios.com \ --cc=npiggin@gmail.com \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.