From: Oliver <oohall@gmail.com> To: Gabriel Paubert <paubert@iram.es> Cc: Sinan Kaya <okaya@codeaurora.org>, "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>, David Laight <David.Laight@aculab.com>, "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" <linuxppc-dev@lists.ozlabs.org> Subject: Re: RFC on writel and writel_relaxed Date: Thu, 22 Mar 2018 20:25:43 +1100 [thread overview] Message-ID: <CAOSf1CE+gsoZB-YNYdx=oNmrdCje4yxiz76hSb4UQ4F5cL8SqQ@mail.gmail.com> (raw) In-Reply-To: <20180322082041.hvaneyfpzdwelely@lt-gp.iram.es> On Thu, Mar 22, 2018 at 7:20 PM, Gabriel Paubert <paubert@iram.es> wrote: > On Thu, Mar 22, 2018 at 04:24:24PM +1100, Oliver wrote: >> On Thu, Mar 22, 2018 at 1:35 AM, David Laight <David.Laight@aculab.com> wrote: >> >> x86 has compiler barrier inside the relaxed() API so that code does not >> >> get reordered. ARM64 architecturally guarantees device writes to be observed >> >> in order. >> > >> > There are places where you don't even need a compile barrier between >> > every write. >> > >> > I had horrid problems getting some ppc code (for a specific embedded SoC) >> > optimised to have no extra barriers. >> > I ended up just writing through 'pointer to volatile' and adding an >> > explicit 'eieio' between the block of writes and status read. >> >> This is what you are supposed to do. For accesses to MMIO (cache >> inhibited + guarded) storage the Power ISA guarantees that load-load >> and store-store pairs of accesses will always occur in program order, >> but there's no implicit ordering between load-store or store-load > > And even for load store, eieio is not always necessary, in the important > case of reading and writing to the same address, when modifying bits in > a control register for example. > > Typically also loads will be moved ahead of stores, but not the other > way around, so in practice you won't notice a missed eieio in this case. > This does not mean you should not insert it. Yep, but it doesn't really help us here. The generic accessors need to cope with the general case. >> pairs. In those cases you need an explicit eieio barrier between the >> two accesses. At the HW level you can think of the CPU as having >> separate queues for MMIO loads and stores. Accesses will be added to >> the respective queue in program order, but there's no synchronisation >> between the two queues. If the CPU is doing write combining it's easy >> to imagine the whole store queue being emptied in one big gulp before >> the load queue is even touched. > > Is write combining allowed on guarded storage? > > <Looking at docs> > From PowerISA_V3.0.pdf, Book2, section 1.6.2 "Caching inhibited": > > "No combining occurs if the storage is also Guarded" Yeah it's not allowed. That's what I get for handwaving examples ;)
WARNING: multiple messages have this Message-ID (diff)
From: Oliver <oohall@gmail.com> To: Gabriel Paubert <paubert@iram.es> Cc: David Laight <David.Laight@aculab.com>, Sinan Kaya <okaya@codeaurora.org>, "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>, "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" <linuxppc-dev@lists.ozlabs.org> Subject: Re: RFC on writel and writel_relaxed Date: Thu, 22 Mar 2018 20:25:43 +1100 [thread overview] Message-ID: <CAOSf1CE+gsoZB-YNYdx=oNmrdCje4yxiz76hSb4UQ4F5cL8SqQ@mail.gmail.com> (raw) In-Reply-To: <20180322082041.hvaneyfpzdwelely@lt-gp.iram.es> On Thu, Mar 22, 2018 at 7:20 PM, Gabriel Paubert <paubert@iram.es> wrote: > On Thu, Mar 22, 2018 at 04:24:24PM +1100, Oliver wrote: >> On Thu, Mar 22, 2018 at 1:35 AM, David Laight <David.Laight@aculab.com> wrote: >> >> x86 has compiler barrier inside the relaxed() API so that code does not >> >> get reordered. ARM64 architecturally guarantees device writes to be observed >> >> in order. >> > >> > There are places where you don't even need a compile barrier between >> > every write. >> > >> > I had horrid problems getting some ppc code (for a specific embedded SoC) >> > optimised to have no extra barriers. >> > I ended up just writing through 'pointer to volatile' and adding an >> > explicit 'eieio' between the block of writes and status read. >> >> This is what you are supposed to do. For accesses to MMIO (cache >> inhibited + guarded) storage the Power ISA guarantees that load-load >> and store-store pairs of accesses will always occur in program order, >> but there's no implicit ordering between load-store or store-load > > And even for load store, eieio is not always necessary, in the important > case of reading and writing to the same address, when modifying bits in > a control register for example. > > Typically also loads will be moved ahead of stores, but not the other > way around, so in practice you won't notice a missed eieio in this case. > This does not mean you should not insert it. Yep, but it doesn't really help us here. The generic accessors need to cope with the general case. >> pairs. In those cases you need an explicit eieio barrier between the >> two accesses. At the HW level you can think of the CPU as having >> separate queues for MMIO loads and stores. Accesses will be added to >> the respective queue in program order, but there's no synchronisation >> between the two queues. If the CPU is doing write combining it's easy >> to imagine the whole store queue being emptied in one big gulp before >> the load queue is even touched. > > Is write combining allowed on guarded storage? > > <Looking at docs> > From PowerISA_V3.0.pdf, Book2, section 1.6.2 "Caching inhibited": > > "No combining occurs if the storage is also Guarded" Yeah it's not allowed. That's what I get for handwaving examples ;)
next prev parent reply other threads:[~2018-03-22 9:25 UTC|newest] Thread overview: 216+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-03-21 3:07 RFC on writel and writel_relaxed Sinan Kaya 2018-03-21 3:40 ` Oliver 2018-03-21 3:40 ` Oliver 2018-03-21 13:53 ` Sinan Kaya 2018-03-21 13:53 ` Sinan Kaya 2018-03-21 13:58 ` Sinan Kaya 2018-03-21 13:58 ` Sinan Kaya 2018-03-26 13:43 ` Arnd Bergmann 2018-03-26 13:43 ` Arnd Bergmann 2018-03-26 16:00 ` Sinan Kaya 2018-03-26 16:00 ` Sinan Kaya 2018-03-21 14:35 ` David Laight 2018-03-21 14:35 ` David Laight 2018-03-21 15:04 ` Sinan Kaya 2018-03-22 5:24 ` Oliver 2018-03-22 8:20 ` Gabriel Paubert 2018-03-22 8:20 ` Gabriel Paubert 2018-03-22 9:25 ` Oliver [this message] 2018-03-22 9:25 ` Oliver 2018-03-22 11:25 ` Gabriel Paubert 2018-03-22 11:25 ` Gabriel Paubert 2018-03-22 10:37 ` David Laight 2018-03-22 10:37 ` David Laight 2018-03-22 4:24 ` Benjamin Herrenschmidt 2018-03-22 4:24 ` Benjamin Herrenschmidt 2018-03-22 10:15 ` Oliver 2018-03-22 10:15 ` Oliver 2018-03-22 13:52 ` Benjamin Herrenschmidt 2018-03-22 13:52 ` Benjamin Herrenschmidt 2018-03-22 17:51 ` Sinan Kaya 2018-03-22 17:51 ` Sinan Kaya 2018-03-23 0:16 ` Benjamin Herrenschmidt 2018-03-23 0:16 ` Benjamin Herrenschmidt 2018-03-23 13:42 ` Sinan Kaya 2018-03-23 13:42 ` Sinan Kaya 2018-03-24 1:22 ` Benjamin Herrenschmidt 2018-03-24 1:22 ` Benjamin Herrenschmidt 2018-03-24 15:06 ` Sinan Kaya 2018-03-24 15:06 ` Sinan Kaya 2018-03-26 11:44 ` Will Deacon 2018-03-26 11:44 ` Will Deacon 2018-03-26 12:11 ` okaya 2018-03-26 12:11 ` okaya 2018-03-26 12:42 ` Sinan Kaya 2018-03-26 12:42 ` Sinan Kaya 2018-03-23 16:35 ` Jason Gunthorpe 2018-03-23 16:35 ` Jason Gunthorpe 2018-03-24 1:23 ` Benjamin Herrenschmidt 2018-03-24 1:23 ` Benjamin Herrenschmidt 2018-03-26 11:08 ` David Laight 2018-03-26 11:08 ` David Laight 2018-03-26 16:54 ` Jason Gunthorpe 2018-03-26 16:54 ` Jason Gunthorpe 2018-03-26 19:44 ` Arnd Bergmann 2018-03-26 19:44 ` Arnd Bergmann 2018-03-26 20:25 ` Jason Gunthorpe 2018-03-26 20:25 ` Jason Gunthorpe 2018-03-26 20:43 ` Arnd Bergmann 2018-03-26 20:43 ` Arnd Bergmann 2018-03-26 21:09 ` Jason Gunthorpe 2018-03-26 21:09 ` Jason Gunthorpe 2018-03-26 21:30 ` Arnd Bergmann 2018-03-26 21:30 ` Arnd Bergmann 2018-03-26 21:46 ` Sinan Kaya 2018-03-26 21:46 ` Sinan Kaya 2018-03-26 22:01 ` Benjamin Herrenschmidt 2018-03-26 22:01 ` Benjamin Herrenschmidt 2018-03-26 22:08 ` Sinan Kaya 2018-03-26 22:08 ` Sinan Kaya 2018-03-26 22:28 ` Benjamin Herrenschmidt 2018-03-26 22:28 ` Benjamin Herrenschmidt 2018-03-26 22:27 ` Jason Gunthorpe 2018-03-26 22:27 ` Jason Gunthorpe 2018-03-26 22:36 ` Benjamin Herrenschmidt 2018-03-26 22:36 ` Benjamin Herrenschmidt 2018-03-26 22:42 ` Benjamin Herrenschmidt 2018-03-26 22:42 ` Benjamin Herrenschmidt 2018-03-26 22:50 ` Jason Gunthorpe 2018-03-26 22:50 ` Jason Gunthorpe 2018-03-26 23:59 ` Benjamin Herrenschmidt 2018-03-26 23:59 ` Benjamin Herrenschmidt 2018-03-27 1:39 ` Jason Gunthorpe 2018-03-27 1:39 ` Jason Gunthorpe 2018-03-27 7:56 ` Arnd Bergmann 2018-03-27 7:56 ` Arnd Bergmann 2018-03-27 8:56 ` Benjamin Herrenschmidt 2018-03-27 8:56 ` Benjamin Herrenschmidt 2018-03-27 9:44 ` Arnd Bergmann 2018-03-27 9:44 ` Arnd Bergmann 2018-03-27 10:00 ` Will Deacon 2018-03-27 10:00 ` Will Deacon 2018-03-27 11:23 ` Benjamin Herrenschmidt 2018-03-27 11:23 ` Benjamin Herrenschmidt 2018-03-27 12:22 ` okaya 2018-03-27 12:22 ` okaya 2018-03-27 14:12 ` Jason Gunthorpe 2018-03-27 14:12 ` Jason Gunthorpe 2018-03-27 21:27 ` Benjamin Herrenschmidt 2018-03-27 21:27 ` Benjamin Herrenschmidt 2018-03-27 9:57 ` Will Deacon 2018-03-27 9:57 ` Will Deacon 2018-03-27 10:05 ` Arnd Bergmann 2018-03-27 10:05 ` Arnd Bergmann 2018-03-27 10:09 ` Will Deacon 2018-03-27 10:09 ` Will Deacon 2018-03-27 10:53 ` Arnd Bergmann 2018-03-27 10:53 ` Arnd Bergmann 2018-03-27 11:02 ` Will Deacon 2018-03-27 11:02 ` Will Deacon 2018-03-27 11:05 ` Arnd Bergmann 2018-03-27 11:05 ` Arnd Bergmann 2018-03-27 11:25 ` Benjamin Herrenschmidt 2018-03-27 11:25 ` Benjamin Herrenschmidt 2018-03-27 13:20 ` David Laight 2018-03-27 13:20 ` David Laight 2018-03-27 13:46 ` Sinan Kaya 2018-03-27 13:46 ` Sinan Kaya 2018-03-27 14:36 ` Will Deacon 2018-03-27 14:36 ` Will Deacon 2018-03-27 21:29 ` Benjamin Herrenschmidt 2018-03-27 21:29 ` Benjamin Herrenschmidt 2018-03-28 8:53 ` Will Deacon 2018-03-28 8:53 ` Will Deacon 2018-03-28 9:00 ` David Laight 2018-03-28 9:00 ` David Laight 2018-03-28 9:09 ` Will Deacon 2018-03-28 9:09 ` Will Deacon 2018-03-28 9:56 ` Benjamin Herrenschmidt 2018-03-28 9:56 ` Benjamin Herrenschmidt 2018-03-28 9:50 ` Benjamin Herrenschmidt 2018-03-28 9:50 ` Benjamin Herrenschmidt 2018-03-28 9:55 ` Arnd Bergmann 2018-03-28 9:55 ` Arnd Bergmann 2018-03-28 10:01 ` Benjamin Herrenschmidt 2018-03-28 10:01 ` Benjamin Herrenschmidt 2018-03-28 10:13 ` Will Deacon 2018-03-28 10:13 ` Will Deacon 2018-03-28 16:57 ` Jason Gunthorpe 2018-03-28 16:57 ` Jason Gunthorpe 2018-03-29 9:19 ` Will Deacon 2018-03-29 9:19 ` Will Deacon 2018-03-29 14:45 ` Jason Gunthorpe 2018-03-29 14:45 ` Jason Gunthorpe 2018-03-29 14:58 ` David Laight 2018-03-29 14:58 ` David Laight 2018-03-29 16:40 ` Jason Gunthorpe 2018-03-29 16:40 ` Jason Gunthorpe 2018-03-27 21:24 ` Benjamin Herrenschmidt 2018-03-27 21:24 ` Benjamin Herrenschmidt 2018-03-27 11:21 ` Benjamin Herrenschmidt 2018-03-27 11:21 ` Benjamin Herrenschmidt 2018-03-27 9:42 ` Will Deacon 2018-03-27 9:42 ` Will Deacon 2018-03-27 11:20 ` Benjamin Herrenschmidt 2018-03-27 11:20 ` Benjamin Herrenschmidt 2018-03-27 11:24 ` Will Deacon 2018-03-27 11:24 ` Will Deacon 2018-03-27 14:24 ` Jason Gunthorpe 2018-03-27 14:24 ` Jason Gunthorpe 2018-03-27 14:16 ` Jason Gunthorpe 2018-03-27 14:16 ` Jason Gunthorpe 2018-03-26 22:00 ` Benjamin Herrenschmidt 2018-03-26 22:00 ` Benjamin Herrenschmidt 2018-03-27 14:46 ` Sinan Kaya 2018-03-27 15:01 ` Jose Abreu 2018-03-27 15:10 ` Will Deacon 2018-03-27 18:54 ` Alexander Duyck 2018-03-27 19:54 ` Arnd Bergmann 2018-03-27 19:54 ` Arnd Bergmann 2018-03-27 20:46 ` Arnd Bergmann 2018-03-27 20:46 ` Arnd Bergmann 2018-03-27 21:33 ` Benjamin Herrenschmidt 2018-03-28 0:39 ` Linus Torvalds 2018-03-28 1:03 ` Benjamin Herrenschmidt 2018-03-28 2:51 ` Linus Torvalds 2018-03-28 3:24 ` Sinan Kaya 2018-03-28 4:41 ` Benjamin Herrenschmidt 2018-03-28 6:14 ` Linus Torvalds 2018-03-28 11:41 ` okaya 2018-03-28 15:13 ` Benjamin Herrenschmidt 2018-03-28 15:55 ` David Miller 2018-03-28 16:23 ` Nicholas Piggin 2018-03-28 21:31 ` Benjamin Herrenschmidt 2018-03-28 22:09 ` Nicholas Piggin 2018-03-29 9:20 ` Will Deacon 2018-03-29 13:56 ` Sinan Kaya 2018-03-29 14:04 ` David Miller 2018-03-29 16:29 ` Arnd Bergmann 2018-03-29 16:59 ` Sinan Kaya 2018-03-30 1:40 ` Benjamin Herrenschmidt 2018-04-02 13:01 ` Sinan Kaya 2018-03-28 4:33 ` Benjamin Herrenschmidt 2018-03-28 6:26 ` Linus Torvalds 2018-03-28 6:42 ` Benjamin Herrenschmidt 2018-03-28 6:53 ` Linus Torvalds 2018-03-28 6:53 ` Linus Torvalds 2018-03-28 6:53 ` Linus Torvalds 2018-03-28 6:56 ` Benjamin Herrenschmidt 2018-03-28 7:11 ` Arnd Bergmann 2018-03-28 7:42 ` Benjamin Herrenschmidt 2018-03-28 9:07 ` Will Deacon 2018-03-28 9:56 ` Benjamin Herrenschmidt 2018-03-28 10:13 ` Aw: " Lino Sanfilippo 2018-03-28 10:20 ` Benjamin Herrenschmidt 2018-03-28 11:30 ` David Laight 2018-03-28 11:30 ` David Laight 2018-03-28 15:12 ` Benjamin Herrenschmidt 2018-03-28 16:16 ` David Laight 2018-03-28 16:16 ` David Laight 2018-03-28 1:21 ` Benjamin Herrenschmidt 2018-03-27 21:35 ` Benjamin Herrenschmidt 2018-03-26 21:26 ` Benjamin Herrenschmidt 2018-03-26 21:26 ` Benjamin Herrenschmidt 2018-03-27 21:54 Alexander Duyck 2018-03-27 22:35 ` Sinan Kaya 2018-03-27 23:43 ` Benjamin Herrenschmidt
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAOSf1CE+gsoZB-YNYdx=oNmrdCje4yxiz76hSb4UQ4F5cL8SqQ@mail.gmail.com' \ --to=oohall@gmail.com \ --cc=David.Laight@aculab.com \ --cc=linux-rdma@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=okaya@codeaurora.org \ --cc=paubert@iram.es \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.