Re: [RFT PATCH v3 12/27] of/address: Add infrastructure to declare MMIO as non-posted

[Arnd Bergmann <arnd@kernel.org>]

On Mon, Mar 8, 2021 at 4:56 PM Rob Herring <robh@kernel.org> wrote:
> On Fri, Mar 5, 2021 at 2:17 PM Arnd Bergmann <arnd@kernel.org> wrote:
> > On Fri, Mar 5, 2021 at 7:18 PM Hector Martin <marcan@marcan.st> wrote:
>
> > > > What's the code path using these functions on the M1 where we need to
> > > > return 'posted'? It's just downstream PCI mappings (PCI memory space),
> > > > right? Those would never hit these paths because they don't have a DT
> > > > node or if they do the memory space is not part of it. So can't the
> > > > check just be:
> > > >
> > > > bool of_mmio_is_nonposted(struct device_node *np)
> > > > {
> > > >      return np && of_machine_is_compatible("apple,arm-platform");
> > > > }
> > >
> > > Yes; the implementation was trying to be generic, but AIUI we don't need
> > > this on M1 because the PCI mappings don't go through this codepath, and
> > > nothing else needs posted mode. My first hack was something not too
> > > unlike this, then I was going to get rid of apple,arm-platform and just
> > > have this be a generic mechanism with the properties, but then we added
> > > the optimization to not do the lookups on other platforms, and now we're
> > > coming full circle... :-)
> >
> > I never liked the idea of having a list of platforms that need a
> > special hack, please let's not go back to that.
>
> I'm a fan of generic solutions as much as anyone, but not when there's
> a single user. Yes, there could be more, but we haven't seen any yet
> and Apple seems to have a knack for doing special things. I'm pretty
> sure posted vs. non-posted has been a possibility with AXI buses from
> the start, so it's not like this is a new thing we're going to see
> frequently on new platforms.

Ok, but if we make it a platform specific bit, I would prefer not
to do the IORESOURCE_MEM_NONPOSTED flag either but
instead keep the logic in the device drivers that call ioremap().

This is obviously more work for the drivers, but at least it keeps
the common code free of the hack while also allowing drivers to
use ioremap_np() intentionally on other platforms.

> The other situation I worry about here is another arch has implicitly
> defaulted to non-posted instead of posted. It could just be non-posted
> was what worked everywhere and Linux couldn't distinguish. Now someone
> sees we have this new posted vs. non-posted handling and can optimize
> some mappings on their platform and we have to have per arch defaults
> (like 'dma-coherent' now).

I think one of the dark secrets of MMIO is that a lot of drivers
get the posted behavior wrong by assuming that a writel() before
a spin_unlock() is protected by that unlock. This may in fact work
on many architectures but is broken on PCI and on local devices
for ARM.

Having a properly working (on non-PCI) ioremap_np() interface
would be nice here, as it could be used to document when drivers
rely on non-posted behavior, and cause the ioremap to fail when
running on architectures that don't support nonposted maps.

       Arnd