Re: [RFT PATCH v3 12/27] of/address: Add infrastructure to declare MMIO as non-posted

[Arnd Bergmann <arnd@kernel.org>]

On Thu, Mar 11, 2021 at 7:10 PM Rob Herring <robh@kernel.org> wrote:
>
> On Thu, Mar 11, 2021 at 9:48 AM Arnd Bergmann <arnd@kernel.org> wrote:
> >
> > On Thu, Mar 11, 2021 at 5:10 PM Rob Herring <robh@kernel.org> wrote:
> > > On Thu, Mar 11, 2021 at 2:12 AM Arnd Bergmann <arnd@kernel.org> wrote:
> > > > On Wed, Mar 10, 2021 at 6:01 PM Rob Herring <robh@kernel.org> wrote:
> > > > Ok, makes sense.
> > > >
> > > > Conceptually, I'd like to then see a check that verifies that the
> > > > property is only set for nodes whose parent also has it set, since
> > > > that is how AXI defines it: A bus can wait for the ack from its
> > > > child node, or it can acknowledge the write to its parent early.
> > > > However, this breaks down as soon as a bus does the early ack:
> > > > all its children by definition use posted writes (as seen by the
> > > > CPU), even if they wait for stores that come from other masters.
> > > >
> > > > Does this make sense to you?
> > >
> > > BTW, I don't think it's clear in this thread, but the current
> > > definition proposed for the spec[1] and schema is 'nonposted-mmio' is
> > > specific to 'simple-bus'. I like this restriction and we can expand
> > > where 'nonposted-mmio' is allowed later if needed.
> >
> > That sounds ok, as long as we can express everything for the mac
> > at the moment. Do we need to explicitly add a description to allow
> > the property in the root node in addition to simple-bus to be able
> > to enforce the rule about parent buses also having it?
>
> IMO it should not be allowed in the root node. That's a failure to
> define a bus node.

My interpretation would be that the root node defines the first bus
connected to the CPU(s) themselves, which may already have
posted writes. If writes on that bus are posted, then no child could
be non-posted.

I suppose it depends a bit on the mental model of what the nodes
refer to. If you say that there cannot be a device with registers
directly on the root node, but every bus defines its own space,
then we don't need this, but I think a lot of machines would break
if you try to enforce the rule that there cannot be devices on
the root node.

> Also, would that mean your memory has to be non-posted!?

Good question. You could argue that this should not be because you
don't want to use ioremap_np() flags but instead want this to be
normal cacheable memory instead of device memory.

On the other hand, you definitely don't want memory stores to be
posted, as that would break coherency between the CPUs when
a wmb() no longer has an effect.

       Arnd