linux-arch.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Hector Martin <marcan@marcan.st>
To: Will Deacon <will@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org,
	Marc Zyngier <maz@kernel.org>, Rob Herring <robh@kernel.org>,
	Arnd Bergmann <arnd@kernel.org>, Olof Johansson <olof@lixom.net>,
	Krzysztof Kozlowski <krzk@kernel.org>,
	Mark Kettenis <mark.kettenis@xs4all.nl>,
	Tony Lindgren <tony@atomide.com>,
	Mohamed Mediouni <mohamed.mediouni@caramail.com>,
	Stan Skowronek <stan@corellium.com>,
	Alexander Graf <graf@amazon.com>,
	Linus Walleij <linus.walleij@linaro.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Andy Shevchenko <andy.shevchenko@gmail.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Christoph Hellwig <hch@infradead.org>,
	"David S. Miller" <davem@davemloft.net>,
	devicetree@vger.kernel.org, linux-serial@vger.kernel.org,
	linux-doc@vger.kernel.org, linux-samsung-soc@vger.kernel.org,
	linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [RFT PATCH v3 16/27] irqchip/apple-aic: Add support for the Apple Interrupt Controller
Date: Thu, 1 Apr 2021 22:16:30 +0900	[thread overview]
Message-ID: <5ff8eef3-6943-d3c8-cd6f-3dcb44158fab@marcan.st> (raw)
In-Reply-To: <20210329120442.GA3636@willie-the-truck>

Hi Will,

On 29/03/2021 21.04, Will Deacon wrote:
>> One CPU still needs to be able to mutate the flags of another CPU to fire an
>> IPI; AIUI the per-cpu ops are *not* atomic for concurrent access by multiple
>> CPUs, and in fact there is no API for that, only for "this CPU".
> 
> Huh, I really thought we had an API for that, but you're right. Oh well! But
> I'd still suggest a per-cpu atomic_t in that case, rather than the array.

Yeah, after digging into the per-cpu stuff earlier and understanding how 
it works, I agree that a per-cpu atomic makes sense here. Switched it to 
that (which simplified out a bunch of smp_processor_id() calls too). Thanks!

>>> I think a more idiomatic (and portable) way to do this would be to use
>>> the relaxed accessors, but with smp_mb__after_atomic() between them. Do you
>>> have a good reason for _not_ doing it like that?
>>
>> Not particularly, other than symmetry with the case below.
> 
> I think it would be better not to rely on arm64-specific ordering unless
> there's a good reason to.

Sounds reasonable, I'll switch to the barrier version.

>> We do need the return data here, and the release semantics (or another
>> barrier before it). But the read below can be made relaxed and a barrier
>> used instead, and then the same patern above except with a plain
>> atomic_or().
> 
> Yes, I think using atomic_fetch_or() followed by atomic_read() would be
> best (obviously with the relevant comments!)

atomic_fetch_or_release is sufficient here (atomic_fetch_or is stronger; 
atomic_fetch_or_relaxed would not be strong enough as this needs to be 
ordered after any writes prior to sending the IPI; in this case release 
semantics also make logical sense).

>> It is ordered, right? As the comment says, it "needs to be ordered after the
>> aic_ic_write() above". atomic_fetch_andnot() is *supposed* to be fully
>> ordered and that should include against the writel_relaxed() on
>> AIC_IPI_FLAG. On ARM it turns out it's not quite fully ordered, but the
>> acquire semantics of the read half are sufficient for this case, as they
>> guarantee the flags are always read after the FIQ has been ACKed.
> 
> Sorry, I missed that the answer to my question was already written in the
> comment. However, I'm still a bit unsure about whether the memory barriers
> give you what you need here. The barrier in atomic_fetch_andnot() will
> order the previous aic_ic_write(AIC_IPI_ACK) for the purposes of other
> CPUs reading those locations, but it doesn't say anything about when the
> interrupt controller actually changes state after the Ack.
> 
> Given that the AIC is mapped Device-nGnRnE, the Arm ARM offers:
> 
>    | Additionally, for Device-nGnRnE memory, a read or write of a Location
>    | in a Memory-mapped peripheral that exhibits side-effects is complete
>    | only when the read or write both:
>    |
>    | * Can begin to affect the state of the Memory-mapped peripheral.
>    | * Can trigger all associated side-effects, whether they affect other
>    |   peripheral devices, PEs, or memory.
> 
> so without AIC documentation I can't tell whether completion of the Ack write
> just begins the process of an Ack (in which case we might need something like
> a read-back), or whether the write response back from the AIC only occurs once
> the Ack has taken effect. Any ideas?

Ahh, you're talking about latency within AIC itself... I obviously don't 
have an authoritative answer to this, though the hardware designer in me 
wants to say this really ought to be single-cycle type stuff that isn't 
internally pipelined in a way that would create races.

I tried to set up an SMP test case for the atomic-to-AIC sequence in 
m1n1, but unfortunately I couldn't hit the race window in deliberately 
racy code (i.e. ack after clearing flags) without widening it even 
further with at least one dummy load in between, and of course I didn't 
experience any races with the proper code either.

What I can say is that a simple set IPI; ack IPI (in adjacent str 
instructions) sequence always yields a cleared IPI, and the converse 
always yields a set IPI. So if there is latency to the operations it 
seems it would at least be the same for sets and acks and would imply 
readbacks block, which should still yield equivalently correct results. 
But of course this is a single-CPU test, so it is not fully 
representative of what could happen in an SMP scenario.

At this point all I can say is I'm inclined to shrug and say we have no 
evidence of this being something that can happen, and it shouldn't in 
sane hardware, and hope for the best :-)

Thanks,
-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

  reply	other threads:[~2021-04-01 17:46 UTC|newest]

Thread overview: 136+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-04 21:38 [RFT PATCH v3 00/27] Apple M1 SoC platform bring-up Hector Martin
2021-03-04 21:38 ` [RFT PATCH v3 01/27] arm64: Cope with CPUs stuck in VHE mode Hector Martin
2021-03-24 18:05   ` Will Deacon
2021-03-24 20:00     ` Marc Zyngier
2021-03-26  7:54       ` Hector Martin
2021-03-04 21:38 ` [RFT PATCH v3 02/27] dt-bindings: vendor-prefixes: Add apple prefix Hector Martin
2021-03-08 20:26   ` Rob Herring
2021-03-04 21:38 ` [RFT PATCH v3 03/27] dt-bindings: arm: apple: Add bindings for Apple ARM platforms Hector Martin
2021-03-05 10:16   ` Linus Walleij
2021-03-08 20:27   ` Rob Herring
2021-03-04 21:38 ` [RFT PATCH v3 04/27] dt-bindings: arm: cpus: Add apple,firestorm & icestorm compatibles Hector Martin
2021-03-08 20:27   ` Rob Herring
2021-03-04 21:38 ` [RFT PATCH v3 05/27] arm64: cputype: Add CPU implementor & types for the Apple M1 cores Hector Martin
2021-03-24 18:13   ` Will Deacon
2021-03-04 21:38 ` [RFT PATCH v3 06/27] dt-bindings: timer: arm,arch_timer: Add interrupt-names support Hector Martin
2021-03-05 10:18   ` Linus Walleij
2021-03-08 11:12   ` Marc Zyngier
2021-03-08 17:14   ` Tony Lindgren
2021-03-08 20:38   ` Rob Herring
2021-03-08 22:42     ` Marc Zyngier
2021-03-09 16:11       ` Rob Herring
2021-03-09 20:28         ` Hector Martin
2021-03-04 21:38 ` [RFT PATCH v3 07/27] arm64: arch_timer: implement support for interrupt-names Hector Martin
2021-03-05 10:19   ` Linus Walleij
2021-03-08 11:13   ` Marc Zyngier
2021-03-04 21:38 ` [RFT PATCH v3 08/27] asm-generic/io.h: Add a non-posted variant of ioremap() Hector Martin
2021-03-05 14:45   ` Andy Shevchenko
2021-03-05 15:19     ` Hector Martin
2021-03-08 11:20   ` Marc Zyngier
2021-03-24 18:12   ` Will Deacon
2021-03-24 19:09     ` Arnd Bergmann
2021-03-25 14:07       ` Hector Martin
2021-03-25 14:49         ` Will Deacon
2021-03-04 21:38 ` [RFT PATCH v3 09/27] docs: driver-api: device-io: Document I/O access functions Hector Martin
2021-03-05 10:22   ` Linus Walleij
2021-03-04 21:38 ` [RFT PATCH v3 10/27] docs: driver-api: device-io: Document ioremap() variants & access funcs Hector Martin
2021-03-05 10:25   ` Linus Walleij
2021-03-05 15:09     ` Andy Shevchenko
2021-03-05 15:51       ` Arnd Bergmann
2021-03-09 20:29         ` Hector Martin
2021-03-04 21:38 ` [RFT PATCH v3 11/27] arm64: Implement ioremap_np() to map MMIO as nGnRnE Hector Martin
2021-03-08 11:22   ` Marc Zyngier
2021-03-24 18:18   ` Will Deacon
2021-03-04 21:38 ` [RFT PATCH v3 12/27] of/address: Add infrastructure to declare MMIO as non-posted Hector Martin
2021-03-05 10:28   ` Linus Walleij
2021-03-05 15:13   ` Andy Shevchenko
2021-03-05 15:55     ` Hector Martin
2021-03-05 16:08       ` Andy Shevchenko
2021-03-05 16:43         ` Arnd Bergmann
2021-03-05 17:19           ` Hector Martin
2021-03-05 16:05     ` Rob Herring
2021-03-05 17:39   ` Rob Herring
2021-03-05 18:18     ` Hector Martin
2021-03-05 21:17       ` Arnd Bergmann
2021-03-08 15:56         ` Rob Herring
2021-03-08 20:29           ` Arnd Bergmann
2021-03-08 21:13             ` Rob Herring
2021-03-08 21:56               ` Arnd Bergmann
2021-03-09 15:48                 ` Rob Herring
2021-03-09 20:23                   ` Hector Martin
2021-03-09 22:06                     ` Rob Herring
2021-03-10  8:26                       ` Hector Martin
2021-03-10 17:01                         ` Rob Herring
2021-03-11  9:12                           ` Arnd Bergmann
2021-03-11 12:11                             ` Hector Martin
2021-03-11 13:35                               ` Arnd Bergmann
2021-03-11 16:07                             ` Rob Herring
2021-03-11 16:48                               ` Arnd Bergmann
2021-03-11 18:10                                 ` Rob Herring
2021-03-12 10:20                                   ` Arnd Bergmann
2021-03-09 11:14               ` Linus Walleij
2021-03-09 12:41                 ` Arnd Bergmann
2021-03-09 15:40                   ` Linus Walleij
2021-03-04 21:38 ` [RFT PATCH v3 13/27] arm64: Add Apple vendor-specific system registers Hector Martin
2021-03-24 18:38   ` Will Deacon
2021-03-24 18:59     ` Mark Rutland
2021-03-24 19:04       ` Will Deacon
2021-03-26  6:23         ` Hector Martin
2021-03-04 21:38 ` [RFT PATCH v3 14/27] arm64: move ICH_ sysreg bits from arm-gic-v3.h to sysreg.h Hector Martin
2021-03-08 11:39   ` Marc Zyngier
2021-03-24 18:23   ` Will Deacon
2021-03-04 21:38 ` [RFT PATCH v3 15/27] dt-bindings: interrupt-controller: Add DT bindings for apple-aic Hector Martin
2021-03-08 21:16   ` Rob Herring
2021-03-04 21:38 ` [RFT PATCH v3 16/27] irqchip/apple-aic: Add support for the Apple Interrupt Controller Hector Martin
2021-03-05 15:05   ` Andy Shevchenko
2021-03-08 11:50     ` Marc Zyngier
2021-03-08 12:02       ` Andy Shevchenko
2021-03-26 13:40     ` Hector Martin
2021-03-08 13:31   ` Marc Zyngier
2021-03-26  7:57     ` Hector Martin
2021-03-24 19:57   ` Will Deacon
2021-03-26  8:58     ` Hector Martin
2021-03-29 12:04       ` Will Deacon
2021-04-01 13:16         ` Hector Martin [this message]
2021-03-04 21:38 ` [RFT PATCH v3 17/27] arm64: Kconfig: Introduce CONFIG_ARCH_APPLE Hector Martin
2021-03-08 15:35   ` Marc Zyngier
2021-03-09 20:30     ` Hector Martin
2021-03-04 21:38 ` [RFT PATCH v3 18/27] tty: serial: samsung_tty: Separate S3C64XX ops structure Hector Martin
2021-03-05 10:30   ` Krzysztof Kozlowski
2021-03-04 21:38 ` [RFT PATCH v3 19/27] tty: serial: samsung_tty: Add ucon_mask parameter Hector Martin
2021-03-05 10:34   ` Krzysztof Kozlowski
2021-03-04 21:38 ` [RFT PATCH v3 20/27] tty: serial: samsung_tty: Add s3c24xx_port_type Hector Martin
2021-03-05 10:49   ` Krzysztof Kozlowski
2021-03-04 21:38 ` [RFT PATCH v3 21/27] tty: serial: samsung_tty: IRQ rework Hector Martin
2021-03-05 10:51   ` Krzysztof Kozlowski
2021-03-05 15:17   ` Andy Shevchenko
2021-03-05 16:16     ` Hector Martin
2021-03-05 16:20       ` Andy Shevchenko
2021-03-05 16:29         ` Hector Martin
2021-03-07 11:34           ` Krzysztof Kozlowski
2021-03-07 16:01             ` Arnd Bergmann
2021-03-07 19:51               ` Krzysztof Kozlowski
2021-03-04 21:38 ` [RFT PATCH v3 22/27] tty: serial: samsung_tty: Use devm_ioremap_resource Hector Martin
2021-03-05 10:54   ` Krzysztof Kozlowski
2021-03-05 15:19     ` Andy Shevchenko
2021-03-04 21:38 ` [RFT PATCH v3 23/27] dt-bindings: serial: samsung: Add apple,s5l-uart compatible Hector Martin
2021-03-08 21:17   ` Rob Herring
2021-03-04 21:38 ` [RFT PATCH v3 24/27] tty: serial: samsung_tty: Add support for Apple UARTs Hector Martin
2021-03-05 10:58   ` Krzysztof Kozlowski
2021-03-05 15:28   ` Andy Shevchenko
2021-03-05 17:04     ` Hector Martin
2021-03-07 11:40       ` Krzysztof Kozlowski
2021-03-04 21:39 ` [RFT PATCH v3 25/27] tty: serial: samsung_tty: Add earlycon " Hector Martin
2021-03-05 10:55   ` Krzysztof Kozlowski
2021-03-10 23:11   ` Linus Walleij
2021-03-04 21:39 ` [RFT PATCH v3 26/27] dt-bindings: display: Add apple,simple-framebuffer Hector Martin
2021-03-08 21:18   ` Rob Herring
2021-03-09 16:37   ` Linus Walleij
2021-03-09 20:35     ` Hector Martin
2021-03-04 21:39 ` [RFT PATCH v3 27/27] arm64: apple: Add initial Apple Mac mini (M1, 2020) devicetree Hector Martin
2021-03-05 11:03   ` Krzysztof Kozlowski
2021-03-05 11:14     ` Hector Martin
2021-03-05 11:45       ` Krzysztof Kozlowski
2021-03-05 15:59       ` Mark Kettenis
2021-03-05 16:50         ` Hector Martin
2021-03-05 10:11 ` [RFT PATCH v3 00/27] Apple M1 SoC platform bring-up Hector Martin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5ff8eef3-6943-d3c8-cd6f-3dcb44158fab@marcan.st \
    --to=marcan@marcan.st \
    --cc=andy.shevchenko@gmail.com \
    --cc=arnd@kernel.org \
    --cc=catalin.marinas@arm.com \
    --cc=corbet@lwn.net \
    --cc=davem@davemloft.net \
    --cc=devicetree@vger.kernel.org \
    --cc=graf@amazon.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hch@infradead.org \
    --cc=krzk@kernel.org \
    --cc=linus.walleij@linaro.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-samsung-soc@vger.kernel.org \
    --cc=linux-serial@vger.kernel.org \
    --cc=mark.kettenis@xs4all.nl \
    --cc=mark.rutland@arm.com \
    --cc=maz@kernel.org \
    --cc=mohamed.mediouni@caramail.com \
    --cc=olof@lixom.net \
    --cc=robh@kernel.org \
    --cc=stan@corellium.com \
    --cc=tony@atomide.com \
    --cc=will@kernel.org \
    --subject='Re: [RFT PATCH v3 16/27] irqchip/apple-aic: Add support for the Apple Interrupt Controller' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox