From: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
To: Robin Murphy <robin.murphy@arm.com>,
Neil Armstrong <narmstrong@baylibre.com>,
Florian Fainelli <f.fainelli@gmail.com>,
Christoph Hellwig <hch@infradead.org>
Cc: devicetree <devicetree@vger.kernel.org>,
Rob Herring <robh@kernel.org>,
linux-pci <linux-pci@vger.kernel.org>,
Bjorn Helgaas <helgaas@kernel.org>,
bcm-kernel-feedback-list <bcm-kernel-feedback-list@broadcom.com>,
Robin Murphy <robin.murphy@arm.con>,
linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
linux-rpi-kernel <linux-rpi-kernel@lists.infradead.org>
Subject: Re: RPi4 can't deal with 64 bit PCI accesses
Date: Thu, 25 Feb 2021 12:35:27 +0100 [thread overview]
Message-ID: <01091991523dac4c0c7e40f40e95c887af84f560.camel@suse.de> (raw)
In-Reply-To: <0cca5246-065b-b52e-7005-b1b5229922a7@arm.com>
[-- Attachment #1: Type: text/plain, Size: 3737 bytes --]
On Thu, 2021-02-25 at 11:10 +0000, Robin Murphy wrote:
> On 2021-02-25 10:29, Neil Armstrong wrote:
> > On 24/02/2021 21:35, Florian Fainelli wrote:
> > >
> > >
> > > On 2/24/2021 12:25 PM, Christoph Hellwig wrote:
> > > > On Wed, Feb 24, 2021 at 08:55:10AM -0800, Florian Fainelli wrote:
> > > > > > Working around kernel I/O accessors is all very well, but another
> > > > > > concern for PCI in particular is when things like framebuffer memory can
> > > > > > get mmap'ed into userspace (or even memremap'ed within the kernel). Even
> > > > > > in AArch32, compiled code may result in 64-bit accesses being generated
> > > > > > depending on how the CPU and interconnect handle LDRD/STRD/LDM/STM/etc.,
> > > > > > so it's basically not safe to ever let that happen at all.
> > > > >
> > > > > Agreed, this makes finding a generic solution a tiny bit harder. Do you
> > > > > have something in mind Nicolas?
> > > >
> > > > The only workable solution is a new
> > > >
> > > > bool 64bit_mmio_supported(void)
>
> Note that to be sufficiently generic this would have to be a per-device
> property - a system could have an affected PCIe root complex but still
> have other devices elsewhere in the SoC that can, or even need to, use
> 64-bit accesses.
Yes, that's what I had in mind myself. All in all, why penalize the rest of
busses in the system. What I'm planning is to introduce a '64bit-mmio-broken'
DT property that'll utimately live somwhere in 'struct device.'
WRT why not defaulting to 32-bit accesses for distro images if they support
RPi4. My *un-educated* guess is that, the performance penalty of checking for a
device flag is (way) lower than having to resort to two distinct write
operations with their assorted memory barriers. I'm sure you can
comment/correct me here.
> > > > check that is used like:
> > > >
> > > > if (64bit_mmio_supported())
> > > > readq(foodev->regs, REG_OFFSET);
> > > > else
> > > > lo_hi_readq(foodev->regs, REG_OFFSET);
> > > >
> > > > where 64bit_mmio_supported() return false for all 32-bit kernels,
> > > > true for all non-broken 64-bit kernels and is an actual function
> > > > for arm64 multiplatforms builds that include te RPi quirk.
> > > >
> > > > The above would then replace the existing magic from the
> > > > <linux/io-64-nonatomic-lo-hi.h> and <linux/io-64-nonatomic-hi-lo.h>
> > > > headers.
> > >
> > > That would work. The use case described by Robin is highly unlikely to
> > > exist on the Pi4 given that you cannot easily access the PCIe bus and
> > > plug an arbitrary GPU, so maybe there is nothing to do for framebuffer
> > > memory.
>
> Framebuffers are only the most obvious example - I don't feel the
> inclination to audit every driver/subsystem that can possibly make a
> non-iomem remapping or userspace mmap of a prefetchable BAR, but I'm
> sure there are more.
IIUC the only solution to the issue here is to disallow mmaping memory
belonging to a broken bus, right? In this case, the function above would do the
trick.
> > Erf, not really, with the compute module ATX/ITX boards are being designed
> > with a full PCIe connector like:
> > https://www.indiegogo.com/projects/over-board-raspberry-pi-4-mini-itx-motherboard/#/
>
> Right, this whole thread looks to have come about due to random
> endpoints getting connected to the exposed bus on compute modules. If it
> was an issue at all for the XHCI on standard Pi 4 boards I don't think
> people would just be starting to notice it now...
Indeed. For the record, here's the original complaint, although I'm sure others
exist: https://github.com/raspberrypi/linux/issues/4158
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
next prev parent reply other threads:[~2021-02-25 11:36 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-22 15:47 RPi4 can't deal with 64 bit PCI accesses Nicolas Saenz Julienne
2021-02-22 16:18 ` Ard Biesheuvel
2021-02-22 16:36 ` Nicolas Saenz Julienne
2021-02-22 16:56 ` Robin Murphy
2021-02-24 16:55 ` Florian Fainelli
2021-02-24 20:25 ` Christoph Hellwig
2021-02-24 20:35 ` Florian Fainelli
2021-02-25 10:29 ` Neil Armstrong
2021-02-25 11:10 ` Robin Murphy
2021-02-25 11:35 ` Nicolas Saenz Julienne [this message]
2021-02-26 5:32 ` Christoph Hellwig
2021-02-25 10:41 ` David Woodhouse
2021-02-22 17:55 ` Russell King - ARM Linux admin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=01091991523dac4c0c7e40f40e95c887af84f560.camel@suse.de \
--to=nsaenzjulienne@suse.de \
--cc=bcm-kernel-feedback-list@broadcom.com \
--cc=devicetree@vger.kernel.org \
--cc=f.fainelli@gmail.com \
--cc=hch@infradead.org \
--cc=helgaas@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-rpi-kernel@lists.infradead.org \
--cc=narmstrong@baylibre.com \
--cc=robh@kernel.org \
--cc=robin.murphy@arm.com \
--cc=robin.murphy@arm.con \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).