Linux-ARM-Kernel Archive on lore.kernel.org
 help / color / Atom feed
From: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
To: linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org,
	linux-kernel@vger.kernel.org
Cc: f.fainelli@gmail.com, arnd@arndb.de, narmstrong@baylibre.com,
	dwmw2@infradead.org, linux@armlinux.org.uk, hch@infradead.org,
	will@kernel.org, robh+dt@kernel.org, catalin.marinas@arm.com,
	robin.murphy@arm.com, ardb@kernel.org,
	Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
Subject: [RFC 00/13] Generic way of dealing with broken 64-bit buses
Date: Fri, 26 Feb 2021 15:02:52 +0100
Message-ID: <20210226140305.26356-1-nsaenzjulienne@suse.de> (raw)

BCM2711, Raspberry Pi 4's arm64 system on chip, contains a PCIe bus that can't
handle 64-bit accesses to its MMIO address space. The issue has already been
discussed here[1], and it turns out BCM2711 isn't the only broken device in the
wild.

In most cases, the solution to this issue is to convert writeq/readq() to into
their lo_hi/hi_lo variants and the eventual introduction of some amount of
locking. But that's not good enough for every device. For example, on some
arm's SMMU configurations atomic 64-bit accesses are mandatory. This series
tries to introduce a mechanism for drivers to be able to ascertain whether or
not they are allowed to perform 64-bit accesses.

The big question is the amount of granularity needed to deal with this
(think here of distro images):

- Build-time: if a broken platform included in the image, disallow any 64-bit
  access. Drivers that need 64-bit accesses could simply bypass the check and
  hope for the best. Imposes a performance penalty on otherwise well behaving
  platforms, and features that depend on 64bit access might be disabled
  unnecessarily. It's simple to implement, yet not very generic/future proof.

- Run-time: allow/disallow 64-bit accesses based on boot time checks (i.e.
  check which platform the kernel is running on). Gets rid of all the negative
  aspects imposed to well-behaving platforms. Well-behaving buses can't coexist
  with broken ones while using all features.

- Per-device: each device has its MMIO access properties and can take decisions
  based on its local bus. That said, I'm not aware of a system that absolutely
  needs this ATM.

This series implements the third option mainly as a proof of concept.
It's my personal preference on how to deal with this. That said, my main
aim ATM is to settle on a general approach.

Regards,
Nicolas

[1] https://lore.kernel.org/linux-arm-kernel/c188698ca0de3ed6c56a0cf7880e1578aa753077.camel@suse.de/

---

Nicolas Saenz Julienne (13):
  dt-bindings: Introduce 64bit-mmio-broken
  driver core: Introduce MMIO configuration
  of: device: Introduce of_mmio_configure()
  driver core: plafrom: Introduce platform_mmio_configure()
  pci: Introduce pci_mmio_configure()
  device core: Introduce dev_64bit_mmio_supported()
  arm64: Mark ARCH_MVEBU as needing broken 64bit MMIO support
  arm64: dts: marvell: armada-ap80x: Mark config-space bus as
    64bit-mmio-broken
  iommu/arm-smmu: Make use of dev_64bit_mmio_supported()
  iommu/arm-smmu-impl: Get rid of Marvell's implementation details
  arm64: Mark ARCH_BCM2835 as needing broken 64bit MMIO support
  ARM: dts: bcm2711: Mark PCIe bus as 64bit-mmio-broken
  scsi: megaraid: Make use of dev_64bit_mmio_supported()

 .../devicetree/bindings/common-properties.txt | 15 +++++++++++
 arch/Kconfig                                  |  8 ++++++
 arch/arm/boot/dts/bcm2711.dtsi                |  1 +
 arch/arm64/Kconfig.platforms                  |  2 ++
 arch/arm64/boot/dts/marvell/armada-ap80x.dtsi |  1 +
 drivers/base/dd.c                             |  6 +++++
 drivers/base/platform.c                       |  9 +++++++
 drivers/iommu/arm/arm-smmu/arm-smmu-impl.c    | 21 ---------------
 drivers/iommu/arm/arm-smmu/arm-smmu.c         |  9 +++++++
 drivers/iommu/arm/arm-smmu/arm-smmu.h         |  9 +++++--
 drivers/of/device.c                           | 19 ++++++++++++++
 drivers/pci/pci-driver.c                      | 26 +++++++++++++++++++
 drivers/scsi/megaraid/megaraid_sas_fusion.c   | 23 ++++++++--------
 include/linux/device.h                        | 20 ++++++++++++++
 include/linux/device/bus.h                    |  3 +++
 include/linux/of_device.h                     |  8 ++++++
 16 files changed, 145 insertions(+), 35 deletions(-)

-- 
2.30.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

             reply index

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-26 14:02 Nicolas Saenz Julienne [this message]
2021-02-26 14:02 ` [RFC 01/13] dt-bindings: Introduce 64bit-mmio-broken Nicolas Saenz Julienne
2021-02-26 14:02 ` [RFC 02/13] driver core: Introduce MMIO configuration Nicolas Saenz Julienne
2021-03-02 11:29   ` Robin Murphy
2021-03-02 14:09     ` Nicolas Saenz Julienne
2021-02-26 14:02 ` [RFC 03/13] of: device: Introduce of_mmio_configure() Nicolas Saenz Julienne
2021-02-26 14:02 ` [RFC 04/13] driver core: plafrom: Introduce platform_mmio_configure() Nicolas Saenz Julienne
2021-02-26 14:02 ` [RFC 05/13] pci: Introduce pci_mmio_configure() Nicolas Saenz Julienne
2021-02-26 14:02 ` [RFC 06/13] device core: Introduce dev_64bit_mmio_supported() Nicolas Saenz Julienne
2021-02-26 14:02 ` [RFC 07/13] arm64: Mark ARCH_MVEBU as needing broken 64bit MMIO support Nicolas Saenz Julienne
2021-02-26 14:03 ` [RFC 08/13] arm64: dts: marvell: armada-ap80x: Mark config-space bus as 64bit-mmio-broken Nicolas Saenz Julienne
2021-02-26 14:03 ` [RFC 09/13] iommu/arm-smmu: Make use of dev_64bit_mmio_supported() Nicolas Saenz Julienne
2021-03-02  9:32   ` Arnd Bergmann
2021-03-02 12:42     ` Nicolas Saenz Julienne
2021-03-02 11:07   ` Robin Murphy
2021-03-02 13:38     ` Nicolas Saenz Julienne
2021-03-03  8:44     ` Arnd Bergmann
2021-02-26 14:03 ` [RFC 10/13] iommu/arm-smmu-impl: Get rid of Marvell's implementation details Nicolas Saenz Julienne
2021-03-02 11:40   ` Robin Murphy
2021-03-02 14:06     ` Nicolas Saenz Julienne
2021-02-26 14:03 ` [RFC 11/13] arm64: Mark ARCH_BCM2835 as needing broken 64bit MMIO support Nicolas Saenz Julienne
2021-02-26 14:03 ` [RFC 12/13] ARM: dts: bcm2711: Mark PCIe bus as 64bit-mmio-broken Nicolas Saenz Julienne
2021-02-26 14:03 ` [RFC 13/13] scsi: megaraid: Make use of dev_64bit_mmio_supported() Nicolas Saenz Julienne
2021-02-26 14:30   ` Arnd Bergmann
2021-02-26 18:06     ` Arnd Bergmann

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210226140305.26356-1-nsaenzjulienne@suse.de \
    --to=nsaenzjulienne@suse.de \
    --cc=ardb@kernel.org \
    --cc=arnd@arndb.de \
    --cc=catalin.marinas@arm.com \
    --cc=devicetree@vger.kernel.org \
    --cc=dwmw2@infradead.org \
    --cc=f.fainelli@gmail.com \
    --cc=hch@infradead.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux@armlinux.org.uk \
    --cc=narmstrong@baylibre.com \
    --cc=robh+dt@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-ARM-Kernel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-arm-kernel/0 linux-arm-kernel/git/0.git
	git clone --mirror https://lore.kernel.org/linux-arm-kernel/1 linux-arm-kernel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-arm-kernel linux-arm-kernel/ https://lore.kernel.org/linux-arm-kernel \
		linux-arm-kernel@lists.infradead.org
	public-inbox-index linux-arm-kernel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.infradead.lists.linux-arm-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git