IOMMU Archive on lore.kernel.org
 help / color / Atom feed
* PCIe DMA addressing issues on Raspberry Pi 4
@ 2019-09-03 15:43 Nicolas Saenz Julienne
  0 siblings, 0 replies; only message in thread
From: Nicolas Saenz Julienne @ 2019-09-03 15:43 UTC (permalink / raw)
  To: Rob Herring, robin.murphy
  Cc: Stefan Wahren, devicetree, mbrugger, iommu, Christoph Hellwig,
	linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 2489 bytes --]

Hi all, sorry for the long read, I kept it as short as possible.

So, the wrapper around the PCIe block available on the Raspberry Pi 4 has a bug
preventing it from accessing anything beyond the first 3G of ram [1]. I'm
trying to figure out the best way to integrate this upstream.

Note that the only thing behind the PCIe bus is an USB3 chip. The bus is not
exposed to users directly.

I see two options:

- Isolate the PCIe block on it's own interconnect. IMO this could be acceptable
  as it's arguable that the bug is not really in the PCIe block.  I set the
  interconnect's dma-range size to 2GB instead of 3GB as dma masks don't play
  well with non power of two DMA constraints:

  buggy-scb {
	compatible = "simple-bus";
	dma-ranges = <0x0 0x00000000 0x0 0x00000000 0x800000000>;

	pcie {
		compatible = "brcm,bcm2711-pcie";
		dma-ranges = <0x02000000 0x0 0x00000000 0x0 0x00000000
			      0x1 0x00000000>;
		[...]

	};
  };

  scb {
	compatible = "simple-bus";
	dma-ranges = <0x0 0x00000000 0x0 0x00000000 0xfc0000000>;

	eth0 {
		[...]
	};

	[...]
  };

  With this setup the PCIe devices should behave correctly as long as they
  don't play with their own DMA masks.

- Configure PCIe's inbound view of memory taking into account the buggy
  interconnect:

  scb {
	compatible = "simple-bus";
	dma-ranges = <0x0 0x00000000 0x0 0x00000000 0xfc000000>;

	pcie {
		compatible = "brcm,bcm2711-pcie";
		dma-ranges = <0x02000000 0x0 0x00000000 0x0 0x00000000
			      0x0 0x80000000>;
		[...]

	};

	eth0 {
		[...]
	};

	[...]
  };

  The downside of this is that of_dma_configure() doesn't play well with PCI
  devices. of_dma_configure() expects a device's OF node, yet the PCI core
  passes the bridge's OF node, as the device has none. The result is
  of_dma_configure() ignores PCI's dma-ranges. Solving this is not trivial.
  IMO the simplest solution would be to create a distinct OF node on PCI
  bridges, child of the actual PCI root complex.  FYI this was already an issue
  some years ago [2].

  This solution also suffers from devices setting their own DMA masks.

If you're curious abot the downstream kernel, they use their custom buffer
bouncing code, which AFAIK is something we're trying to get rid of.

Any comments? Alternative solutions?

Thanks,
Nicolas

[1] https://www.spinics.net/lists/arm-kernel/msg740693.html
[2] https://patchwork.kernel.org/patch/9650345/#20294961


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, back to index

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-03 15:43 PCIe DMA addressing issues on Raspberry Pi 4 Nicolas Saenz Julienne

IOMMU Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-iommu/0 linux-iommu/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-iommu linux-iommu/ https://lore.kernel.org/linux-iommu \
		iommu@lists.linux-foundation.org iommu@archiver.kernel.org
	public-inbox-index linux-iommu


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.linux-foundation.lists.iommu


AGPL code for this site: git clone https://public-inbox.org/ public-inbox