From: Mark Kettenis <mark.kettenis@xs4all.nl>
To: "Sven Peter" <sven@svenpeter.dev>
Cc: iommu@lists.linux-foundation.org, joro@8bytes.org,
will@kernel.org, robin.murphy@arm.com, robh+dt@kernel.org,
arnd@kernel.org, marcan@marcan.st, maz@kernel.org,
mohamed.mediouni@caramail.com, stan@corellium.com,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, devicetree@vger.kernel.org
Subject: Re: [PATCH 0/3] Apple M1 DART IOMMU driver
Date: Sun, 21 Mar 2021 19:35:04 +0100 (CET) [thread overview]
Message-ID: <c1bcc0be8ae6e500@bloch.sibelius.xs4all.nl> (raw)
In-Reply-To: <8360b3b3-296c-450d-abc3-bb47159bf4e1@www.fastmail.com> (sven@svenpeter.dev)
> Date: Sun, 21 Mar 2021 18:22:15 +0100
> From: "Sven Peter" <sven@svenpeter.dev>
>
> Hi Mark,
>
> On 21. Mar 2021, at 17:00, Mark Kettenis <mark.kettenis@xs4all.nl>
> wrote:
>
> I don't think the first option is going to work for PCIe. PCIe
> devices will have to use "iommu-map" properties to map PCI devices to
> the right iommu, and the currently implementation seems to assume that
> #iommu-cells = <1>. The devictree binding[1] doesn't explicitly state
> that it relies on #iommu-cells = <1>, but it isn't clear how the
> rid-base to iommu-base mapping mechanism would work when that isn't
> the case.
>
> Now the PCIe DARTs are simpler and seem to have only one "instance"
> per DART. So if we keep #iommu-cells = <1> for those, you'd still be
> fine using the first approach.
>
> Good point, I guess that only leaves option two for now then.
> Having some DARTs use cells = <1> and others <2> sounds confusing to me.
Guess we do need to understand a little bit better how the USB DART
actually works. My hypothesis (based on our discussion on #asahi) is
that the XHCI host controller and the peripheral controller of the
DWC3 block use different DMA "streams" that are handled by the
different sub-DARTs.
The Corellium folks use a DART + sub-DART model in their driver and a
single node in the device tree that represents both. That might sense
since the error registers and interrupts are shared. Maybe it would
make sense to select the appropriate sub-DART based on the DMA stream
ID?
> As I mentioned before, not all DARTs support the full 32-bit aperture.
> In particular the PCIe DARTs support a smaller address-space. It is
> not clear whether this is a restriction of the PCIe host controller or
> the DART, but the Apple Device Tree has "vm-base" and "vm-size"
> properties that encode the base address and size of the aperture.
> These single-cell properties which is probably why for the USB DARTs
> only "vm-base" is given; since "vm-base" is 0, a 32-bit number
> wouldn't be able to encode the full aperture size. We could make them
> 64-bit numbers in the Linux device tree though and always be explicit
> about the size. Older Sun SPARC machines used a single "virtual-dma"
> property to encode the aperture. We could do someting similar. You
> would use this property to initialize domain->geometry.aperture_start
> and domain->geometry.aperture_end in diff 3/3 of this series.
>
> I think it would make sense to include this in this series, as this
> would make adding support for PCIe very easy, and PCIe gives you
> aupport for network (both wired and wireless) and the type-A USB ports
> on the mini.
>
> Agreed, I'd ideally like to converge on a device tree binding
> that won't have to change in the near future.
>
> I've tried to use an address space larger than 32bit and that seems to
> work for parts of the dwc3 controller but breaks for the xhci parts because
> the upper lines don't seem to be connected there (e.g. if xhci tries to
> use <0x1 0xffff0000> I get a fault for <0 0xffff0000>).
>
> Looking at other iommu drivers I have found the following two similar
> bindings:
Ah, missed those.
> qcom uses a ranges property with a 64bit address and 32 bit size [1]
>
> apps_iommu: iommu@1e20000 {
> ...
> ranges = <0 0x1e20000 0x40000>;
> ...
> };
Doesn't sound like a very good idea to me to repurpose "ranges" for
this...
>
> and tegra seems to support a dma-window property with 32bit address
> and size [2]
>
> smmu {
> [...]
> dma-window = <0 0x40000000>; /* IOVA start & length */
> [...]
> };
>
> I believe there already is of_get_dma_window to handle parsing this
> in the common iommu code [3] but I can't find any place using it.
> It's a little bit more complex that we need since it allows to specify the
> number of cells for both the address and the size but it should allow us to
> express all possible configurations:
>
> usb_dart {
> [ ... ]
> #dma-address-cells = <1>;
> #dma-size-cells = <2>;
> dma-window = <0 0x1 0x0>;
> [ ... ]
> };
> pcie_dart {
> [ ... ]
> #dma-address-cells = <1>;
> #dma-size-cells = <1>;
> dma-window = <0x100000 0x3fe00000>;
> [ ... ]
> };
>
> where #dma-address-cells and #dma-size-cells default to
> #address-cells and #size-cells respectively if I understand
> the code correctly. That way we could also just always use
> a 64bit address and size in the DT, e.g.
>
> pcie_dart {
> [ ... ]
> dma-window = <0 0x100000 0 0x3fe00000>;
> [ ... ]
> };
That sounds like a serious contender to me! Hopefully one of the
Linux kernel developers can give this some sort of blessing.
I think it would make sense for us to just rely on the #address-cells
and #size-cells defaults for the M1 device tree.
next prev parent reply other threads:[~2021-03-21 18:35 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-20 15:19 [PATCH 0/3] Apple M1 DART IOMMU driver Sven Peter
2021-03-20 15:19 ` [PATCH 1/3] iommu: io-pgtable: add DART pagetable format Sven Peter
2021-03-24 16:37 ` Robin Murphy
2021-03-25 20:47 ` Sven Peter
2021-03-20 15:20 ` [PATCH 2/3] dt-bindings: iommu: add DART iommu bindings Sven Peter
2021-03-22 0:15 ` Rob Herring
2021-03-22 18:16 ` Sven Peter
2021-03-21 16:00 ` [PATCH 0/3] Apple M1 DART IOMMU driver Mark Kettenis
2021-03-21 17:28 ` Sven Peter
[not found] ` <8360b3b3-296c-450d-abc3-bb47159bf4e1@www.fastmail.com>
2021-03-21 18:35 ` Mark Kettenis [this message]
2021-03-22 22:17 ` Sven Peter
2021-03-23 20:00 ` Mark Kettenis
2021-03-23 21:03 ` Sven Peter
2021-03-23 20:53 ` Rob Herring
2021-03-23 22:33 ` Mark Kettenis
2021-03-25 7:53 ` Sven Peter
2021-03-25 11:50 ` Robin Murphy
2021-03-25 20:49 ` Sven Peter
2021-03-27 15:33 ` Sven Peter
2021-03-25 21:41 ` Arnd Bergmann
2021-03-26 15:59 ` Mark Kettenis
2021-03-26 16:09 ` Arnd Bergmann
2021-03-26 16:10 ` Sven Peter
2021-03-26 16:38 ` Arnd Bergmann
2021-03-26 17:06 ` Sven Peter
2021-03-26 17:26 ` Mark Kettenis
2021-03-26 17:34 ` Robin Murphy
2021-03-26 17:51 ` Sven Peter
2021-03-26 19:59 ` Arnd Bergmann
2021-03-26 21:16 ` Mark Kettenis
2021-03-27 15:30 ` Sven Peter
2021-03-26 20:03 ` Arnd Bergmann
2021-03-26 21:13 ` Mark Kettenis
2021-03-24 15:29 ` Robin Murphy
2021-03-25 7:58 ` Sven Peter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c1bcc0be8ae6e500@bloch.sibelius.xs4all.nl \
--to=mark.kettenis@xs4all.nl \
--cc=arnd@kernel.org \
--cc=devicetree@vger.kernel.org \
--cc=iommu@lists.linux-foundation.org \
--cc=joro@8bytes.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=marcan@marcan.st \
--cc=maz@kernel.org \
--cc=mohamed.mediouni@caramail.com \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=stan@corellium.com \
--cc=sven@svenpeter.dev \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).