IOMMU Archive on lore.kernel.org
 help / color / Atom feed
From: Tomasz Figa <tfiga@chromium.org>
To: Rob Herring <robh@kernel.org>, Robin Murphy <robin.murphy@arm.com>
Cc: linux-devicetree <devicetree@vger.kernel.org>,
	heikki.krogerus@linux.intel.com,
	Saravana Kannan <saravanak@google.com>,
	suzuki.poulose@arm.com, Greg KH <gregkh@linuxfoundation.org>,
	lkml <linux-kernel@vger.kernel.org>,
	Bartosz Golaszewski <bgolaszewski@baylibre.com>,
	"list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>,
	Joerg Roedel <joro@8bytes.org>,
	" <iommu@lists.linux-foundation.org>,
	Nicolas Boichat <drinkcat@chromium.org>,
	Claire Chang <tientzu@chromium.org>,
	dan.j.williams@intel.com, Thierry Reding <treding@nvidia.com>,
	Frank Rowand <frowand.list@gmail.com>,
	Christoph Hellwig <hch@lst.de>
Subject: Re: [RFC v2 4/5] dt-bindings: of: Add plumbing for restricted DMA pool
Date: Tue, 11 Aug 2020 11:15:35 +0200
Message-ID: <CAAFQd5AfbN0V3sGCs8vhmeD-MNn3bTvfWBCaT-OY3hgjBUs-LQ@mail.gmail.com> (raw)
In-Reply-To: <CAAFQd5Cm+ZGx9ia2sAdvHQC6zC1U=+9AWs7iW7o-qE4g7wZgsw@mail.gmail.com>

On Mon, Aug 3, 2020 at 5:15 PM Tomasz Figa <tfiga@chromium.org> wrote:
>
> Hi Claire and Rob,
>
> On Mon, Aug 3, 2020 at 4:26 PM Claire Chang <tientzu@chromium.org> wrote:
> >
> > On Sat, Aug 1, 2020 at 4:58 AM Rob Herring <robh@kernel.org> wrote:
> > >
> > > On Tue, Jul 28, 2020 at 01:01:39PM +0800, Claire Chang wrote:
> > > > Introduce the new compatible string, device-swiotlb-pool, for restricted
> > > > DMA. One can specify the address and length of the device swiotlb memory
> > > > region by device-swiotlb-pool in the device tree.
> > > >
> > > > Signed-off-by: Claire Chang <tientzu@chromium.org>
> > > > ---
> > > >  .../reserved-memory/reserved-memory.txt       | 35 +++++++++++++++++++
> > > >  1 file changed, 35 insertions(+)
> > > >
> > > > diff --git a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt
> > > > index 4dd20de6977f..78850896e1d0 100644
> > > > --- a/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt
> > > > +++ b/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt
> > > > @@ -51,6 +51,24 @@ compatible (optional) - standard definition
> > > >            used as a shared pool of DMA buffers for a set of devices. It can
> > > >            be used by an operating system to instantiate the necessary pool
> > > >            management subsystem if necessary.
> > > > +        - device-swiotlb-pool: This indicates a region of memory meant to be
> > >
> > > swiotlb is a Linux thing. The binding should be independent.
> > Got it. Thanks for pointing this out.
> >
> > >
> > > > +          used as a pool of device swiotlb buffers for a given device. When
> > > > +          using this, the no-map and reusable properties must not be set, so the
> > > > +          operating system can create a virtual mapping that will be used for
> > > > +          synchronization. Also, there must be a restricted-dma property in the
> > > > +          device node to specify the indexes of reserved-memory nodes. One can
> > > > +          specify two reserved-memory nodes in the device tree. One with
> > > > +          shared-dma-pool to handle the coherent DMA buffer allocation, and
> > > > +          another one with device-swiotlb-pool for regular DMA to/from system
> > > > +          memory, which would be subject to bouncing. The main purpose for
> > > > +          restricted DMA is to mitigate the lack of DMA access control on
> > > > +          systems without an IOMMU, which could result in the DMA accessing the
> > > > +          system memory at unexpected times and/or unexpected addresses,
> > > > +          possibly leading to data leakage or corruption. The feature on its own
> > > > +          provides a basic level of protection against the DMA overwriting buffer
> > > > +          contents at unexpected times. However, to protect against general data
> > > > +          leakage and system memory corruption, the system needs to provide a
> > > > +          way to restrict the DMA to a predefined memory region.
> > >
> > > I'm pretty sure we already support per device carveouts and I don't
> > > understand how this is different.
> > We use this to bounce streaming DMA in and out of a specially allocated region.
> > I'll try to merge this with the existing one (i.e., shared-dma-pool)
> > to see if that
> > makes things clearer.
> >
>
> Indeed, from the firmware point of view, this is just a carveout, for
> which we have the "shared-dma-pool" compatible string defined already.
>
> However, depending on the device and firmware setup, the way the
> carevout is used may change. I can see the following scenarios:
>
> 1) coherent DMA (dma_alloc_*) within a reserved pool and no
> non-coherent DMA (dma_map_*).
>
> This is how the "memory-region" property is handled today in Linux for
> devices which can only DMA from/to the given memory region. However,
> I'm not sure if no non-coherent DMA is actually enforced in any way by
> the DMA subsystem.
>
> 2) coherent DMA from a reserved pool and non-coherent DMA from system memory
>
> This is the case for the systems which have some dedicated part of
> memory which is guaranteed to be coherent with the DMA, but still can
> do non-coherent DMA to any part of the system memory. Linux handles it
> the same way as 1), which is what made me believe that 1) might not
> actually be handled correctly.
>
> 3) coherent DMA and bounced non-coherent DMA within a reserved pool
> 4) coherent DMA within one pool and bounced non-coherent within another pool
>
> These are the two cases we're interested in. Basically they make it
> possible for non-coherent DMA from arbitrary system memory to be
> bounced through a reserved pool, which the device has access to. The
> current series implements 4), but I'd argue that it:
>
> - is problematic from the firmware point of view, because on most of
> the systems, both pools would be just some carveouts and the fact that
> Linux would use one for coherent and the other for non-coherent DMA
> would be an OS implementation detail,
> - suffers from the static memory split between coherent and
> non-coherent DMA, which could either result in some wasted memory or
> the DMA stopped working after a kernel update if the driver changes
> its allocation pattern,
>
> and so we should rather go with 3).
>
> Now, from the firmware point of view, it doesn't matter how the OS
> uses the carveout, but I think it's still necessary to tell the OS
> about the device DMA capability. Right now we use "memory-region" for
> any kind of reserved memory, but looking at the above scenarios, there
> are 2 cases:
>
> a) the memory region is preferred for the device, e.g. it enables
> coherency, but the device can still DMA across the rest of the system
> memory. This is the case in scenario 2) and is kind of assumed in the
> Linux DMA subsystem, although it's certainly not the case for a lot of
> hardware, even if they use the "memory-region" binding.
>
> b) the memory region is the only region that the device can access.
> This is the case in scenarios 1), 3) and 4).
>
> For this, I'd like to propose a "restricted-dma-region" (feel free to
> suggest a better name) binding, which is explicitly specified to be
> the only DMA-able memory for this device and make Linux use the given
> pool for coherent DMA allocations and bouncing non-coherent DMA.
>
> What do you think?

Rob, Robin, we'd appreciate your feedback on this when you have a
chance to take a look again. Thanks!

Best regards,
Tomasz

>
> Best regards,
> Tomasz
>
> > >
> > > What is the last sentence supposed to imply? You need an IOMMU?
> > The main purpose is to mitigate the lack of DMA access control on
> > systems without an IOMMU.
> > For example, we plan to use this plus a MPU for our PCIe WiFi which is
> > not behind an IOMMU.
> >
> > >
> > > >          - vendor specific string in the form <vendor>,[<device>-]<usage>
> > > >  no-map (optional) - empty property
> > > >      - Indicates the operating system must not create a virtual mapping
> > > > @@ -117,6 +135,16 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB).
> > > >                       compatible = "acme,multimedia-memory";
> > > >                       reg = <0x77000000 0x4000000>;
> > > >               };
> > > > +
> > > > +             wifi_coherent_mem_region: wifi_coherent_mem_region {
> > > > +                     compatible = "shared-dma-pool";
> > > > +                     reg = <0x50000000 0x400000>;
> > > > +             };
> > > > +
> > > > +             wifi_device_swiotlb_region: wifi_device_swiotlb_region {
> > > > +                     compatible = "device-swiotlb-pool";
> > > > +                     reg = <0x50400000 0x4000000>;
> > > > +             };
> > > >       };
> > > >
> > > >       /* ... */
> > > > @@ -135,4 +163,11 @@ one for multimedia processing (named multimedia-memory@77000000, 64MiB).
> > > >               memory-region = <&multimedia_reserved>;
> > > >               /* ... */
> > > >       };
> > > > +
> > > > +     pcie_wifi: pcie_wifi@0,0 {
> > > > +             memory-region = <&wifi_coherent_mem_region>,
> > > > +                      <&wifi_device_swiotlb_region>;
> > > > +             restricted-dma = <0>, <1>;
> > > > +             /* ... */
> > > > +     };
> > > >  };
> > > > --
> > > > 2.28.0.rc0.142.g3c755180ce-goog
> > > >
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

  reply index

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-28  5:01 [RFC v2 0/5] Restricted DMA Claire Chang
2020-07-28  5:01 ` [RFC v2 1/5] swiotlb: Add io_tlb_mem struct Claire Chang
2020-07-28  5:01 ` [RFC v2 2/5] swiotlb: Add device swiotlb pool Claire Chang
2020-07-28  5:01 ` [RFC v2 3/5] swiotlb: Use device swiotlb pool if available Claire Chang
2020-07-28  5:01 ` [RFC v2 4/5] dt-bindings: of: Add plumbing for restricted DMA pool Claire Chang
2020-07-31 20:58   ` Rob Herring
2020-08-03 14:26     ` Claire Chang
2020-08-03 15:15       ` Tomasz Figa
2020-08-11  9:15         ` Tomasz Figa [this message]
2020-08-24 17:24           ` Tomasz Figa
2020-09-08  9:49             ` Claire Chang
2020-07-28  5:01 ` [RFC v2 5/5] " Claire Chang
2020-07-28 11:59 ` [RFC v2 0/5] Restricted DMA Claire Chang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAAFQd5AfbN0V3sGCs8vhmeD-MNn3bTvfWBCaT-OY3hgjBUs-LQ@mail.gmail.com \
    --to=tfiga@chromium.org \
    --cc=bgolaszewski@baylibre.com \
    --cc=dan.j.williams@intel.com \
    --cc=devicetree@vger.kernel.org \
    --cc=drinkcat@chromium.org \
    --cc=frowand.list@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hch@lst.de \
    --cc=heikki.krogerus@linux.intel.com \
    --cc=iommu@lists.linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=robh@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=saravanak@google.com \
    --cc=suzuki.poulose@arm.com \
    --cc=tientzu@chromium.org \
    --cc=treding@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

IOMMU Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-iommu/0 linux-iommu/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-iommu linux-iommu/ https://lore.kernel.org/linux-iommu \
		iommu@lists.linux-foundation.org
	public-inbox-index linux-iommu

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.linux-foundation.lists.iommu


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git