From: Claire Chang <tientzu@chromium.org>
To: Florian Fainelli <f.fainelli@gmail.com>
Cc: heikki.krogerus@linux.intel.com, peterz@infradead.org,
benh@kernel.crashing.org, grant.likely@arm.com, paulus@samba.org,
Frank Rowand <frowand.list@gmail.com>,
mingo@kernel.org, sstabellini@kernel.org,
Saravana Kannan <saravanak@google.com>,
mpe@ellerman.id.au, rafael.j.wysocki@intel.com,
Christoph Hellwig <hch@lst.de>,
Bartosz Golaszewski <bgolaszewski@baylibre.com>,
xen-devel@lists.xenproject.org,
Thierry Reding <treding@nvidia.com>,
linux-devicetree <devicetree@vger.kernel.org>,
will@kernel.org, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
dan.j.williams@intel.com, linuxppc-dev@lists.ozlabs.org,
Rob Herring <robh+dt@kernel.org>,
boris.ostrovsky@oracle.com,
Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
jgross@suse.com, Nicolas Boichat <drinkcat@chromium.org>,
Greg KH <gregkh@linuxfoundation.org>,
rdunlap@infradead.org, lkml <linux-kernel@vger.kernel.org>,
"list@263.net:IOMMU DRIVERS" <iommu@lists.linux-foundation.org>,
Jim Quinlan <james.quinlan@broadcom.com>,
xypron.glpk@gmx.de, Robin Murphy <robin.murphy@arm.com>
Subject: Re: [RFC PATCH v3 0/6] Restricted DMA
Date: Tue, 12 Jan 2021 15:48:16 +0800 [thread overview]
Message-ID: <CALiNf29_PmLJTVLksSHp3NFAaL52idqehSMOtatJ=jaM2Muq1g@mail.gmail.com> (raw)
In-Reply-To: <78871151-947d-b085-db03-0d0bd0b55632@gmail.com>
On Fri, Jan 8, 2021 at 1:59 AM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
> On 1/7/21 9:42 AM, Claire Chang wrote:
>
> >> Can you explain how ATF gets involved and to what extent it does help,
> >> besides enforcing a secure region from the ARM CPU's perpsective? Does
> >> the PCIe root complex not have an IOMMU but can somehow be denied access
> >> to a region that is marked NS=0 in the ARM CPU's MMU? If so, that is
> >> still some sort of basic protection that the HW enforces, right?
> >
> > We need the ATF support for memory MPU (memory protection unit).
> > Restricted DMA (with reserved-memory in dts) makes sure the predefined memory
> > region is for PCIe DMA only, but we still need MPU to locks down PCIe access to
> > that specific regions.
>
> OK so you do have a protection unit of some sort to enforce which region
> in DRAM the PCIE bridge is allowed to access, that makes sense,
> otherwise the restricted DMA region would only be a hint but nothing you
> can really enforce. This is almost entirely analogous to our systems then.
Here is the example of setting the MPU:
https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/mediatek/mt8183/drivers/emi_mpu/emi_mpu.c#L132
>
> There may be some value in standardizing on an ARM SMCCC call then since
> you already support two different SoC vendors.
>
> >
> >>
> >> On Broadcom STB SoCs we have had something similar for a while however
> >> and while we don't have an IOMMU for the PCIe bridge, we do have a a
> >> basic protection mechanism whereby we can configure a region in DRAM to
> >> be PCIe read/write and CPU read/write which then gets used as the PCIe
> >> inbound region for the PCIe EP. By default the PCIe bridge is not
> >> allowed access to DRAM so we must call into a security agent to allow
> >> the PCIe bridge to access the designated DRAM region.
> >>
> >> We have done this using a private CMA area region assigned via Device
> >> Tree, assigned with a and requiring the PCIe EP driver to use
> >> dma_alloc_from_contiguous() in order to allocate from this device
> >> private CMA area. The only drawback with that approach is that it
> >> requires knowing how much memory you need up front for buffers and DMA
> >> descriptors that the PCIe EP will need to process. The problem is that
> >> it requires driver modifications and that does not scale over the number
> >> of PCIe EP drivers, some we absolutely do not control, but there is no
> >> need to bounce buffer. Your approach scales better across PCIe EP
> >> drivers however it does require bounce buffering which could be a
> >> performance hit.
> >
> > Only the streaming DMA (map/unmap) needs bounce buffering.
>
> True, and typically only on transmit since you don't really control
> where the sk_buff are allocated from, right? On RX since you need to
> hand buffer addresses to the WLAN chip prior to DMA, you can allocate
> them from a pool that already falls within the restricted DMA region, right?
>
Right, but applying bounce buffering to RX will make it more secure.
The device won't be able to modify the content after unmap. Just like what
iommu_unmap does.
> > I also added alloc/free support in this series
> > (https://lore.kernel.org/patchwork/patch/1360995/), so dma_direct_alloc() will
> > try to allocate memory from the predefined memory region.
> >
> > As for the performance hit, it should be similar to the default swiotlb.
> > Here are my experiment results. Both SoCs lack IOMMU for PCIe.
> >
> > PCIe wifi vht80 throughput -
> >
> > MTK SoC tcp_tx tcp_rx udp_tx udp_rx
> > w/o Restricted DMA 244.1 134.66 312.56 350.79
> > w/ Restricted DMA 246.95 136.59 363.21 351.99
> >
> > Rockchip SoC tcp_tx tcp_rx udp_tx udp_rx
> > w/o Restricted DMA 237.87 133.86 288.28 361.88
> > w/ Restricted DMA 256.01 130.95 292.28 353.19
>
> How come you get better throughput with restricted DMA? Is it because
> doing DMA to/from a contiguous region allows for better grouping of
> transactions from the DRAM controller's perspective somehow?
I'm not sure, but actually, enabling the default swiotlb for wifi also helps the
throughput a little bit for me.
>
> >
> > The CPU usage doesn't increase too much either.
> > Although I didn't measure the CPU usage very precisely, it's ~3% with a single
> > big core (Cortex-A72) and ~5% with a single small core (Cortex-A53).
> >
> > Thanks!
> >
> >>
> >> Thanks!
> >> --
> >> Florian
>
>
> --
> Florian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-01-12 7:56 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-06 3:41 [RFC PATCH v3 0/6] Restricted DMA Claire Chang
2021-01-06 3:41 ` [RFC PATCH v3 1/6] swiotlb: Add io_tlb_mem struct Claire Chang
2021-01-13 11:50 ` Christoph Hellwig
2021-01-06 3:41 ` [RFC PATCH v3 2/6] swiotlb: Add restricted DMA pool Claire Chang
2021-01-06 7:50 ` Greg KH
2021-01-13 11:51 ` Christoph Hellwig
2021-01-13 12:29 ` Greg KH
2021-01-13 12:37 ` Christoph Hellwig
2021-01-06 18:52 ` Konrad Rzeszutek Wilk
2021-01-07 17:39 ` Claire Chang
2021-01-07 17:57 ` Konrad Rzeszutek Wilk
2021-01-07 18:09 ` Florian Fainelli
2021-01-07 21:19 ` Konrad Rzeszutek Wilk
2021-01-12 23:52 ` Florian Fainelli
2021-01-25 5:26 ` Jon Masters
2021-01-13 1:53 ` Robin Murphy
2021-01-13 0:03 ` Florian Fainelli
2021-01-13 13:59 ` Nicolas Saenz Julienne
2021-01-13 15:27 ` Robin Murphy
2021-01-13 17:43 ` Florian Fainelli
2021-01-13 18:03 ` Robin Murphy
2021-01-13 12:42 ` Christoph Hellwig
2021-01-14 9:06 ` Claire Chang
2021-01-06 3:41 ` [RFC PATCH v3 3/6] swiotlb: Use restricted DMA pool if available Claire Chang
2021-01-12 23:39 ` Florian Fainelli
2021-01-13 12:44 ` Christoph Hellwig
2021-01-06 3:41 ` [RFC PATCH v3 4/6] swiotlb: Add restricted DMA alloc/free support Claire Chang
2021-01-12 23:41 ` Florian Fainelli
2021-01-13 12:48 ` Christoph Hellwig
2021-01-13 18:27 ` Robin Murphy
2021-01-13 18:32 ` Christoph Hellwig
2021-01-06 3:41 ` [RFC PATCH v3 5/6] dt-bindings: of: Add restricted DMA pool Claire Chang
2021-01-06 18:57 ` Konrad Rzeszutek Wilk
2021-01-07 17:39 ` Claire Chang
2021-01-07 18:00 ` Konrad Rzeszutek Wilk
2021-01-07 18:14 ` Florian Fainelli
2021-01-12 7:47 ` Claire Chang
2021-01-20 16:53 ` Rob Herring
2021-01-20 17:30 ` Robin Murphy
2021-01-20 21:31 ` Rob Herring
2021-01-21 1:09 ` Robin Murphy
2021-01-21 15:48 ` Rob Herring
2021-01-21 17:29 ` Robin Murphy
2021-01-06 3:41 ` [RFC PATCH v3 6/6] of: Add plumbing for " Claire Chang
2021-01-12 23:48 ` Florian Fainelli
2021-01-14 9:08 ` Claire Chang
2021-01-14 18:52 ` Florian Fainelli
2021-01-15 3:46 ` Claire Chang
2021-01-06 18:48 ` [RFC PATCH v3 0/6] Restricted DMA Florian Fainelli
2021-01-07 17:38 ` Claire Chang
2021-01-07 17:42 ` Claire Chang
2021-01-07 17:59 ` Florian Fainelli
2021-01-12 7:48 ` Claire Chang [this message]
2021-01-12 18:01 ` Florian Fainelli
2021-01-13 2:29 ` Tomasz Figa
2021-01-13 3:56 ` Florian Fainelli
2021-01-13 4:25 ` Tomasz Figa
2021-01-13 4:41 ` Florian Fainelli
2021-02-09 6:27 ` Claire Chang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CALiNf29_PmLJTVLksSHp3NFAaL52idqehSMOtatJ=jaM2Muq1g@mail.gmail.com' \
--to=tientzu@chromium.org \
--cc=andriy.shevchenko@linux.intel.com \
--cc=benh@kernel.crashing.org \
--cc=bgolaszewski@baylibre.com \
--cc=boris.ostrovsky@oracle.com \
--cc=dan.j.williams@intel.com \
--cc=devicetree@vger.kernel.org \
--cc=drinkcat@chromium.org \
--cc=f.fainelli@gmail.com \
--cc=frowand.list@gmail.com \
--cc=grant.likely@arm.com \
--cc=gregkh@linuxfoundation.org \
--cc=hch@lst.de \
--cc=heikki.krogerus@linux.intel.com \
--cc=iommu@lists.linux-foundation.org \
--cc=james.quinlan@broadcom.com \
--cc=jgross@suse.com \
--cc=konrad.wilk@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mingo@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=paulus@samba.org \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=rdunlap@infradead.org \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=saravanak@google.com \
--cc=sstabellini@kernel.org \
--cc=treding@nvidia.com \
--cc=will@kernel.org \
--cc=xen-devel@lists.xenproject.org \
--cc=xypron.glpk@gmx.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).