From: Robin Murphy <robin.murphy@arm.com>
To: Russell King - ARM Linux admin <linux@armlinux.org.uk>,
linux-arm-kernel@lists.infradead.org,
Will Deacon <will@kernel.org>
Subject: Re: SMMU performance
Date: Mon, 30 Sep 2019 12:45:28 +0100 [thread overview]
Message-ID: <cf3ac700-1954-8800-6ce4-9983ab27707b@arm.com> (raw)
In-Reply-To: <20190930110047.GF25745@shell.armlinux.org.uk>
Hi Russell,
On 30/09/2019 12:00, Russell King - ARM Linux admin wrote:
> Hi,
>
> While using iperf on a platform using the ARM SMMU (v2), I notice the
> following behaviour on v5.3 with Will's iommu patch set merged, kernel
> lock debugging disabled.
>
> With iommu.passthrough=1, three consecutive runs:
> [ 3] 0.0-10.0 sec 4.51 GBytes 3.87 Gbits/sec
> [ 3] 0.0-10.0 sec 4.53 GBytes 3.89 Gbits/sec
> [ 3] 0.0-10.0 sec 4.49 GBytes 3.86 Gbits/sec
>
> With iommu.passthrough=0:
> [ 3] 0.0-10.0 sec 1.77 GBytes 1.52 Gbits/sec
> [ 3] 0.0-10.0 sec 1.82 GBytes 1.56 Gbits/sec
> [ 3] 0.0-10.0 sec 1.69 GBytes 1.45 Gbits/sec
>
> Running perf record -a -g ... followed by perf report --no-children
> shows:
>
> - 15.72% iperf [kernel.vmlinux] [k] _raw_spin_unlock_irqrestor
> - _raw_spin_unlock_irqrestore
> - 8.95% arm_smmu_tlb_sync_context
> arm_smmu_iotlb_sync
> - __iommu_dma_unmap
> + 4.54% iommu_dma_unmap_sg
> + 4.41% iommu_dma_unmap_page
> - 2.92% alloc_iova_fast
> - iommu_dma_alloc_iova.isra.26
> + 1.54% iommu_dma_map_sg
> + 1.38% __iommu_dma_map
> - 2.64% free_iova_fast
> iommu_dma_free_iova
> - __iommu_dma_unmap
> + 1.35% iommu_dma_unmap_sg
> + 1.29% iommu_dma_unmap_page
>
> which seems to be pointing to the SMMU code as a bottleneck.
>
> Will suggests that his iommu changes (in his for-joerg/arm-smmu/updates
> branch), allows IOMMU driver modifications that may have a beneficial
> effect. Any thoughts?
We default to synchronous invalidation on unmaps, since it gives the
greatest degree of security against misbehaving devices (and proves
quite useful for smoking out dodgy drivers too). If you're happy with
deferred invalidation as x86 defaults to, try "iommu.strict=0" - that
should avoid the main serialising bottleneck. As for the IOVA allocation
overhead, that's probably about as low as it's likely to get now - what
remains is the inevitable "doing anything vs. doing nothing" tradeoff.
The major changes in 5.4 are for SMMUv3, so won't impact your platform.
Robin.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-09-30 11:45 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-30 11:00 SMMU performance Russell King - ARM Linux admin
2019-09-30 11:45 ` Robin Murphy [this message]
2019-09-30 11:54 ` Will Deacon
2019-09-30 12:00 ` Robin Murphy
2019-10-02 9:02 ` Will Deacon
2019-10-02 11:09 ` Robin Murphy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cf3ac700-1954-8800-6ce4-9983ab27707b@arm.com \
--to=robin.murphy@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux@armlinux.org.uk \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).