From: Robin Murphy <robin.murphy@arm.com>
To: Christoph Hellwig <hch@lst.de>,
Jonathan Lemon <jonathan.lemon@gmail.com>
Cc: netdev@vger.kernel.org, iommu@lists.linux-foundation.org,
"Björn Töpel" <bjorn.topel@intel.com>
Subject: Re: the XSK buffer pool needs be to reverted
Date: Mon, 29 Jun 2020 14:15:16 +0100 [thread overview]
Message-ID: <e43ab7b9-22f5-75c3-c9e6-f1eb18d57148@arm.com> (raw)
In-Reply-To: <20200627070236.GA11854@lst.de>
On 2020-06-27 08:02, Christoph Hellwig wrote:
> On Fri, Jun 26, 2020 at 01:54:12PM -0700, Jonathan Lemon wrote:
>> On Fri, Jun 26, 2020 at 09:47:25AM +0200, Christoph Hellwig wrote:
>>>
>>> Note that this is somewhat urgent, as various of the APIs that the code
>>> is abusing are slated to go away for Linux 5.9, so this addition comes
>>> at a really bad time.
>>
>> Could you elaborate on what is upcoming here?
>
> Moving all these calls out of line, and adding a bypass flag to avoid
> the indirect function call for IOMMUs in direct mapped mode.
>
>> Also, on a semi-related note, are there limitations on how many pages
>> can be left mapped by the iommu? Some of the page pool work involves
>> leaving the pages mapped instead of constantly mapping/unmapping them.
>
> There are, but I think for all modern IOMMUs they are so big that they
> don't matter. Maintaines of the individual IOMMU drivers might know
> more.
Right - I don't know too much about older and more esoteric stuff like
POWER TCE, but for modern pagetable-based stuff like Intel VT-d, AMD-Vi,
and Arm SMMU, the only "limits" are such that legitimate DMA API use
should never get anywhere near them (you'd run out of RAM for actual
buffers long beforehand). The most vaguely-realistic concern might be a
pathological system topology where some old 32-bit PCI device doesn't
have ACS isolation from your high-performance NIC such that they have to
share an address space, where the NIC might happen to steal all the low
addresses and prevent the soundcard or whatever from being able to map a
usable buffer.
With an IOMMU, you typically really *want* to keep a full working set's
worth of pages mapped, since dma_map/unmap are expensive while dma_sync
is somewhere between relatively cheap and free. With no IOMMU it makes
no real difference from the DMA API perspective since map/unmap are
effectively no more than the equivalent sync operations anyway (I'm
assuming we're not talking about the kind of constrained hardware that
might need SWIOTLB).
>> On a heavily loaded box with iommu enabled, it seems that quite often
>> there is contention on the iova_lock. Are there known issues in this
>> area?
>
> I'll have to defer to the IOMMU maintainers, and for that you'll need
> to say what code you are using. Current mainlaine doesn't even have
> an iova_lock anywhere.
Again I can't speak for non-mainstream stuff outside drivers/iommu, but
it's been over 4 years now since merging the initial scalability work
for the generic IOVA allocator there that focused on minimising lock
contention, and it's had considerable evaluation and tweaking since. But
if we can achieve the goal of efficiently recycling mapped buffers then
we shouldn't need to go anywhere near IOVA allocation either way except
when expanding the pool.
Robin.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2020-06-29 13:15 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-26 7:47 the XSK buffer pool needs be to reverted Christoph Hellwig
2020-06-26 12:22 ` Björn Töpel
2020-06-26 12:41 ` Christoph Hellwig
2020-06-26 12:45 ` Björn Töpel
2020-06-26 20:54 ` Jonathan Lemon
2020-06-27 7:02 ` Christoph Hellwig
2020-06-29 13:15 ` Robin Murphy [this message]
2020-06-30 19:08 ` Jonathan Lemon
2020-07-01 9:46 ` Robin Murphy
2020-07-06 19:59 ` Jonathan Lemon
2020-07-07 17:35 ` Robin Murphy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e43ab7b9-22f5-75c3-c9e6-f1eb18d57148@arm.com \
--to=robin.murphy@arm.com \
--cc=bjorn.topel@intel.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux-foundation.org \
--cc=jonathan.lemon@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).