From: Niklas Schnelle <schnelle@linux.ibm.com>
To: Joerg Roedel <joro@8bytes.org>,
Matthew Rosato <mjrosato@linux.ibm.com>,
Will Deacon <will@kernel.org>,
Wenjia Zhang <wenjia@linux.ibm.com>,
Robin Murphy <robin.murphy@arm.com>,
Jason Gunthorpe <jgg@ziepe.ca>
Cc: Gerd Bayer <gbayer@linux.ibm.com>,
Julian Ruess <julianr@linux.ibm.com>,
Pierre Morel <pmorel@linux.ibm.com>,
Alexandra Winter <wintera@linux.ibm.com>,
Heiko Carstens <hca@linux.ibm.com>,
Vasily Gorbik <gor@linux.ibm.com>,
Alexander Gordeev <agordeev@linux.ibm.com>,
Christian Borntraeger <borntraeger@linux.ibm.com>,
Sven Schnelle <svens@linux.ibm.com>,
Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
Hector Martin <marcan@marcan.st>, Sven Peter <sven@svenpeter.dev>,
Alyssa Rosenzweig <alyssa@rosenzweig.io>,
David Woodhouse <dwmw2@infradead.org>,
Lu Baolu <baolu.lu@linux.intel.com>,
Andy Gross <agross@kernel.org>,
Bjorn Andersson <andersson@kernel.org>,
Konrad Dybcio <konrad.dybcio@linaro.org>,
Yong Wu <yong.wu@mediatek.com>,
Matthias Brugger <matthias.bgg@gmail.com>,
AngeloGioacchino Del Regno
<angelogioacchino.delregno@collabora.com>,
Gerald Schaefer <gerald.schaefer@linux.ibm.com>,
Orson Zhai <orsonzhai@gmail.com>,
Baolin Wang <baolin.wang@linux.alibaba.com>,
Chunyan Zhang <zhang.lyra@gmail.com>,
Chen-Yu Tsai <wens@csie.org>,
Jernej Skrabec <jernej.skrabec@gmail.com>,
Samuel Holland <samuel@sholland.org>,
Thierry Reding <thierry.reding@gmail.com>,
Krishna Reddy <vdumpa@nvidia.com>,
Jonathan Hunter <jonathanh@nvidia.com>,
Jonathan Corbet <corbet@lwn.net>,
linux-s390@vger.kernel.org, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org, iommu@lists.linux.dev,
asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org,
linux-arm-msm@vger.kernel.org,
linux-mediatek@lists.infradead.org, linux-sunxi@lists.linux.dev,
linux-tegra@vger.kernel.org, linux-doc@vger.kernel.org
Subject: Re: [PATCH v10 5/6] iommu/dma: Allow a single FQ in addition to per-CPU FQs
Date: Thu, 22 Jun 2023 14:47:18 +0200 [thread overview]
Message-ID: <2bd0da80754331e00f66c724138d9bbf157e1565.camel@linux.ibm.com> (raw)
In-Reply-To: <20230310-dma_iommu-v10-5-f1fbd8310854@linux.ibm.com>
On Wed, 2023-05-24 at 16:53 +0200, Niklas Schnelle wrote:
> In some virtualized environments, including s390 paged memory guests,
> IOTLB flushes are used to update IOMMU shadow tables. Due to this, they
> are much more expensive than in typical bare metal environments or
> non-paged s390 guests. In addition they may parallelize poorly in
> virtualized environments. This changes the trade off for flushing IOVAs
> such that minimizing the number of IOTLB flushes trumps any benefit of
> cheaper queuing operations or increased paralellism.
>
> In this scenario per-CPU flush queues pose several problems. Firstly
> per-CPU memory is often quite limited prohibiting larger queues.
> Secondly collecting IOVAs per-CPU but flushing via a global timeout
> reduces the number of IOVAs flushed for each timeout especially on s390
> where PCI interrupts may not be bound to a specific CPU.
>
> Let's introduce a single flush queue mode that reuses the same queue
> logic but only allocates a single global queue. This mode is selected by
> dma-iommu if a newly introduced .shadow_on_flush flag is set in struct
> dev_iommu. As a first user the s390 IOMMU driver sets this flag during
> probe_device. With the unchanged small FQ size and timeouts this setting
> is worse than per-CPU queues but a follow up patch will make the FQ size
> and timeout variable. Together this allows the common IOVA flushing code
> to more closely resemble the global flush behavior used on s390's
> previous internal DMA API implementation.
>
> Link: https://lore.kernel.org/linux-iommu/3e402947-61f9-b7e8-1414-fde006257b6f@arm.com/
> Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com> #s390
> Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
> ---
Gentle ping. As stated in the cover letter this version aimed to
resolve the only outstanding feedback of removing the .tune_dma_iommu()
op in favor of a .shadow_on_flush flag in struct dev_iommu. This then
let's the dma-iommu choose a single queue and larger timeouts and IOVA
counts. This leaves the dma-iommu with full responsibility for the
settings.
Thanks,
Niklas
next prev parent reply other threads:[~2023-06-22 12:48 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-24 14:53 [PATCH v10 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing Niklas Schnelle
2023-05-24 14:53 ` [PATCH v10 1/6] s390/ism: Set DMA coherent mask Niklas Schnelle
2023-05-24 14:53 ` [PATCH v10 2/6] iommu: Allow .iotlb_sync_map to fail and handle s390's -ENOMEM return Niklas Schnelle
2023-05-24 14:53 ` [PATCH v10 3/6] s390/pci: prepare is_passed_through() for dma-iommu Niklas Schnelle
2023-05-24 14:53 ` [PATCH v10 4/6] s390/pci: Use dma-iommu layer Niklas Schnelle
2023-05-24 14:53 ` [PATCH v10 5/6] iommu/dma: Allow a single FQ in addition to per-CPU FQs Niklas Schnelle
2023-06-22 12:47 ` Niklas Schnelle [this message]
2023-05-24 14:53 ` [PATCH v10 6/6] iommu/dma: Use a large flush queue and timeout for shadow_on_flush Niklas Schnelle
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2bd0da80754331e00f66c724138d9bbf157e1565.camel@linux.ibm.com \
--to=schnelle@linux.ibm.com \
--cc=agordeev@linux.ibm.com \
--cc=agross@kernel.org \
--cc=alyssa@rosenzweig.io \
--cc=andersson@kernel.org \
--cc=angelogioacchino.delregno@collabora.com \
--cc=asahi@lists.linux.dev \
--cc=baolin.wang@linux.alibaba.com \
--cc=baolu.lu@linux.intel.com \
--cc=borntraeger@linux.ibm.com \
--cc=corbet@lwn.net \
--cc=dwmw2@infradead.org \
--cc=gbayer@linux.ibm.com \
--cc=gerald.schaefer@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=iommu@lists.linux.dev \
--cc=jernej.skrabec@gmail.com \
--cc=jgg@ziepe.ca \
--cc=jonathanh@nvidia.com \
--cc=joro@8bytes.org \
--cc=julianr@linux.ibm.com \
--cc=konrad.dybcio@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mediatek@lists.infradead.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-sunxi@lists.linux.dev \
--cc=linux-tegra@vger.kernel.org \
--cc=marcan@marcan.st \
--cc=matthias.bgg@gmail.com \
--cc=mjrosato@linux.ibm.com \
--cc=netdev@vger.kernel.org \
--cc=orsonzhai@gmail.com \
--cc=pmorel@linux.ibm.com \
--cc=robin.murphy@arm.com \
--cc=samuel@sholland.org \
--cc=suravee.suthikulpanit@amd.com \
--cc=sven@svenpeter.dev \
--cc=svens@linux.ibm.com \
--cc=thierry.reding@gmail.com \
--cc=vdumpa@nvidia.com \
--cc=wenjia@linux.ibm.com \
--cc=wens@csie.org \
--cc=will@kernel.org \
--cc=wintera@linux.ibm.com \
--cc=yong.wu@mediatek.com \
--cc=zhang.lyra@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).