From: Shenming Lu <lushenming@huawei.com>
To: Alex Williamson <alex.williamson@redhat.com>,
Cornelia Huck <cohuck@redhat.com>, Will Deacon <will@kernel.org>,
Robin Murphy <robin.murphy@arm.com>,
Joerg Roedel <joro@8bytes.org>,
Jean-Philippe Brucker <jean-philippe@linaro.org>,
Eric Auger <eric.auger@redhat.com>, <kvm@vger.kernel.org>,
<linux-kernel@vger.kernel.org>,
<linux-arm-kernel@lists.infradead.org>,
<iommu@lists.linux-foundation.org>, <linux-api@vger.kernel.org>
Cc: Kevin Tian <kevin.tian@intel.com>,
Lu Baolu <baolu.lu@linux.intel.com>, <yi.l.liu@intel.com>,
Christoph Hellwig <hch@infradead.org>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Barry Song <song.bao.hua@hisilicon.com>,
<wanghaibin.wang@huawei.com>, <yuzenghui@huawei.com>
Subject: Re: [RFC PATCH v3 0/8] Add IOPF support for VFIO passthrough
Date: Mon, 26 Apr 2021 09:41:23 +0800 [thread overview]
Message-ID: <cb9584fd-c7f5-8cac-8c63-219ded2ef9db@huawei.com> (raw)
In-Reply-To: <20210409034420.1799-1-lushenming@huawei.com>
On 2021/4/9 11:44, Shenming Lu wrote:
> Hi,
>
> Requesting for your comments and suggestions. :-)
Kind ping...
>
> The static pinning and mapping problem in VFIO and possible solutions
> have been discussed a lot [1, 2]. One of the solutions is to add I/O
> Page Fault support for VFIO devices. Different from those relatively
> complicated software approaches such as presenting a vIOMMU that provides
> the DMA buffer information (might include para-virtualized optimizations),
> IOPF mainly depends on the hardware faulting capability, such as the PCIe
> PRI extension or Arm SMMU stall model. What's more, the IOPF support in
> the IOMMU driver has already been implemented in SVA [3]. So we add IOPF
> support for VFIO passthrough based on the IOPF part of SVA in this series.
>
> We have measured its performance with UADK [4] (passthrough an accelerator
> to a VM(1U16G)) on Hisilicon Kunpeng920 board (and compared with host SVA):
>
> Run hisi_sec_test...
> - with varying sending times and message lengths
> - with/without IOPF enabled (speed slowdown)
>
> when msg_len = 1MB (and PREMAP_LEN (in Patch 4) = 1):
> slowdown (num of faults)
> times VFIO IOPF host SVA
> 1 63.4% (518) 82.8% (512)
> 100 22.9% (1058) 47.9% (1024)
> 1000 2.6% (1071) 8.5% (1024)
>
> when msg_len = 10MB (and PREMAP_LEN = 512):
> slowdown (num of faults)
> times VFIO IOPF
> 1 32.6% (13)
> 100 3.5% (26)
> 1000 1.6% (26)
>
> History:
>
> v2 -> v3
> - Nit fixes.
> - No reason to disable reporting the unrecoverable faults. (baolu)
> - Maintain a global IOPF enabled group list.
> - Split the pre-mapping optimization to be a separate patch.
> - Add selective faulting support (use vfio_pin_pages to indicate the
> non-faultable scope and add a new struct vfio_range to record it,
> untested). (Kevin)
>
> v1 -> v2
> - Numerous improvements following the suggestions. Thanks a lot to all
> of you.
>
> Note that PRI is not supported at the moment since there is no hardware.
>
> Links:
> [1] Lesokhin I, et al. Page Fault Support for Network Controllers. In ASPLOS,
> 2016.
> [2] Tian K, et al. coIOMMU: A Virtual IOMMU with Cooperative DMA Buffer Tracking
> for Efficient Memory Management in Direct I/O. In USENIX ATC, 2020.
> [3] https://patchwork.kernel.org/project/linux-arm-kernel/cover/20210401154718.307519-1-jean-philippe@linaro.org/
> [4] https://github.com/Linaro/uadk
>
> Thanks,
> Shenming
>
>
> Shenming Lu (8):
> iommu: Evolve the device fault reporting framework
> vfio/type1: Add a page fault handler
> vfio/type1: Add an MMU notifier to avoid pinning
> vfio/type1: Pre-map more pages than requested in the IOPF handling
> vfio/type1: VFIO_IOMMU_ENABLE_IOPF
> vfio/type1: No need to statically pin and map if IOPF enabled
> vfio/type1: Add selective DMA faulting support
> vfio: Add nested IOPF support
>
> .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 3 +-
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 18 +-
> drivers/iommu/iommu.c | 56 +-
> drivers/vfio/vfio.c | 85 +-
> drivers/vfio/vfio_iommu_type1.c | 1000 ++++++++++++++++-
> include/linux/iommu.h | 19 +-
> include/linux/vfio.h | 13 +
> include/uapi/linux/iommu.h | 4 +
> include/uapi/linux/vfio.h | 6 +
> 9 files changed, 1181 insertions(+), 23 deletions(-)
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-04-26 1:44 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-09 3:44 [RFC PATCH v3 0/8] Add IOPF support for VFIO passthrough Shenming Lu
2021-04-09 3:44 ` [RFC PATCH v3 1/8] iommu: Evolve the device fault reporting framework Shenming Lu
2021-05-18 18:58 ` Alex Williamson
2021-05-21 6:37 ` Shenming Lu
2021-04-09 3:44 ` [RFC PATCH v3 2/8] vfio/type1: Add a page fault handler Shenming Lu
2021-05-18 18:58 ` Alex Williamson
2021-05-21 6:38 ` Shenming Lu
2021-05-24 22:11 ` Alex Williamson
2021-05-27 11:16 ` Shenming Lu
2021-04-09 3:44 ` [RFC PATCH v3 3/8] vfio/type1: Add an MMU notifier to avoid pinning Shenming Lu
2021-05-18 18:58 ` Alex Williamson
2021-05-21 6:37 ` Shenming Lu
2021-04-09 3:44 ` [RFC PATCH v3 4/8] vfio/type1: Pre-map more pages than requested in the IOPF handling Shenming Lu
2021-05-18 18:58 ` Alex Williamson
2021-05-21 6:37 ` Shenming Lu
2021-04-09 3:44 ` [RFC PATCH v3 5/8] vfio/type1: VFIO_IOMMU_ENABLE_IOPF Shenming Lu
2021-05-18 18:58 ` Alex Williamson
2021-05-21 6:38 ` Shenming Lu
2021-05-24 22:11 ` Alex Williamson
2021-05-27 11:15 ` Shenming Lu
2021-04-09 3:44 ` [RFC PATCH v3 6/8] vfio/type1: No need to statically pin and map if IOPF enabled Shenming Lu
2021-05-18 18:58 ` Alex Williamson
2021-05-21 6:39 ` Shenming Lu
2021-04-09 3:44 ` [RFC PATCH v3 7/8] vfio/type1: Add selective DMA faulting support Shenming Lu
2021-05-18 18:58 ` Alex Williamson
2021-05-21 6:39 ` Shenming Lu
2021-04-09 3:44 ` [RFC PATCH v3 8/8] vfio: Add nested IOPF support Shenming Lu
2021-05-18 18:58 ` Alex Williamson
2021-05-21 7:59 ` Shenming Lu
2021-05-24 13:11 ` Shenming Lu
2021-05-24 22:11 ` Alex Williamson
2021-05-27 11:03 ` Shenming Lu
2021-05-27 11:18 ` Lu Baolu
2021-06-01 4:36 ` Shenming Lu
2021-04-26 1:41 ` Shenming Lu [this message]
2021-05-11 11:30 ` [RFC PATCH v3 0/8] Add IOPF support for VFIO passthrough Shenming Lu
2021-05-18 18:57 ` Alex Williamson
2021-05-21 6:37 ` Shenming Lu
2021-05-24 22:11 ` Alex Williamson
2021-05-27 11:25 ` Shenming Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cb9584fd-c7f5-8cac-8c63-219ded2ef9db@huawei.com \
--to=lushenming@huawei.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=alex.williamson@redhat.com \
--cc=baolu.lu@linux.intel.com \
--cc=cohuck@redhat.com \
--cc=eric.auger@redhat.com \
--cc=hch@infradead.org \
--cc=iommu@lists.linux-foundation.org \
--cc=jean-philippe@linaro.org \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=robin.murphy@arm.com \
--cc=song.bao.hua@hisilicon.com \
--cc=wanghaibin.wang@huawei.com \
--cc=will@kernel.org \
--cc=yi.l.liu@intel.com \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).