From: Yi Liu <yi.l.liu@intel.com>
To: joro@8bytes.org, alex.williamson@redhat.com, jgg@nvidia.com,
kevin.tian@intel.com, robin.murphy@arm.com,
baolu.lu@linux.intel.com
Cc: cohuck@redhat.com, eric.auger@redhat.com, nicolinc@nvidia.com,
kvm@vger.kernel.org, mjrosato@linux.ibm.com,
chao.p.peng@linux.intel.com, yi.l.liu@intel.com,
yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com,
shameerali.kolothum.thodi@huawei.com, lulu@redhat.com,
suravee.suthikulpanit@amd.com, iommu@lists.linux.dev,
linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
zhenzhong.duan@intel.com, joao.m.martins@oracle.com
Subject: [PATCH v4 00/17] iommufd: Add nesting infrastructure
Date: Thu, 21 Sep 2023 00:51:21 -0700 [thread overview]
Message-ID: <20230921075138.124099-1-yi.l.liu@intel.com> (raw)
Nested translation is a hardware feature that is supported by many modern
IOMMU hardwares. It has two stages (stage-1, stage-2) address translation
to get access to the physical address. stage-1 translation table is owned
by userspace (e.g. by a guest OS), while stage-2 is owned by kernel. Changes
to stage-1 translation table should be followed by an IOTLB invalidation.
Take Intel VT-d as an example, the stage-1 translation table is I/O page
table. As the below diagram shows, guest I/O page table pointer in GPA
(guest physical address) is passed to host and be used to perform the stage-1
address translation. Along with it, modifications to present mappings in the
guest I/O page table should be followed with an IOTLB invalidation.
.-------------. .---------------------------.
| vIOMMU | | Guest I/O page table |
| | '---------------------------'
.----------------/
| PASID Entry |--- PASID cache flush --+
'-------------' |
| | V
| | I/O page table pointer in GPA
'-------------'
Guest
------| Shadow |---------------------------|--------
v v v
Host
.-------------. .------------------------.
| pIOMMU | | FS for GIOVA->GPA |
| | '------------------------'
.----------------/ |
| PASID Entry | V (Nested xlate)
'----------------\.----------------------------------.
| | | SS for GPA->HPA, unmanaged domain|
| | '----------------------------------'
'-------------'
Where:
- FS = First stage page tables
- SS = Second stage page tables
<Intel VT-d Nested translation>
In IOMMUFD, all the translation tables are tracked by hw_pagetable (hwpt)
and each has an iommu_domain allocated from iommu driver. So in this series
hw_pagetable and iommu_domain means the same thing if no special note.
IOMMUFD has already supported allocating hw_pagetable that is linked with
an IOAS. However, nesting requires IOMMUFD to allow allocating hw_pagetable
with driver specific parameters and interface to sync stage-1 IOTLB as user
owns the stage-1 translation table.
This series is based on the iommu hw info reporting series [1]. It first
extends domain_alloc_user to allocate domains with user data and adds new
op for invalidate stage-1 IOTLB for user-managed domains, then extends the
IOMMUFD internal infrastructure to accept user_data and parent hwpt, relay
the user_data/parent to iommu core to allocate user-managed iommu_domain.
After it, extends the ioctl IOMMU_HWPT_ALLOC to accept user data and stage-2
hwpt ID. Along with it, ioctl IOMMU_HWPT_INVALIDATE is added to invalidate
stage-1 IOTLB. This is needed for user-managed hwpts. Selftest is added as
well to cover the new ioctls.
Complete code can be found in [2], QEMU could can be found in [3].
At last, this is a team work together with Nicolin Chen, Lu Baolu. Thanks
them for the help. ^_^. Look forward to your feedbacks.
[1] https://lore.kernel.org/linux-iommu/20230818101033.4100-1-yi.l.liu@intel.com/#r - merged
[2] https://github.com/yiliu1765/iommufd/tree/iommufd_nesting
[3] https://github.com/yiliu1765/qemu/tree/zhenzhong/wip/iommufd_nesting_rfcv1
Change log:
v4:
- Separate HWPT alloc/destroy/abort functions between user-managed HWPTs
and kernel-managed HWPTs
- Rework invalidate uAPI to be a multi-request array-based design
- Add a struct iommu_user_data_array and a helper for driver to sanitize
and copy the entry data from user space invalidation array
- Add a patch fixing TEST_LENGTH() in selftest program
- Drop IOMMU_RESV_IOVA_RANGES patches
- Update kdoc and inline comments
- Drop the code to add IOMMU_RESV_SW_MSI to kernel-managed HWPT in nested translation,
this does not change the rule that resv regions should only be added to the
kernel-managed HWPT. The IOMMU_RESV_SW_MSI stuff will be added in later series
as it is needed only by SMMU so far.
v3: https://lore.kernel.org/linux-iommu/20230724110406.107212-1-yi.l.liu@intel.com/
- Add new uAPI things in alphabetical order
- Pass in "enum iommu_hwpt_type hwpt_type" to op->domain_alloc_user for
sanity, replacing the previous op->domain_alloc_user_data_len solution
- Return ERR_PTR from domain_alloc_user instead of NULL
- Only add IOMMU_RESV_SW_MSI to kernel-managed HWPT in nested translation (Kevin)
- Add IOMMU_RESV_IOVA_RANGES to report resv iova ranges to userspace hence
userspace is able to exclude the ranges in the stage-1 HWPT (e.g. guest I/O
page table). (Kevin)
- Add selftest coverage for the new IOMMU_RESV_IOVA_RANGES ioctl
- Minor changes per Kevin's inputs
v2: https://lore.kernel.org/linux-iommu/20230511143844.22693-1-yi.l.liu@intel.com/
- Add union iommu_domain_user_data to include all user data structures to avoid
passing void * in kernel APIs.
- Add iommu op to return user data length for user domain allocation
- Rename struct iommu_hwpt_alloc::data_type to be hwpt_type
- Store the invalidation data length in iommu_domain_ops::cache_invalidate_user_data_len
- Convert cache_invalidate_user op to be int instead of void
- Remove @data_type in struct iommu_hwpt_invalidate
- Remove out_hwpt_type_bitmap in struct iommu_hw_info hence drop patch 08 of v1
v1: https://lore.kernel.org/linux-iommu/20230309080910.607396-1-yi.l.liu@intel.com/
Thanks,
Yi Liu
Lu Baolu (1):
iommu: Add nested domain support
Nicolin Chen (12):
iommufd: Unite all kernel-managed members into a struct
iommufd: Separate kernel-managed HWPT alloc/destroy/abort functions
iommufd: Add shared alloc_fn function pointer and mutex pointer
iommufd: Add user-managed hw_pagetable support
iommufd: Always setup MSI and anforce cc on kernel-managed domains
iommufd/device: Add helpers to enforce/remove device reserved regions
iommufd/selftest: Rework TEST_LENGTH to test min_size explicitly
iommufd/selftest: Add nested domain allocation for mock domain
iommufd/selftest: Add coverage for IOMMU_HWPT_ALLOC with nested HWPTs
iommufd/selftest: Add mock_domain_cache_invalidate_user support
iommufd/selftest: Add IOMMU_TEST_OP_MD_CHECK_IOTLB test op
iommufd/selftest: Add coverage for IOMMU_HWPT_INVALIDATE ioctl
Yi Liu (4):
iommu: Add hwpt_type with user_data for domain_alloc_user op
iommufd: Pass in hwpt_type/user_data to iommufd_hw_pagetable_alloc()
iommufd: Support IOMMU_HWPT_ALLOC allocation with user data
iommufd: Add IOMMU_HWPT_INVALIDATE
drivers/iommu/intel/iommu.c | 5 +-
drivers/iommu/iommufd/device.c | 51 +++-
drivers/iommu/iommufd/hw_pagetable.c | 257 ++++++++++++++++--
drivers/iommu/iommufd/iommufd_private.h | 59 +++-
drivers/iommu/iommufd/iommufd_test.h | 40 +++
drivers/iommu/iommufd/main.c | 3 +
drivers/iommu/iommufd/selftest.c | 184 ++++++++++++-
include/linux/iommu.h | 110 +++++++-
include/uapi/linux/iommufd.h | 60 +++-
tools/testing/selftests/iommu/iommufd.c | 209 +++++++++++++-
.../selftests/iommu/iommufd_fail_nth.c | 3 +-
tools/testing/selftests/iommu/iommufd_utils.h | 91 ++++++-
12 files changed, 998 insertions(+), 74 deletions(-)
--
2.34.1
next reply other threads:[~2023-09-21 7:52 UTC|newest]
Thread overview: 106+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-21 7:51 Yi Liu [this message]
2023-09-21 7:51 ` [PATCH v4 01/17] iommu: Add hwpt_type with user_data for domain_alloc_user op Yi Liu
2023-09-21 12:10 ` Baolu Lu
2023-09-21 20:58 ` Nicolin Chen
2023-09-25 13:05 ` Jason Gunthorpe
2023-09-25 18:17 ` Nicolin Chen
2023-09-25 6:22 ` Yi Liu
2023-09-25 8:01 ` Baolu Lu
2023-09-25 13:00 ` Jason Gunthorpe
2023-09-21 12:12 ` Baolu Lu
2023-09-21 16:44 ` Jason Gunthorpe
2023-09-22 9:47 ` Robin Murphy
2023-09-25 6:17 ` Yi Liu
2023-09-25 12:59 ` Jason Gunthorpe
2023-09-26 6:37 ` Tian, Kevin
2023-09-26 16:29 ` Jason Gunthorpe
2023-09-27 1:08 ` Tian, Kevin
2023-10-07 9:38 ` Yi Liu
2023-09-26 6:56 ` Tian, Kevin
2023-10-12 9:12 ` Yi Liu
2023-10-13 11:42 ` Yi Liu
2023-10-13 22:22 ` Nicolin Chen
2023-10-10 16:58 ` Jason Gunthorpe
2023-10-12 9:11 ` Yi Liu
2023-10-12 13:39 ` Jason Gunthorpe
2023-10-13 4:33 ` Yi Liu
2023-10-13 14:04 ` Jason Gunthorpe
2023-10-13 17:56 ` Nicolin Chen
2023-10-16 3:28 ` Yi Liu
2023-10-16 11:54 ` Jason Gunthorpe
2023-10-16 18:17 ` Nicolin Chen
2023-10-17 8:51 ` Yi Liu
2023-10-17 9:28 ` Tian, Kevin
2023-10-18 6:12 ` Yi Liu
2023-10-18 16:37 ` Jason Gunthorpe
2023-10-18 16:51 ` Nicolin Chen
2023-10-13 0:34 ` Nicolin Chen
2023-10-13 14:03 ` Jason Gunthorpe
2023-09-21 7:51 ` [PATCH v4 02/17] iommu: Add nested domain support Yi Liu
2023-10-10 17:21 ` Jason Gunthorpe
2023-10-12 9:13 ` Yi Liu
2023-10-14 0:47 ` Nicolin Chen
2023-09-21 7:51 ` [PATCH v4 03/17] iommufd: Unite all kernel-managed members into a struct Yi Liu
2023-09-26 7:46 ` Tian, Kevin
2023-10-07 10:08 ` Yan Zhao
2023-10-09 4:13 ` Yi Liu
2023-10-09 5:13 ` Yan Zhao
2023-10-10 3:26 ` Yi Liu
2023-09-21 7:51 ` [PATCH v4 04/17] iommufd: Pass in hwpt_type/user_data to iommufd_hw_pagetable_alloc() Yi Liu
2023-09-21 7:51 ` [PATCH v4 05/17] iommufd: Separate kernel-managed HWPT alloc/destroy/abort functions Yi Liu
2023-09-26 7:57 ` Tian, Kevin
2023-10-10 18:49 ` Jason Gunthorpe
2023-10-12 19:09 ` Jason Gunthorpe
2023-10-13 7:13 ` Tian, Kevin
2023-10-13 14:05 ` Jason Gunthorpe
2023-10-16 8:26 ` Tian, Kevin
2023-10-16 12:02 ` Jason Gunthorpe
2023-09-21 7:51 ` [PATCH v4 06/17] iommufd: Add shared alloc_fn function pointer and mutex pointer Yi Liu
2023-09-21 7:51 ` [PATCH v4 07/17] iommufd: Add user-managed hw_pagetable support Yi Liu
2023-09-26 8:14 ` Tian, Kevin
2023-10-14 0:08 ` Nicolin Chen
2023-09-21 7:51 ` [PATCH v4 08/17] iommufd: Always setup MSI and anforce cc on kernel-managed domains Yi Liu
2023-09-26 8:16 ` Tian, Kevin
2023-10-14 0:44 ` Nicolin Chen
2023-10-16 8:48 ` Tian, Kevin
2023-10-16 11:57 ` Jason Gunthorpe
2023-10-17 8:52 ` Tian, Kevin
2023-10-17 15:53 ` Jason Gunthorpe
2023-10-17 19:58 ` Nicolin Chen
2023-10-18 16:51 ` Jason Gunthorpe
2023-10-19 1:56 ` Tian, Kevin
2023-10-19 23:53 ` Jason Gunthorpe
2023-10-20 2:43 ` Tian, Kevin
2023-10-20 13:55 ` Jason Gunthorpe
2023-10-20 18:59 ` Nicolin Chen
2023-10-21 16:38 ` Jason Gunthorpe
2023-10-23 0:18 ` Nicolin Chen
2023-10-23 2:53 ` Tian, Kevin
2023-10-23 18:42 ` Nicolin Chen
2023-10-23 13:59 ` Jason Gunthorpe
2023-10-23 18:49 ` Nicolin Chen
2023-10-18 2:32 ` Baolu Lu
2023-10-17 9:05 ` Tian, Kevin
2023-09-21 7:51 ` [PATCH v4 09/17] iommufd/device: Add helpers to enforce/remove device reserved regions Yi Liu
2023-10-07 7:20 ` Yan Zhao
2023-10-07 9:27 ` Nicolin Chen
2023-09-21 7:51 ` [PATCH v4 10/17] iommufd: Support IOMMU_HWPT_ALLOC allocation with user data Yi Liu
2023-10-13 15:19 ` Jason Gunthorpe
2023-10-13 20:58 ` Nicolin Chen
2023-10-14 0:07 ` Jason Gunthorpe
2023-10-14 0:51 ` Nicolin Chen
2023-10-16 7:03 ` Yi Liu
2023-10-16 11:59 ` Jason Gunthorpe
2023-10-16 18:44 ` Nicolin Chen
2023-10-17 8:55 ` Yi Liu
2023-10-17 15:50 ` Jason Gunthorpe
2023-10-17 19:32 ` Nicolin Chen
2023-09-21 7:51 ` [PATCH v4 11/17] iommufd: Add IOMMU_HWPT_INVALIDATE Yi Liu
2023-09-21 7:51 ` [PATCH v4 12/17] iommufd/selftest: Rework TEST_LENGTH to test min_size explicitly Yi Liu
2023-09-21 7:51 ` [PATCH v4 13/17] iommufd/selftest: Add nested domain allocation for mock domain Yi Liu
2023-09-21 7:51 ` [PATCH v4 14/17] iommufd/selftest: Add coverage for IOMMU_HWPT_ALLOC with nested HWPTs Yi Liu
2023-09-21 7:51 ` [PATCH v4 15/17] iommufd/selftest: Add mock_domain_cache_invalidate_user support Yi Liu
2023-09-21 7:51 ` [PATCH v4 16/17] iommufd/selftest: Add IOMMU_TEST_OP_MD_CHECK_IOTLB test op Yi Liu
2023-09-21 7:51 ` [PATCH v4 17/17] iommufd/selftest: Add coverage for IOMMU_HWPT_INVALIDATE ioctl Yi Liu
2023-10-10 16:53 ` [PATCH v4 00/17] iommufd: Add nesting infrastructure Jason Gunthorpe
2023-10-12 8:45 ` Yi Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230921075138.124099-1-yi.l.liu@intel.com \
--to=yi.l.liu@intel.com \
--cc=alex.williamson@redhat.com \
--cc=baolu.lu@linux.intel.com \
--cc=chao.p.peng@linux.intel.com \
--cc=cohuck@redhat.com \
--cc=eric.auger@redhat.com \
--cc=iommu@lists.linux.dev \
--cc=jasowang@redhat.com \
--cc=jgg@nvidia.com \
--cc=joao.m.martins@oracle.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=lulu@redhat.com \
--cc=mjrosato@linux.ibm.com \
--cc=nicolinc@nvidia.com \
--cc=peterx@redhat.com \
--cc=robin.murphy@arm.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=suravee.suthikulpanit@amd.com \
--cc=yi.y.sun@linux.intel.com \
--cc=zhenzhong.duan@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).