From: Nicolin Chen <nicolinc@nvidia.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: <will@kernel.org>, <robin.murphy@arm.com>, <kevin.tian@intel.com>,
<suravee.suthikulpanit@amd.com>, <joro@8bytes.org>,
<linux-kernel@vger.kernel.org>, <iommu@lists.linux.dev>,
<linux-arm-kernel@lists.infradead.org>,
<linux-tegra@vger.kernel.org>, <yi.l.liu@intel.com>,
<eric.auger@redhat.com>, <vasant.hegde@amd.com>,
<jon.grimm@amd.com>, <santosh.shukla@amd.com>,
<Dhaval.Giani@amd.com>, <shameerali.kolothum.thodi@huawei.com>
Subject: Re: [PATCH RFCv1 05/14] iommufd: Add IOMMUFD_OBJ_VIOMMU and IOMMUFD_CMD_VIOMMU_ALLOC
Date: Tue, 14 May 2024 18:20:06 -0700 [thread overview]
Message-ID: <ZkQNxkYv23z7i6e0@nvidia.com> (raw)
In-Reply-To: <ZkOFkfHhG2h2fv/c@nvidia.com>
On Tue, May 14, 2024 at 12:38:57PM -0300, Jason Gunthorpe wrote:
> > > > +
> > > > +/**
> > > > + * enum iommu_viommu_type - VIOMMU Type
> > > > + * @IOMMU_VIOMMU_TEGRA241_CMDQV: NVIDIA Tegra241 CMDQV Extension for SMMUv3
> > > > + */
> > > > +enum iommu_viommu_type {
> > > > + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
> > > > +};
> > >
> > > At least the 241 line should be in a following patch
> >
> > It's for the "enum iommu_viommu_type" mentioned in the following
> > structure. Yi told me that you don't like an empty enum, and he
> > did something like this in HWPT_INVALIDATE series:
> > https://lore.kernel.org/linux-iommu/20240111041015.47920-3-yi.l.liu@intel.com/
>
> I suspect 0 should be reserved as a non-set value for some
> basic sanity in all these driver type enums.
We have an IOMMU_HWPT_DATA_NONE for HWPT_ALLOC to compatible
with an S2 hwpt, since it doesn't need a data.
Maybe we can have an IOMMU_VIOMMU_TYPE_DEFAULT to be 0, for
an IOMMU driver (e.g. VT-d) that doesn't need to handle nor
be aware of any viommu object?
So, VMM can have a unified "attach-to-viommu" practice with
different IOMMUs, v.s. some still doing "attach-to-s2"?
> > > So, to make this all work perfectly we need approx the following
> > > - S2 sharing across instances in ARM - meaning the VMID is allocated
> > > at attach not domain alloc
> > > - S2 hwpt is refcounted by the VIOMMU in the iommufd layer
> > > - VIOMMU is refcounted by every nesting child in the iommufd layer
> > > - The nesting child holds a pointer to both the S2 and the VIOMMU
> > > (viommu optional)
> > > - When the nesting child attaches to a device the STE will source the
> > > VMID from the VIOMMU if present otherwise from the S2
> > > - "RID" attach (ie naked S2) will have to be done with a Nesting
> > > Child using a vSTE that indicates Identity. Then the attach logic
> > > will have enough information to get the VMID from the VIOMMU
> >
> > What is this RID attach (naked S2) case? S1DSS_BYPASS + SVA?
>
> No, when the guest installs a vSTE that simply says bypass with no CD
> table pointer. That should result in a pSTE that is the S2 with on CD
> pointer.
>
> I was originally thinking that the VMM would simply directly attach
> the S2 HWPT in this caes, but given the above issue with the VMID lifetime
> it makes more sense to 'attach' the viommu which holds the correct
> VMID.
>
> The issue with direct attach the S2 HWPT is the VMID lifetime, as it
> would have to borrow the VMID from the viommu but then the lifetime
> becomes more complex as it has to live beyond VIOMMU destruction. Not
> unsolvable but it seems easier to just avoid it entirely.
That makes a lot sense. I'd need to go through QEMU code and
see how we will accommodate these two more naturally: likely
the QEMU core should allocate an S2 HWPT for a VM, while the
viommu code should allocate a VIOMMU for each instance.
Thanks
Nicolin
next prev parent reply other threads:[~2024-05-15 1:20 UTC|newest]
Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-13 3:46 [PATCH RFCv1 00/14] Add Tegra241 (Grace) CMDQV Support (part 2/2) Nicolin Chen
2024-04-13 3:46 ` [PATCH RFCv1 01/14] iommufd: Move iommufd_object to public iommufd header Nicolin Chen
2024-05-12 13:21 ` Jason Gunthorpe
2024-05-12 22:40 ` Nicolin Chen
2024-05-13 22:30 ` Jason Gunthorpe
2024-04-13 3:46 ` [PATCH RFCv1 02/14] iommufd: Swap _iommufd_object_alloc and __iommufd_object_alloc Nicolin Chen
2024-05-12 13:26 ` Jason Gunthorpe
2024-05-13 2:29 ` Nicolin Chen
2024-05-13 3:44 ` Nicolin Chen
2024-05-13 22:30 ` Jason Gunthorpe
2024-04-13 3:47 ` [PATCH RFCv1 03/14] iommufd: Prepare for viommu structures and functions Nicolin Chen
2024-05-12 13:42 ` Jason Gunthorpe
2024-05-13 2:35 ` Nicolin Chen
2024-04-13 3:47 ` [PATCH RFCv1 04/14] iommufd: Add struct iommufd_viommu and iommufd_viommu_ops Nicolin Chen
2024-05-12 14:03 ` Jason Gunthorpe
2024-05-13 3:34 ` Nicolin Chen
2024-05-14 15:55 ` Jason Gunthorpe
2024-05-22 8:58 ` Tian, Kevin
2024-05-22 9:57 ` Baolu Lu
2024-05-22 13:39 ` Jason Gunthorpe
2024-05-23 1:43 ` Tian, Kevin
2024-05-23 4:01 ` Nicolin Chen
2024-05-23 5:40 ` Tian, Kevin
2024-05-23 12:58 ` Jason Gunthorpe
2024-05-24 2:16 ` Tian, Kevin
2024-05-24 13:03 ` Jason Gunthorpe
2024-05-24 2:36 ` Tian, Kevin
2024-04-13 3:47 ` [PATCH RFCv1 05/14] iommufd: Add IOMMUFD_OBJ_VIOMMU and IOMMUFD_CMD_VIOMMU_ALLOC Nicolin Chen
2024-05-12 14:27 ` Jason Gunthorpe
2024-05-13 4:33 ` Nicolin Chen
2024-05-14 15:38 ` Jason Gunthorpe
2024-05-15 1:20 ` Nicolin Chen [this message]
2024-05-21 18:05 ` Jason Gunthorpe
2024-05-22 0:13 ` Nicolin Chen
2024-05-22 16:46 ` Jason Gunthorpe
2024-04-13 3:47 ` [PATCH RFCv1 06/14] iommufd/selftest: Add IOMMU_VIOMMU_ALLOC test coverage Nicolin Chen
2024-04-13 3:47 ` [PATCH RFCv1 07/14] iommufd: Add viommu set/unset_dev_id ops Nicolin Chen
2024-05-12 14:46 ` Jason Gunthorpe
2024-05-13 4:39 ` Nicolin Chen
2024-05-14 15:53 ` Jason Gunthorpe
2024-05-15 1:59 ` Nicolin Chen
2024-05-21 18:24 ` Jason Gunthorpe
2024-05-21 22:27 ` Nicolin Chen
2024-05-22 13:59 ` Jason Gunthorpe
2024-05-23 6:19 ` Tian, Kevin
2024-05-23 15:01 ` Jason Gunthorpe
2024-05-24 2:21 ` Tian, Kevin
2024-05-24 3:26 ` Nicolin Chen
2024-05-24 5:24 ` Tian, Kevin
2024-05-24 5:57 ` Nicolin Chen
2024-05-24 7:21 ` Tian, Kevin
2024-05-24 13:12 ` Jason Gunthorpe
2024-05-24 13:05 ` Jason Gunthorpe
2024-05-23 5:44 ` Tian, Kevin
2024-05-23 6:09 ` Nicolin Chen
2024-05-23 6:22 ` Tian, Kevin
2024-05-23 13:33 ` Jason Gunthorpe
2024-05-12 14:51 ` Jason Gunthorpe
2024-04-13 3:47 ` [PATCH RFCv1 08/14] iommufd: Add IOMMU_VIOMMU_SET_DEV_ID ioctl Nicolin Chen
2024-05-12 14:58 ` Jason Gunthorpe
2024-05-13 5:24 ` Nicolin Chen
2024-05-17 5:14 ` Nicolin Chen
2024-05-21 18:30 ` Jason Gunthorpe
2024-05-22 2:15 ` Nicolin Chen
2024-05-23 6:42 ` Tian, Kevin
2024-05-24 5:40 ` Nicolin Chen
2024-05-24 7:13 ` Tian, Kevin
2024-05-24 13:19 ` Jason Gunthorpe
2024-05-27 1:08 ` Tian, Kevin
2024-05-28 20:22 ` Nicolin Chen
2024-05-28 20:33 ` Nicolin Chen
2024-05-29 2:58 ` Tian, Kevin
2024-05-29 3:20 ` Nicolin Chen
2024-05-28 20:30 ` Nicolin Chen
2024-05-24 13:20 ` Jason Gunthorpe
2024-04-13 3:47 ` [PATCH RFCv1 09/14] iommufd/selftest: Add IOMMU_VIOMMU_SET_DEV_ID test coverage Nicolin Chen
2024-04-13 3:47 ` [PATCH RFCv1 10/14] iommufd/selftest: Add IOMMU_TEST_OP_MV_CHECK_DEV_ID Nicolin Chen
2024-04-13 3:47 ` [PATCH RFCv1 11/14] iommufd: Add struct iommufd_vqueue and its related viommu ops Nicolin Chen
2024-04-13 3:47 ` [PATCH RFCv1 12/14] iommufd: Add IOMMUFD_OBJ_VQUEUE and IOMMUFD_CMD_VQUEUE_ALLOC Nicolin Chen
2024-05-12 15:02 ` Jason Gunthorpe
2024-05-13 4:41 ` Nicolin Chen
2024-05-13 22:36 ` Jason Gunthorpe
2024-05-23 6:57 ` Tian, Kevin
2024-05-24 4:42 ` Nicolin Chen
2024-05-24 5:26 ` Tian, Kevin
2024-05-24 6:03 ` Nicolin Chen
2024-05-23 7:05 ` Tian, Kevin
2024-04-13 3:47 ` [PATCH RFCv1 13/14] iommufd: Add mmap infrastructure Nicolin Chen
2024-05-12 15:19 ` Jason Gunthorpe
2024-05-13 4:43 ` Nicolin Chen
2024-04-13 3:47 ` [PATCH RFCv1 14/14] iommu/tegra241-cmdqv: Add user-space use support Nicolin Chen
2024-05-22 8:40 ` [PATCH RFCv1 00/14] Add Tegra241 (Grace) CMDQV Support (part 2/2) Tian, Kevin
2024-05-22 16:48 ` Jason Gunthorpe
2024-05-22 19:47 ` Nicolin Chen
2024-05-22 23:28 ` Jason Gunthorpe
2024-05-22 23:43 ` Tian, Kevin
2024-05-23 3:09 ` Nicolin Chen
2024-05-23 12:48 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZkQNxkYv23z7i6e0@nvidia.com \
--to=nicolinc@nvidia.com \
--cc=Dhaval.Giani@amd.com \
--cc=eric.auger@redhat.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=jon.grimm@amd.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=robin.murphy@arm.com \
--cc=santosh.shukla@amd.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=suravee.suthikulpanit@amd.com \
--cc=vasant.hegde@amd.com \
--cc=will@kernel.org \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).