Re: [PATCH v2 00/11] Connect VFIO to IOMMUFD

From: Jason Gunthorpe <jgg@nvidia.com>
To: Matthew Rosato <mjrosato@linux.ibm.com>
Cc: kvm@vger.kernel.org, Kevin Tian <kevin.tian@intel.com>,
	dri-devel@lists.freedesktop.org,
	Vineeth Vijayan <vneethv@linux.ibm.com>,
	Diana Craciun <diana.craciun@oss.nxp.com>,
	Alexander Gordeev <agordeev@linux.ibm.com>,
	Longfang Liu <liulongfang@huawei.com>,
	linux-s390@vger.kernel.org, "Liu, Yi L" <yi.l.liu@intel.com>,
	Will Deacon <will@kernel.org>, Joerg Roedel <joro@8bytes.org>,
	Halil Pasic <pasic@linux.ibm.com>,
	iommu@lists.linux.dev, Nicolin Chen <nicolinc@nvidia.com>,
	Christian Borntraeger <borntraeger@linux.ibm.com>,
	intel-gfx@lists.freedesktop.org, Zhi Wang <zhi.a.wang@intel.com>,
	Jason Herne <jjherne@linux.ibm.com>,
	Eric Farman <farman@linux.ibm.com>,
	Vasily Gorbik <gor@linux.ibm.com>,
	Heiko Carstens <hca@linux.ibm.com>,
	Eric Auger <eric.auger@redhat.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	Harald Freudenberger <freude@linux.ibm.com>,
	Rodrigo Vivi <rodrigo.vivi@intel.com>,
	intel-gvt-dev@lists.freedesktop.org,
	Tony Krowiak <akrowiak@linux.ibm.com>,
	Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
	Yishai Hadas <yishaih@nvidia.com>,
	Cornelia Huck <cohuck@redhat.com>,
	Peter Oberparleiter <oberpar@linux.ibm.com>,
	Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>,
	Sven Schnelle <svens@linux.ibm.com>,
	Robin Murphy <robin.murphy@arm.com>,
	Lu Baolu <baolu.lu@linux.intel.com>
Subject: Re: [PATCH v2 00/11] Connect VFIO to IOMMUFD
Date: Mon, 14 Nov 2022 10:23:35 -0400	[thread overview]
Message-ID: <Y3JPZwEHjGDU4Oyq@nvidia.com> (raw)
In-Reply-To: <f54c2a93-539f-4520-0561-fbe5cea7772f@linux.ibm.com>

On Thu, Nov 10, 2022 at 10:01:13PM -0500, Matthew Rosato wrote:
> On 11/7/22 7:52 PM, Jason Gunthorpe wrote:
> > This series provides an alternative container layer for VFIO implemented
> > using iommufd. This is optional, if CONFIG_IOMMUFD is not set then it will
> > not be compiled in.
> > 
> > At this point iommufd can be injected by passing in a iommfd FD to
> > VFIO_GROUP_SET_CONTAINER which will use the VFIO compat layer in iommufd
> > to obtain the compat IOAS and then connect up all the VFIO drivers as
> > appropriate.
> > 
> > This is temporary stopping point, a following series will provide a way to
> > directly open a VFIO device FD and directly connect it to IOMMUFD using
> > native ioctls that can expose the IOMMUFD features like hwpt, future
> > vPASID and dynamic attachment.
> > 
> > This series, in compat mode, has passed all the qemu tests we have
> > available, including the test suites for the Intel GVT mdev. Aside from
> > the temporary limitation with P2P memory this is belived to be fully
> > compatible with VFIO.
> 
> AFAICT there is no equivalent means to specify
> vfio_iommu_type1.dma_entry_limit when using iommufd; looks like
> we'll just always get the default 65535.

No, there is no arbitary limit on iommufd

> Was this because you envision the limit being not applicable for
> iommufd (limits will be enforced via either means and eventually we
> won't want to ) or was it an oversight?

The limit here is primarily about limiting userspace abuse of the
interface.

iommufd is using GFP_KERNEL_ACCOUNT which shifts the responsiblity to
cgroups, which is similar to how KVM works.

So, for a VM sandbox you'd set a cgroup limit and if a hostile
userspace in the sanbox decides to try to OOM the system it will hit
that limit, regardless of which kernel APIs it tries to abuse.

This work is not entirely complete as we also need the iommu driver to
use GFP_KERNEL_ACCOUNT for allocations connected to the iommu_domain,
particularly for allocations of the IO page tables themselves - which
can be quite big.

Jason