Re: [Qemu-devel] [RFC PATCH 7/8] VFIO: Add new IOCTL for IOMMU TLB invalidate propagation - Jean-Philippe Brucker

From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: "Liu, Yi L" <yi.l.liu@intel.com>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"Liu, Yi L" <yi.l.liu@linux.intel.com>,
	"Lan, Tianyu" <tianyu.lan@intel.com>,
	"Raj, Ashok" <ashok.raj@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"jasowang@redhat.com" <jasowang@redhat.com>,
	Will Deacon <Will.Deacon@arm.com>,
	"peterx@redhat.com" <peterx@redhat.com>,
	"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"Pan, Jacob jun" <jacob.jun.pan@intel.com>,
	Joerg Roedel <joro@8bytes.org>
Subject: Re: [Qemu-devel] [RFC PATCH 7/8] VFIO: Add new IOCTL for IOMMU TLB invalidate propagation
Date: Tue, 18 Jul 2017 16:03:15 +0100	[thread overview]
Message-ID: <c049a793-9d0e-8c2e-6c2b-f8d1625cfef9@arm.com> (raw)
In-Reply-To: <20170718082907.69893dda@w520.home>

On 18/07/17 15:29, Alex Williamson wrote:
> On Tue, 18 Jul 2017 10:38:40 +0100
> Jean-Philippe Brucker <jean-philippe.brucker@arm.com> wrote:
> 
>> On 17/07/17 23:45, Alex Williamson wrote:
>> [..]
>>>>>
>>>>> How does a user learn which model(s) are supported by the interface?
>>>>> How do they learn which ops are supported?  Perhaps a good use for one of those
>>>>> flag bits in the outer data structure is "probe".    
>>>>
>>>> My initial plan to user fills it, if the underlying HW doesn't support the
>>>> model, it refuses to service it. User should get a failure and stop to use
>>>> it. But your suggestion to have a probe or kinds of query makes sense.
>>>> How about we add one more operation for such purpose? Besides the
>>>> supported model query, I'd like to add more. E.g the HW IOMMU capabilities.  
>>>
>>> We also have VFIO_IOMMU_GET_INFO where the structure can be extended
>>> for missing capabilities.  Depending on the capability you want to
>>> describe, this might be a better, existing interface for it.  
>>
>> It might become hairy when physical IOMMUs start supporting multiple
>> formats, or when we want to describe multiple page table formats in
>> addition to PASID tables. I was wondering if sysfs iommu_group would be
>> better suited for this kind of hardware description with variable-length
>> properties, but a new ioctl-based probing mechanism would work as well.
> 
> Would different groups have different properties?  Perhaps it's related
> to the iommu hardware unit supporting the device, which could host one
> or more groups.  Each device already has a link to its iommu where we
> could add info (/sys/class/iommu/).

Yes, /sys/class/iommu might be better than iommu_group for PASID and page
table formats.

>> Other things that we'll want to describe are fault reporting capability
>> and PASID range, which would fit better in vfio_device_info.
> 
> Why?  The per device PASID info is in a PCI capability, so wouldn't
> this be iommu info?  Isn't the fault reporting also via the iommu?

Ah yes, I missed that. If userspace can read the PASID and PRI
capabilities it should be enough.

Inspecting individual device capability might help userspace decide how to
combine multiple devices in a container. For example, if it puts
PASID-capable and non-PASID-capable device in the same container, the
container probably wouldn't support PASID (but would still support MAP/UNMAP).

I'm not sure how it will work for platform devices though, some integrated
devices on ARM will support features resembling PASID and PRI. I suspect
these will need ACPI/DT description anyway, that userspace would access
via sysfs.

Thanks,
Jean