From: Auger Eric <eric.auger@redhat.com>
To: Robin Murphy <robin.murphy@arm.com>,
eric.auger.pro@gmail.com, iommu@lists.linux-foundation.org,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
kvmarm@lists.cs.columbia.edu, joro@8bytes.org,
alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com,
yi.l.liu@intel.com, jean-philippe.brucker@arm.com,
will.deacon@arm.com
Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com,
christoffer.dall@arm.com, peter.maydell@linaro.org,
vincent.stehle@arm.com
Subject: Re: [PATCH v7 06/23] iommu: Introduce bind/unbind_guest_msi
Date: Fri, 10 May 2019 16:35:15 +0200 [thread overview]
Message-ID: <af36c889-8405-6de6-0256-4c157dc17aea@redhat.com> (raw)
In-Reply-To: <a11e6535-9e9e-ed8d-9d19-1f9d895effa6@arm.com>
Hi Robin,
On 5/8/19 3:59 PM, Robin Murphy wrote:
> On 08/04/2019 13:18, Eric Auger wrote:
>> On ARM, MSI are translated by the SMMU. An IOVA is allocated
>> for each MSI doorbell. If both the host and the guest are exposed
>> with SMMUs, we end up with 2 different IOVAs allocated by each.
>> guest allocates an IOVA (gIOVA) to map onto the guest MSI
>> doorbell (gDB). The Host allocates another IOVA (hIOVA) to map
>> onto the physical doorbell (hDB).
>>
>> So we end up with 2 untied mappings:
>> S1 S2
>> gIOVA -> gDB
>> hIOVA -> hDB
>>
>> Currently the PCI device is programmed by the host with hIOVA
>> as MSI doorbell. So this does not work.
>>
>> This patch introduces an API to pass gIOVA/gDB to the host so
>> that gIOVA can be reused by the host instead of re-allocating
>> a new IOVA. So the goal is to create the following nested mapping:
>>
>> S1 S2
>> gIOVA -> gDB -> hDB
>>
>> and program the PCI device with gIOVA MSI doorbell.
>>
>> In case we have several devices attached to this nested domain
>> (devices belonging to the same group), they cannot be isolated
>> on guest side either. So they should also end up in the same domain
>> on guest side. We will enforce that all the devices attached to
>> the host iommu domain use the same physical doorbell and similarly
>> a single virtual doorbell mapping gets registered (1 single
>> virtual doorbell is used on guest as well).
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>> v6 -> v7:
>> - remove the device handle parameter.
>> - Add comments saying there can only be a single MSI binding
>> registered per iommu_domain
>> v5 -> v6:
>> -fix compile issue when IOMMU_API is not set
>>
>> v3 -> v4:
>> - add unbind
>>
>> v2 -> v3:
>> - add a struct device handle
>> ---
>> drivers/iommu/iommu.c | 37 +++++++++++++++++++++++++++++++++++++
>> include/linux/iommu.h | 23 +++++++++++++++++++++++
>> 2 files changed, 60 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index 6d6cb4005ca5..0d160bbd6f81 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -1575,6 +1575,43 @@ static void __iommu_detach_device(struct
>> iommu_domain *domain,
>> trace_detach_device_from_domain(dev);
>> }
>> +/**
>> + * iommu_bind_guest_msi - Passes the stage1 GIOVA/GPA mapping of a
>> + * virtual doorbell
>> + *
>> + * @domain: iommu domain the stage 1 mapping will be attached to
>> + * @iova: iova allocated by the guest
>> + * @gpa: guest physical address of the virtual doorbell
>> + * @size: granule size used for the mapping
>> + *
>> + * The associated IOVA can be reused by the host to create a nested
>> + * stage2 binding mapping translating into the physical doorbell used
>> + * by the devices attached to the domain.
>> + *
>> + * All devices within the domain must share the same physical doorbell.
>> + * A single MSI GIOVA/GPA mapping can be attached to an iommu_domain.
>> + */
>> +
>> +int iommu_bind_guest_msi(struct iommu_domain *domain,
>> + dma_addr_t giova, phys_addr_t gpa, size_t size)
>> +{
>> + if (unlikely(!domain->ops->bind_guest_msi))
>> + return -ENODEV;
>> +
>> + return domain->ops->bind_guest_msi(domain, giova, gpa, size);
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_bind_guest_msi);
>> +
>> +void iommu_unbind_guest_msi(struct iommu_domain *domain,
>> + dma_addr_t iova)
>> +{
>> + if (unlikely(!domain->ops->unbind_guest_msi))
>> + return;
>> +
>> + domain->ops->unbind_guest_msi(domain, iova);
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_unbind_guest_msi);
>> +
>> void iommu_detach_device(struct iommu_domain *domain, struct device
>> *dev)
>> {
>> struct iommu_group *group;
>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
>> index 7c7c6bad1420..a2f3f964ead2 100644
>> --- a/include/linux/iommu.h
>> +++ b/include/linux/iommu.h
>> @@ -192,6 +192,8 @@ struct iommu_resv_region {
>> * @attach_pasid_table: attach a pasid table
>> * @detach_pasid_table: detach the pasid table
>> * @cache_invalidate: invalidate translation caches
>> + * @bind_guest_msi: provides a stage1 giova/gpa MSI doorbell mapping
>> + * @unbind_guest_msi: withdraw a stage1 giova/gpa MSI doorbell mapping
>> * @pgsize_bitmap: bitmap of all possible supported page sizes
>> */
>> struct iommu_ops {
>> @@ -243,6 +245,10 @@ struct iommu_ops {
>> int (*cache_invalidate)(struct iommu_domain *domain, struct
>> device *dev,
>> struct iommu_cache_invalidate_info *inv_info);
>> + int (*bind_guest_msi)(struct iommu_domain *domain,
>> + dma_addr_t giova, phys_addr_t gpa, size_t size);
>> + void (*unbind_guest_msi)(struct iommu_domain *domain, dma_addr_t
>> giova);
>> +
>> unsigned long pgsize_bitmap;
>> };
>> @@ -356,6 +362,11 @@ extern void iommu_detach_pasid_table(struct
>> iommu_domain *domain);
>> extern int iommu_cache_invalidate(struct iommu_domain *domain,
>> struct device *dev,
>> struct iommu_cache_invalidate_info *inv_info);
>> +extern int iommu_bind_guest_msi(struct iommu_domain *domain,
>> + dma_addr_t giova, phys_addr_t gpa, size_t size);
>> +extern void iommu_unbind_guest_msi(struct iommu_domain *domain,
>> + dma_addr_t giova);
>> +
>> extern struct iommu_domain *iommu_get_domain_for_dev(struct device
>> *dev);
>> extern struct iommu_domain *iommu_get_dma_domain(struct device *dev);
>> extern int iommu_map(struct iommu_domain *domain, unsigned long iova,
>> @@ -812,6 +823,18 @@ iommu_cache_invalidate(struct iommu_domain *domain,
>> return -ENODEV;
>> }
>> +static inline
>> +int iommu_bind_guest_msi(struct iommu_domain *domain,
>> + dma_addr_t giova, phys_addr_t gpa, size_t size)
>> +{
>> + return -ENODEV;
>> +}
>> +static inline
>> +int iommu_unbind_guest_msi(struct iommu_domain *domain, dma_addr_t
>> giova)
>> +{
>> + return -ENODEV;
>
> It's less of a problem than mismatching the other way round, but for
> consistency this should return void like the real version.
thank you for spotting this.
Eric
>
> Robin.
>
>> +}
>> +
>> #endif /* CONFIG_IOMMU_API */
>> #ifdef CONFIG_IOMMU_DEBUGFS
>>
next prev parent reply other threads:[~2019-05-10 14:35 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-08 12:18 [PATCH v7 00/23] SMMUv3 Nested Stage Setup Eric Auger
2019-04-08 12:18 ` [PATCH v7 01/23] driver core: add per device iommu param Eric Auger
2019-04-08 12:18 ` [PATCH v7 02/23] iommu: introduce device fault data Eric Auger
2019-04-08 12:18 ` [PATCH v7 03/23] iommu: introduce device fault report API Eric Auger
2019-04-08 12:18 ` [PATCH v7 04/23] iommu: Introduce attach/detach_pasid_table API Eric Auger
2019-05-15 12:09 ` Jean-Philippe Brucker
2019-05-15 13:06 ` Auger Eric
2019-05-15 15:57 ` Jean-Philippe Brucker
2019-04-08 12:18 ` [PATCH v7 05/23] iommu: Introduce cache_invalidate API Eric Auger
2019-05-01 10:38 ` Jean-Philippe Brucker
2019-05-02 6:58 ` Auger Eric
2019-05-02 10:53 ` Jean-Philippe Brucker
2019-05-02 16:46 ` Jacob Pan
2019-05-07 11:45 ` Jean-Philippe Brucker
2019-04-08 12:18 ` [PATCH v7 06/23] iommu: Introduce bind/unbind_guest_msi Eric Auger
2019-05-08 13:59 ` Robin Murphy
2019-05-10 14:35 ` Auger Eric [this message]
2019-04-08 12:18 ` [PATCH v7 07/23] vfio: VFIO_IOMMU_ATTACH/DETACH_PASID_TABLE Eric Auger
2019-04-08 12:18 ` [PATCH v7 08/23] vfio: VFIO_IOMMU_CACHE_INVALIDATE Eric Auger
2019-04-08 12:18 ` [PATCH v7 09/23] vfio: VFIO_IOMMU_BIND/UNBIND_MSI Eric Auger
2019-04-08 12:18 ` [PATCH v7 10/23] iommu/arm-smmu-v3: Link domains and devices Eric Auger
2019-04-08 12:18 ` [PATCH v7 11/23] iommu/arm-smmu-v3: Maintain a SID->device structure Eric Auger
2019-05-08 14:05 ` Robin Murphy
2019-05-08 18:31 ` Jean-Philippe Brucker
2019-04-08 12:19 ` [PATCH v7 12/23] iommu/smmuv3: Get prepared for nested stage support Eric Auger
2019-05-08 14:24 ` Robin Murphy
2019-05-10 14:34 ` Auger Eric
2019-05-13 11:43 ` Robin Murphy
2019-05-13 14:40 ` Auger Eric
2019-04-08 12:19 ` [PATCH v7 13/23] iommu/smmuv3: Implement attach/detach_pasid_table Eric Auger
2019-05-08 14:38 ` Robin Murphy
2019-05-10 14:35 ` Auger Eric
2019-05-13 12:04 ` Robin Murphy
2019-04-08 12:19 ` [PATCH v7 14/23] iommu/smmuv3: Implement cache_invalidate Eric Auger
2019-05-08 15:01 ` Robin Murphy
2019-05-13 12:16 ` Auger Eric
2019-05-13 14:01 ` Robin Murphy
2019-05-13 14:04 ` Auger Eric
2019-04-08 12:19 ` [PATCH v7 15/23] dma-iommu: Implement NESTED_MSI cookie Eric Auger
2019-05-08 16:42 ` Robin Murphy
2019-04-08 12:19 ` [PATCH v7 16/23] iommu/smmuv3: Nested mode single MSI doorbell per domain enforcement Eric Auger
2019-04-08 12:19 ` [PATCH v7 17/23] iommu/smmuv3: Implement bind/unbind_guest_msi Eric Auger
2019-04-08 12:19 ` [PATCH v7 18/23] iommu/smmuv3: Report non recoverable faults Eric Auger
2019-05-08 17:20 ` Robin Murphy
2019-05-13 7:46 ` Auger Eric
2019-05-13 11:54 ` Robin Murphy
2019-05-13 12:32 ` Auger Eric
2019-05-13 13:47 ` Robin Murphy
2019-04-08 12:19 ` [PATCH v7 19/23] vfio-pci: Add a new VFIO_REGION_TYPE_NESTED region type Eric Auger
2019-04-08 12:19 ` [PATCH v7 20/23] vfio-pci: Register an iommu fault handler Eric Auger
2019-04-08 12:19 ` [PATCH v7 21/23] vfio_pci: Allow to mmap the fault queue Eric Auger
2019-04-08 12:19 ` [PATCH v7 22/23] vfio-pci: Add VFIO_PCI_DMA_FAULT_IRQ_INDEX Eric Auger
2019-04-08 12:19 ` [PATCH v7 23/23] vfio: Document nested stage control Eric Auger
2019-04-30 7:09 ` [PATCH v7 00/23] SMMUv3 Nested Stage Setup Auger Eric
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=af36c889-8405-6de6-0256-4c157dc17aea@redhat.com \
--to=eric.auger@redhat.com \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=christoffer.dall@arm.com \
--cc=eric.auger.pro@gmail.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jean-philippe.brucker@arm.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-kernel@vger.kernel.org \
--cc=marc.zyngier@arm.com \
--cc=peter.maydell@linaro.org \
--cc=robin.murphy@arm.com \
--cc=vincent.stehle@arm.com \
--cc=will.deacon@arm.com \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).