* [PATCH 2/2] vfio/type1: Give hardware MSI regions precedence
2017-07-27 14:54 [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions Robin Murphy
@ 2017-07-27 14:54 ` Robin Murphy
2017-07-30 10:39 ` Auger Eric
2017-07-30 10:39 ` [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions Auger Eric
` (2 subsequent siblings)
3 siblings, 1 reply; 6+ messages in thread
From: Robin Murphy @ 2017-07-27 14:54 UTC (permalink / raw)
To: alex.williamson
Cc: kvm, linux-kernel, linux-arm-kernel, eric.auger,
shameerali.kolothum.thodi, marc.zyngier
If the IOMMU driver advertises 'real' reserved regions for MSIs, but
still includes the software-managed region as well, we are currently
blind to the former and will configure the IOMMU domain to map MSIs into
the latter, which is unlikely to work as expected.
Since it would take a ridiculous hardware topology for both regions to
be valid (which would be rather difficult to support in general), we
should be safe to assume that the presence of any hardware regions makes
the software region irrelevant. However, the IOMMU driver might still
advertise the software region by default, particularly if the hardware
regions are filled in elsewhere by generic code, so it might not be fair
for VFIO to be super-strict about not mixing them. To that end, make
vfio_iommu_has_sw_msi() robust against the presence of both region types
at once, so that we end up doing what is almost certainly right, rather
than what is almost certainly wrong.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
---
drivers/vfio/vfio_iommu_type1.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
index 2328be628f21..92155cce926d 100644
--- a/drivers/vfio/vfio_iommu_type1.c
+++ b/drivers/vfio/vfio_iommu_type1.c
@@ -1169,13 +1169,21 @@ static bool vfio_iommu_has_sw_msi(struct iommu_group *group, phys_addr_t *base)
INIT_LIST_HEAD(&group_resv_regions);
iommu_get_group_resv_regions(group, &group_resv_regions);
list_for_each_entry(region, &group_resv_regions, list) {
+ /*
+ * The presence of any 'real' MSI regions should take
+ * precedence over the software-managed one if the
+ * IOMMU driver happens to advertise both types.
+ */
+ if (region->type == IOMMU_RESV_MSI) {
+ ret = false;
+ break;
+ }
+
if (region->type == IOMMU_RESV_SW_MSI) {
*base = region->start;
ret = true;
- goto out;
}
}
-out:
list_for_each_entry_safe(region, next, &group_resv_regions, list)
kfree(region);
return ret;
--
2.12.2.dirty
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 2/2] vfio/type1: Give hardware MSI regions precedence
2017-07-27 14:54 ` [PATCH 2/2] vfio/type1: Give hardware MSI regions precedence Robin Murphy
@ 2017-07-30 10:39 ` Auger Eric
0 siblings, 0 replies; 6+ messages in thread
From: Auger Eric @ 2017-07-30 10:39 UTC (permalink / raw)
To: Robin Murphy, alex.williamson
Cc: kvm, linux-kernel, linux-arm-kernel, shameerali.kolothum.thodi,
marc.zyngier
Hi Robin,
On 27/07/2017 16:54, Robin Murphy wrote:
> If the IOMMU driver advertises 'real' reserved regions for MSIs, but
> still includes the software-managed region as well, we are currently
> blind to the former and will configure the IOMMU domain to map MSIs into
> the latter, which is unlikely to work as expected.
>
> Since it would take a ridiculous hardware topology for both regions to
> be valid (which would be rather difficult to support in general), we
> should be safe to assume that the presence of any hardware regions makes
> the software region irrelevant. However, the IOMMU driver might still
> advertise the software region by default, particularly if the hardware
> regions are filled in elsewhere by generic code, so it might not be fair
> for VFIO to be super-strict about not mixing them. To that end, make
> vfio_iommu_has_sw_msi() robust against the presence of both region types
> at once, so that we end up doing what is almost certainly right, rather
> than what is almost certainly wrong.
>
> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Thanks
Eric
> ---
> drivers/vfio/vfio_iommu_type1.c | 12 ++++++++++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 2328be628f21..92155cce926d 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -1169,13 +1169,21 @@ static bool vfio_iommu_has_sw_msi(struct iommu_group *group, phys_addr_t *base)
> INIT_LIST_HEAD(&group_resv_regions);
> iommu_get_group_resv_regions(group, &group_resv_regions);
> list_for_each_entry(region, &group_resv_regions, list) {
> + /*
> + * The presence of any 'real' MSI regions should take
> + * precedence over the software-managed one if the
> + * IOMMU driver happens to advertise both types.
> + */
> + if (region->type == IOMMU_RESV_MSI) {
> + ret = false;
> + break;
> + }
> +
> if (region->type == IOMMU_RESV_SW_MSI) {
> *base = region->start;
> ret = true;
> - goto out;
> }
> }
> -out:
> list_for_each_entry_safe(region, next, &group_resv_regions, list)
> kfree(region);
> return ret;
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions
2017-07-27 14:54 [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions Robin Murphy
2017-07-27 14:54 ` [PATCH 2/2] vfio/type1: Give hardware MSI regions precedence Robin Murphy
@ 2017-07-30 10:39 ` Auger Eric
2017-07-31 15:59 ` Shameerali Kolothum Thodi
2017-08-10 19:16 ` Alex Williamson
3 siblings, 0 replies; 6+ messages in thread
From: Auger Eric @ 2017-07-30 10:39 UTC (permalink / raw)
To: Robin Murphy, alex.williamson
Cc: kvm, marc.zyngier, linux-kernel, shameerali.kolothum.thodi,
linux-arm-kernel
Hi Robin,
On 27/07/2017 16:54, Robin Murphy wrote:
> For ARM-based systems with a GICv3 ITS to provide interrupt isolation,
> but hardware limitations which are worked around by having MSIs bypass
> SMMU translation (e.g. HiSilicon Hip06/Hip07), VFIO neglects to check
> for the IRQ_DOMAIN_FLAG_MSI_REMAP capability, (and thus erroneously
> demands unsafe_interrupts) if a software-managed MSI region is absent.
>
> Fix this by always checking for isolation capability at both the IRQ
> domain and IOMMU domain levels, rather than predicating that on whether
> MSIs require an IOMMU mapping (which was always slightly tenuous logic).
>
> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Thanks
Eric
> ---
> drivers/vfio/vfio_iommu_type1.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 8549cb111627..2328be628f21 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -1265,8 +1265,8 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,
> INIT_LIST_HEAD(&domain->group_list);
> list_add(&group->next, &domain->group_list);
>
> - msi_remap = resv_msi ? irq_domain_check_msi_remap() :
> - iommu_capable(bus, IOMMU_CAP_INTR_REMAP);
> + msi_remap = irq_domain_check_msi_remap() ||
> + iommu_capable(bus, IOMMU_CAP_INTR_REMAP);
>
> if (!allow_unsafe_interrupts && !msi_remap) {
> pr_warn("%s: No interrupt remapping support. Use the module param \"allow_unsafe_interrupts\" to enable VFIO IOMMU support on this platform\n",
>
^ permalink raw reply [flat|nested] 6+ messages in thread* RE: [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions
2017-07-27 14:54 [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions Robin Murphy
2017-07-27 14:54 ` [PATCH 2/2] vfio/type1: Give hardware MSI regions precedence Robin Murphy
2017-07-30 10:39 ` [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions Auger Eric
@ 2017-07-31 15:59 ` Shameerali Kolothum Thodi
2017-08-10 19:16 ` Alex Williamson
3 siblings, 0 replies; 6+ messages in thread
From: Shameerali Kolothum Thodi @ 2017-07-31 15:59 UTC (permalink / raw)
To: Robin Murphy, alex.williamson
Cc: kvm, linux-kernel, linux-arm-kernel, eric.auger, marc.zyngier, Linuxarm
> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: Thursday, July 27, 2017 3:54 PM
> To: alex.williamson@redhat.com
> Cc: kvm@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
> kernel@lists.infradead.org; eric.auger@redhat.com; Shameerali Kolothum
> Thodi; marc.zyngier@arm.com
> Subject: [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions
>
> For ARM-based systems with a GICv3 ITS to provide interrupt isolation,
> but hardware limitations which are worked around by having MSIs bypass
> SMMU translation (e.g. HiSilicon Hip06/Hip07), VFIO neglects to check
> for the IRQ_DOMAIN_FLAG_MSI_REMAP capability, (and thus erroneously
> demands unsafe_interrupts) if a software-managed MSI region is absent.
>
> Fix this by always checking for isolation capability at both the IRQ
> domain and IOMMU domain levels, rather than predicating that on whether
> MSIs require an IOMMU mapping (which was always slightly tenuous logic).
>
> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
> ---
> drivers/vfio/vfio_iommu_type1.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/vfio/vfio_iommu_type1.c
> b/drivers/vfio/vfio_iommu_type1.c
> index 8549cb111627..2328be628f21 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -1265,8 +1265,8 @@ static int vfio_iommu_type1_attach_group(void
> *iommu_data,
> INIT_LIST_HEAD(&domain->group_list);
> list_add(&group->next, &domain->group_list);
>
> - msi_remap = resv_msi ? irq_domain_check_msi_remap() :
> - iommu_capable(bus,
> IOMMU_CAP_INTR_REMAP);
> + msi_remap = irq_domain_check_msi_remap() ||
> + iommu_capable(bus, IOMMU_CAP_INTR_REMAP);
>
> if (!allow_unsafe_interrupts && !msi_remap) {
> pr_warn("%s: No interrupt remapping support. Use the
> module param \"allow_unsafe_interrupts\" to enable VFIO IOMMU support
> on this platform\n",
Verified these patches(1 and 2) on HiSilicon Hip07 platform along with HW MSI resv
region patches[1]. Please feel free to add,
Tested-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Thanks,
Shameer
1. https://www.spinics.net/lists/arm-kernel/msg595847.html
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions
2017-07-27 14:54 [PATCH 1/2] vfio/type1: Cope with hardware MSI reserved regions Robin Murphy
` (2 preceding siblings ...)
2017-07-31 15:59 ` Shameerali Kolothum Thodi
@ 2017-08-10 19:16 ` Alex Williamson
3 siblings, 0 replies; 6+ messages in thread
From: Alex Williamson @ 2017-08-10 19:16 UTC (permalink / raw)
To: Robin Murphy
Cc: kvm, linux-kernel, linux-arm-kernel, eric.auger,
shameerali.kolothum.thodi, marc.zyngier
On Thu, 27 Jul 2017 15:54:13 +0100
Robin Murphy <robin.murphy@arm.com> wrote:
> For ARM-based systems with a GICv3 ITS to provide interrupt isolation,
> but hardware limitations which are worked around by having MSIs bypass
> SMMU translation (e.g. HiSilicon Hip06/Hip07), VFIO neglects to check
> for the IRQ_DOMAIN_FLAG_MSI_REMAP capability, (and thus erroneously
> demands unsafe_interrupts) if a software-managed MSI region is absent.
>
> Fix this by always checking for isolation capability at both the IRQ
> domain and IOMMU domain levels, rather than predicating that on whether
> MSIs require an IOMMU mapping (which was always slightly tenuous logic).
>
> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
> ---
> drivers/vfio/vfio_iommu_type1.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
I've pushed these to my next branch with Eric's R-b and Shameerali's
T-b for v4.14. Thanks,
Alex
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 8549cb111627..2328be628f21 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -1265,8 +1265,8 @@ static int vfio_iommu_type1_attach_group(void *iommu_data,
> INIT_LIST_HEAD(&domain->group_list);
> list_add(&group->next, &domain->group_list);
>
> - msi_remap = resv_msi ? irq_domain_check_msi_remap() :
> - iommu_capable(bus, IOMMU_CAP_INTR_REMAP);
> + msi_remap = irq_domain_check_msi_remap() ||
> + iommu_capable(bus, IOMMU_CAP_INTR_REMAP);
>
> if (!allow_unsafe_interrupts && !msi_remap) {
> pr_warn("%s: No interrupt remapping support. Use the module param \"allow_unsafe_interrupts\" to enable VFIO IOMMU support on this platform\n",
^ permalink raw reply [flat|nested] 6+ messages in thread