Re: [PATCH v6 09/22] vfio: VFIO_IOMMU_BIND/UNBIND_MSI

From: Marc Zyngier <marc.zyngier@arm.com>
To: "Vincent Stehlé" <vincent.stehle@arm.com>,
	"Auger Eric" <eric.auger@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>,
	eric.auger.pro@gmail.com, iommu@lists.linux-foundation.org,
	linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
	kvmarm@lists.cs.columbia.edu, joro@8bytes.org,
	jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com,
	jean-philippe.brucker@arm.com, will.deacon@arm.com,
	robin.murphy@arm.com, kevin.tian@intel.com, ashok.raj@intel.com,
	christoffer.dall@arm.com, peter.maydell@linaro.org
Subject: Re: [PATCH v6 09/22] vfio: VFIO_IOMMU_BIND/UNBIND_MSI
Date: Wed, 10 Apr 2019 14:15:41 +0100	[thread overview]
Message-ID: <73e3c98a-2a2c-83ad-1bbb-7f7368cce274@arm.com> (raw)
In-Reply-To: <20190410123531.GA19023@debian>

Hi Vincent,

On 10/04/2019 13:35, Vincent Stehlé wrote:
> On Thu, Apr 04, 2019 at 08:55:25AM +0200, Auger Eric wrote:
>> Hi Marc, Robin, Alex,
> (..)
>> Do you think this is a reasonable assumption to consider devices within
>> the same host iommu group share the same MSI doorbell?
> 
> Hi Eric,
> 
> I am not sure this assumption always hold.
> 
> Marc, Robin and Alex can correct me, but for example I think the following
> topology is valid for Arm systems:
> 
>  +------------+  +------------+
>  | Endpoint A |  | Endpoint B |
>  +------------+  +------------+
>             v     v
>           /---------\
>          |  Non-ACS  |
>          |  Switch   |
>           \---------/
>                v
>        +---------------+
>        |     PCIe      |
>        | Root Complex  |
>        +---------------+
>                v
>          +-----------+
>          |   SMMU    |
>          +-----------+
>                v
>   +--------------------------+
>   |   System interconnect    |
>   +--------------------------+
>         v              v
>   +-----------+  +-----------+
>   |   ITS A   |  |   ITS B   |
>   +-----------+  +-----------+
> 
> All PCIe Endpoints and ITS could be in the same ITS Group 0, meaning
> devices could send their MSI at any ITS in hardware.
> 
> For Linux the two PCIe Endpoints would be in the same iommu group, because
> the switch in this example does not support ACS.
> 
> I think the devicetree msi-map property could be used to "map" the RID of
> Endpoint A to ITS A and the RID of Endpoint B to ITS B, which would violate
> the assumption.
> 
> See the monolithic example in [1], the example system in [2], appendices
> D, E and F in [3] and the msi-map property in [4].

I think we are all in agreement that this is a possible topology. It is
just that it doesn't exist in any real-life implementation we know of
(the ITS tends to be close to the RC and not downstream of the
interconnect).

Given the complexity of what we're trying to put together, I'd rather
start with a small step which supports commonly implemented topology,
and later address the odd ones if they actually crop up.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...