Re: [PATCH v4] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding <thierry.reding@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4231 bytes --]

On Thu, Jul 10, 2014 at 11:23:34AM +0100, Will Deacon wrote:
> On Thu, Jul 10, 2014 at 10:49:10AM +0100, Thierry Reding wrote:
> > On Wed, Jul 09, 2014 at 07:10:48PM +0100, Will Deacon wrote:
> > > On Wed, Jul 09, 2014 at 03:21:27PM +0100, Thierry Reding wrote:
> > > > Anything beyond that (e.g. logical grouping of masters) isn't directly
> > > > within the scope of the binding (it doesn't describe hardware but some
> > > > policy pertaining to some specific use-case).
> > > 
> > > This *is* for hardware. I can use PCI as an example, but this could equally
> > > apply to other types of bus. If you have a bunch of PCI master devices
> > > sitting being a non-transparent bridge, they can end up sharing the same
> > > master device ID (requester ID). This means that there is no way in the
> > > IOMMU to initialise a translation for one of these devices without also
> > > affecting the others. We currently have iommu_groups to deal with this, but
> > > it *is* a property of the hardware and we absolutely need a way to describe
> > > it. I'm happy to add it later, but we need to think about it now to avoid
> > > merging something that can't easily be extended.
> > > 
> > > For PCI, the topology is probable but even then, we need this information to
> > > describe the resulting master device ID emitted by the bridge for the
> > > upstream group. One way to do this with your binding would be to treat all
> > > of the upstream masters as having the same device ID.
> > 
> > Yes, I think that makes most sense. After all from the IOMMU's point of
> > view requests from all devices behind the bridge will originate from the
> > same ID.
> > 
> > So technically it's not really correct to encode the master ID within
> > each of the devices, but rather they should be inheriting the ID from
> > the non-transparent bridge.
> 
> Indeed. Is that possible with your binding, or would we just duplicate the
> IDs between the masters?

No, the binding only describes direct relationships between the IOMMU
and masters. There's no way to translate them inbetween or inherit them.

I'm wondering how this could be described in device tree, though.
Perhaps something like this:

	iommu {
		#iommu-cells = <1>;
	};

	bridge {
		iommus = <&/iommu 42>;
		#iommu-cells = <0>;

		device@0 {
			iommus = <&/bridge>;
		};

		device@1 {
			iommus = <&/bridge>;
		};

		...
	};

? That way some code could walk up the IOMMU tree to resolve this. Or
perhaps even easier:

	iommu {
		#iommu-cells = <1>;
	};

	bridge {
		iommus = <&/iommu 42>;

		device@0 {
			...
		};

		device@1 {
			...
		};

		...
	};

And we could enhance the binding by defining that the iommus node is
inherited by devices on a bus, which by what you're saying would be the
sensible thing to do anyway.

In the second example above, the presence of an iommus property in the
bridge would indicate that it's non-transparent regarding IOMMU
translation and therefore the master ID should be inherited. Devices
could still override by providing their own iommus property, though I'd
be a little surprised if there ever was hardware like that.

> > > With virtualisation, we may want to assign a group of devices to a guest but
> > > without emulating the bridge. This would need something the device-tree to
> > > describe that they are grouped together.
> > 
> > But that's also a software decision, isn't it? Virtualization doesn't
> > have anything to do with the hardware description. Or am I missing
> > something? Of course I guess you could generate a DTB for the guest and
> > group device together, in which case you're pretty much free to do what
> > you want since you're essentially defining your own hardware.
> 
> If you're doing device passthrough and you want to allow the guest to
> program the IOMMU, I think that virtualisation is directly related to the
> hardware description, since the guest will be bound by physical properties
> of the system.

Evidently you know much better what the requirements are here and what
will actually be required. I guess we'll need to have more discussions
along with examples of use-cases.

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]