Re: [RFC PATCH v2 07/26] ARM: GICv3 ITS: introduce host LPI array

From: Julien Grall <julien.grall@arm.com>
To: Stefano Stabellini <sstabellini@kernel.org>,
	Andre Przywara <andre.przywara@arm.com>
Cc: xen-devel@lists.xenproject.org, Vijay Kilari <vijay.kilari@gmail.com>
Subject: Re: [RFC PATCH v2 07/26] ARM: GICv3 ITS: introduce host LPI array
Date: Fri, 20 Jan 2017 13:00:20 +0100	[thread overview]
Message-ID: <b37550ab-b67c-41f6-9e26-baba7533a5a5@arm.com> (raw)
In-Reply-To: <alpine.DEB.2.10.1701061139450.2866@sstabellini-ThinkPad-X260>

Hi Stefano,

Sorry for the late answer, still going through my e-mail backlog.

On 06/01/2017 21:20, Stefano Stabellini wrote:
> On Fri, 6 Jan 2017, Andre Przywara wrote:
>>> It is also possible to end up calling mapti with an inexistent eventid
>>> for host_devid. Could that be a problem?
>>
>> Not at all. Actually there is no such thing as a "nonexistent event ID",
>> because the event ID will be written by the device as the payload to the
>> MSI doorbell address, probably because it learned about it by the
>> driver. So if we provision an ITTE with an event ID which the device
>> will never send, that LPI will just never fire.
>> Since Xen (in contrast to the driver in the domain) has no idea how many
>> and which MSIs the device will use, we just allocate a bunch of them.
>> The upper limit (32 atm) is something we probably need to still think
>> about, though.
>> I tried to learn a limit from Linux ("nvecs" in its_create_device()
>> seems to be the source), but couldn't find anything useful other than 32.
>> We will learn about exceeding this as soon as a domain tries to map a
>> virtual LPI with an event ID higher than 31, however it's too late to
>> fix this then. We can bark when this happens to learn if any device ever
>> does this during our testing to get some heuristic data.
>>
>> Eventually all boils down to Xen getting more information from Dom0
>> about the required number of MSIs. We could then even limit the
>> allocation to less than 32, if that helps.
>
> Originally Julien and I thought that Xen should map events up to the
> theoretically maximum for each device, but we realized that they were
> too many: an MSI-X capable device can generate up to 2048 different
> events.
>
> Xen needs to find out the exact number of events for each device. The
> information can either be provided by the guest, or the hypervisor needs
> to figure it out on its own.
>
> With Julien's PCI Passthrough work, Xen will become able to read the
> amount of events a device is capable of generating, so in the long term
> this problem should be easy to solve. But Julien's work might land one
> or two Xen releases after ITS.
>
> In the meantime, we can extend an existing PHYSDEVOP hypercall or add a
> new one. Julien, do you agree?

PHYSDEVOP are stable ABI and some are already being in used by Linux 
even on ARM.

I am not in favor of adding a PHYSDEVOP which may not be necessary in a 
couple of the release. Furthermore, there are already some issues with 
how device are being added in the ITS. Indeed, the PHYSDEVOP operations 
will provide the RID (can be deduced from the BDF), but the deviceID may 
not be equal to the RID on some platform. This is where IORT (on ACPI) 
and msi-map (on DT) comes from. The plumbing will be added during the 
PCI work.

So I would be more in favor to hardcode the device info (DeviceID, 
number of max number of MSIs) per platform until we get PCI passthrough 
working.

Note that the PCI work will not include any thoughts regarding platform 
device supporting MSI. So we may end up to add a new PHYSDEVO.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel