Re: [RFC PATCH v2 03/26] ARM: GICv3 ITS: allocate device and collection table

[Andre Przywara <andre.przywara@arm.com>]

Hi,

On 20/01/17 12:18, Julien Grall wrote:
> Hi Andre,
> 
> On 20/01/2017 12:27, Andre Przywara wrote:
>> On 20/01/17 11:12, Julien Grall wrote:
>>> Hello,
>>>
>>> On 04/01/2017 22:47, Stefano Stabellini wrote:
>>>> On Thu, 22 Dec 2016, Andre Przywara wrote:
>>>>> Each ITS maps a pair of a DeviceID (usually the PCI b/d/f triplet) and
>>>>> an EventID (the MSI payload or interrupt ID) to a pair of LPI number
>>>>> and collection ID, which points to the target CPU.
>>>>> This mapping is stored in the device and collection tables, which
>>>>> software
>>>>> has to provide for the ITS to use.
>>>>> Allocate the required memory and hand it the ITS.
>>>>> The maximum number of devices is limited to a compile-time constant
>>>>> exposed in Kconfig.
>>>>>
>>>>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>>>>> ---
>>>>>  xen/arch/arm/Kconfig          |   6 +++
>>>>>  xen/arch/arm/gic-its.c        | 114
>>>>> ++++++++++++++++++++++++++++++++++++++++++
>>>>>  xen/arch/arm/gic-v3.c         |   5 ++
>>>>>  xen/include/asm-arm/bitops.h  |   1 +
>>>>>  xen/include/asm-arm/gic-its.h |  51 ++++++++++++++++++-
>>>>>  5 files changed, 176 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>>>>> index a7d941c..a369305 100644
>>>>> --- a/xen/arch/arm/Kconfig
>>>>> +++ b/xen/arch/arm/Kconfig
>>>>> @@ -59,6 +59,12 @@ config MAX_HOST_LPI_BITS
>>>>>            This can be overriden on the command line with the
>>>>> max_lpi_bits
>>>>>            parameter.
>>>>>
>>>>> +config MAX_ITS_PCI_BUSSES
>>>>> +        depends on HAS_ITS
>>>>> +        int "Number of PCI busses the ITS supports (4)"
>>>>> +        range 1 1024
>>>>> +        default "4"
>>>>
>>>> Given that any kind of devices can be behind an ITS, including non-PCI
>>>> devices, I think it is best to call this MAX_PHYS_ITS_DEVICES.
>>>
>>> I don't think we should limit the number of devices supported at
>>> compilation time. We aim to have a single Xen binary running on any
>>> board. The best would be to guess the number at runtime, but I would be
>>> fine with a command line options.
>>
>> Yes, so I changed this already to specify the number of devices and will
>> now also add a command line option.
>> So do you want to get rid of the Kconfig entirely to just go with
>> #define-ing the default value in the code?
>> Or do we still want to have this easily accessible way of changing the
>> default, at least for the first time to simplify experimentation?
> 
> Before answering to your questions, I'd like to understand why you need
> to have this value hardcoded or coming from the command line? Can't this
> be figured out at runtime?
> 
> Also what is this issue to use the number of ID bits presented in TYPER?
> Is it because your series is only handling flat table, so a large amount
> of memory would be used?

My understanding is that is the TYPER value just specifies the number of
device ID bits the ITS supports, without necessarily requiring software
to allocate tables to cover all theoretically possible devices. Instead
the expectation is that an ITS driver has some clue about how many
devices the system has or will need - which is not true in the (current)
Xen case, I am afraid.
A good ITS implementation might just implement all architected 32 bits,
which would require 32GB of a flat device table :-(
Also the indirect table feature is optional - and it happens to be that
it is _not_ available on the Cavium ThunderX, IIRC, despite it having
sparse device IDs.
And Linux limits all BASER tables to be at most 16MB.

So I think having some kind of upper limit would be good to have in any
case, whether this has to be a compile time option or not is another
question.

Cheers,
Andre.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel