Re: [PATCH v6 2/7] xen/arm: add pci-domain for disabled devices

[Julien Grall <julien@xen.org>]

On 18/11/2021 07:13, Oleksandr Andrushchenko wrote:
> Hi, Julien!

Hi Oleksandr,

> On 17.11.21 23:33, Julien Grall wrote:
>> Hi Oleksandr,
>>
>> On 17/11/2021 06:56, Oleksandr Andrushchenko wrote:
>>> Hi, Julien!
>>>
>>> On 16.11.21 20:48, Julien Grall wrote:
>>>> Hi Oleksander,
>>>>
>>>> On 05/11/2021 06:33, Oleksandr Andrushchenko wrote:
>>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> If a PCI host bridge device is present in the device tree, but is
>>>>> disabled, then its PCI host bridge driver was not instantiated.
>>>>> This results in the failure of the pci_get_host_bridge_segment()
>>>>> and the following panic during Xen start:
>>>>>
>>>>> (XEN) Device tree generation failed (-22).
>>>>> (XEN)
>>>>> (XEN) ****************************************
>>>>> (XEN) Panic on CPU 0:
>>>>> (XEN) Could not set up DOM0 guest OS
>>>>> (XEN) ****************************************
>>>>>
>>>>> Fix this by adding "linux,pci-domain" property for all device tree nodes
>>>>> which have "pci" device type, so we know which segments will be used by
>>>>> the guest for which bridges.
>>>>>
>>>>> Fixes: 4cfab4425d39 ("xen/arm: Add linux,pci-domain property for hwdom if not available.")
>>>>>
>>>>> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>>>>
>>>>> ---
>>>>> New in v6
>>>>> ---
>>>>>     xen/arch/arm/domain_build.c        | 15 ++++++++++++++-
>>>>>     xen/arch/arm/pci/pci-host-common.c |  2 +-
>>>>>     xen/include/asm-arm/pci.h          |  8 ++++++++
>>>>>     3 files changed, 23 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>>>>> index 491f5e2c316e..f7fcb1400c19 100644
>>>>> --- a/xen/arch/arm/domain_build.c
>>>>> +++ b/xen/arch/arm/domain_build.c
>>>>> @@ -753,9 +753,22 @@ static int __init write_properties(struct domain *d, struct kernel_info *kinfo,
>>>>>             {
>>>>>                 uint16_t segment;
>>>>>     +            /*
>>>>> +             * The node doesn't have "linux,pci-domain" property and it is
>>>>> +             * possible that:
>>>>> +             *  - Xen only has drivers for a part of the host bridges
>>>>> +             *  - some host bridges are disabled
>>>>> +             * Make sure we insert the correct "linux,pci-domain" property
>>>>> +             * in any case, so we know which segments will be used
>>>>> +             * by Linux for which bridges.
>>>>
>>>> The check above will check the node type is "pci". AFAICT, this would also cover PCI devices. I am not aware of any issue to add "linux,pci-domain" for them. However, this feels a bit odd.
>>>>
>>>>   From my understanding, a PCI device would always be described as a child of the hostbridges. So I would rework the 'if' to also check if the parent type is not "pci".
>>>>
>>> We may have "bridge -> bridge -> device" topology as well.
>>
>> Do you have an example of Device-Tree?
> No, I don't have at hand, but I can imagine this can relatively easy be done with QEMU > Even if not, do you assume this topology can never happen?

I think it is not possible to describe them in the Device-Tree because 
you wouldn't be able to translate the "regs" of the nested hostbridge to 
an actual MMIO address.

In fact, looking at dt_device_get_address(), I think Xen would reject 
such DT.

>>
>>> So, I prefer to have the check as it is.
>>
>> I don't really like the idea to spuriously add "linux,pci-domain" to PCI DT node. But if there are no other solution, then this should at least be mentionned in the commit message and code.
> I am fine with any solution here, I just want that to be defined and implemented.
> Please let me know the final decision on this and how we proceed

I would prefer to go with my way. This can be refined in the future if 
we find Device-Tree that matches what you wrote.

Cheers,

-- 
Julien Grall