Re: [PATCH v6 2/3] xen/arm: Enable the existing x86 virtual PCI support for ARM.

[Luca Fancellu <luca.fancellu@arm.com>]

> On 15 Oct 2021, at 14:47, Roger Pau Monné <roger.pau@citrix.com> wrote:
> 
> On Fri, Oct 15, 2021 at 10:48:41AM +0000, Bertrand Marquis wrote:
>> Hi Jan,
>> 
>>> On 15 Oct 2021, at 11:41, Jan Beulich <jbeulich@suse.com> wrote:
>>> 
>>> On 15.10.2021 12:33, Bertrand Marquis wrote:
>>>>> On 15 Oct 2021, at 11:24, Jan Beulich <jbeulich@suse.com> wrote:
>>>>> On 15.10.2021 11:52, Bertrand Marquis wrote:
>>>>>>> On 15 Oct 2021, at 09:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
>>>>>>> On Thu, Oct 14, 2021 at 03:49:50PM +0100, Bertrand Marquis wrote:
>>>>>>>> @@ -752,6 +752,19 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
>>>>>>>> 
>>>>>>>>  check_pdev(pdev);
>>>>>>>> 
>>>>>>>> +#ifdef CONFIG_ARM
>>>>>>>> +    /*
>>>>>>>> +     * On ARM PCI devices discovery will be done by Dom0. Add vpci handler when
>>>>>>>> +     * Dom0 inform XEN to add the PCI devices in XEN.
>>>>>>>> +     */
>>>>>>>> +    ret = vpci_add_handlers(pdev);
>>>>>>>> +    if ( ret )
>>>>>>>> +    {
>>>>>>>> +        printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
>>>>>>>> +        goto out;
>>>>>>>> +    }
>>>>>>>> +#endif
>>>>>>> 
>>>>>>> I think vpci_add_handlers should be called after checking that
>>>>>>> pdev->domain is != NULL, so I would move this chunk a bit below.
>>>>>> 
>>>>>> On arm this would prevent the dom0less use case or to have the PCI
>>>>>> bus enumerated from an other domain.
>>>>>> @oleksandr: can you comment on this one, you might have a better
>>>>>> answer than me on this ?
>>>>> 
>>>>> Well, without Xen doing the enumeration, some other entity would need
>>>>> to do so, including the reporting to Xen. Obviously without a Dom0 it
>>>>> would be ambiguous which domain to assign the device to; perhaps it
>>>>> should be the caller in this case? That would make that caller domain
>>>>> a pseudo-hwdom though, as far as PCI is concerned, which may not be
>>>>> desirable according to my (limited) understanding of dom0less.
>>>> 
>>>> This is not really related to this patch but the plan is the following:
>>>> - enumeration would have to be done by the firmware or boot loader before
>>>> - xen will have some code to detect PCI devices
>>>> - dom0less can be used to assign PCI devices to guest
>>>> 
>>>> Anyway does not change the fact that this must be called when domain is
>>>> not NULL and I will fix that.
>>> 
>>> Since we now all seem to agree that the NULL would have been a problem,
>>> may I ask in how far any of this has actually been tested?
>> 
>> With the whole serie currently on gitlab we have extensively tested passing
>> through PCI devices on Arm in several configuration (number of device, MSI,
>> MSI-X) and check that PCI was still functional on x86.
>> 
>> With the patches pushed to Xen right now it was checked that:
>> - xen compiles properly on arm32, arm64 and x86
>> - xen compiles properly with VPCI activated (using a patch) on arm32 and arm64
>> - xen on x86 is functionnal (using basic test on QEMU)
>> - xen on arm64 is functionnal (with some extensive tests on different targets)
> 
> I thinks it's unlikely, but since I haven't checked myself, could you
> see if the vpci user-space test harness (tools/tests/vpci) still
> builds and functions properly?

Hi Roger,

Bertrand is going to send the v7, however I’m keen to do this test but unfortunately
I’ve never used it before and I don’t have an x86 environment, do you have any
resource that I can read or some guidance?

Cheers,
Luca

> 
> Thanks, Roger.
>