Re: [PATCH v6 2/3] xen/arm: Enable the existing x86 virtual PCI support for ARM.

[Bertrand Marquis <Bertrand.Marquis@arm.com>]

> On 15 Oct 2021, at 11:24, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 15.10.2021 11:52, Bertrand Marquis wrote:
>>> On 15 Oct 2021, at 09:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
>>> On Thu, Oct 14, 2021 at 03:49:50PM +0100, Bertrand Marquis wrote:
>>>> @@ -752,6 +752,19 @@ int pci_add_device(u16 seg, u8 bus, u8 devfn,
>>>> 
>>>>    check_pdev(pdev);
>>>> 
>>>> +#ifdef CONFIG_ARM
>>>> +    /*
>>>> +     * On ARM PCI devices discovery will be done by Dom0. Add vpci handler when
>>>> +     * Dom0 inform XEN to add the PCI devices in XEN.
>>>> +     */
>>>> +    ret = vpci_add_handlers(pdev);
>>>> +    if ( ret )
>>>> +    {
>>>> +        printk(XENLOG_ERR "Setup of vPCI failed: %d\n", ret);
>>>> +        goto out;
>>>> +    }
>>>> +#endif
>>> 
>>> I think vpci_add_handlers should be called after checking that
>>> pdev->domain is != NULL, so I would move this chunk a bit below.
>> 
>> On arm this would prevent the dom0less use case or to have the PCI
>> bus enumerated from an other domain.
>> @oleksandr: can you comment on this one, you might have a better
>> answer than me on this ?
> 
> Well, without Xen doing the enumeration, some other entity would need
> to do so, including the reporting to Xen. Obviously without a Dom0 it
> would be ambiguous which domain to assign the device to; perhaps it
> should be the caller in this case? That would make that caller domain
> a pseudo-hwdom though, as far as PCI is concerned, which may not be
> desirable according to my (limited) understanding of dom0less.

This is not really related to this patch but the plan is the following:
- enumeration would have to be done by the firmware or boot loader before
- xen will have some code to detect PCI devices
- dom0less can be used to assign PCI devices to guest

Anyway does not change the fact that this must be called when domain is not NULL and I will fix that.

> 
>>>> @@ -784,6 +797,9 @@ out:
>>>>                   &PCI_SBDF(seg, bus, slot, func));
>>>>        }
>>>>    }
>>>> +    else if ( pdev )
>>>> +        pci_cleanup_msi(pdev);
>>> 
>>> I'm slightly lost at why you add this chunk, is this strictly related
>>> to the patch?
>> 
>> This was discussed a lot in previous version of the patch and
>> requested by Stefano. The idea here is that as soon as handlers
>> are added some bits might be modified in the PCI config space
>> leading possibly to msi interrupts. So it is safer to cleanup on the
>> error path. For references please see discussion on v4 and v5 where
>> this was actually added (to much references as the discussion was
>> long so here [1] and [2] are the patchwork thread).
>> 
>> [1] https://patchwork.kernel.org/project/xen-devel/patch/9bdca2cda5d2e83f94dc2423e55714273539760a.1633540842.git.rahul.singh@arm.com/
>> [2] https://patchwork.kernel.org/project/xen-devel/patch/f093de681c2560a7196895bcd666ef8840885c1d.1633340795.git.rahul.singh@arm.com/
> 
> The addition of this call has repeatedly raised questions. This is a
> good indication that sufficient discussion thereof has been lacking
> from the patch description.

Yes and I will remove it as it only impacts x86 right now.
If this is needed, we will have to do it while adding MSI support on Arm.

Regards
Bertrand

> 
> Jan
>