Re: [PATCH v5 00/11] PCI devices passthrough on Arm

[Rahul Singh <Rahul.Singh@arm.com>]

Hi Stefano,

> On 7 Oct 2021, at 8:54 pm, Stefano Stabellini <sstabellini@kernel.org> wrote:
> 
> I committed patches 2-7.
> 
> I made two small adjustments on commit:
> - patch #3: bool_t/bool
> - patch #7: drop _XEN_DOMCTL_CDF_vpci

Thank you very much. I will be on leave from tomorrow. Someone from my 
team will take care of the rest of the patch and reply to the comments.

Regards,
Rahul
> 
> Cheers,
> 
> Stefano
> 
> 
> On Wed, 6 Oct 2021, Rahul Singh wrote:
>> Hello All,
>> 
>> The purpose of this patch series is to add PCI passthrough support to Xen on
>> Arm. PCI passthrough support on ARM is the collaboration work between EPAM and
>> ARM. ARM submitted the partial RFC [1][2] last year to get early feedback. We
>> tried to fix all the comments and added more features to this patch series.
>> 
>> Working POC with all the features can be found at [3]. Working POC is tested
>> on x86 so that there will be no regression on x86. Design presentation can be
>> found at [4]
>> 
>> PCI passthrough support is divided into different patches. This patch series
>> includes following features: 
>> 
>> Preparatory work to implement the PCI passthrough support for the ARM:
>> - Refactor MSI code.
>> - Fixed compilation error when HAS_PCI enabled for ARM.
>> 
>> Discovering PCI Host Bridge in XEN:
>> - PCI init to initialize the PCI driver.
>> - PCI host bridge discovery in XEN and map the PCI ECAM configuration space to
>>  the XEN memory.
>> - PCI access functions.
>> 
>> Discovering PCI devices:
>> - To support the PCI passthrough, XEN should be aware of the PCI
>>  devices.
>> - Hardware domain is in charge of doing the PCI enumeration and will discover
>>  the PCI devices and then communicate to the XEN via a hypercall to add the
>>  PCI devices in XEN.
>> 
>> Enable the existing x86 virtual PCI support for ARM:
>> - Add VPCI trap handler for each of the PCI device added for config space
>>  access.
>> - Register the trap handler in XEN for each of the host bridge PCI ECAM config
>>  space access.
>> 
>> Emulated PCI device tree node in libxl:
>> - Create a virtual PCI device tree node in libxl to enable the guest OS to
>>  discover the virtual PCI during guest boot.
>> 
>> This patch series does not inlcude the following features. Following features
>> will be send for review in the next version of the patch series once initial
>> patch series merged.
>> 
>> - VPCI support for DOMU guests (Non-identity mappings guest view of the BARs)
>> - Virtual bus topology implementation
>> - IOMMU related changes (generic, SMMUv2, SMMUv3)
>> - MSI support for DOMU guests.
>> - Virual ITS support for DOMU guests
>> 
>> [1] https://lists.xenproject.org/archives/html/xen-devel/2020-07/msg01184.html
>> [2] https://lists.xenproject.org/archives/html/xen-devel/2020-07/threads.html#01184
>> [3] https://gitlab.com/rahsingh/xen-integration/-/tree/arm_pci_v4_integrate
>> [4] https://static.sched.com/hosted_files/xen2021/e4/PCI_Device_Passthrough_On_Arm.pdf
>> 
>> Oleksandr Andrushchenko (1):
>>  xen/arm: Add support for Xilinx ZynqMP PCI host controller
>> 
>> Rahul Singh (10):
>>  xen/arm: xc_domain_ioport_permission(..) not supported on ARM.
>>  xen/arm: Add PHYSDEVOP_pci_device_(*add/remove) support for ARM
>>  xen/arm: Add cmdline boot option "pci-passthrough = <boolean>"
>>  xen/arm: PCI host bridge discovery within XEN on ARM
>>  xen/arm: Implement pci access functions
>>  xen/domctl: Introduce XEN_DOMCTL_CDF_vpci flag
>>  xen/arm: Enable the existing x86 virtual PCI support for ARM.
>>  xen/arm: Transitional change to build HAS_VPCI on ARM.
>>  arm/libxl: Emulated PCI device tree node in libxl
>>  xen/arm: Add linux,pci-domain property for hwdom if not available.
>> 
>> docs/misc/xen-command-line.pandoc   |   7 +
>> tools/include/libxl.h               |   6 +
>> tools/libs/ctrl/xc_domain.c         |   9 +
>> tools/libs/light/libxl_arm.c        | 111 +++++++++++
>> tools/libs/light/libxl_types.idl    |   1 +
>> tools/ocaml/libs/xc/xenctrl.ml      |   1 +
>> tools/ocaml/libs/xc/xenctrl.mli     |   1 +
>> xen/arch/arm/Makefile               |   1 +
>> xen/arch/arm/domain.c               |   8 +-
>> xen/arch/arm/domain_build.c         |  19 ++
>> xen/arch/arm/pci/Makefile           |   5 +
>> xen/arch/arm/pci/ecam.c             |  61 ++++++
>> xen/arch/arm/pci/pci-access.c       | 140 ++++++++++++++
>> xen/arch/arm/pci/pci-host-common.c  | 287 ++++++++++++++++++++++++++++
>> xen/arch/arm/pci/pci-host-generic.c |  48 +++++
>> xen/arch/arm/pci/pci-host-zynqmp.c  |  65 +++++++
>> xen/arch/arm/pci/pci.c              |  12 ++
>> xen/arch/arm/physdev.c              |   6 +-
>> xen/arch/arm/vpci.c                 | 102 ++++++++++
>> xen/arch/arm/vpci.h                 |  36 ++++
>> xen/arch/x86/domain.c               |   6 +
>> xen/arch/x86/physdev.c              |  52 +----
>> xen/arch/x86/x86_64/physdev.c       |   2 +-
>> xen/common/domain.c                 |   2 +-
>> xen/drivers/passthrough/pci.c       |  18 ++
>> xen/drivers/pci/Makefile            |   1 +
>> xen/drivers/pci/physdev.c           |  86 +++++++++
>> xen/drivers/vpci/Makefile           |   3 +-
>> xen/drivers/vpci/header.c           |   2 +
>> xen/include/asm-arm/domain.h        |   7 +-
>> xen/include/asm-arm/pci.h           |  85 ++++++++
>> xen/include/asm-x86/pci.h           |   8 +-
>> xen/include/public/arch-arm.h       |  21 +-
>> xen/include/public/domctl.h         |   4 +-
>> xen/include/xen/hypercall.h         |   4 +
>> xen/include/xen/pci.h               |   2 +
>> 36 files changed, 1168 insertions(+), 61 deletions(-)
>> create mode 100644 xen/arch/arm/pci/ecam.c
>> create mode 100644 xen/arch/arm/pci/pci-access.c
>> create mode 100644 xen/arch/arm/pci/pci-host-common.c
>> create mode 100644 xen/arch/arm/pci/pci-host-generic.c
>> create mode 100644 xen/arch/arm/pci/pci-host-zynqmp.c
>> create mode 100644 xen/arch/arm/vpci.c
>> create mode 100644 xen/arch/arm/vpci.h
>> create mode 100644 xen/drivers/pci/physdev.c
>> 
>> -- 
>> 2.25.1
>>