[PATCH v5 00/14] PCI devices passthrough on Arm, part 3

* [PATCH v5 00/14] PCI devices passthrough on Arm, part 3
@ 2021-11-25 11:02 Oleksandr Andrushchenko
  2021-11-25 11:02 ` [PATCH v5 01/14] rangeset: add RANGESETF_no_print flag Oleksandr Andrushchenko
                   ` (14 more replies)
  0 siblings, 15 replies; 130+ messages in thread
From: Oleksandr Andrushchenko @ 2021-11-25 11:02 UTC (permalink / raw)
  To: xen-devel
  Cc: julien, sstabellini, oleksandr_tyshchenko, volodymyr_babchuk,
	Artem_Mygaiev, roger.pau, jbeulich, andrew.cooper3,
	george.dunlap, paul, bertrand.marquis, rahul.singh,
	Oleksandr Andrushchenko

From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>

Hi, all!

1. This patch series is focusing on vPCI and adds support for non-identity
PCI BAR mappings which is required while passing through a PCI device to
a guest. The highlights are:

- Add relevant vpci register handlers when assigning PCI device to a domain
  and remove those when de-assigning. This allows having different
  handlers for different domains, e.g. hwdom and other guests.

- Emulate guest BAR register values based on physical BAR values.
  This allows creating a guest view of the registers and emulates
  size and properties probe as it is done during PCI device enumeration by
  the guest.

- Instead of handling a single range set, that contains all the memory
  regions of all the BARs and ROM, have them per BAR.

- Take into account guest's BAR view and program its p2m accordingly:
  gfn is guest's view of the BAR and mfn is the physical BAR value as set
  up by the host bridge in the hardware domain.
  This way hardware doamin sees physical BAR values and guest sees
  emulated ones.

2. The series also adds support for virtual PCI bus topology for guests:
 - We emulate a single host bridge for the guest, so segment is always 0.
 - The implementation is limited to 32 devices which are allowed on
   a single PCI bus.
 - The virtual bus number is set to 0, so virtual devices are seen
   as embedded endpoints behind the root complex.

3. The series has complete re-work of the locking scheme used/absent before with
the help of the work started by Roger [1]:
[PATCH v5 03/13] vpci: move lock outside of struct vpci

This way the lock can be used to check whether vpci is present, and
removal can be performed while holding the lock, in order to make
sure there are no accesses to the contents of the vpci struct.
Previously removal could race with vpci_read for example, since the
lock was dropped prior to freeing pdev->vpci.
This also solves synchronization issues between all vPCI code entities
which could run in parallel.

4. There is an outstanding TODO left unimplemented by this series:
for unprivileged guests vpci_{read|write} need to be re-worked
to not passthrough accesses to the registers not explicitly handled
by the corresponding vPCI handlers: without fixing that passthrough
to guests is completely unsafe as Xen allows them full access to
the registers.

Xen needs to be sure that every register a guest accesses is not
going to cause the system to malfunction, so Xen needs to keep a
list of the registers it is safe for a guest to access.

For example, we should only expose the PCI capabilities that we know
are safe for a guest to use, i.e.: MSI and MSI-X initially.
The rest of the capabilities should be blocked from guest access,
unless we audit them and declare safe for a guest to access.

As a reference we might want to look at the approach currently used
by QEMU in order to do PCI passthrough. A very limited set of PCI
capabilities known to be safe for untrusted access are exposed to the
guest and registers need to be explicitly handled or else access is
rejected. Xen needs a fairly similar model in vPCI or else none of
this will be safe for unprivileged access.

5. The series was also tested on:
 - x86 PVH Dom0 and doesn't break it.
 - x86 HVM with PCI passthrough to DomU and doesn't break it.

Thank you,
Oleksandr

[1] https://lore.kernel.org/xen-devel/20180717094830.54806-2-roger.pau@citrix.com/

Oleksandr Andrushchenko (13):
  rangeset: add RANGESETF_no_print flag
  vpci: fix function attributes for vpci_process_pending
  vpci: cancel pending map/unmap on vpci removal
  vpci: add hooks for PCI device assign/de-assign
  vpci/header: implement guest BAR register handlers
  vpci/header: handle p2m range sets per BAR
  vpci/header: program p2m with guest BAR view
  vpci/header: emulate PCI_COMMAND register for guests
  vpci/header: reset the command register when adding devices
  vpci: add initial support for virtual PCI bus topology
  xen/arm: translate virtual PCI bus topology for guests
  xen/arm: account IO handlers for emulated PCI MSI-X
  vpci: add TODO for the registers not explicitly handled

Roger Pau Monne (1):
  vpci: move lock outside of struct vpci

 tools/tests/vpci/emul.h       |   5 +-
 tools/tests/vpci/main.c       |   4 +-
 xen/arch/arm/vpci.c           |  33 +++-
 xen/arch/x86/hvm/vmsi.c       |   8 +-
 xen/common/rangeset.c         |   5 +-
 xen/drivers/Kconfig           |   4 +
 xen/drivers/passthrough/pci.c |  11 ++
 xen/drivers/vpci/header.c     | 352 +++++++++++++++++++++++++++-------
 xen/drivers/vpci/msi.c        |  11 +-
 xen/drivers/vpci/msix.c       |   8 +-
 xen/drivers/vpci/vpci.c       | 252 +++++++++++++++++++++---
 xen/include/xen/pci.h         |   6 +
 xen/include/xen/rangeset.h    |   7 +-
 xen/include/xen/sched.h       |   8 +
 xen/include/xen/vpci.h        |  47 ++++-
 15 files changed, 644 insertions(+), 117 deletions(-)

-- 
2.25.1

^ permalink raw reply	[flat|nested] 130+ messages in thread