Re: [PATCH 1/2] docs/designs/launch: hyperlaunch design document

[Christopher Clark <christopher.w.clark@gmail.com>]

On Thu, Mar 18, 2021 at 9:43 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
>
> Just took a quick look at it.
>
> On Mon, Mar 15, 2021 at 11:18:13PM -0400, Daniel P. Smith wrote:
> > + +---------------+-----------+------------+-----------+-------------+---------------------+
> > + | **Xen Dom0**  | **Linux** | **Late**   | **Jail**  | **Xen**     | **Xen Hyperlaunch** |
> > + | **(Classic)** | **KVM**   | **HW Dom** | **house** | **dom0less**+---------+-----------+
> > + |               |           |            |           |             | Static  | Dynamic   |
> > + +===============+===========+============+===========+=============+=========+===========+
> > + | Hypervisor able to launch multiple VMs during host boot                                |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |     Y     |       Y     |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Hypervisor supports Static Partitioning                                                |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |     Y     |       Y     |    Y    |           |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Able to launch VMs dynamically after host boot                                         |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |       Y       |     Y     |      Y*    |     Y     |       Y*    |         |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Supports strong isolation between all VMs started at host boot                         |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |     Y     |       Y     |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Enables flexible sequencing of VM start during host boot                               |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |           |             |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Prevent all-powerful static root domain being launched at boot                         |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |           |       Y*    |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Operates without a Highly-privileged management VM (eg. Dom0)                          |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |      Y*    |           |       Y*    |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Operates without a privileged toolstack VM (Control Domain)                            |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |           |       Y*    |    Y    |           |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Extensible VM configuration applied before launch of VMs at host boot                  |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |           |             |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Flexible granular assignment of permissions and functions to VMs                       |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |           |             |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | Supports extensible VM measurement architecture for DRTM and attestation               |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |           |             |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + | PCI passthrough configured at host boot                                                |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
> > + |               |           |            |           |             |    Y    |     Y     |
> > + +---------------+-----------+------------+-----------+-------------+---------+-----------+
>
> I'm curious about this, I assume this is done using vPCI so that
> there's no hardware domain (and user-space device model) involved for
> PCI passthrough?

That would be an incorrect assumption. See below for why.

> I'm also not sure how you are going to handle things like SR-IOV
> devices. Right now SR-IOV capability is setup and initialized by the
> hardware domain, and the new virtual devices are notified to Xen once
> setup is done. Do you plan to move those bits into Xen, so that it can
> setup and initialize the SR-IOV capability?

While you could do it with the vPCI, as you point out this will not work
for SR-IOV. With hyperlaunch, these cases will require the use of a boot
domain, which is for all intents and purposes, a lightweight/restricted
toolstack domain.

The boot domain will have to do the necessary operations to ensure that
when startup is finished, PCI passthrough will be successfully setup.
Note, this may have to include the boot domain unpausing the hardware
domain to help complete the setup before the boot domain can exit and
allow the remaining domains to come online.

Both of these cases are in scope for our customer funding the work,
so we will be giving these use cases extra scrutiny, though it is
not like we intend to leave anything that works today unusable.

thanks

Christopher

>
> Thanks, Roger.