Nested Virtualization of Hyper-V on Xen Not Working

* Nested Virtualization of Hyper-V on Xen Not Working
@ 2021-07-21  4:09 Xentrigued
  2021-07-25  1:32 ` Rob Townley
  2021-07-26 13:55 ` Andrew Cooper
  0 siblings, 2 replies; 5+ messages in thread
From: Xentrigued @ 2021-07-21  4:09 UTC (permalink / raw)
  To: xen-devel

RATIONALE: Features in recent versions of Windows now REQUIRE Hyper-V
support to work.  In particular, Windows Containers, Sandbox, Docker Desktop
and the Windows Subsystem for Linux version 2 (WSL2).  Running Windows in a
VM as a development and test platform is currently a common requirement for
various user segments and will likely become necessary for production in the
future.  Nested virtualization of Hyper-V currently works on VMware ESXi,
Microsoft Hyper-V and KVM-based hypervisors.  This puts Xen and its
derivatives at a disadvantage when choosing a hypervisor.

WHAT IS NOT WORKING?  Provided the requirements set forth in:
https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen have been met,
an hvm guest running Windows 10 PRO Version 21H1 x64 shows that all four
requirements for running Hyper-V are available using the msinfo32.exe or
systeminfo.exe commands.  More granular knowledge of the CPU capabilities
exposed to the guest can be observed using the Sysinternals Coreinfo64.exe
command.  CPUID flags present appear to mirror those on other working nested
hypervisor configurations.

Enabling Windows Features for Hyper-V, Virtual Machine Platform, etc. all
appear to work without error.  However, after the finishing reboot, Hyper-V
is simply not active.  This--despite the fact that vmcompute.exe (Hyper-V
host compute service) is running and there are no errors in the logs.  In
addition, all four Hyper-V prerequisites continue to show as available.

By contrast, after the finishing reboot of an analogous Windows VM running
on ESXi, the four prerequisites are reversed:  hypervisor is now active;
vmx, ept and urg (unrestricted guest) are all off as viewed with the
Coreinfo64.exe -v command.  Furthermore, all functions requiring Hyper-V are
now active and working as expected.

This deficiency has been observed in two test setups running Xen 4.15 from
source and XCP-ng 8.2, both running on Intel with all of the latest,
generally available patches.  We presume that the same behavior is present
on Citrix Hypervisor 8.2 as well.

SUMMATION:
Clearly, much effort has already been expended to support the Viridian
enlightenments that optimize running Windows on Xen.  It also looks like a
significant amount of effort has been put forth to advance nested
virtualization in general.

Therefore, if it would be helpful, I am willing to perform testing and
provide feedback and logs as appropriate in order to help get this working.

While my day job is managing a heterogeneous collection of systems running
on various hypervisors, I have learned the rudiments of integrating patches
and rebuilding Xen from source so could no doubt be useful in assisting you
with this worthwhile endeavor.

^ permalink raw reply	[flat|nested] 5+ messages in thread