Booting Xen dom0 on Hyper-V

Booting Xen dom0 on Hyper-V

[Anish Gupta]

Hello all,
I am new to Azure/Hyper-V environment and trying to boot Xen PV.  Dom0 Linux on Xen doesn’t detect hv_vmbus and boot fails. I get the same result even with Xen HYPERV_GUEST config. Here is the console logs with latest Xen 4.15.3 and Linux 5.18:

Loading Xen 4.15.3 ...
Loading Linux 5.18.7+ ...
Loading initial ramdisk ...
 Xen 4.15.3
(XEN) Xen version 4.15.3 (root@i2ltiotchnquhby5irpcvxfz2h.xx.internal.cloudapp.net) (gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)) debug=n Mon Aug  8 17:48:08 UTC 2022
(XEN) Latest ChangeSet: Mon Jun 27 15:16:56 2022 +0200 git:cc3329f
(XEN) build-id: 25b7db08246b91bfd5e2302898263ae47cda7473
(XEN) Bootloader: GRUB 2.02~beta2
(XEN) Command line: placeholder dom0_mem=4096M,max:4096M com1=115200,8n1 console=com1,vga dom0_vcpus_pin=true dom0_max_vcpus=4 loglvl=all guest_loglvl=all noreboot=true
(XEN) Xen image load base address: 0
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 106 (0x6a), Stepping 6 (raw 000606a6)
(XEN) Xen-e820 RAM map:
(XEN)  [0000000000000000, 000000000009fbff] (usable)
(XEN)  [000000000009fc00, 000000000009ffff] (reserved)
(XEN)  [00000000000e0000, 00000000000fffff] (reserved)
(XEN)  [0000000000100000, 000000003ffeffff] (usable)
(XEN)  [000000003fff0000, 000000003fffefff] (ACPI data)
(XEN)  [000000003ffff000, 000000003fffffff] (ACPI NVS)
(XEN)  [0000000100000000, 0000000fdfffffff] (usable)
(XEN)  [0000001000000000, 00000010dfffffff] (usable)
….
(XEN) System RAM: 65535MB (67108412kB)
….
<<< Linux booting >>>
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] Xen: [mem 0x0000000000000000-0x000000000009efff] usable
[    0.000000] Xen: [mem 0x000000000009fc00-0x00000000000fffff] reserved
[    0.000000] Xen: [mem 0x0000000000100000-0x000000003ffeffff] usable
[    0.000000] Xen: [mem 0x000000003fff0000-0x000000003fffefff] ACPI data
[    0.000000] Xen: [mem 0x000000003ffff000-0x000000003fffffff] ACPI NVS
[    0.000000] Xen: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] Xen: [mem 0x00000000fee00000-0x00000000feefffff] reserved
[    0.000000] Xen: [mem 0x0000000100000000-0x00000001c0070fff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.3 present.
[    0.000000] DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008  12/07/2018
[    0.000000] Hypervisor detected: Xen PV <<<
…
[  197.060965] dracut-initqueue[311]: Warning: Could not boot.
[  197.067020] dracut-initqueue[311]: Warning: /dev/disk/by-uuid/c8396182-9335-4bf7-a963-f14e85f41f23 does not exist
         Starting Setup Virtual Console...
[  OK  ] Started Setup Virtual Console.
         Starting Dracut Emergency Shell...
Warning: /dev/disk/by-uuid/c8396182-9335-4bf7-a963-f14e85f41f23 does not exist

Generating "/run/initramfs/rdsosreport.txt"


Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.


dracut:/# 


As an experiment I did force load hv_vmbus but then hit a page fault. I will really appreciate guidance in getting it working. 

Was Xen PV ever supported in any version of Xen + Linux on Azure/Hyper-V?

Thanks in advance for your help!

Regards,
Anish

Re: Booting Xen dom0 on Hyper-V

[Wei Liu]

Hi Anish

On Tue, Aug 09, 2022 at 10:10:03PM +0530, Anish Gupta wrote:
> Hello all,
> I am new to Azure/Hyper-V environment and trying to boot Xen PV.  Dom0 Linux on Xen doesn’t detect hv_vmbus and boot fails. I get the same result even with Xen HYPERV_GUEST config. Here is the console logs with latest Xen 4.15.3 and Linux 5.18:
> 
[...]
> [    0.000000] SMBIOS 2.3 present.
> [    0.000000] DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008  12/07/2018
> [    0.000000] Hypervisor detected: Xen PV <<<
> …

This is expected.

> [  197.060965] dracut-initqueue[311]: Warning: Could not boot.
> [  197.067020] dracut-initqueue[311]: Warning: /dev/disk/by-uuid/c8396182-9335-4bf7-a963-f14e85f41f23 does not exist
>          Starting Setup Virtual Console...
> [  OK  ] Started Setup Virtual Console.
>          Starting Dracut Emergency Shell...
> Warning: /dev/disk/by-uuid/c8396182-9335-4bf7-a963-f14e85f41f23 does not exist
> 
> Generating "/run/initramfs/rdsosreport.txt"
> 
> 
> Entering emergency mode. Exit the shell to continue.
> Type "journalctl" to view system logs.
> You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
> after mounting them and attach it to a bug report.
> 
> 
> dracut:/# 
> 
> 
> As an experiment I did force load hv_vmbus but then hit a page fault. I will really appreciate guidance in getting it working. 
> 

What did the backtrace show?

> Was Xen PV ever supported in any version of Xen + Linux on Azure/Hyper-V?

I never got the time to make it work with VMBus based devices.

The initial development work was done on a Gen1 VM locally with emulated
hard drive and network adapter. Azure Gen2 VMs don't have emulated
devices.

In hv_acpi_init, you can see that if the hypervisor personality is not
Hyper-V then that function returns -ENODEV.

I think you will need to find a way to initialize VMBus somehow.

Fundamentally, you're trying to achieve mixed-personalities in a single
system. I think it is going to require non-trivial investment to make it
work properly.

Thanks,
Wei.

Re: Booting Xen dom0 on Hyper-V

[Wei Liu]

Hi Anish

Please use reply-all in the future. And please use bottom-posting style
and wrap your lines to about 72 characters wide.

I added the list back.

On Thu, Aug 11, 2022 at 06:45:27AM +0530, Anish Gupta wrote:
> Hi Wei,
> 
> Thanks again for your reply.
> 
> >The initial development work was done on a Gen1 VM locally with emulated
> > 
> > hard drive and network adapter. Azure Gen2 VMs don't have emulated
> > devices.
> 
> 
> Do you remember the Gen1 VM type with emulated device that you used? I
> am new to Azure and still trying to figure out things. If I know the
> instance type, I can work with Azure support team to get my hand on to
> one of those old VMs and use that as base to start digging for vmbus.

No. The VM I used was on my local machine. I did not do my development
with an Azure VM, because that would be impossible to debug (at least at
the time for me).

Probably when you pick a Gen1 image on Azure you will get a Gen1 VM. Or
you can ask directly the support team how you can get a Gen1 VM.

Thanks,
Wei.

> 
> Regards,
> Anish