Re: Problem creating domU guest on arm64

[Marco Cacre <marco.cacre@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3977 bytes --]

Hi  all,



I did several attempts but I have always problems with disk backend setup
during creation of domU domain.



The latest attempt was using deban arm64 image with this configuration:



memory = 512
name = "debian"
vcpus = 1
maxvcpus = 1

kernel = "/home/xen/vmlinuz"
ramdisk = "/home/xen/initrd.img"
disk = ["phy:/dev/loop0,xvda,rw"]
extra = "root=/dev/xvda rw console=hvc0 earlyprintk=xen"



kernel and ramdisk boot up successfully and I can attach to the domU
console, but when I try to define the disk backend with the debian rootfs
the creation of the domain fails.



rootfs is attached to loop device as follow:



losetup /dev/loop0 /home/xen/rootfs.img



I attached to this mail the verbose trace of the command xl create



Thanks a lot in advance for any kind of help

Best Regards

Marco

Il giorno ven 15 apr 2022 alle ore 12:56 Marco Cacre <marco.cacre@gmail.com>
ha scritto:

> Hi all,
>
> I'm facing always the same problem (described below) trying to create a
> Xen guest domain on rockpro64 (arm64).
>
> I have successfully built uboot, xen (4.14.4) and dom0 linux kernel with
> xen tool stack, these two last starting from buildroot (with linux kernel
> version 5.15.33 and xen toolstack version 4.14.4)
>
> These are the process of Xen:
>
> 177 root    /usr/sbin/xenwatchdogd 30 15
> 203 root    /usr/sbin/xenstored --pid-file /var/run/xen/xenstored.pid
> 208 root    /usr/sbin/xenconsoled --pid-file=/var/run/xen/xenconsoled.pid
> 214 root    /usr/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach
> -name dom0 -nographic -M xenpv -daemonize -monitor /dev/null
> -serial/dev/nul -parallel/dev/null -pidfile/var/run/xen/qemu-dom0.pid
>
> and the drivers loaded as a module:
> # lsmod
> Module                  Size  Used by    Tainted: G
> xen_netback            65536  0
> xen_blkback            49152  0
>
> Xen and dom0 Image boot up successfully, at the end:
> ================================================
> >xl list
> Name                                        ID  Mem VCPUs State Time(s)
> Domain-0                                    0  1024    2    r-----
>  4000.3
>
> # uname -a
> Linux rockpro64 5.15.33 #3 SMP PREEMPT Fri Apr 15 11:01:37 CEST 2022
> aarch64 GNU/Linux
> ================================================
>
> when I try to create a domU domain I have problem with disk backend, I
> have made several attempts trying with different domU linux and I always
> get the following error :
>
> ================================================
> Parsing config from ./alpine_xen_domu.config
>
> libxl: error: libxl_aoutils.c:539:async_exec_timeout: killing execution of
> /etc/xen/scripts/block add because of timeout
> libxl: error: libxl_create.c:1608:domcreate_launch_dm: Domain 2:unable to
> add disk devices
> libxl: error: libxl_aoutils.c:539:async_exec_timeout: killing execution of
> /etc/xen/scripts/block remove because of timeout
> libxl: error: libxl_domain.c:1182:libxl__destroy_domid: Domain
> 2:Non-existant domain
> libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain 2:Unable
> to destroy guest
> libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain 2:Destruction
> of domain failed
> ================================================
>
> The last try was with alpine domU linux with the following config:
> ================================================
>
> # Alpine Linux PV DomU
> # Kernel paths for install
> kernel = "/home/xen/vmlinuz-virt"
> ramdisk = "/home/xen/initramfs-virt"
> extra="modules=loop,squashfs console=hvc0"
> # Path to HDD and iso file
> disk = [ 'format=raw, vdev=xvda, access=w, target=/data/a1.img',
>         'format=raw, vdev=xvdc, access=r, devtype=cdrom,
> target=/home/xen/alpine-virt-3.15.3-armv7.iso'  ]
> # Network configuration
> vif = ['bridge=br0']
> # DomU settings
>
> memory = 512
>
> name = "alpine-a1"
>
> vcpus = 1
>
> maxvcpus = 1
> ================================================
>
> thanks a lot in advance
> Best regards
> Marco
>

[-- Attachment #1.2: Type: text/html, Size: 5963 bytes --]

[-- Attachment #2: xl_create_trace.txt --]
[-- Type: text/plain, Size: 12082 bytes --]

# xl -vvv create ./linux.config 
Parsing config from ./linux.config
libxl: debug: libxl_create.c:2017:do_domain_create: ao 0x55b786e280: create: how=(nil) callback=(nil) poller=0x55b786e3d0
libxl: detail: libxl_create.c:622:libxl__domain_make: passthrough: disabled
libxl: debug: libxl_arm.c:71:libxl__arch_domain_prepare_config: Configure the domain
libxl: debug: libxl_arm.c:74:libxl__arch_domain_prepare_config:  - Allocate 0 SPIs
libxl: debug: libxl_device.c:379:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=unknown
libxl: debug: libxl_device.c:413:libxl__device_disk_set_backend: Disk vdev=xvda, using backend phy
libxl: debug: libxl_create.c:1279:initiate_domain_create: Domain 6:running bootloader
libxl: debug: libxl_bootloader.c:334:libxl__bootloader_run: Domain 6:no bootloader configured, using user supplied kernel
libxl: debug: libxl_event.c:864:libxl__ev_xswatch_deregister: watch w=0x55b786fa58: deregister unregistered
libxl: debug: libxl_sched.c:81:libxl__set_vcpuaffinity: Domain 6:New soft affinity for vcpu 0 has unreachable cpus
domainbuilder: detail: xc_dom_allocate: cmdline="root=/dev/xvda rw console=hvc0 earlyprintk=xen", features=""
domainbuilder: detail: xc_dom_kernel_file: filename="/home/xen/vmlinuz"
domainbuilder: detail: xc_dom_malloc_filemap    : 26889 kB
domainbuilder: detail: xc_dom_module_file: filename="/home/xen/initrd.img"
domainbuilder: detail: xc_dom_malloc_filemap    : 40 MB
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.14, caps xen-3.0-aarch64 xen-3.0-armv7l 
domainbuilder: detail: xc_dom_rambase_init: RAM starts at 40000
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... 
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM64) loader ... 
domainbuilder: detail: loader probe OK
domainbuilder: detail: xc_dom_parse_zimage64_kernel: called
domainbuilder: detail: xc_dom_parse_zimage64_kernel: xen-3.0-aarch64: 0x40000000 -> 0x41a42780
libxl: debug: libxl_arm.c:872:libxl__prepare_dtb: constructing DTB for Xen version 4.14 guest
libxl: debug: libxl_arm.c:874:libxl__prepare_dtb:  - vGIC version: V3
libxl: debug: libxl_arm.c:312:make_chosen_node: /chosen/bootargs = root=/dev/xvda rw console=hvc0 earlyprintk=xen
libxl: debug: libxl_arm.c:319:make_chosen_node: /chosen adding placeholder linux,initrd properties
libxl: debug: libxl_arm.c:459:make_memory_nodes: Creating placeholder node /memory@40000000
libxl: debug: libxl_arm.c:459:make_memory_nodes: Creating placeholder node /memory@200000000
libxl: debug: libxl_arm.c:975:libxl__prepare_dtb: fdt total size 1338
domainbuilder: detail: xc_dom_devicetree_mem: called
libxl: debug: libxl_arm.c:1020:libxl__arch_domain_init_hw_description: Generating ACPI tables is disabled by user.
domainbuilder: detail: xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x20000 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: set_mode: guest xen-3.0-aarch64, address size 64
domainbuilder: detail: populate_guest_memory: populating RAM @ 0000000040000000-0000000060000000 (512MB)
domainbuilder: detail: populate_one_size: populated 0x100/0x100 entries with shift 9
domainbuilder: detail: meminit: placing boot modules at 0x48000000
domainbuilder: detail: meminit: ramdisk: 0x48000000 -> 0x4a81f000
domainbuilder: detail: meminit: devicetree: 0x4a81f000 -> 0x4a820000
libxl: debug: libxl_arm.c:1087:libxl__arch_domain_finalise_hw_description: /chosen updating initrd properties to cover 48000000-4a81f000
libxl: debug: libxl_arm.c:1054:finalise_one_node: Populating placeholder node /memory@40000000
libxl: debug: libxl_arm.c:1048:finalise_one_node: Nopping out placeholder node /memory@200000000
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_malloc            : 157 kB
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x40000+0x1a43 at 0x7fa073c000
domainbuilder: detail: xc_dom_alloc_segment:   kernel       : 0x40000000 -> 0x41a43000  (pfn 0x40000 + 0x1a43 pages)
domainbuilder: detail: xc_dom_load_zimage_kernel: called
domainbuilder: detail: xc_dom_load_zimage_kernel: kernel seg 0x40000000-0x41a43000
domainbuilder: detail: xc_dom_load_zimage_kernel: copy 27535232 bytes from blob 0x7fa499e000 to dst 0x7fa073c000
domainbuilder: detail: xc_dom_malloc            : 240 kB
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x48000+0x281f at 0x7f9df1d000
domainbuilder: detail: xc_dom_alloc_segment:   module0      : 0x48000000 -> 0x4a81f000  (pfn 0x48000 + 0x281f pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x4a81f+0x1 at 0x7fa6715000
domainbuilder: detail: xc_dom_alloc_segment:   devicetree   : 0x4a81f000 -> 0x4a820000  (pfn 0x4a81f + 0x1 pages)
domainbuilder: detail: alloc_magic_pages: called
domainbuilder: detail: xc_dom_build_image  : virt_alloc_end : 0x4a820000
domainbuilder: detail: xc_dom_build_image  : virt_pgtab_end : 0x0
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: bootearly: doing nothing
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-aarch64 <= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-armv7l
domainbuilder: detail: start_info_arm: called
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail:    allocated
domainbuilder: detail:       malloc             : 401 kB
domainbuilder: detail:       anon mmap          : 0 bytes
domainbuilder: detail:    mapped
domainbuilder: detail:       file mmap          : 66 MB
domainbuilder: detail:       domU mmap          : 66 MB
domainbuilder: detail: vcpu_arm64: called
domainbuilder: detail: DTB 4a81f000
domainbuilder: detail: Initial state CPSR 0x1c5 PC 0x40000000
domainbuilder: detail: compat_gnttab_hvm_seed: d6: pfn=0x38000
domainbuilder: detail: xc_dom_set_gnttab_entry: d6 gnt[0] -> d0 0x39000
domainbuilder: detail: xc_dom_set_gnttab_entry: d6 gnt[1] -> d0 0x39001
domainbuilder: detail: xc_dom_release: called
libxl: debug: libxl_device.c:379:libxl__device_disk_set_backend: Disk vdev=xvda spec.backend=phy
libxl: debug: libxl_event.c:813:libxl__ev_xswatch_register: watch w=0x55b7874050 wpath=/local/domain/0/backend/vbd/6/51712/state token=3/0: register slotnum=3
libxl: debug: libxl_create.c:2055:do_domain_create: ao 0x55b786e280: inprogress: poller=0x55b786e3d0, flags=i
libxl: debug: libxl_event.c:750:watchfd_callback: watch w=0x55b7874050 wpath=/local/domain/0/backend/vbd/6/51712/state token=3/0: event epath=/local/domain/0/backend/vbd/6/51712/state
libxl: debug: libxl_event.c:1052:devstate_callback: backend /local/domain/0/backend/vbd/6/51712/state wanted state 2 ok
libxl: debug: libxl_event.c:850:libxl__ev_xswatch_deregister: watch w=0x55b7874050 wpath=/local/domain/0/backend/vbd/6/51712/state token=3/0: deregister slotnum=3
libxl: debug: libxl_device.c:1090:device_backend_callback: Domain 6:calling device_backend_cleanup
libxl: debug: libxl_event.c:864:libxl__ev_xswatch_deregister: watch w=0x55b7874050: deregister unregistered
libxl: debug: libxl_linux.c:194:libxl__hotplug_disk: Domain 6:Args and environment ready
libxl: debug: libxl_device.c:1191:device_hotplug: Domain 6:calling hotplug script: /etc/xen/scripts/block add
libxl: debug: libxl_device.c:1192:device_hotplug: Domain 6:extra args:
libxl: debug: libxl_device.c:1200:device_hotplug: Domain 6:env:
libxl: debug: libxl_device.c:1207:device_hotplug: Domain 6:	script: /etc/xen/scripts/block
libxl: debug: libxl_device.c:1207:device_hotplug: Domain 6:	XENBUS_TYPE: vbd
libxl: debug: libxl_device.c:1207:device_hotplug: Domain 6:	XENBUS_PATH: backend/vbd/6/51712
libxl: debug: libxl_device.c:1207:device_hotplug: Domain 6:	XENBUS_BASE_PATH: backend
libxl: debug: libxl_aoutils.c:593:libxl__async_exec_start: forking to execute: /etc/xen/scripts/block add 
libxl: debug: libxl_event.c:720:watchfd_callback: watch epath=/local/domain/0/backend/vbd/6/51712/state token=3/0: empty slot
libxl: error: libxl_aoutils.c:539:async_exec_timeout: killing execution of /etc/xen/scripts/block add because of timeout
libxl: debug: libxl_event.c:864:libxl__ev_xswatch_deregister: watch w=0x55b7874150: deregister unregistered
libxl: debug: libxl_event.c:864:libxl__ev_xswatch_deregister: watch w=0x55b7874150: deregister unregistered
libxl: error: libxl_create.c:1608:domcreate_launch_dm: Domain 6:unable to add disk devices
libxl: debug: libxl_event.c:864:libxl__ev_xswatch_deregister: watch w=0x55b7871408: deregister unregistered
libxl: debug: libxl_event.c:813:libxl__ev_xswatch_register: watch w=0x55b7872100 wpath=/local/domain/0/backend/vbd/6/51712/state token=3/1: register slotnum=3
libxl: debug: libxl_event.c:750:watchfd_callback: watch w=0x55b7872100 wpath=/local/domain/0/backend/vbd/6/51712/state token=3/1: event epath=/local/domain/0/backend/vbd/6/51712/state
libxl: debug: libxl_event.c:1052:devstate_callback: backend /local/domain/0/backend/vbd/6/51712/state wanted state 6 ok
libxl: debug: libxl_event.c:850:libxl__ev_xswatch_deregister: watch w=0x55b7872100 wpath=/local/domain/0/backend/vbd/6/51712/state token=3/1: deregister slotnum=3
libxl: debug: libxl_device.c:1090:device_backend_callback: Domain 6:calling device_backend_cleanup
libxl: debug: libxl_event.c:864:libxl__ev_xswatch_deregister: watch w=0x55b7872100: deregister unregistered
libxl: debug: libxl_linux.c:194:libxl__hotplug_disk: Domain 6:Args and environment ready
libxl: debug: libxl_device.c:1191:device_hotplug: Domain 6:calling hotplug script: /etc/xen/scripts/block remove
libxl: debug: libxl_device.c:1192:device_hotplug: Domain 6:extra args:
libxl: debug: libxl_device.c:1200:device_hotplug: Domain 6:env:
libxl: debug: libxl_device.c:1207:device_hotplug: Domain 6:	script: /etc/xen/scripts/block
libxl: debug: libxl_device.c:1207:device_hotplug: Domain 6:	XENBUS_TYPE: vbd
libxl: debug: libxl_device.c:1207:device_hotplug: Domain 6:	XENBUS_PATH: backend/vbd/6/51712
libxl: debug: libxl_device.c:1207:device_hotplug: Domain 6:	XENBUS_BASE_PATH: backend
libxl: debug: libxl_aoutils.c:593:libxl__async_exec_start: forking to execute: /etc/xen/scripts/block remove 
libxl: debug: libxl_event.c:864:libxl__ev_xswatch_deregister: watch w=0x55b7872200: deregister unregistered
libxl: debug: libxl_linux.c:211:libxl__get_hotplug_script_info: Domain 6:num_exec 1, not running hotplug scripts
libxl: debug: libxl_device.c:1176:device_hotplug: Domain 6:No hotplug script to execute
libxl: debug: libxl_event.c:864:libxl__ev_xswatch_deregister: watch w=0x55b7872200: deregister unregistered
libxl: debug: libxl_domain.c:1596:devices_destroy_cb: Domain 6:Forked pid 24145 for destroy of domain
libxl: debug: libxl_event.c:2066:libxl__ao_complete: ao 0x55b786e280: complete, rc=-3
libxl: debug: libxl_event.c:2035:libxl__ao__destroy: ao 0x55b786e280: destroy
libxl: debug: libxl_domain.c:1045:libxl_domain_destroy: Domain 6:ao 0x55b786e280: create: how=(nil) callback=(nil) poller=0x55b786e3d0
libxl: error: libxl_domain.c:1182:libxl__destroy_domid: Domain 6:Non-existant domain
libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain 6:Unable to destroy guest
libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain 6:Destruction of domain failed
libxl: debug: libxl_event.c:2066:libxl__ao_complete: ao 0x55b786e280: complete, rc=-21
libxl: debug: libxl_domain.c:1054:libxl_domain_destroy: Domain 6:ao 0x55b786e280: inprogress: poller=0x55b786e3d0, flags=ic
libxl: debug: libxl_event.c:2035:libxl__ao__destroy: ao 0x55b786e280: destroy
xencall:buffer: debug: total allocations:117 total releases:117
xencall:buffer: debug: current allocations:0 maximum allocations:3
xencall:buffer: debug: cache current size:3
xencall:buffer: debug: cache hits:102 misses:3 toobig:12
xencall:buffer: debug: total allocations:0 total releases:0
xencall:buffer: debug: current allocations:0 maximum allocations:0
xencall:buffer: debug: cache current size:0
xencall:buffer: debug: cache hits:0 misses:0 toobig:0