Re: HEADS UP: Imported Xen 4.7 and blkback changes

Re: HEADS UP: Imported Xen 4.7 and blkback changes

[Marcin Cieslak]

On Fri, 3 Jun 2016, Roger Pau Monné wrote:

> Hello,
> 
> First of all, this message is only relevant to those that use FreeBSD as 
> Dom0 (host), not as a DomU (guest), so don't panic.
> 
> I've imported the latest Xen version (4.7-rc4) into the ports tree, it's 
> still not the final version, but it's quite close, so we better start 
> testing it to make sure it works fine with FreeBSD.

Thank you Roger, this is excellent. Are xen-tools-devel 4.5 now?
Looks confusing.

I have also tried building xen-tools (4.7) without python
and qemu configure reported this.

Also got this:

===>   Registering installation for xen-tools-4.7.0
(xen-tools-4.7.0) /usr/ports/sysutils/xen-tools/work/stage//usr/local/lib/python2.7/site-packages/xen/lowlevel/xc.so - required shared library libxenctrl.so.4.5 not found
(xen-tools-4.7.0) /usr/ports/sysutils/xen-tools/work/stage//usr/local/lib/python2.7/site-packages/xen/lowlevel/xc.so - required shared library libxenguest.so.4.5 not found
(xen-tools-4.7.0) /usr/ports/sysutils/xen-tools/work/stage//usr/local/lib/xen/bin/qemu-system-i386 - required shared library libxenctrl.so.4.5 not found
(xen-tools-4.7.0) /usr/ports/sysutils/xen-tools/work/stage//usr/local/lib/xen/bin/qemu-system-i386 - required shared library libxenguest.so.4.5 not found
Installing xen-tools-4.7.0...

Adding Python manually and rebuilding seems to fix those issues.

It seems to work, I only wish we've had ports from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192012 committed...

Marcin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7 and blkback changes - domU respawning on_crash

[Marcin Cieslak]

On Wed, 8 Jun 2016, Marcin Cieslak wrote:

> On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> 
> > Hello,
> > 
> > First of all, this message is only relevant to those that use FreeBSD as 
> > Dom0 (host), not as a DomU (guest), so don't panic.
> > 
> > I've imported the latest Xen version (4.7-rc4) into the ports tree, it's 
> > still not the final version, but it's quite close, so we better start 
> > testing it to make sure it works fine with FreeBSD.

One issue maybe unrelated to FreeBSD:

This domain:

builder = "hvm"
memory = 4096
vcpus = 2
name = "Windows2016"
disk = [
        '/dev/zvol/zroot/windows0,raw,hda,w',
        '/dev/zvol/zroot/vs2013,raw,hdb,w',
#        '/root/win/install.iso,raw,hdc:cdrom,r'
]
boot = "c" # Boot to hard disk image
vnc = 2
#vnclisten = "0.0.0.0"
usbdevice = 'tablet'
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
acpi = 1
bios = 'ovmf'
vif = [ 'bridge=bridge0,mac=00:16:3e:5d:0d:48' ]
videoram=16
vga = "stdvga"

crashes because I didn't have ovmf image:

(d203) HVM Loader
(d203) Detected Xen v4.7.0-rc
(d203) Xenbus rings @0xfeffc000, event channel 1
(d203) Unknown BIOS ovmf, no ROM image found
(d203) *** HVMLoader bug at hvmloader.c:229
(d203) *** HVMLoader crashed.

But I seem unable to kill it with "xl destroy" - it keeps
respawning again:

Windows2016                                211  4079     1     --p---       0.0
Windows2016                                213  4096     1     --psc-       0.0
(disappears)
Windows2016                                221  4096     1     --psc-       0.0
(null)                                     221   147     1     --psc-       0.0
...
...

I have finally managed to snatch it by issuing this a few times, after
changing the "on_crash" to 'destroy':

# xl config-update Windows2016 xen/windows-run.cfg
WARNING: xl now has better capability to manage domain configuration, avoid using this command when possible
setting dom243 configuration


Marcin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Marcin Cieslak]

On Fri, 3 Jun 2016, Roger Pau Monné wrote:

> One of the more relevant changes in 4.7 regarding FreeBSD is the support for 
> block hotplug scripts. This means that we now have the option to use 
> backends different than simple block or regular files, provided that someone 
> writes the proper hotplug scripts to attach them (I've heard there are some 
> iSCSI hotplug scripts around). This however requires changes in blkback, so 
> if you plan to use the Xen 4.7 port, please make sure that you are running a 
> kernel that contains revision r301269 (or any later version). The same also 

I am running it with r301685 and the HVM guests have some trouble with
block devices.

SeaBIOS does not find /dev/zvol/zroot/freebsd1,raw,xvda,w to boot FreeBSD
from, after chaging to "hda" I get up to the kernel mountroot prompt
(Xen block devices seem to be detected in dmesg).

What used to be Windows 2016 domU with /dev/zvol/zroot/windows0,raw,hda,w
ends up in the Tianocore UEFI shell. Block devices seem to be available,
I can even list the fs0: partition, but no booting further possible.

Marcin

# more freebsd.cfg 
builder = "hvm"
name = "FreeBSD"
disk = [
        '/dev/zvol/zroot/freebsd1,raw,xvda,w',
        '/dev/zvol/zroot/freebsd2,raw,xvdb,w'
]
boot = "c"
#usbdevice = 'tablet'
#nographics = 1
serial = [ "file:/tmp/boot.log" ]
vnc = 1
#vnclisten = '0.0.0.0'
vif = ['bridge=bridge0,mac=00:02:04:08:fd:f0']
memory=2048
vcpus=1
vga = "cirrus"
videoram = 16

# more windows-run.cfg 
builder = "hvm"
memory = 4096
vcpus = 2
name = "Windows2016"
disk = [
        '/dev/zvol/zroot/windows0,raw,hda,w',
        '/dev/zvol/zroot/vs2013,raw,hdb,w',
#        '/root/win/install.iso,raw,hdc:cdrom,r'
]
boot = "c" # Boot to hard disk image
vnc = 2
#vnclisten = "0.0.0.0"
usbdevice = 'tablet'
on_poweroff = 'destroy'
on_reboot = 'restart'
#on_crash = 'restart'
on_crash = 'destroy'
acpi = 1
bios = 'ovmf'
vif = [ 'bridge=bridge0,mac=00:16:3e:5d:0d:48' ]
videoram=16
vga = "stdvga"

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7 and blkback changes

[Roger Pau Monné]

On Wed, Jun 08, 2016 at 10:18:09PM +0000, Marcin Cieslak wrote:
> On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> 
> > Hello,
> > 
> > First of all, this message is only relevant to those that use FreeBSD as 
> > Dom0 (host), not as a DomU (guest), so don't panic.
> > 
> > I've imported the latest Xen version (4.7-rc4) into the ports tree, it's 
> > still not the final version, but it's quite close, so we better start 
> > testing it to make sure it works fine with FreeBSD.
> 
> Thank you Roger, this is excellent. Are xen-tools-devel 4.5 now?
> Looks confusing.
> 
> I have also tried building xen-tools (4.7) without python
> and qemu configure reported this.

Python is required, you cannot get rid of it.
 
> Also got this:
> 
> ===>   Registering installation for xen-tools-4.7.0
> (xen-tools-4.7.0) /usr/ports/sysutils/xen-tools/work/stage//usr/local/lib/python2.7/site-packages/xen/lowlevel/xc.so - required shared library libxenctrl.so.4.5 not found
> (xen-tools-4.7.0) /usr/ports/sysutils/xen-tools/work/stage//usr/local/lib/python2.7/site-packages/xen/lowlevel/xc.so - required shared library libxenguest.so.4.5 not found
> (xen-tools-4.7.0) /usr/ports/sysutils/xen-tools/work/stage//usr/local/lib/xen/bin/qemu-system-i386 - required shared library libxenctrl.so.4.5 not found
> (xen-tools-4.7.0) /usr/ports/sysutils/xen-tools/work/stage//usr/local/lib/xen/bin/qemu-system-i386 - required shared library libxenguest.so.4.5 not found
> Installing xen-tools-4.7.0...
> 
> Adding Python manually and rebuilding seems to fix those issues.

AFAICT, this is caused by compiling the new xen-tools-4.7 port while still 
having the xen-tools-4.5 port installed. Due to the ports build environment 
the first path searched for libraries is /usr/local/lib instead of the one 
passed on the command line by the Xen build system.

> It seems to work, I only wish we've had ports from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192012 committed...

My plate is completely full right now, so I cannot really pick this up. 
However if you manage to get the ports committed I should be able to add 
them to the xen-tools package.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7 and blkback changes - domU respawning on_crash

[Roger Pau Monné]

On Wed, Jun 08, 2016 at 10:35:22PM +0000, Marcin Cieslak wrote:
> On Wed, 8 Jun 2016, Marcin Cieslak wrote:
> 
> > On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> > 
> > > Hello,
> > > 
> > > First of all, this message is only relevant to those that use FreeBSD as 
> > > Dom0 (host), not as a DomU (guest), so don't panic.
> > > 
> > > I've imported the latest Xen version (4.7-rc4) into the ports tree, it's 
> > > still not the final version, but it's quite close, so we better start 
> > > testing it to make sure it works fine with FreeBSD.
> 
> One issue maybe unrelated to FreeBSD:
> 
> This domain:
> 
> builder = "hvm"
> memory = 4096
> vcpus = 2
> name = "Windows2016"
> disk = [
>         '/dev/zvol/zroot/windows0,raw,hda,w',
>         '/dev/zvol/zroot/vs2013,raw,hdb,w',
> #        '/root/win/install.iso,raw,hdc:cdrom,r'
> ]
> boot = "c" # Boot to hard disk image
> vnc = 2
> #vnclisten = "0.0.0.0"
> usbdevice = 'tablet'
> on_poweroff = 'destroy'
> on_reboot = 'restart'
> on_crash = 'restart'
> acpi = 1
> bios = 'ovmf'
> vif = [ 'bridge=bridge0,mac=00:16:3e:5d:0d:48' ]
> videoram=16
> vga = "stdvga"
> 
> crashes because I didn't have ovmf image:
> 
> (d203) HVM Loader
> (d203) Detected Xen v4.7.0-rc
> (d203) Xenbus rings @0xfeffc000, event channel 1
> (d203) Unknown BIOS ovmf, no ROM image found
> (d203) *** HVMLoader bug at hvmloader.c:229
> (d203) *** HVMLoader crashed.
> 
> But I seem unable to kill it with "xl destroy" - it keeps
> respawning again:
> 
> Windows2016                                211  4079     1     --p---       0.0
> Windows2016                                213  4096     1     --psc-       0.0
> (disappears)
> Windows2016                                221  4096     1     --psc-       0.0
> (null)                                     221   147     1     --psc-       0.0
> ...
> ...
> 
> I have finally managed to snatch it by issuing this a few times, after
> changing the "on_crash" to 'destroy':
> 
> # xl config-update Windows2016 xen/windows-run.cfg
> WARNING: xl now has better capability to manage domain configuration, avoid using this command when possible
> setting dom243 configuration

The problem is that the domain crashed so early on boot that you weren't 
able to destroy it, and kept rebooting due to the "on_crash = 'restart'" 
option. IIRC I've also used the following hacky rune in order to terminate 
this kind of domains: "while [ 1 ]; do xl destroy <name>; done", but your 
solution seems better.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Roger Pau Monné]

On Thu, Jun 09, 2016 at 12:16:59AM +0000, Marcin Cieslak wrote:
> On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> 
> > One of the more relevant changes in 4.7 regarding FreeBSD is the support for 
> > block hotplug scripts. This means that we now have the option to use 
> > backends different than simple block or regular files, provided that someone 
> > writes the proper hotplug scripts to attach them (I've heard there are some 
> > iSCSI hotplug scripts around). This however requires changes in blkback, so 
> > if you plan to use the Xen 4.7 port, please make sure that you are running a 
> > kernel that contains revision r301269 (or any later version). The same also 
> 
> I am running it with r301685 and the HVM guests have some trouble with
> block devices.
> 
> SeaBIOS does not find /dev/zvol/zroot/freebsd1,raw,xvda,w to boot FreeBSD
> from, after chaging to "hda" I get up to the kernel mountroot prompt
> (Xen block devices seem to be detected in dmesg).

Yes, this is intentional, see:

https://marc.info/?l=xen-devel&m=144482080812353

Have you checked if you need to change your /etc/fstab to correctly point 
to the new device? Does FreeBSD correctly list the disk(s) at the mountroot 
prompt when issuing a "?" command?

Also, can you paste the output of `xenstore-ls -fp` from Dom0 when this 
happens?
 
> What used to be Windows 2016 domU with /dev/zvol/zroot/windows0,raw,hda,w
> ends up in the Tianocore UEFI shell. Block devices seem to be available,
> I can even list the fs0: partition, but no booting further possible.

Hm, I've never used Tianocore myself, so I'm not sure what the issue could 
be here. Again, can you post the output of `xenstore-ls -fp` from Dom0 when 
this happens? At least this will show if the PV block device has been 
correctly attached.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Wei Liu]

On Thu, Jun 09, 2016 at 10:03:43AM +0200, Roger Pau Monné wrote:
> On Thu, Jun 09, 2016 at 12:16:59AM +0000, Marcin Cieslak wrote:
> > On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> > 
> > > One of the more relevant changes in 4.7 regarding FreeBSD is the support for 
> > > block hotplug scripts. This means that we now have the option to use 
> > > backends different than simple block or regular files, provided that someone 
> > > writes the proper hotplug scripts to attach them (I've heard there are some 
> > > iSCSI hotplug scripts around). This however requires changes in blkback, so 
> > > if you plan to use the Xen 4.7 port, please make sure that you are running a 
> > > kernel that contains revision r301269 (or any later version). The same also 
> > 
> > I am running it with r301685 and the HVM guests have some trouble with
> > block devices.
> > 
> > SeaBIOS does not find /dev/zvol/zroot/freebsd1,raw,xvda,w to boot FreeBSD
> > from, after chaging to "hda" I get up to the kernel mountroot prompt
> > (Xen block devices seem to be detected in dmesg).
> 
> Yes, this is intentional, see:
> 
> https://marc.info/?l=xen-devel&m=144482080812353
> 

Note that I reverted that change yesterday. 4.7 will behave as before.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Marcin Cieslak]

On Thu, 9 Jun 2016, Roger Pau Monné wrote:

> On Thu, Jun 09, 2016 at 12:16:59AM +0000, Marcin Cieslak wrote:
> > On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> > 
> > > One of the more relevant changes in 4.7 regarding FreeBSD is the support for 
> > > block hotplug scripts. This means that we now have the option to use 
> > > backends different than simple block or regular files, provided that someone 
> > > writes the proper hotplug scripts to attach them (I've heard there are some 
> > > iSCSI hotplug scripts around). This however requires changes in blkback, so 
> > > if you plan to use the Xen 4.7 port, please make sure that you are running a 
> > > kernel that contains revision r301269 (or any later version). The same also 
> > 
> > I am running it with r301685 and the HVM guests have some trouble with
> > block devices.
> > 
> > SeaBIOS does not find /dev/zvol/zroot/freebsd1,raw,xvda,w to boot FreeBSD
> > from, after chaging to "hda" I get up to the kernel mountroot prompt
> > (Xen block devices seem to be detected in dmesg).
> 
> Yes, this is intentional, see:
> 
> https://marc.info/?l=xen-devel&m=144482080812353

those guests worked fine with 4.5, that's why I am surprised.

xbd0 and xbd1 show up in dmesg, mouting root from xbd0p2 fails, but 
ada0p2 seems to work.

I remember that during my previous attempts ZFS ate most of
my machine's memory (dom0 was too small I think) and the
symptom was very similar if not identical.

One thing which struck me is that I was able to fully but
one Linux HVM domU. I am also toying with OpenFirmware
which boots from floppy as a HVM guest.

> Have you checked if you need to change your /etc/fstab to correctly point 
> to the new device? Does FreeBSD correctly list the disk(s) at the mountroot 
> prompt when issuing a "?" command?

"?" does not work - it mostly causes a panic, the console is slow, but I managed
to switch it to /dev/ada0p2, dmesg below:

Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #1 r298620: Tue Apr 26 13:21:50 UTC 2016
    root@o.saper.info:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0)
WARNING: WITNESS option enabled, expect reduced performance.
VT(vga): text 80x25
XEN: Hypervisor version 4.7 detected.
CPU: Intel(R) Xeon(R) CPU E31245 @ 3.30GHz (3300.08-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206a7  Family=0x6  Model=0x2a  Stepping=7
  Features=0x17c3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,ACPI,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x9fba2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,HV>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  XSAVE Features=0x1<XSAVEOPT>
Hypervisor: Origin = "XenVMMXenVMM"
real memory  = 2130706432 (2032 MB)
avail memory = 2018213888 (1924 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <Xen HVM>
WARNING: L1 data cache covers less APIC IDs than a core
0 < 1
WARNING: L2 data cache covers less APIC IDs than a core
0 < 1
WARNING: L3 data cache covers less APIC IDs than a core
0 < 1
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
random: unblocking device.
ioapic0: Changing APIC ID to 1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-47 on motherboard
random: entropy device external interface
kbd1 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff80f0ffb0, 0) error 19
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <Xen> on motherboard
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 62500000 Hz quality 950
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0xb008-0xb00b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc200-0xc20f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
pci0: <bridge> at device 1.3 (no driver attached)
xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xf0000000-0xf0ffffff irq 24 at device 2.0 on pci0
vgapci0: <VGA-compatible display> mem 0xf1000000-0xf1ffffff,0xf2050000-0xf2050fff at device 3.0 on pci0
vgapci0: Boot video device
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
xenpv0: <Xen PV bus> on motherboard
granttable0: <Xen Grant-table Device> on xenpv0
xen_et0: <Xen PV Clock> on xenpv0
Event timer "XENTIMER" frequency 1000000000 Hz quality 950
Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
xenstore0: <XenStore> on xenpv0
evtchn0: <Xen event channel user-space device> on xenpv0
privcmd0: <Xen privileged interface user-space device> on xenpv0
debug0: <Xen debug handler> on xenpv0
orm0: <ISA Option ROM> at iomem 0xed800-0xeffff on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
fdc0: No FDOUT register!
ppc0: cannot reserve I/O port range
Timecounters tick every 1.000 msec
xenballoon0: <Xen Balloon Device> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
xs_dev0: <Xenstore user-space device> on xenstore0
xenbusb_front0: <Xen Frontend Devices> on xenstore0
xenbusb_add_device: Device device/suspend/event-channel ignored. State 6
xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
xbd0: 16384MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
xbd0: attaching as ada0
xbd0: features: flush, write_barrier
xbd0: synchronize cache commands enabled.
xn0: Ethernet address: 00:02:04:08:fd:f0
xenbusb_back0: <Xen Backend Devices> on xenstore0
xbd1: 40960MB <Virtual Block Device> at device/vbd/832 on xenbusb_front0
xbd1: attaching as ada1
xbd1: features: flush, write_barrier
xbd1: synchronize cache commands enabled.
xn0: backend features: feature-sg
SMP: AP CPU #1 Launched!
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/xbd0p2 [rw]...
mountroot: waiting for device /dev/xbd0p2...
Mounting from ufs:/dev/xbd0p2 failed with error 19.

Loader variables:
  vfs.root.mountfrom=ufs:/dev/xbd0p2
  vfs.root.mountfrom.options=rw

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:tank
        cd9660:/dev/cd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot> Trying to mount root from ufs:/dev/ada0p2 []...
xn0: link state changed to DOWN
xn0: link state changed to UP

> Also, can you paste the output of `xenstore-ls -fp` from Dom0 when this 
> happens?

Windows:

/tool = ""   (n0)
/tool/xenstored = ""   (n0)
/vm = ""   (n0)
/vm/744e8146-2e7b-11e6-80cd-5404a6685907 = ""   (n0,r279)
/vm/744e8146-2e7b-11e6-80cd-5404a6685907/name = "Windows2016"   (n0,r279)
/vm/744e8146-2e7b-11e6-80cd-5404a6685907/uuid = "744e8146-2e7b-11e6-80cd-5404a6685907"   (n0,r279)
/vm/744e8146-2e7b-11e6-80cd-5404a6685907/rtc = ""   (n0,r279)
/vm/744e8146-2e7b-11e6-80cd-5404a6685907/rtc/timeoffset = ""   (n0,r279)
/vm/744e8146-2e7b-11e6-80cd-5404a6685907/image = ""   (n0,r279)
/vm/744e8146-2e7b-11e6-80cd-5404a6685907/image/ostype = "hvm"   (n0,r279)
/vm/744e8146-2e7b-11e6-80cd-5404a6685907/start_time = "1465501844.89"   (n0,r279)
/libxl = ""   (n0)
/libxl/279 = ""   (n0)
/libxl/279/dm-version = "qemu_xen"   (n0)
/local = ""   (n0)
/local/domain = ""   (n0)
/local/domain/0 = ""   (n0)
/local/domain/0/domid = "0"   (n0)
/local/domain/0/name = "Domain-0"   (n0)
/local/domain/0/device-model = ""   (n0)
/local/domain/0/device-model/279 = ""   (n0)
/local/domain/0/device-model/279/state = "running"   (n0)
/local/domain/0/device-model/279/physmap = ""   (n0)
/local/domain/0/device-model/279/physmap/10f000000 = ""   (n0)
/local/domain/0/device-model/279/physmap/10f000000/start_addr = "f1000000"   (n0)
/local/domain/0/device-model/279/physmap/10f000000/size = "1000000"   (n0)
/local/domain/0/device-model/279/physmap/10f000000/name = "vga.vram"   (n0)
/local/domain/0/backend = ""   (n0)
/local/domain/0/backend/vbd = ""   (n0)
/local/domain/0/backend/vbd/279 = ""   (n0)
/local/domain/0/backend/vbd/279/768 = ""   (n0,r279)
/local/domain/0/backend/vbd/279/768/frontend = "/local/domain/279/device/vbd/768"   (n0,r279)
/local/domain/0/backend/vbd/279/768/params = "/dev/zvol/zroot/windows0"   (n0,r279)
/local/domain/0/backend/vbd/279/768/script = "/usr/local/etc/xen/scripts/block"   (n0,r279)
/local/domain/0/backend/vbd/279/768/frontend-id = "279"   (n0,r279)
/local/domain/0/backend/vbd/279/768/online = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/768/removable = "0"   (n0,r279)
/local/domain/0/backend/vbd/279/768/bootable = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/768/state = "3"   (n0,r279)
/local/domain/0/backend/vbd/279/768/dev = "hda"   (n0,r279)
/local/domain/0/backend/vbd/279/768/type = "phy"   (n0,r279)
/local/domain/0/backend/vbd/279/768/mode = "w"   (n0,r279)
/local/domain/0/backend/vbd/279/768/device-type = "disk"   (n0,r279)
/local/domain/0/backend/vbd/279/768/discard-enable = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/768/feature-barrier = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/768/feature-flush-cache = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/768/max-ring-page-order = "5"   (n0,r279)
/local/domain/0/backend/vbd/279/768/physical-device-path = "/dev/zvol/zroot/windows0"   (n0,r279)
/local/domain/0/backend/vbd/279/768/hotplug-status = "connected"   (n0,r279)
/local/domain/0/backend/vbd/279/832 = ""   (n0,r279)
/local/domain/0/backend/vbd/279/832/frontend = "/local/domain/279/device/vbd/832"   (n0,r279)
/local/domain/0/backend/vbd/279/832/params = "/dev/zvol/zroot/vs2013"   (n0,r279)
/local/domain/0/backend/vbd/279/832/script = "/usr/local/etc/xen/scripts/block"   (n0,r279)
/local/domain/0/backend/vbd/279/832/frontend-id = "279"   (n0,r279)
/local/domain/0/backend/vbd/279/832/online = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/832/removable = "0"   (n0,r279)
/local/domain/0/backend/vbd/279/832/bootable = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/832/state = "3"   (n0,r279)
/local/domain/0/backend/vbd/279/832/dev = "hdb"   (n0,r279)
/local/domain/0/backend/vbd/279/832/type = "phy"   (n0,r279)
/local/domain/0/backend/vbd/279/832/mode = "w"   (n0,r279)
/local/domain/0/backend/vbd/279/832/device-type = "disk"   (n0,r279)
/local/domain/0/backend/vbd/279/832/discard-enable = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/832/feature-barrier = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/832/feature-flush-cache = "1"   (n0,r279)
/local/domain/0/backend/vbd/279/832/max-ring-page-order = "5"   (n0,r279)
/local/domain/0/backend/vbd/279/832/physical-device-path = "/dev/zvol/zroot/vs2013"   (n0,r279)
/local/domain/0/backend/vbd/279/832/hotplug-status = "connected"   (n0,r279)
/local/domain/0/backend/console = ""   (n0)
/local/domain/0/backend/console/279 = ""   (n0)
/local/domain/0/backend/console/279/0 = ""   (n0,r279)
/local/domain/0/backend/console/279/0/frontend = "/local/domain/279/console"   (n0,r279)
/local/domain/0/backend/console/279/0/frontend-id = "279"   (n0,r279)
/local/domain/0/backend/console/279/0/online = "1"   (n0,r279)
/local/domain/0/backend/console/279/0/state = "1"   (n0,r279)
/local/domain/0/backend/console/279/0/protocol = "vt100"   (n0,r279)
/local/domain/0/backend/vkbd = ""   (n0)
/local/domain/0/backend/vkbd/279 = ""   (n0)
/local/domain/0/backend/vkbd/279/0 = ""   (n0,r279)
/local/domain/0/backend/vkbd/279/0/frontend = "/local/domain/279/device/vkbd/0"   (n0,r279)
/local/domain/0/backend/vkbd/279/0/frontend-id = "279"   (n0,r279)
/local/domain/0/backend/vkbd/279/0/online = "1"   (n0,r279)
/local/domain/0/backend/vkbd/279/0/state = "2"   (n0,r279)
/local/domain/0/backend/vkbd/279/0/feature-abs-pointer = "1"   (n0,r279)
/local/domain/0/backend/vkbd/279/0/hotplug-status = "connected"   (n0,r279)
/local/domain/0/backend/vif = ""   (n0)
/local/domain/0/backend/vif/279 = ""   (n0)
/local/domain/0/backend/vif/279/0 = ""   (n0,r279)
/local/domain/0/backend/vif/279/0/frontend = "/local/domain/279/device/vif/0"   (n0,r279)
/local/domain/0/backend/vif/279/0/frontend-id = "279"   (n0,r279)
/local/domain/0/backend/vif/279/0/online = "1"   (n0,r279)
/local/domain/0/backend/vif/279/0/state = "2"   (n0,r279)
/local/domain/0/backend/vif/279/0/script = "/usr/local/etc/xen/scripts/vif-bridge"   (n0,r279)
/local/domain/0/backend/vif/279/0/mac = "00:16:3e:5d:0d:48"   (n0,r279)
/local/domain/0/backend/vif/279/0/bridge = "bridge0"   (n0,r279)
/local/domain/0/backend/vif/279/0/handle = "0"   (n0,r279)
/local/domain/0/backend/vif/279/0/type = "vif_ioemu"   (n0,r279)
/local/domain/0/backend/vif/279/0/hotplug-status = "connected"   (n0,r279)
/local/domain/0/backend/vif/279/0/feature-sg = "1"   (n0,r279)
/local/domain/0/backend/vif/279/0/feature-gso-tcpv4 = "0"   (n0,r279)
/local/domain/0/backend/vif/279/0/feature-rx-copy = "1"   (n0,r279)
/local/domain/0/backend/vif/279/0/feature-rx-flip = "0"   (n0,r279)
/local/domain/279 = ""   (n0,r279)
/local/domain/279/vm = "/vm/744e8146-2e7b-11e6-80cd-5404a6685907"   (n0,r279)
/local/domain/279/name = "Windows2016"   (n0,r279)
/local/domain/279/cpu = ""   (n0,r279)
/local/domain/279/cpu/0 = ""   (n0,r279)
/local/domain/279/cpu/0/availability = "online"   (n0,r279)
/local/domain/279/cpu/1 = ""   (n0,r279)
/local/domain/279/cpu/1/availability = "online"   (n0,r279)
/local/domain/279/memory = ""   (n0,r279)
/local/domain/279/memory/static-max = "4194304"   (n0,r279)
/local/domain/279/memory/target = "4177920"   (n0,r279)
/local/domain/279/memory/videoram = "16384"   (n0,r279)
/local/domain/279/device = ""   (n0,r279)
/local/domain/279/device/suspend = ""   (n0,r279)
/local/domain/279/device/suspend/event-channel = ""   (n279)
/local/domain/279/device/vbd = ""   (n0,r279)
/local/domain/279/device/vbd/768 = ""   (n279,r0)
/local/domain/279/device/vbd/768/backend = "/local/domain/0/backend/vbd/279/768"   (n279,r0)
/local/domain/279/device/vbd/768/backend-id = "0"   (n279,r0)
/local/domain/279/device/vbd/768/state = "1"   (n279,r0)
/local/domain/279/device/vbd/768/virtual-device = "768"   (n279,r0)
/local/domain/279/device/vbd/768/device-type = "disk"   (n279,r0)
/local/domain/279/device/vbd/832 = ""   (n279,r0)
/local/domain/279/device/vbd/832/backend = "/local/domain/0/backend/vbd/279/832"   (n279,r0)
/local/domain/279/device/vbd/832/backend-id = "0"   (n279,r0)
/local/domain/279/device/vbd/832/state = "1"   (n279,r0)
/local/domain/279/device/vbd/832/virtual-device = "832"   (n279,r0)
/local/domain/279/device/vbd/832/device-type = "disk"   (n279,r0)
/local/domain/279/device/vkbd = ""   (n0,r279)
/local/domain/279/device/vkbd/0 = ""   (n279,r0)
/local/domain/279/device/vkbd/0/backend = "/local/domain/0/backend/vkbd/279/0"   (n279,r0)
/local/domain/279/device/vkbd/0/backend-id = "0"   (n279,r0)
/local/domain/279/device/vkbd/0/state = "1"   (n279,r0)
/local/domain/279/device/vif = ""   (n0,r279)
/local/domain/279/device/vif/0 = ""   (n279,r0)
/local/domain/279/device/vif/0/backend = "/local/domain/0/backend/vif/279/0"   (n279,r0)
/local/domain/279/device/vif/0/backend-id = "0"   (n279,r0)
/local/domain/279/device/vif/0/state = "1"   (n279,r0)
/local/domain/279/device/vif/0/handle = "0"   (n279,r0)
/local/domain/279/device/vif/0/mac = "00:16:3e:5d:0d:48"   (n279,r0)
/local/domain/279/control = ""   (n0,r279)
/local/domain/279/control/shutdown = ""   (n279)
/local/domain/279/control/platform-feature-multiprocessor-suspend = "1"   (n0,r279)
/local/domain/279/control/platform-feature-xs_reset_watches = "1"   (n0,r279)
/local/domain/279/hvmloader = ""   (n0,r279)
/local/domain/279/hvmloader/bios = "ovmf"   (n0,r279)
/local/domain/279/hvmloader/allow-memory-relocate = "0"   (n0,r279)
/local/domain/279/data = ""   (n279)
/local/domain/279/drivers = ""   (n279)
/local/domain/279/feature = ""   (n279)
/local/domain/279/attr = ""   (n279)
/local/domain/279/domid = "279"   (n0,r279)
/local/domain/279/store = ""   (n0,r279)
/local/domain/279/store/port = "1"   (n0,r279)
/local/domain/279/store/ring-ref = "1044476"   (n0,r279)
/local/domain/279/platform = ""   (n0,r279)
/local/domain/279/platform/acpi = "1"   (n0,r279)
/local/domain/279/platform/acpi_s3 = "1"   (n0,r279)
/local/domain/279/platform/acpi_s4 = "1"   (n0,r279)
/local/domain/279/console = ""   (n0,r279)
/local/domain/279/console/backend = "/local/domain/0/backend/console/279/0"   (n0,r279)
/local/domain/279/console/backend-id = "0"   (n279,r0)
/local/domain/279/console/limit = "1048576"   (n0,r279)
/local/domain/279/console/type = "xenconsoled"   (n0,r279)
/local/domain/279/console/output = "pty"   (n0,r279)
/local/domain/279/console/tty = "/dev/pts/1"   (n0,r279)
/local/domain/279/console/port = "2"   (n0,r279)
/local/domain/279/console/ring-ref = "1044479"   (n0,r279)
/local/domain/279/console/vnc-listen = "127.0.0.1"   (n0,r279)
/local/domain/279/console/vnc-port = "5900"   (n0,r279)
/local/domain/279/image = ""   (n0,r279)
/local/domain/279/image/device-model-pid = "78745"   (n0,r279)

FreeBSD:

/tool = ""   (n0)
/tool/xenstored = ""   (n0)
/vm = ""   (n0)
/vm/faefcbb0-2e7b-11e6-80cd-5404a6685907 = ""   (n0,r281)
/vm/faefcbb0-2e7b-11e6-80cd-5404a6685907/name = "FreeBSD"   (n0,r281)
/vm/faefcbb0-2e7b-11e6-80cd-5404a6685907/uuid = "faefcbb0-2e7b-11e6-80cd-5404a6685907"   (n0,r281)
/vm/faefcbb0-2e7b-11e6-80cd-5404a6685907/rtc = ""   (n0,r281)
/vm/faefcbb0-2e7b-11e6-80cd-5404a6685907/rtc/timeoffset = ""   (n0,r281)
/vm/faefcbb0-2e7b-11e6-80cd-5404a6685907/image = ""   (n0,r281)
/vm/faefcbb0-2e7b-11e6-80cd-5404a6685907/image/ostype = "hvm"   (n0,r281)
/vm/faefcbb0-2e7b-11e6-80cd-5404a6685907/start_time = "1465502070.75"   (n0,r281)
/vm/7bbb396a-2f43-11e6-8abe-5404a6685907 = ""   (n0,r11)
/vm/7bbb396a-2f43-11e6-8abe-5404a6685907/name = "FreeBSD"   (n0,r11)
/vm/7bbb396a-2f43-11e6-8abe-5404a6685907/uuid = "7bbb396a-2f43-11e6-8abe-5404a6685907"   (n0,r11)
/vm/7bbb396a-2f43-11e6-8abe-5404a6685907/rtc = ""   (n0,r11)
/vm/7bbb396a-2f43-11e6-8abe-5404a6685907/rtc/timeoffset = ""   (n0,r11)
/vm/7bbb396a-2f43-11e6-8abe-5404a6685907/image = ""   (n0,r11)
/vm/7bbb396a-2f43-11e6-8abe-5404a6685907/image/ostype = "hvm"   (n0,r11)
/vm/7bbb396a-2f43-11e6-8abe-5404a6685907/start_time = "1465587756.69"   (n0,r11)
/libxl = ""   (n0)
/libxl/281 = ""   (n0)
/libxl/281/dm-version = "qemu_xen"   (n0)
/libxl/11 = ""   (n0)
/libxl/11/dm-version = "qemu_xen"   (n0)
/local = ""   (n0)
/local/domain = ""   (n0)
/local/domain/0 = ""   (n0)
/local/domain/0/domid = "0"   (n0)
/local/domain/0/name = "Domain-0"   (n0)
/local/domain/0/device-model = ""   (n0)
/local/domain/0/device-model/11 = ""   (n0)
/local/domain/0/device-model/11/state = "running"   (n0)
/local/domain/0/device-model/11/physmap = ""   (n0)
/local/domain/0/device-model/11/physmap/7f000000 = ""   (n0)
/local/domain/0/device-model/11/physmap/7f000000/start_addr = "f1000000"   (n0)
/local/domain/0/device-model/11/physmap/7f000000/size = "1000000"   (n0)
/local/domain/0/device-model/11/physmap/7f000000/name = "vga.vram"   (n0)
/local/domain/0/backend = ""   (n0)
/local/domain/0/backend/vbd = ""   (n0)
/local/domain/0/backend/vbd/11 = ""   (n0)
/local/domain/0/backend/vbd/11/768 = ""   (n0,r11)
/local/domain/0/backend/vbd/11/768/frontend = "/local/domain/11/device/vbd/768"   (n0,r11)
/local/domain/0/backend/vbd/11/768/params = "/dev/zvol/zroot/freebsd1"   (n0,r11)
/local/domain/0/backend/vbd/11/768/script = "/usr/local/etc/xen/scripts/block"   (n0,r11)
/local/domain/0/backend/vbd/11/768/frontend-id = "11"   (n0,r11)
/local/domain/0/backend/vbd/11/768/online = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/768/removable = "0"   (n0,r11)
/local/domain/0/backend/vbd/11/768/bootable = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/768/state = "4"   (n0,r11)
/local/domain/0/backend/vbd/11/768/dev = "hda"   (n0,r11)
/local/domain/0/backend/vbd/11/768/type = "phy"   (n0,r11)
/local/domain/0/backend/vbd/11/768/mode = "w"   (n0,r11)
/local/domain/0/backend/vbd/11/768/device-type = "disk"   (n0,r11)
/local/domain/0/backend/vbd/11/768/discard-enable = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/768/feature-barrier = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/768/feature-flush-cache = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/768/max-ring-page-order = "5"   (n0,r11)
/local/domain/0/backend/vbd/11/768/physical-device-path = "/dev/zvol/zroot/freebsd1"   (n0,r11)
/local/domain/0/backend/vbd/11/768/hotplug-status = "connected"   (n0,r11)
/local/domain/0/backend/vbd/11/768/sectors = "33554432"   (n0,r11)
/local/domain/0/backend/vbd/11/768/info = "0"   (n0,r11)
/local/domain/0/backend/vbd/11/768/sector-size = "512"   (n0,r11)
/local/domain/0/backend/vbd/11/832 = ""   (n0,r11)
/local/domain/0/backend/vbd/11/832/frontend = "/local/domain/11/device/vbd/832"   (n0,r11)
/local/domain/0/backend/vbd/11/832/params = "/dev/zvol/zroot/freebsd2"   (n0,r11)
/local/domain/0/backend/vbd/11/832/script = "/usr/local/etc/xen/scripts/block"   (n0,r11)
/local/domain/0/backend/vbd/11/832/frontend-id = "11"   (n0,r11)
/local/domain/0/backend/vbd/11/832/online = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/832/removable = "0"   (n0,r11)
/local/domain/0/backend/vbd/11/832/bootable = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/832/state = "4"   (n0,r11)
/local/domain/0/backend/vbd/11/832/dev = "hdb"   (n0,r11)
/local/domain/0/backend/vbd/11/832/type = "phy"   (n0,r11)
/local/domain/0/backend/vbd/11/832/mode = "w"   (n0,r11)
/local/domain/0/backend/vbd/11/832/device-type = "disk"   (n0,r11)
/local/domain/0/backend/vbd/11/832/discard-enable = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/832/feature-barrier = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/832/feature-flush-cache = "1"   (n0,r11)
/local/domain/0/backend/vbd/11/832/max-ring-page-order = "5"   (n0,r11)
/local/domain/0/backend/vbd/11/832/physical-device-path = "/dev/zvol/zroot/freebsd2"   (n0,r11)
/local/domain/0/backend/vbd/11/832/hotplug-status = "connected"   (n0,r11)
/local/domain/0/backend/vbd/11/832/sectors = "83886080"   (n0,r11)
/local/domain/0/backend/vbd/11/832/info = "0"   (n0,r11)
/local/domain/0/backend/vbd/11/832/sector-size = "512"   (n0,r11)
/local/domain/0/backend/console = ""   (n0)
/local/domain/0/backend/console/11 = ""   (n0)
/local/domain/0/backend/console/11/0 = ""   (n0,r11)
/local/domain/0/backend/console/11/0/frontend = "/local/domain/11/console"   (n0,r11)
/local/domain/0/backend/console/11/0/frontend-id = "11"   (n0,r11)
/local/domain/0/backend/console/11/0/online = "1"   (n0,r11)
/local/domain/0/backend/console/11/0/state = "1"   (n0,r11)
/local/domain/0/backend/console/11/0/protocol = "vt100"   (n0,r11)
/local/domain/0/backend/vkbd = ""   (n0)
/local/domain/0/backend/vkbd/11 = ""   (n0)
/local/domain/0/backend/vkbd/11/0 = ""   (n0,r11)
/local/domain/0/backend/vkbd/11/0/frontend = "/local/domain/11/device/vkbd/0"   (n0,r11)
/local/domain/0/backend/vkbd/11/0/frontend-id = "11"   (n0,r11)
/local/domain/0/backend/vkbd/11/0/online = "1"   (n0,r11)
/local/domain/0/backend/vkbd/11/0/state = "2"   (n0,r11)
/local/domain/0/backend/vkbd/11/0/feature-abs-pointer = "1"   (n0,r11)
/local/domain/0/backend/vkbd/11/0/hotplug-status = "connected"   (n0,r11)
/local/domain/0/backend/vif = ""   (n0)
/local/domain/0/backend/vif/11 = ""   (n0)
/local/domain/0/backend/vif/11/0 = ""   (n0,r11)
/local/domain/0/backend/vif/11/0/frontend = "/local/domain/11/device/vif/0"   (n0,r11)
/local/domain/0/backend/vif/11/0/frontend-id = "11"   (n0,r11)
/local/domain/0/backend/vif/11/0/online = "1"   (n0,r11)
/local/domain/0/backend/vif/11/0/state = "4"   (n0,r11)
/local/domain/0/backend/vif/11/0/script = "/usr/local/etc/xen/scripts/vif-bridge"   (n0,r11)
/local/domain/0/backend/vif/11/0/mac = "00:02:04:08:fd:f0"   (n0,r11)
/local/domain/0/backend/vif/11/0/bridge = "bridge0"   (n0,r11)
/local/domain/0/backend/vif/11/0/handle = "0"   (n0,r11)
/local/domain/0/backend/vif/11/0/type = "vif_ioemu"   (n0,r11)
/local/domain/0/backend/vif/11/0/hotplug-status = "connected"   (n0,r11)
/local/domain/0/backend/vif/11/0/feature-sg = "1"   (n0,r11)
/local/domain/0/backend/vif/11/0/feature-gso-tcpv4 = "0"   (n0,r11)
/local/domain/0/backend/vif/11/0/feature-rx-copy = "1"   (n0,r11)
/local/domain/0/backend/vif/11/0/feature-rx-flip = "0"   (n0,r11)
/local/domain/11 = ""   (n0,r11)
/local/domain/11/vm = "/vm/7bbb396a-2f43-11e6-8abe-5404a6685907"   (n0,r11)
/local/domain/11/name = "FreeBSD"   (n0,r11)
/local/domain/11/cpu = ""   (n0,r11)
/local/domain/11/cpu/0 = ""   (n0,r11)
/local/domain/11/cpu/0/availability = "online"   (n0,r11)
/local/domain/11/cpu/1 = ""   (n0,r11)
/local/domain/11/cpu/1/availability = "online"   (n0,r11)
/local/domain/11/memory = ""   (n0,r11)
/local/domain/11/memory/static-max = "2097152"   (n0,r11)
/local/domain/11/memory/target = "2080768"   (n0,r11)
/local/domain/11/memory/videoram = "16384"   (n0,r11)
/local/domain/11/device = ""   (n0,r11)
/local/domain/11/device/suspend = ""   (n0,r11)
/local/domain/11/device/suspend/event-channel = ""   (n11)
/local/domain/11/device/vbd = ""   (n0,r11)
/local/domain/11/device/vbd/768 = ""   (n11,r0)
/local/domain/11/device/vbd/768/backend = "/local/domain/0/backend/vbd/11/768"   (n11,r0)
/local/domain/11/device/vbd/768/backend-id = "0"   (n11,r0)
/local/domain/11/device/vbd/768/state = "4"   (n11,r0)
/local/domain/11/device/vbd/768/virtual-device = "768"   (n11,r0)
/local/domain/11/device/vbd/768/device-type = "disk"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref0 = "8"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref1 = "9"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref2 = "10"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref3 = "11"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref4 = "12"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref5 = "13"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref6 = "14"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref7 = "15"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref8 = "16"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref9 = "17"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref10 = "18"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref11 = "19"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref12 = "20"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref13 = "21"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref14 = "22"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref15 = "23"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref16 = "24"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref17 = "25"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref18 = "26"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref19 = "27"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref20 = "28"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref21 = "29"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref22 = "30"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref23 = "31"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref24 = "32"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref25 = "33"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref26 = "34"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref27 = "35"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref28 = "36"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref29 = "37"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref30 = "38"   (n11,r0)
/local/domain/11/device/vbd/768/ring-ref31 = "39"   (n11,r0)
/local/domain/11/device/vbd/768/num-ring-pages = "32"   (n11,r0)
/local/domain/11/device/vbd/768/ring-page-order = "5"   (n11,r0)
/local/domain/11/device/vbd/768/event-channel = "10"   (n11,r0)
/local/domain/11/device/vbd/768/protocol = "x86_64-abi"   (n11,r0)
/local/domain/11/device/vbd/832 = ""   (n11,r0)
/local/domain/11/device/vbd/832/backend = "/local/domain/0/backend/vbd/11/832"   (n11,r0)
/local/domain/11/device/vbd/832/backend-id = "0"   (n11,r0)
/local/domain/11/device/vbd/832/state = "4"   (n11,r0)
/local/domain/11/device/vbd/832/virtual-device = "832"   (n11,r0)
/local/domain/11/device/vbd/832/device-type = "disk"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref0 = "40"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref1 = "41"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref2 = "42"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref3 = "43"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref4 = "44"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref5 = "45"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref6 = "46"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref7 = "47"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref8 = "48"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref9 = "49"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref10 = "50"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref11 = "51"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref12 = "52"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref13 = "53"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref14 = "54"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref15 = "55"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref16 = "56"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref17 = "57"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref18 = "58"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref19 = "59"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref20 = "60"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref21 = "61"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref22 = "62"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref23 = "63"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref24 = "64"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref25 = "65"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref26 = "66"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref27 = "67"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref28 = "68"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref29 = "69"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref30 = "70"   (n11,r0)
/local/domain/11/device/vbd/832/ring-ref31 = "71"   (n11,r0)
/local/domain/11/device/vbd/832/num-ring-pages = "32"   (n11,r0)
/local/domain/11/device/vbd/832/ring-page-order = "5"   (n11,r0)
/local/domain/11/device/vbd/832/event-channel = "11"   (n11,r0)
/local/domain/11/device/vbd/832/protocol = "x86_64-abi"   (n11,r0)
/local/domain/11/device/vkbd = ""   (n0,r11)
/local/domain/11/device/vkbd/0 = ""   (n11,r0)
/local/domain/11/device/vkbd/0/backend = "/local/domain/0/backend/vkbd/11/0"   (n11,r0)
/local/domain/11/device/vkbd/0/backend-id = "0"   (n11,r0)
/local/domain/11/device/vkbd/0/state = "1"   (n11,r0)
/local/domain/11/device/vif = ""   (n0,r11)
/local/domain/11/device/vif/0 = ""   (n11,r0)
/local/domain/11/device/vif/0/backend = "/local/domain/0/backend/vif/11/0"   (n11,r0)
/local/domain/11/device/vif/0/backend-id = "0"   (n11,r0)
/local/domain/11/device/vif/0/state = "4"   (n11,r0)
/local/domain/11/device/vif/0/handle = "0"   (n11,r0)
/local/domain/11/device/vif/0/mac = "00:02:04:08:fd:f0"   (n11,r0)
/local/domain/11/device/vif/0/tx-ring-ref = "768"   (n11,r0)
/local/domain/11/device/vif/0/rx-ring-ref = "329"   (n11,r0)
/local/domain/11/device/vif/0/event-channel = "12"   (n11,r0)
/local/domain/11/device/vif/0/request-rx-copy = "1"   (n11,r0)
/local/domain/11/device/vif/0/feature-rx-notify = "1"   (n11,r0)
/local/domain/11/device/vif/0/feature-sg = "1"   (n11,r0)
/local/domain/11/device/vif/0/feature-gso-tcpv4 = "1"   (n11,r0)
/local/domain/11/control = ""   (n0,r11)
/local/domain/11/control/shutdown = ""   (n11)
/local/domain/11/control/platform-feature-multiprocessor-suspend = "1"   (n0,r11)
/local/domain/11/control/platform-feature-xs_reset_watches = "1"   (n0,r11)
/local/domain/11/hvmloader = ""   (n0,r11)
/local/domain/11/hvmloader/bios = "seabios"   (n0,r11)
/local/domain/11/hvmloader/allow-memory-relocate = "0"   (n0,r11)
/local/domain/11/data = ""   (n11)
/local/domain/11/drivers = ""   (n11)
/local/domain/11/feature = ""   (n11)
/local/domain/11/attr = ""   (n11)
/local/domain/11/domid = "11"   (n0,r11)
/local/domain/11/store = ""   (n0,r11)
/local/domain/11/store/port = "1"   (n0,r11)
/local/domain/11/store/ring-ref = "1044476"   (n0,r11)
/local/domain/11/platform = ""   (n0,r11)
/local/domain/11/platform/acpi = "1"   (n0,r11)
/local/domain/11/platform/acpi_s3 = "1"   (n0,r11)
/local/domain/11/platform/acpi_s4 = "1"   (n0,r11)
/local/domain/11/console = ""   (n0,r11)
/local/domain/11/console/backend = "/local/domain/0/backend/console/11/0"   (n0,r11)
/local/domain/11/console/backend-id = "0"   (n11,r0)
/local/domain/11/console/limit = "1048576"   (n0,r11)
/local/domain/11/console/type = "xenconsoled"   (n0,r11)
/local/domain/11/console/output = "pty"   (n0,r11)
/local/domain/11/console/tty = "/dev/pts/1"   (n0,r11)
/local/domain/11/console/port = "2"   (n0,r11)
/local/domain/11/console/ring-ref = "1044479"   (n0,r11)
/local/domain/11/console/vnc-listen = "127.0.0.1"   (n0,r11)
/local/domain/11/console/vnc-port = "5900"   (n0,r11)
/local/domain/11/image = ""   (n0,r11)
/local/domain/11/image/device-model-pid = "6311"   (n0,r11)

> > What used to be Windows 2016 domU with /dev/zvol/zroot/windows0,raw,hda,w
> > ends up in the Tianocore UEFI shell. Block devices seem to be available,
> > I can even list the fs0: partition, but no booting further possible.
> 
> Hm, I've never used Tianocore myself, so I'm not sure what the issue could 
> be here. Again, can you post the output of `xenstore-ls -fp` from Dom0 when 
> this happens? At least this will show if the PV block device has been 
> correctly attached.

See above.

How do I rollback to 4.5.2, do I need to downgrade the FreeBSD kernel
as well?

Marcin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7 and blkback changes

[Marcin Cieslak]

On Thu, 9 Jun 2016, Roger Pau Monné wrote:

> On Wed, Jun 08, 2016 at 10:18:09PM +0000, Marcin Cieslak wrote:
> > On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> > 
> > > Hello,
> > > 
> > > First of all, this message is only relevant to those that use FreeBSD as 
> > > Dom0 (host), not as a DomU (guest), so don't panic.
> > > 
> > > I've imported the latest Xen version (4.7-rc4) into the ports tree, it's 
> > > still not the final version, but it's quite close, so we better start 
> > > testing it to make sure it works fine with FreeBSD.
> > 
> > Thank you Roger, this is excellent. Are xen-tools-devel 4.5 now?
> > Looks confusing.
> > 
> > I have also tried building xen-tools (4.7) without python
> > and qemu configure reported this.
> 
> Python is required, you cannot get rid of it.

Strange, seems USES= python in the port's Makefile didn't work for me and
continued happily after.

Marcin


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Roger Pau Monné]

On Fri, Jun 10, 2016 at 09:38:59PM +0000, Marcin Cieslak wrote:
> On Thu, 9 Jun 2016, Roger Pau Monné wrote:
> 
> > On Thu, Jun 09, 2016 at 12:16:59AM +0000, Marcin Cieslak wrote:
> > > On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> > > 
> > > > One of the more relevant changes in 4.7 regarding FreeBSD is the support for 
> > > > block hotplug scripts. This means that we now have the option to use 
> > > > backends different than simple block or regular files, provided that someone 
> > > > writes the proper hotplug scripts to attach them (I've heard there are some 
> > > > iSCSI hotplug scripts around). This however requires changes in blkback, so 
> > > > if you plan to use the Xen 4.7 port, please make sure that you are running a 
> > > > kernel that contains revision r301269 (or any later version). The same also 
> > > 
> > > I am running it with r301685 and the HVM guests have some trouble with
> > > block devices.
> > > 
> > > SeaBIOS does not find /dev/zvol/zroot/freebsd1,raw,xvda,w to boot FreeBSD
> > > from, after chaging to "hda" I get up to the kernel mountroot prompt
> > > (Xen block devices seem to be detected in dmesg).
> > 
> > Yes, this is intentional, see:
> > 
> > https://marc.info/?l=xen-devel&m=144482080812353
> 
> those guests worked fine with 4.5, that's why I am surprised.
> 
> xbd0 and xbd1 show up in dmesg, mouting root from xbd0p2 fails, but 
> ada0p2 seems to work.
> 
> I remember that during my previous attempts ZFS ate most of
> my machine's memory (dom0 was too small I think) and the
> symptom was very similar if not identical.
> 
> One thing which struck me is that I was able to fully but
> one Linux HVM domU. I am also toying with OpenFirmware
> which boots from floppy as a HVM guest.
> 
> > Have you checked if you need to change your /etc/fstab to correctly point 
> > to the new device? Does FreeBSD correctly list the disk(s) at the mountroot 
> > prompt when issuing a "?" command?
> 
> "?" does not work - it mostly causes a panic, the console is slow, but I managed
> to switch it to /dev/ada0p2, dmesg below:

This has now been reverted, so when I import the new RC this should be fixed 
and you won't need to change anything.
 
> Copyright (c) 1992-2016 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> 	The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 11.0-CURRENT #1 r298620: Tue Apr 26 13:21:50 UTC 2016
>     root@o.saper.info:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0)
> WARNING: WITNESS option enabled, expect reduced performance.
> VT(vga): text 80x25
> XEN: Hypervisor version 4.7 detected.
> CPU: Intel(R) Xeon(R) CPU E31245 @ 3.30GHz (3300.08-MHz K8-class CPU)
>   Origin="GenuineIntel"  Id=0x206a7  Family=0x6  Model=0x2a  Stepping=7
>   Features=0x17c3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,ACPI,MMX,FXSR,SSE,SSE2,HTT>
>   Features2=0x9fba2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,HV>
>   AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
>   AMD Features2=0x1<LAHF>
>   XSAVE Features=0x1<XSAVEOPT>
> Hypervisor: Origin = "XenVMMXenVMM"
> real memory  = 2130706432 (2032 MB)
> avail memory = 2018213888 (1924 MB)
> Event timer "LAPIC" quality 400
> ACPI APIC Table: <Xen HVM>
> WARNING: L1 data cache covers less APIC IDs than a core
> 0 < 1
> WARNING: L2 data cache covers less APIC IDs than a core
> 0 < 1
> WARNING: L3 data cache covers less APIC IDs than a core
> 0 < 1
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> FreeBSD/SMP: 1 package(s) x 2 core(s)
> random: unblocking device.
> ioapic0: Changing APIC ID to 1
> MADT: Forcing active-low polarity and level trigger for SCI
> ioapic0 <Version 1.1> irqs 0-47 on motherboard
> random: entropy device external interface
> kbd1 at kbdmux0
> netmap: loaded module
> module_register_init: MOD_LOAD (vesa, 0xffffffff80f0ffb0, 0) error 19
> vtvga0: <VT VGA driver> on motherboard
> cryptosoft0: <software crypto> on motherboard
> acpi0: <Xen> on motherboard
> acpi0: Power Button (fixed)
> acpi0: Sleep Button (fixed)
> cpu0: <ACPI CPU> on acpi0
> cpu1: <ACPI CPU> on acpi0
> hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
> Timecounter "HPET" frequency 62500000 Hz quality 950
> attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
> Timecounter "i8254" frequency 1193182 Hz quality 0
> Event timer "i8254" frequency 1193182 Hz quality 100
> atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
> Event timer "RTC" frequency 32768 Hz quality 0
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
> acpi_timer0: <32-bit timer at 3.579545MHz> port 0xb008-0xb00b on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> isab0: <PCI-ISA bridge> at device 1.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc200-0xc20f at device 1.1 on pci0
> ata0: <ATA channel> at channel 0 on atapci0
> ata1: <ATA channel> at channel 1 on atapci0
> pci0: <bridge> at device 1.3 (no driver attached)
> xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xf0000000-0xf0ffffff irq 24 at device 2.0 on pci0
> vgapci0: <VGA-compatible display> mem 0xf1000000-0xf1ffffff,0xf2050000-0xf2050fff at device 3.0 on pci0
> vgapci0: Boot video device
> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> psm0: <PS/2 Mouse> irq 12 on atkbdc0
> psm0: [GIANT-LOCKED]
> psm0: model IntelliMouse Explorer, device ID 4
> fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
> fdc0: does not respond
> device_attach: fdc0 attach returned 6
> xenpv0: <Xen PV bus> on motherboard
> granttable0: <Xen Grant-table Device> on xenpv0
> xen_et0: <Xen PV Clock> on xenpv0
> Event timer "XENTIMER" frequency 1000000000 Hz quality 950
> Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
> xenstore0: <XenStore> on xenpv0
> evtchn0: <Xen event channel user-space device> on xenpv0
> privcmd0: <Xen privileged interface user-space device> on xenpv0
> debug0: <Xen debug handler> on xenpv0
> orm0: <ISA Option ROM> at iomem 0xed800-0xeffff on isa0
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> fdc0: No FDOUT register!
> ppc0: cannot reserve I/O port range
> Timecounters tick every 1.000 msec
> xenballoon0: <Xen Balloon Device> on xenstore0
> xctrl0: <Xen Control Device> on xenstore0
> xs_dev0: <Xenstore user-space device> on xenstore0
> xenbusb_front0: <Xen Frontend Devices> on xenstore0
> xenbusb_add_device: Device device/suspend/event-channel ignored. State 6
> xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
> xbd0: 16384MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
> xbd0: attaching as ada0
> xbd0: features: flush, write_barrier
> xbd0: synchronize cache commands enabled.
> xn0: Ethernet address: 00:02:04:08:fd:f0
> xenbusb_back0: <Xen Backend Devices> on xenstore0
> xbd1: 40960MB <Virtual Block Device> at device/vbd/832 on xenbusb_front0
> xbd1: attaching as ada1
> xbd1: features: flush, write_barrier
> xbd1: synchronize cache commands enabled.
> xn0: backend features: feature-sg
> SMP: AP CPU #1 Launched!
> WARNING: WITNESS option enabled, expect reduced performance.
> Trying to mount root from ufs:/dev/xbd0p2 [rw]...
> mountroot: waiting for device /dev/xbd0p2...
> Mounting from ufs:/dev/xbd0p2 failed with error 19.
> 
> Loader variables:
>   vfs.root.mountfrom=ufs:/dev/xbd0p2
>   vfs.root.mountfrom.options=rw
> 
> Manual root filesystem specification:
>   <fstype>:<device> [options]
>       Mount <device> using filesystem <fstype>
>       and with the specified (optional) option list.
> 
>     eg. ufs:/dev/da0s1a
>         zfs:tank
>         cd9660:/dev/cd0 ro
>           (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
> 
>   ?               List valid disk boot devices
>   .               Yield 1 second (for background tasks)
>   <empty line>    Abort manual input
> 
> mountroot> Trying to mount root from ufs:/dev/ada0p2 []...
> xn0: link state changed to DOWN
> xn0: link state changed to UP
> 
> > Also, can you paste the output of `xenstore-ls -fp` from Dom0 when this 
> > happens?
> 
> Windows:

[...]

> /local/domain/279/device/vbd/768/backend = "/local/domain/0/backend/vbd/279/768"   (n279,r0)
> /local/domain/279/device/vbd/768/backend-id = "0"   (n279,r0)
> /local/domain/279/device/vbd/768/state = "1"   (n279,r0)
> /local/domain/279/device/vbd/768/virtual-device = "768"   (n279,r0)
> /local/domain/279/device/vbd/768/device-type = "disk"   (n279,r0)
> /local/domain/279/device/vbd/832 = ""   (n279,r0)
> /local/domain/279/device/vbd/832/backend = "/local/domain/0/backend/vbd/279/832"   (n279,r0)
> /local/domain/279/device/vbd/832/backend-id = "0"   (n279,r0)
> /local/domain/279/device/vbd/832/state = "1"   (n279,r0)
> /local/domain/279/device/vbd/832/virtual-device = "832"   (n279,r0)
> /local/domain/279/device/vbd/832/device-type = "disk"   (n279,r0)
> /local/domain/279/device/vkbd = ""   (n0,r279)
> /local/domain/279/device/vkbd/0 = ""   (n279,r0)
> /local/domain/279/device/vkbd/0/backend = "/local/domain/0/backend/vkbd/279/0"   (n279,r0)
> /local/domain/279/device/vkbd/0/backend-id = "0"   (n279,r0)
> /local/domain/279/device/vkbd/0/state = "1"   (n279,r0)
> /local/domain/279/device/vif = ""   (n0,r279)
> /local/domain/279/device/vif/0 = ""   (n279,r0)
> /local/domain/279/device/vif/0/backend = "/local/domain/0/backend/vif/279/0"   (n279,r0)
> /local/domain/279/device/vif/0/backend-id = "0"   (n279,r0)
> /local/domain/279/device/vif/0/state = "1"   (n279,r0)
> /local/domain/279/device/vif/0/handle = "0"   (n279,r0)
> /local/domain/279/device/vif/0/mac = "00:16:3e:5d:0d:48"   (n279,r0)

It seems like Windows PV drivers don't attach at all, or are you running 
Windows without the PV drivers?

[...]

> FreeBSD:

[...]

> /local/domain/11/device/vbd/768/backend = "/local/domain/0/backend/vbd/11/768"   (n11,r0)
> /local/domain/11/device/vbd/768/backend-id = "0"   (n11,r0)
> /local/domain/11/device/vbd/768/state = "4"   (n11,r0)
> /local/domain/11/device/vbd/768/virtual-device = "768"   (n11,r0)
> /local/domain/11/device/vbd/768/device-type = "disk"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref0 = "8"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref1 = "9"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref2 = "10"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref3 = "11"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref4 = "12"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref5 = "13"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref6 = "14"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref7 = "15"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref8 = "16"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref9 = "17"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref10 = "18"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref11 = "19"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref12 = "20"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref13 = "21"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref14 = "22"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref15 = "23"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref16 = "24"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref17 = "25"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref18 = "26"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref19 = "27"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref20 = "28"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref21 = "29"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref22 = "30"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref23 = "31"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref24 = "32"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref25 = "33"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref26 = "34"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref27 = "35"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref28 = "36"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref29 = "37"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref30 = "38"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-ref31 = "39"   (n11,r0)
> /local/domain/11/device/vbd/768/num-ring-pages = "32"   (n11,r0)
> /local/domain/11/device/vbd/768/ring-page-order = "5"   (n11,r0)
> /local/domain/11/device/vbd/768/event-channel = "10"   (n11,r0)
> /local/domain/11/device/vbd/768/protocol = "x86_64-abi"   (n11,r0)
> /local/domain/11/device/vbd/832 = ""   (n11,r0)
> /local/domain/11/device/vbd/832/backend = "/local/domain/0/backend/vbd/11/832"   (n11,r0)
> /local/domain/11/device/vbd/832/backend-id = "0"   (n11,r0)
> /local/domain/11/device/vbd/832/state = "4"   (n11,r0)
> /local/domain/11/device/vbd/832/virtual-device = "832"   (n11,r0)
> /local/domain/11/device/vbd/832/device-type = "disk"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref0 = "40"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref1 = "41"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref2 = "42"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref3 = "43"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref4 = "44"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref5 = "45"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref6 = "46"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref7 = "47"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref8 = "48"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref9 = "49"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref10 = "50"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref11 = "51"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref12 = "52"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref13 = "53"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref14 = "54"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref15 = "55"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref16 = "56"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref17 = "57"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref18 = "58"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref19 = "59"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref20 = "60"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref21 = "61"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref22 = "62"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref23 = "63"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref24 = "64"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref25 = "65"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref26 = "66"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref27 = "67"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref28 = "68"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref29 = "69"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref30 = "70"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-ref31 = "71"   (n11,r0)
> /local/domain/11/device/vbd/832/num-ring-pages = "32"   (n11,r0)
> /local/domain/11/device/vbd/832/ring-page-order = "5"   (n11,r0)
> /local/domain/11/device/vbd/832/event-channel = "11"   (n11,r0)
> /local/domain/11/device/vbd/832/protocol = "x86_64-abi"   (n11,r0)

Everything seems fine regarding FreeBSD, PV devices have attached correctly 
AFAICT.

[...]
> /local/domain/11/console/vnc-port = "5900"   (n0,r11)
> /local/domain/11/image = ""   (n0,r11)
> /local/domain/11/image/device-model-pid = "6311"   (n0,r11)
> 
> > > What used to be Windows 2016 domU with /dev/zvol/zroot/windows0,raw,hda,w
> > > ends up in the Tianocore UEFI shell. Block devices seem to be available,
> > > I can even list the fs0: partition, but no booting further possible.
> > 
> > Hm, I've never used Tianocore myself, so I'm not sure what the issue could 
> > be here. Again, can you post the output of `xenstore-ls -fp` from Dom0 when 
> > this happens? At least this will show if the PV block device has been 
> > correctly attached.
> 
> See above.
> 
> How do I rollback to 4.5.2, do I need to downgrade the FreeBSD kernel
> as well?

Yes, you will need to switch back to an older kernel, but I thing we should 
aim to solve this, or else you will get stuck with an older Xen and FreeBSD 
kernel.

Since you mention that the console is very slow, if you run 'top' on Dom0, 
do you see any process (eg: qemu) taking a lot of CPU time?

Also, do you see Dom0 consuming a lot of CPU if you run 'xentop'?

Thanks.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7 and blkback changes

[Roger Pau Monné]

On Fri, Jun 10, 2016 at 09:41:04PM +0000, Marcin Cieslak wrote:
> On Thu, 9 Jun 2016, Roger Pau Monné wrote:
> 
> > On Wed, Jun 08, 2016 at 10:18:09PM +0000, Marcin Cieslak wrote:
> > > On Fri, 3 Jun 2016, Roger Pau Monné wrote:
> > > 
> > > > Hello,
> > > > 
> > > > First of all, this message is only relevant to those that use FreeBSD as 
> > > > Dom0 (host), not as a DomU (guest), so don't panic.
> > > > 
> > > > I've imported the latest Xen version (4.7-rc4) into the ports tree, it's 
> > > > still not the final version, but it's quite close, so we better start 
> > > > testing it to make sure it works fine with FreeBSD.
> > > 
> > > Thank you Roger, this is excellent. Are xen-tools-devel 4.5 now?
> > > Looks confusing.
> > > 
> > > I have also tried building xen-tools (4.7) without python
> > > and qemu configure reported this.
> > 
> > Python is required, you cannot get rid of it.
> 
> Strange, seems USES= python in the port's Makefile didn't work for me and
> continued happily after.

Have you been able to build the tools successfully? I'm afraid I cannot 
provide much help regarding this, since I don't know that much about the 
ports build system :(.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Marcin Cieslak]

On Mon, 13 Jun 2016, Roger Pau Monné wrote:

> Since you mention that the console is very slow, if you run 'top' on Dom0, 
> do you see any process (eg: qemu) taking a lot of CPU time?
> 
> Also, do you see Dom0 consuming a lot of CPU if you run 'xentop'?

Just quick info regarding console problem at the mountroot prompt:

- it was also very slow and behaving strange with older kernel
and 4.5.2.

- it works better when I use dual console - one can type on the 
serial console and see the results immediately, vidconsole
is slower but thins seem to work (at least I get no panics
due to "cannot mount root" anymore).

- will check top

Marcin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Marcin Cieslak]

On Mon, 13 Jun 2016, Roger Pau Monné wrote:

> On Fri, Jun 10, 2016 at 09:38:59PM +0000, Marcin Cieslak wrote:
> > "?" does not work - it mostly causes a panic, the console is slow, but I managed
> > to switch it to /dev/ada0p2, dmesg below:
> 
> This has now been reverted, so when I import the new RC this should be fixed 
> and you won't need to change anything.

I am confused now - so with a new Xen kernel (not yet in ports) I can use
/dev/xbd* devices again? They are in fact missing - xbd driver says "attaching as ada0"
and I can mount it only as /dev/ada

> It seems like Windows PV drivers don't attach at all, or are you running 
> Windows without the PV drivers?

Yes, I have. Those Windows partitions used to work properly without changes
under xen 4.5. But we are too early - the problem is that even ovmf
does not se them drives now, this is before Windows boots.

> Since you mention that the console is very slow, if you run 'top' on Dom0, 
> do you see any process (eg: qemu) taking a lot of CPU time?

Yes,

 1635 root          7 100    0   241M   101M RUN      7:16  91.14% qemu-system-i386

or more

> Also, do you see Dom0 consuming a lot of CPU if you run 'xentop'?

  Domain-0 -----r        446  100.0    4194300   25.2   no limit       n/a     1    0        0        0    0        0        0        0          0          0    0

Marcin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Roger Pau Monné]

On Mon, Jun 13, 2016 at 07:26:30PM +0000, Marcin Cieslak wrote:
> On Mon, 13 Jun 2016, Roger Pau Monné wrote:
> 
> > On Fri, Jun 10, 2016 at 09:38:59PM +0000, Marcin Cieslak wrote:
> > > "?" does not work - it mostly causes a panic, the console is slow, but I managed
> > > to switch it to /dev/ada0p2, dmesg below:
> > 
> > This has now been reverted, so when I import the new RC this should be fixed 
> > and you won't need to change anything.
> 
> I am confused now - so with a new Xen kernel (not yet in ports) I can use
> /dev/xbd* devices again? They are in fact missing - xbd driver says "attaching as ada0"
> and I can mount it only as /dev/ada
> 
> > It seems like Windows PV drivers don't attach at all, or are you running 
> > Windows without the PV drivers?
> 
> Yes, I have. Those Windows partitions used to work properly without changes
> under xen 4.5. But we are too early - the problem is that even ovmf
> does not se them drives now, this is before Windows boots.
> 
> > Since you mention that the console is very slow, if you run 'top' on Dom0, 
> > do you see any process (eg: qemu) taking a lot of CPU time?
> 
> Yes,
> 
>  1635 root          7 100    0   241M   101M RUN      7:16  91.14% qemu-system-i386
> 
> or more
> 
> > Also, do you see Dom0 consuming a lot of CPU if you run 'xentop'?
> 
>   Domain-0 -----r        446  100.0    4194300   25.2   no limit       n/a     1    0        0        0    0        0        0        0          0          0    0
> 

Hello,

I've just imported Xen 4.7.0-rc6 into the ports tree, could you give it a 
try when you have a moment?

FWIW, I'm going on vacations until the 4th of July, and I will be mostly 
AFK.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HEADS UP: Imported Xen 4.7: no blkback

[Marcin Cieslak]

On Thu, 16 Jun 2016, Roger Pau Monné wrote:

> I've just imported Xen 4.7.0-rc6 into the ports tree, could you give it a 
> try when you have a moment?

Yes, of course. A quick test shows no change - Windows get stuck
at the UEFI shell with some block devices listed (I use "hda" for
Windows), SeaBIOS cannot boot FreeBSD with "xvda". FreeBSD
works with "hda", but I have to mount "/dev/ada0p1".

Marcin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

HEADS UP: Imported Xen 4.7 and blkback changes

[Roger Pau Monné]

Hello,

First of all, this message is only relevant to those that use FreeBSD as 
Dom0 (host), not as a DomU (guest), so don't panic.

I've imported the latest Xen version (4.7-rc4) into the ports tree, it's 
still not the final version, but it's quite close, so we better start 
testing it to make sure it works fine with FreeBSD.

One of the more relevant changes in 4.7 regarding FreeBSD is the support for 
block hotplug scripts. This means that we now have the option to use 
backends different than simple block or regular files, provided that someone 
writes the proper hotplug scripts to attach them (I've heard there are some 
iSCSI hotplug scripts around). This however requires changes in blkback, so 
if you plan to use the Xen 4.7 port, please make sure that you are running a 
kernel that contains revision r301269 (or any later version). The same also 
works the other way around, if you update your kernel past r301269, you will 
have to run Xen 4.7 or else disks are not going to work.

Also, and depending on your hardware, you will also need to add 
hw.pci.mcfg=0 to your /boot/loader.conf file after updating to Xen 4.7.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel