* libxl: problem with devices in PV
@ 2011-07-20 11:01 Roger Pau Monné
2011-07-20 11:37 ` Stefano Stabellini
0 siblings, 1 reply; 12+ messages in thread
From: Roger Pau Monné @ 2011-07-20 11:01 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1: Type: text/plain, Size: 17817 bytes --]
Hello,
I'm trying to run PV machines using the new libxenlight toolstack on
NetBSD. So far, I've been able to connect to the domu using the
console (I was unable to do so before). I'm attaching a little patch
that removes setting the consback to IOEMU when detecting a qdisk
(that was preventing the domu from even booting). With the
introduction of libxenlight, Xen doesn't use vbd for disk backend
anymore, it uses qdisk, which I assume Qemu automatically attaches to
running guests. It works fine with HVM guests, but it seems to fail
with PV guests. The config file I'm using is:
bootloader = '/usr/xen42/bin/pygrub'
vcpus = '1'
memory = '30720'
root = '/dev/xvda2 ro'
disk = [
'file:/home/xen/debian/disk.img,xvda2,w',
'file:/home/xen/debian/swap.img,xvda1,w',
]
name = 'debian'
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
And the output from booting:
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 2.6.32-5-xen-amd64 (Debian
2.6.32-34squeeze1) (dannf@debian.org) (gcc version 4.3.5 (Debian
4.3.5-4) ) #1 SMP Thu May 19 01:16:47 UTC 2011
[ 0.000000] Command line: root=/dev/xvda2 ro root=/dev/xvda2 ro
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Centaur CentaurHauls
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] released 0 pages of unused memory
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
[ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
[ 0.000000] Xen: 0000000000100000 - 0000000780000000 (usable)
[ 0.000000] DMI not present or invalid.
[ 0.000000] last_pfn = 0x780000 max_arch_pfn = 0x400000000
[ 0.000000] last_pfn = 0x100000 max_arch_pfn = 0x400000000
[ 0.000000] init_memory_mapping: 0000000000000000-0000000100000000
[ 0.000000] init_memory_mapping: 0000000100000000-0000000780000000
[ 0.000000] RAMDISK: 016ba000 - 02feb000
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at 0000000000000000-0000000780000000
[ 0.000000] Bootmem setup node 0 0000000000000000-0000000780000000
[ 0.000000] NODE_DATA [0000000000008000 - 000000000000ffff]
[ 0.000000] bootmap [00000000008c7000 - 00000000009b6fff] pages f0
[ 0.000000] (8 early reservations) ==> bootmem [0000000000 - 0780000000]
[ 0.000000] #0 [0000000000 - 0000001000] BIOS data page ==>
[0000000000 - 0000001000]
[ 0.000000] #1 [0006bee000 - 0006c29000] XEN PAGETABLES ==>
[0006bee000 - 0006c29000]
[ 0.000000] #2 [0000006000 - 0000008000] TRAMPOLINE ==>
[0000006000 - 0000008000]
[ 0.000000] #3 [0001000000 - 00016999d4] TEXT DATA BSS ==>
[0001000000 - 00016999d4]
[ 0.000000] #4 [00016ba000 - 0002feb000] RAMDISK ==>
[00016ba000 - 0002feb000]
[ 0.000000] #5 [0002feb000 - 0006bee000] XEN START INFO ==>
[0002feb000 - 0006bee000]
[ 0.000000] #6 [0000100000 - 00008c7000] PGTABLE ==>
[0000100000 - 00008c7000]
[ 0.000000] #7 [0006c29000 - 000a043000] PGTABLE ==>
[0006c29000 - 000a043000]
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000000 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal 0x00100000 -> 0x00780000
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[2] active PFN ranges
[ 0.000000] 0: 0x00000000 -> 0x000000a0
[ 0.000000] 0: 0x00000100 -> 0x00780000
[ 0.000000] SFI: Simple Firmware Interface v0.7 http://simplefirmware.org
[ 0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.000000] No local APIC present
[ 0.000000] APIC: disable apic facility
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[ 0.000000] PCI: Warning: Cannot find a gap in the 32bit address range
[ 0.000000] PCI: Unassigned devices with 32bit resource registers may break!
[ 0.000000] Allocating PCI resources starting at 780100000 (gap:
780100000:400000)
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.2-unstable (preserve-AD)
[ 0.000000] NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:1 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 30 pages/cpu @ffff88002804f000 s90328
r8192 d24360 u122880
[ 0.000000] pcpu-alloc: s90328 r8192 d24360 u122880 alloc=30*4096
[ 0.000000] pcpu-alloc: [0] 0
[ 0.000000] Xen: using vcpu_info placement
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on.
Total pages: 7754710
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: root=/dev/xvda2 ro root=/dev/xvda2 ro
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Initializing CPU#0
[ 0.000000] Checking aperture...
[ 0.000000] No AGP bridge found
[ 0.000000] Memory: 30803904k/31457280k available (3149k kernel
code, 384k absent, 652992k reserved, 1907k data, 604k init)
[ 0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0,
CPUs=1, Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] NR_IRQS:4352 nr_irqs:512
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] console [hvc0] enabled
[ 0.000000] installing Xen timer for CPU 0
[ 0.000000] Detected 2394.046 MHz processor.
[ 0.004000] Calibrating delay loop (skipped), value calculated
using timer frequency.. 4788.09 BogoMIPS (lpj=9576184)
[ 0.004000] Security Framework initialized
[ 0.004000] SELinux: Disabled at boot.
[ 0.005680] Dentry cache hash table entries: 4194304 (order: 13,
33554432 bytes)
[ 0.014625] Inode-cache hash table entries: 2097152 (order: 12,
16777216 bytes)
[ 0.017597] Mount-cache hash table entries: 256
[ 0.017761] Initializing cgroup subsys ns
[ 0.017769] Initializing cgroup subsys cpuacct
[ 0.017776] Initializing cgroup subsys devices
[ 0.017782] Initializing cgroup subsys freezer
[ 0.017786] Initializing cgroup subsys net_cls
[ 0.017827] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.017835] CPU: L2 cache: 256K
[ 0.017838] CPU: L3 cache: 12288K
[ 0.017844] CPU 0/0x20 -> Node 0
[ 0.017848] CPU: Unsupported number of siblings 32
[ 0.017854] Performance Events: unsupported p6 CPU model 44 no PMU
driver, software events only.
[ 0.017874] SMP alternatives: switching to UP code
[ 0.020000] Freeing SMP alternatives: 28k freed
[ 0.020192] Brought up 1 CPUs
[ 0.020309] devtmpfs: initialized
[ 0.024342] Grant table initialized
[ 0.024351] regulator: core version 0.5
[ 0.024403] NET: Registered protocol family 16
[ 0.025064] PCI: setting up Xen PCI frontend stub
[ 0.026729] bio: create slab <bio-0> at 0
[ 0.026798] ACPI: Interpreter disabled.
[ 0.026833] xen_balloon: Initialising balloon driver with page order 0.
[ 0.026878] vgaarb: loaded
[ 0.026939] PCI: System does not support PCI
[ 0.026944] PCI: System does not support PCI
[ 0.027019] Switching to clocksource xen
[ 0.028001] pnp: PnP ACPI: disabled
[ 0.028001] NET: Registered protocol family 2
[ 0.028001] IP route cache hash table entries: 524288 (order: 10,
4194304 bytes)
[ 0.028460] TCP established hash table entries: 524288 (order: 11,
8388608 bytes)
[ 0.030137] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 0.030331] TCP: Hash tables configured (established 524288 bind 65536)
[ 0.030337] TCP reno registered
[ 0.030405] NET: Registered protocol family 1
[ 0.030463] Unpacking initramfs...
[ 0.051340] Freeing initrd memory: 25796k freed
[ 0.058949] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.058964] DMA: Placing 64MB software IO TLB between
ffff880020000000 - ffff880024000000
[ 0.058970] DMA: software IO TLB at phys 0x20000000 - 0x24000000
[ 0.059058] platform rtc_cmos: registered platform RTC device (no
PNP device found)
[ 0.059272] audit: initializing netlink socket (disabled)
[ 0.059288] type=2000 audit(1311165823.857:1): initialized
[ 0.064506] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.065744] VFS: Disk quotas dquot_6.5.2
[ 0.065796] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.065877] msgmni has been set to 32768
[ 0.066057] alg: No test for stdrng (krng)
[ 0.066112] Block layer SCSI generic (bsg) driver version 0.4
loaded (major 253)
[ 0.066119] io scheduler noop registered
[ 0.066122] io scheduler anticipatory registered
[ 0.066127] io scheduler deadline registered
[ 0.066157] io scheduler cfq registered (default)
[ 0.072642] registering netback
[ 0.074001] Linux agpgart interface v0.103
[ 0.074032] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.173825] input: Macintosh mouse button emulation as
/devices/virtual/input/input0
[ 0.173872] PNP: No PS/2 controller found. Probing ports directly.
[ 0.174689] i8042.c: No controller found.
[ 0.174750] mice: PS/2 mouse device common for all mice
[ 0.174829] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[ 0.174883] cpuidle: using governor ladder
[ 0.174890] cpuidle: using governor menu
[ 0.174897] No iBFT detected.
[ 0.175118] TCP cubic registered
[ 0.175221] NET: Registered protocol family 10
[ 0.175788] Mobile IPv6
[ 0.175797] NET: Registered protocol family 17
[ 0.175889] registered taskstats version 1
[ 5.272069] XENBUS: Waiting for devices to initialise:
295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s....
udevd[46]: worker [53] unexpectedly returned with status 0x0100
udevd[46]: worker [53] failed while handling '/devices/vbd-51713'
udevd[46]: worker [54] unexpectedly returned with status 0x0100
udevd[46]: worker [54] failed while handling '/devices/vbd-51714'
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Waiting for root file system ...
120s...115s...110s...105s...100s...95s...90s...done.
Gave up waiting for root device. Common problems:
- Boot args (cat /proc/cmdline)
- Check rootdelay= (did the system wait long enough?)
- Check root= (did the system wait for the right device?)
- Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/xvda2 does not exist. Dropping to a shell!
BusyBox v1.17.1 (Debian 1:1.17.1-8) built-in shell (ash)
Enter 'help' for a list of built-in commands.
/bin/sh: can't access tty; job control turned off
(initramfs)
[ 600.488236] XENBUS: Timeout connecting to device: device/vbd/51714
(local state 3, remote state 1)
[ 600.488328] XENBUS: Timeout connecting to device: device/vbd/51713
(local state 3, remote state 1)
>From what I see, the guest is unable to find the disk, which is
strange, because HVM guests work fine, and the disk is attached
correctly. Does this also happen when using file:/ backend (qdisk)
under Linux also? Is it a problem with Qemu or is it related to libxl?
I'm also attaching the output from xenstore, although I think it is ok
(or at least is the same as the one I have when running a HVM
machine):
/local/domain/0/backend = "" (n0)
/local/domain/0/backend/qdisk = "" (n0)
/local/domain/0/backend/qdisk/1 = "" (n0)
/local/domain/0/backend/qdisk/1/51714 = "" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/frontend =
"/local/domain/1/device/vbd/51714" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/params =
"aio:/home/xen/debian/disk.img" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/frontend-id = "1" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/online = "1" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/removable = "0" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/bootable = "1" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/state = "1" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/dev = "xvda2" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/type = "qdisk" (n0,r1)
/local/domain/0/backend/qdisk/1/51714/mode = "w" (n0,r1)
/local/domain/0/backend/qdisk/1/51713 = "" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/frontend =
"/local/domain/1/device/vbd/51713" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/params =
"aio:/home/xen/debian/swap.img" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/frontend-id = "1" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/online = "1" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/removable = "0" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/bootable = "1" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/state = "1" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/dev = "xvda1" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/type = "qdisk" (n0,r1)
/local/domain/0/backend/qdisk/1/51713/mode = "w" (n0,r1)
/local/domain/0/backend/console = "" (n0)
/local/domain/0/backend/console/1 = "" (n0)
/local/domain/0/backend/console/1/0 = "" (n0,r1)
/local/domain/0/backend/console/1/0/frontend =
"/local/domain/1/console" (n0,r1)
/local/domain/0/backend/console/1/0/frontend-id = "1" (n0,r1)
/local/domain/0/backend/console/1/0/online = "1" (n0,r1)
/local/domain/0/backend/console/1/0/state = "1" (n0,r1)
/local/domain/0/backend/console/1/0/domain = "debian" (n0,r1)
/local/domain/0/backend/console/1/0/protocol = "vt100" (n0,r1)
/local/domain/0/backend/pci = "" (n0)
/local/domain/0/backend/pci/1 = "" (n0)
/local/domain/0/backend/pci/1/0 = "" (n0,r1)
/local/domain/0/backend/pci/1/0/frontend =
"/local/domain/1/device/pci/0" (n0,r1)
/local/domain/0/backend/pci/1/0/frontend-id = "1" (n0,r1)
/local/domain/0/backend/pci/1/0/online = "1" (n0,r1)
/local/domain/0/backend/pci/1/0/state = "1" (n0,r1)
/local/domain/0/backend/pci/1/0/domain = "debian" (n0,r1)
/local/domain/0/backend/pci/1/0/num_devs = "0" (n0,r1)
/local/domain/0/device-model = "" (n0)
/local/domain/0/device-model/1 = "" (n0)
/local/domain/0/device-model/1/disable_pf = "1" (n0)
/local/domain/0/device-model/1/state = "running" (n0)
/local/domain/1 = "" (n0,r1)
/local/domain/1/vm = "/vm/4bfc3adf-cdb2-e011-a177-1803730a9a51" (n0,r1)
/local/domain/1/name = "debian" (n0,r1)
/local/domain/1/control = "" (n0,r1)
/local/domain/1/control/shutdown = "" (n1)
/local/domain/1/control/platform-feature-multiprocessor-suspend = "1" (n0,r1)
/local/domain/1/device = "" (n0,r1)
/local/domain/1/device/suspend = "" (n1)
/local/domain/1/device/suspend/event-channel = "" (n1)
/local/domain/1/device/vbd = "" (n0,r1)
/local/domain/1/device/vbd/51714 = "" (n1,r0)
/local/domain/1/device/vbd/51714/backend =
"/local/domain/0/backend/qdisk/1/51714" (n1,r0)
/local/domain/1/device/vbd/51714/backend-id = "0" (n1,r0)
/local/domain/1/device/vbd/51714/state = "3" (n1,r0)
/local/domain/1/device/vbd/51714/virtual-device = "51714" (n1,r0)
/local/domain/1/device/vbd/51714/device-type = "disk" (n1,r0)
/local/domain/1/device/vbd/51714/ring-ref = "9" (n1,r0)
/local/domain/1/device/vbd/51714/event-channel = "9" (n1,r0)
/local/domain/1/device/vbd/51714/protocol = "x86_64-abi" (n1,r0)
/local/domain/1/device/vbd/51713 = "" (n1,r0)
/local/domain/1/device/vbd/51713/backend =
"/local/domain/0/backend/qdisk/1/51713" (n1,r0)
/local/domain/1/device/vbd/51713/backend-id = "0" (n1,r0)
/local/domain/1/device/vbd/51713/state = "3" (n1,r0)
/local/domain/1/device/vbd/51713/virtual-device = "51713" (n1,r0)
/local/domain/1/device/vbd/51713/device-type = "disk" (n1,r0)
/local/domain/1/device/vbd/51713/ring-ref = "10" (n1,r0)
/local/domain/1/device/vbd/51713/event-channel = "10" (n1,r0)
/local/domain/1/device/vbd/51713/protocol = "x86_64-abi" (n1,r0)
/local/domain/1/device/pci = "" (n0,r1)
/local/domain/1/device/pci/0 = "" (n1,r0)
/local/domain/1/device/pci/0/backend =
"/local/domain/0/backend/pci/1/0" (n1,r0)
/local/domain/1/device/pci/0/backend-id = "0" (n1,r0)
/local/domain/1/device/pci/0/state = "3" (n1,r0)
/local/domain/1/device/pci/0/pci-op-ref = "8" (n1,r0)
/local/domain/1/device/pci/0/event-channel = "8" (n1,r0)
/local/domain/1/device/pci/0/magic = "7" (n1,r0)
/local/domain/1/data = "" (n1)
/local/domain/1/cpu = "" (n0,r1)
/local/domain/1/cpu/0 = "" (n0,r1)
/local/domain/1/cpu/0/availability = "online" (n0,r1)
/local/domain/1/memory = "" (n0,r1)
/local/domain/1/memory/static-max = "31457280" (n0,r1)
/local/domain/1/memory/target = "31457280" (n0,r1)
/local/domain/1/memory/videoram = "0" (n0,r1)
/local/domain/1/error = "" (n0,r1)
/local/domain/1/drivers = "" (n0,r1)
/local/domain/1/attr = "" (n0,r1)
/local/domain/1/messages = "" (n0,r1)
/local/domain/1/domid = "1" (n0,r1)
/local/domain/1/store = "" (n0,r1)
/local/domain/1/store/port = "1" (n0,r1)
/local/domain/1/store/ring-ref = "6454766" (n0,r1)
/local/domain/1/console = "" (n1,r0)
/local/domain/1/console/backend =
"/local/domain/0/backend/console/1/0" (n1,r0)
/local/domain/1/console/backend-id = "0" (n1,r0)
/local/domain/1/console/limit = "1048576" (n1,r0)
/local/domain/1/console/type = "xenconsoled" (n1,r0)
/local/domain/1/console/output = "pty" (n1,r0)
/local/domain/1/console/port = "2" (n1,r0)
/local/domain/1/console/ring-ref = "8569076" (n1,r0)
/local/domain/1/console/tty = "/dev/pts/2" (n1,r0)
/local/domain/1/hvmloader = "" (n0,r1)
/local/domain/1/hvmloader/bios = "rombios" (n0,r1)
/local/domain/1/image = "" (n0,r1)
/local/domain/1/image/device-model-pid = "230" (n0,r1)
Let's see if someone can shed some light over this.
Thanks again, Roger.
[-- Attachment #2: patch-console --]
[-- Type: application/octet-stream, Size: 603 bytes --]
diff -r e298ce67777e tools/libxl/libxl_create.c
--- a/tools/libxl/libxl_create.c Mon Jul 18 14:38:31 2011 +0100
+++ b/tools/libxl/libxl_create.c Wed Jul 20 14:57:29 2011 +0200
@@ -509,8 +509,8 @@
d_config->num_vfbs, d_config->vfbs,
d_config->num_disks, &d_config->disks[0]);
- if (need_qemu)
- console.consback = LIBXL_CONSOLE_BACKEND_IOEMU;
+ //if (need_qemu)
+ // console.consback = LIBXL_CONSOLE_BACKEND_IOEMU;
libxl__device_console_add(gc, domid, &console, &state);
libxl_device_console_destroy(&console);
[-- Attachment #3: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-20 11:01 libxl: problem with devices in PV Roger Pau Monné
@ 2011-07-20 11:37 ` Stefano Stabellini
2011-07-20 12:54 ` Ian Campbell
0 siblings, 1 reply; 12+ messages in thread
From: Stefano Stabellini @ 2011-07-20 11:37 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel
[-- Attachment #1: Type: text/plain, Size: 12455 bytes --]
On Wed, 20 Jul 2011, Roger Pau Monné wrote:
> Hello,
>
> I'm trying to run PV machines using the new libxenlight toolstack on
> NetBSD. So far, I've been able to connect to the domu using the
> console (I was unable to do so before). I'm attaching a little patch
> that removes setting the consback to IOEMU when detecting a qdisk
> (that was preventing the domu from even booting). With the
> introduction of libxenlight, Xen doesn't use vbd for disk backend
> anymore, it uses qdisk, which I assume Qemu automatically attaches to
> running guests. It works fine with HVM guests, but it seems to fail
> with PV guests. The config file I'm using is:
>
Xen uses qdisk only when blktap is not available.
I suggest you install blktap if you can because it is significantly
faster than qdisk at the moment.
Otherwise if you use a physical partition and specify phy: in the config
file you should get the kernel based blkback that is the fastest option
available.
> bootloader = '/usr/xen42/bin/pygrub'
> vcpus = '1'
> memory = '30720'
>
> root = '/dev/xvda2 ro'
> disk = [
> 'file:/home/xen/debian/disk.img,xvda2,w',
> 'file:/home/xen/debian/swap.img,xvda1,w',
> ]
>
> name = 'debian'
>
> on_poweroff = 'destroy'
> on_reboot = 'restart'
> on_crash = 'restart'
>
> And the output from booting:
>
> [ 0.000000] Initializing cgroup subsys cpuset
> [ 0.000000] Initializing cgroup subsys cpu
> [ 0.000000] Linux version 2.6.32-5-xen-amd64 (Debian
> 2.6.32-34squeeze1) (dannf@debian.org) (gcc version 4.3.5 (Debian
> 4.3.5-4) ) #1 SMP Thu May 19 01:16:47 UTC 2011
> [ 0.000000] Command line: root=/dev/xvda2 ro root=/dev/xvda2 ro
> [ 0.000000] KERNEL supported cpus:
> [ 0.000000] Intel GenuineIntel
> [ 0.000000] AMD AuthenticAMD
> [ 0.000000] Centaur CentaurHauls
> [ 0.000000] ACPI in unprivileged domain disabled
> [ 0.000000] released 0 pages of unused memory
> [ 0.000000] BIOS-provided physical RAM map:
> [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
> [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
> [ 0.000000] Xen: 0000000000100000 - 0000000780000000 (usable)
> [ 0.000000] DMI not present or invalid.
> [ 0.000000] last_pfn = 0x780000 max_arch_pfn = 0x400000000
> [ 0.000000] last_pfn = 0x100000 max_arch_pfn = 0x400000000
> [ 0.000000] init_memory_mapping: 0000000000000000-0000000100000000
> [ 0.000000] init_memory_mapping: 0000000100000000-0000000780000000
> [ 0.000000] RAMDISK: 016ba000 - 02feb000
> [ 0.000000] No NUMA configuration found
> [ 0.000000] Faking a node at 0000000000000000-0000000780000000
> [ 0.000000] Bootmem setup node 0 0000000000000000-0000000780000000
> [ 0.000000] NODE_DATA [0000000000008000 - 000000000000ffff]
> [ 0.000000] bootmap [00000000008c7000 - 00000000009b6fff] pages f0
> [ 0.000000] (8 early reservations) ==> bootmem [0000000000 - 0780000000]
> [ 0.000000] #0 [0000000000 - 0000001000] BIOS data page ==>
> [0000000000 - 0000001000]
> [ 0.000000] #1 [0006bee000 - 0006c29000] XEN PAGETABLES ==>
> [0006bee000 - 0006c29000]
> [ 0.000000] #2 [0000006000 - 0000008000] TRAMPOLINE ==>
> [0000006000 - 0000008000]
> [ 0.000000] #3 [0001000000 - 00016999d4] TEXT DATA BSS ==>
> [0001000000 - 00016999d4]
> [ 0.000000] #4 [00016ba000 - 0002feb000] RAMDISK ==>
> [00016ba000 - 0002feb000]
> [ 0.000000] #5 [0002feb000 - 0006bee000] XEN START INFO ==>
> [0002feb000 - 0006bee000]
> [ 0.000000] #6 [0000100000 - 00008c7000] PGTABLE ==>
> [0000100000 - 00008c7000]
> [ 0.000000] #7 [0006c29000 - 000a043000] PGTABLE ==>
> [0006c29000 - 000a043000]
> [ 0.000000] Zone PFN ranges:
> [ 0.000000] DMA 0x00000000 -> 0x00001000
> [ 0.000000] DMA32 0x00001000 -> 0x00100000
> [ 0.000000] Normal 0x00100000 -> 0x00780000
> [ 0.000000] Movable zone start PFN for each node
> [ 0.000000] early_node_map[2] active PFN ranges
> [ 0.000000] 0: 0x00000000 -> 0x000000a0
> [ 0.000000] 0: 0x00000100 -> 0x00780000
> [ 0.000000] SFI: Simple Firmware Interface v0.7 http://simplefirmware.org
> [ 0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
> [ 0.000000] No local APIC present
> [ 0.000000] APIC: disable apic facility
> [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
> [ 0.000000] PCI: Warning: Cannot find a gap in the 32bit address range
> [ 0.000000] PCI: Unassigned devices with 32bit resource registers may break!
> [ 0.000000] Allocating PCI resources starting at 780100000 (gap:
> 780100000:400000)
> [ 0.000000] Booting paravirtualized kernel on Xen
> [ 0.000000] Xen version: 4.2-unstable (preserve-AD)
> [ 0.000000] NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:1 nr_node_ids:1
> [ 0.000000] PERCPU: Embedded 30 pages/cpu @ffff88002804f000 s90328
> r8192 d24360 u122880
> [ 0.000000] pcpu-alloc: s90328 r8192 d24360 u122880 alloc=30*4096
> [ 0.000000] pcpu-alloc: [0] 0
> [ 0.000000] Xen: using vcpu_info placement
> [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on.
> Total pages: 7754710
> [ 0.000000] Policy zone: Normal
> [ 0.000000] Kernel command line: root=/dev/xvda2 ro root=/dev/xvda2 ro
> [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [ 0.000000] Initializing CPU#0
> [ 0.000000] Checking aperture...
> [ 0.000000] No AGP bridge found
> [ 0.000000] Memory: 30803904k/31457280k available (3149k kernel
> code, 384k absent, 652992k reserved, 1907k data, 604k init)
> [ 0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0,
> CPUs=1, Nodes=1
> [ 0.000000] Hierarchical RCU implementation.
> [ 0.000000] NR_IRQS:4352 nr_irqs:512
> [ 0.000000] Console: colour dummy device 80x25
> [ 0.000000] console [tty0] enabled
> [ 0.000000] console [hvc0] enabled
> [ 0.000000] installing Xen timer for CPU 0
> [ 0.000000] Detected 2394.046 MHz processor.
> [ 0.004000] Calibrating delay loop (skipped), value calculated
> using timer frequency.. 4788.09 BogoMIPS (lpj=9576184)
> [ 0.004000] Security Framework initialized
> [ 0.004000] SELinux: Disabled at boot.
> [ 0.005680] Dentry cache hash table entries: 4194304 (order: 13,
> 33554432 bytes)
> [ 0.014625] Inode-cache hash table entries: 2097152 (order: 12,
> 16777216 bytes)
> [ 0.017597] Mount-cache hash table entries: 256
> [ 0.017761] Initializing cgroup subsys ns
> [ 0.017769] Initializing cgroup subsys cpuacct
> [ 0.017776] Initializing cgroup subsys devices
> [ 0.017782] Initializing cgroup subsys freezer
> [ 0.017786] Initializing cgroup subsys net_cls
> [ 0.017827] CPU: L1 I cache: 32K, L1 D cache: 32K
> [ 0.017835] CPU: L2 cache: 256K
> [ 0.017838] CPU: L3 cache: 12288K
> [ 0.017844] CPU 0/0x20 -> Node 0
> [ 0.017848] CPU: Unsupported number of siblings 32
> [ 0.017854] Performance Events: unsupported p6 CPU model 44 no PMU
> driver, software events only.
> [ 0.017874] SMP alternatives: switching to UP code
> [ 0.020000] Freeing SMP alternatives: 28k freed
> [ 0.020192] Brought up 1 CPUs
> [ 0.020309] devtmpfs: initialized
> [ 0.024342] Grant table initialized
> [ 0.024351] regulator: core version 0.5
> [ 0.024403] NET: Registered protocol family 16
> [ 0.025064] PCI: setting up Xen PCI frontend stub
> [ 0.026729] bio: create slab <bio-0> at 0
> [ 0.026798] ACPI: Interpreter disabled.
> [ 0.026833] xen_balloon: Initialising balloon driver with page order 0.
> [ 0.026878] vgaarb: loaded
> [ 0.026939] PCI: System does not support PCI
> [ 0.026944] PCI: System does not support PCI
> [ 0.027019] Switching to clocksource xen
> [ 0.028001] pnp: PnP ACPI: disabled
> [ 0.028001] NET: Registered protocol family 2
> [ 0.028001] IP route cache hash table entries: 524288 (order: 10,
> 4194304 bytes)
> [ 0.028460] TCP established hash table entries: 524288 (order: 11,
> 8388608 bytes)
> [ 0.030137] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
> [ 0.030331] TCP: Hash tables configured (established 524288 bind 65536)
> [ 0.030337] TCP reno registered
> [ 0.030405] NET: Registered protocol family 1
> [ 0.030463] Unpacking initramfs...
> [ 0.051340] Freeing initrd memory: 25796k freed
> [ 0.058949] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
> [ 0.058964] DMA: Placing 64MB software IO TLB between
> ffff880020000000 - ffff880024000000
> [ 0.058970] DMA: software IO TLB at phys 0x20000000 - 0x24000000
> [ 0.059058] platform rtc_cmos: registered platform RTC device (no
> PNP device found)
> [ 0.059272] audit: initializing netlink socket (disabled)
> [ 0.059288] type=2000 audit(1311165823.857:1): initialized
> [ 0.064506] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [ 0.065744] VFS: Disk quotas dquot_6.5.2
> [ 0.065796] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [ 0.065877] msgmni has been set to 32768
> [ 0.066057] alg: No test for stdrng (krng)
> [ 0.066112] Block layer SCSI generic (bsg) driver version 0.4
> loaded (major 253)
> [ 0.066119] io scheduler noop registered
> [ 0.066122] io scheduler anticipatory registered
> [ 0.066127] io scheduler deadline registered
> [ 0.066157] io scheduler cfq registered (default)
> [ 0.072642] registering netback
> [ 0.074001] Linux agpgart interface v0.103
> [ 0.074032] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [ 0.173825] input: Macintosh mouse button emulation as
> /devices/virtual/input/input0
> [ 0.173872] PNP: No PS/2 controller found. Probing ports directly.
> [ 0.174689] i8042.c: No controller found.
> [ 0.174750] mice: PS/2 mouse device common for all mice
> [ 0.174829] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
> [ 0.174883] cpuidle: using governor ladder
> [ 0.174890] cpuidle: using governor menu
> [ 0.174897] No iBFT detected.
> [ 0.175118] TCP cubic registered
> [ 0.175221] NET: Registered protocol family 10
> [ 0.175788] Mobile IPv6
> [ 0.175797] NET: Registered protocol family 17
> [ 0.175889] registered taskstats version 1
> [ 5.272069] XENBUS: Waiting for devices to initialise:
> 295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s....
> udevd[46]: worker [53] unexpectedly returned with status 0x0100
> udevd[46]: worker [53] failed while handling '/devices/vbd-51713'
> udevd[46]: worker [54] unexpectedly returned with status 0x0100
> udevd[46]: worker [54] failed while handling '/devices/vbd-51714'
> Begin: Loading essential drivers ... done.
> Begin: Running /scripts/init-premount ... done.
> Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
> Begin: Waiting for root file system ...
> 120s...115s...110s...105s...100s...95s...90s...done.
> Gave up waiting for root device. Common problems:
> - Boot args (cat /proc/cmdline)
> - Check rootdelay= (did the system wait long enough?)
> - Check root= (did the system wait for the right device?)
> - Missing modules (cat /proc/modules; ls /dev)
> ALERT! /dev/xvda2 does not exist. Dropping to a shell!
>
>
> BusyBox v1.17.1 (Debian 1:1.17.1-8) built-in shell (ash)
> Enter 'help' for a list of built-in commands.
>
> /bin/sh: can't access tty; job control turned off
> (initramfs)
> [ 600.488236] XENBUS: Timeout connecting to device: device/vbd/51714
> (local state 3, remote state 1)
> [ 600.488328] XENBUS: Timeout connecting to device: device/vbd/51713
> (local state 3, remote state 1)
>
> >From what I see, the guest is unable to find the disk, which is
> strange, because HVM guests work fine, and the disk is attached
> correctly. Does this also happen when using file:/ backend (qdisk)
> under Linux also? Is it a problem with Qemu or is it related to libxl?
It doesn't happen under Linux; it is probably a problem with Qemu. Qemu
uses the gntdev device (/dev/xen/gntdev) to open the grant table
(necessary to implement xen backends in userspace).
There doesn't seem anything equivalent on NetBSD, hence it fails...
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-20 11:37 ` Stefano Stabellini
@ 2011-07-20 12:54 ` Ian Campbell
2011-07-20 13:10 ` Roger Pau Monné
0 siblings, 1 reply; 12+ messages in thread
From: Ian Campbell @ 2011-07-20 12:54 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: Roger Pau Monné, xen-devel
On Wed, 2011-07-20 at 12:37 +0100, Stefano Stabellini wrote:
> On Wed, 20 Jul 2011, Roger Pau Monné wrote:
> > Hello,
> >
> > I'm trying to run PV machines using the new libxenlight toolstack on
> > NetBSD. So far, I've been able to connect to the domu using the
> > console (I was unable to do so before). I'm attaching a little patch
> > that removes setting the consback to IOEMU when detecting a qdisk
> > (that was preventing the domu from even booting). With the
> > introduction of libxenlight, Xen doesn't use vbd for disk backend
> > anymore, it uses qdisk, which I assume Qemu automatically attaches to
> > running guests. It works fine with HVM guests, but it seems to fail
> > with PV guests. The config file I'm using is:
> >
>
> Xen uses qdisk only when blktap is not available.
> I suggest you install blktap if you can because it is significantly
> faster than qdisk at the moment.
This is a NetBSD host so I don't think blktap is an option.
> Otherwise if you use a physical partition and specify phy: in the config
> file you should get the kernel based blkback that is the fastest option
> available.
phy: would be worth a go since it should tie into NetBSD's equivalent of
blkback, whereas file: presumably goes to qdisk in the absence of
blktap.
[...]
> > >From what I see, the guest is unable to find the disk, which is
> > strange, because HVM guests work fine, and the disk is attached
> > correctly. Does this also happen when using file:/ backend (qdisk)
> > under Linux also? Is it a problem with Qemu or is it related to libxl?
>
> It doesn't happen under Linux; it is probably a problem with Qemu. Qemu
> uses the gntdev device (/dev/xen/gntdev) to open the grant table
> (necessary to implement xen backends in userspace).
> There doesn't seem anything equivalent on NetBSD, hence it fails...
That makes some sense. libxl should really detect that qdisk isn't an
option under NetBSD (like it does for blktap) and abort if that is the
case. This could be a case of a simple check in
tools/libxl/libxl_device.c:disk_try_backend. (could be as simple as an
#ifdef __NetBSD__ or we could e.g. add -DHAVE_QEMU_BACKENDS to the build
infrastructure where appropriate).
Presumably if libxl isn't trying to start a qdisk it won't try and use a
qemu based console either (which also won't work under NetBSD) and the
original patch from this thread becomes unnecessary.
Ian.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-20 12:54 ` Ian Campbell
@ 2011-07-20 13:10 ` Roger Pau Monné
2011-07-20 13:30 ` Ian Campbell
0 siblings, 1 reply; 12+ messages in thread
From: Roger Pau Monné @ 2011-07-20 13:10 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, Stefano Stabellini
2011/7/20 Ian Campbell <Ian.Campbell@citrix.com>:
> On Wed, 2011-07-20 at 12:37 +0100, Stefano Stabellini wrote:
>> On Wed, 20 Jul 2011, Roger Pau Monné wrote:
>> > Hello,
>> >
>> > I'm trying to run PV machines using the new libxenlight toolstack on
>> > NetBSD. So far, I've been able to connect to the domu using the
>> > console (I was unable to do so before). I'm attaching a little patch
>> > that removes setting the consback to IOEMU when detecting a qdisk
>> > (that was preventing the domu from even booting). With the
>> > introduction of libxenlight, Xen doesn't use vbd for disk backend
>> > anymore, it uses qdisk, which I assume Qemu automatically attaches to
>> > running guests. It works fine with HVM guests, but it seems to fail
>> > with PV guests. The config file I'm using is:
>> >
>>
>> Xen uses qdisk only when blktap is not available.
>> I suggest you install blktap if you can because it is significantly
>> faster than qdisk at the moment.
>
> This is a NetBSD host so I don't think blktap is an option.
NetBSD has the vnd driver, which provides a disk interface to a file
(it's basically a loop device):
http://netbsd.gw.com/cgi-bin/man-cgi?vnd+4+NetBSD-current
It is used with xend and xenbackendd to mount raw disks. It is not the
fancy blktap2, altough something similar to blktap2 *could* be
implemented *easily* using the RUMP Anykernel I think:
http://www.netbsd.org/docs/rump/index.html
Don't know for sure, but I think it should be possible to port the
blktap2 driver or something very similar using this.
>
>> Otherwise if you use a physical partition and specify phy: in the config
>> file you should get the kernel based blkback that is the fastest option
>> available.
>
> phy: would be worth a go since it should tie into NetBSD's equivalent of
> blkback, whereas file: presumably goes to qdisk in the absence of
> blktap.
Haven't tried phy:/ with unstable, but I supose it works, since it
worked in previous versions.
>
> [...]
>> > >From what I see, the guest is unable to find the disk, which is
>> > strange, because HVM guests work fine, and the disk is attached
>> > correctly. Does this also happen when using file:/ backend (qdisk)
>> > under Linux also? Is it a problem with Qemu or is it related to libxl?
>>
>> It doesn't happen under Linux; it is probably a problem with Qemu. Qemu
>> uses the gntdev device (/dev/xen/gntdev) to open the grant table
>> (necessary to implement xen backends in userspace).
>> There doesn't seem anything equivalent on NetBSD, hence it fails...
>
> That makes some sense. libxl should really detect that qdisk isn't an
> option under NetBSD (like it does for blktap) and abort if that is the
> case. This could be a case of a simple check in
> tools/libxl/libxl_device.c:disk_try_backend. (could be as simple as an
> #ifdef __NetBSD__ or we could e.g. add -DHAVE_QEMU_BACKENDS to the build
> infrastructure where appropriate).
>
> Presumably if libxl isn't trying to start a qdisk it won't try and use a
> qemu based console either (which also won't work under NetBSD) and the
> original patch from this thread becomes unnecessary.
What I don't get is why qdisk work with HVM machines but not with PV
machines, shouldn't it be the same?
>
> Ian.
>
>
Also, I saw that on Linux the loop device was used to mount images in
the past, but with libxl the loop device isn't used anymore right? I
think that using the vnd device is the only way to get file images to
work under NetBSD.
Thanks, Roger.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-20 13:10 ` Roger Pau Monné
@ 2011-07-20 13:30 ` Ian Campbell
2011-07-20 14:27 ` Christoph Egger
0 siblings, 1 reply; 12+ messages in thread
From: Ian Campbell @ 2011-07-20 13:30 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel, Stefano Stabellini
On Wed, 2011-07-20 at 14:10 +0100, Roger Pau Monné wrote:
> 2011/7/20 Ian Campbell <Ian.Campbell@citrix.com>:
> > On Wed, 2011-07-20 at 12:37 +0100, Stefano Stabellini wrote:
> >> On Wed, 20 Jul 2011, Roger Pau Monné wrote:
> >> > Hello,
> >> >
> >> > I'm trying to run PV machines using the new libxenlight toolstack on
> >> > NetBSD. So far, I've been able to connect to the domu using the
> >> > console (I was unable to do so before). I'm attaching a little patch
> >> > that removes setting the consback to IOEMU when detecting a qdisk
> >> > (that was preventing the domu from even booting). With the
> >> > introduction of libxenlight, Xen doesn't use vbd for disk backend
> >> > anymore, it uses qdisk, which I assume Qemu automatically attaches to
> >> > running guests. It works fine with HVM guests, but it seems to fail
> >> > with PV guests. The config file I'm using is:
> >> >
> >>
> >> Xen uses qdisk only when blktap is not available.
> >> I suggest you install blktap if you can because it is significantly
> >> faster than qdisk at the moment.
> >
> > This is a NetBSD host so I don't think blktap is an option.
>
> NetBSD has the vnd driver, which provides a disk interface to a file
> (it's basically a loop device):
>
> http://netbsd.gw.com/cgi-bin/man-cgi?vnd+4+NetBSD-current
>
> It is used with xend and xenbackendd to mount raw disks.
If I understand correctly (which is remotish possibility):
xenbackendd watches the backend/vbd directory and creates disks based on
it. A "vbd" directory roughly corresponds to a "phy:" type device
configuration in libxl.
xend, at least on Linux, internally handles "file:" by setting up a loop
device and translating the config into a "phy:"==vbd refering to
the /dev/loopN device. I suspect that on NetBSD xend used to just create
the vbd referring to the file directly and let xenbackendd take care of
any necessary loop back stuff.
libxl on the other hand does not do this loop device magic but instead
palms "file:" off onto either qdisk or blktap and furthermore libxl
requires that a "phy:" disk configuration refers to a block device and
not a file.
The upshot is that the backend selection logic in libxl is apparently
not correct for NetBSD which _can_ handle files passed as "phy:" due to
xenbackendd doing the right thing via the vnd driver.
Fortunately Ian Jackson recently cleaned up the backend selection logic
in libxl and it should be much cleaner and easier to express these sorts
of system specific things, presumably by patching disk_try_backend to
allow files as well as block devices for LIBXL_DISK_BACKEND_PHY on
NetBSD.
> >> Otherwise if you use a physical partition and specify phy: in the config
> >> file you should get the kernel based blkback that is the fastest option
> >> available.
> >
> > phy: would be worth a go since it should tie into NetBSD's equivalent of
> > blkback, whereas file: presumably goes to qdisk in the absence of
> > blktap.
>
> Haven't tried phy:/ with unstable, but I supose it works, since it
> worked in previous versions.
Since you are passing a file name a block device it is possible that
libxl today might reject it when used with phy:/...
[...]
> What I don't get is why qdisk work with HVM machines but not with PV
> machines, shouldn't it be the same?
I suspect that in the HVM case you aren't really getting a qdisk (a PV
backend) but actually an emulated disk device. Or rather you are
probably getting both but the guest only tries to use the emulated one
and hence the PV backend never tries to initialise and so never falls
over due to lack of gntdev.
Can you confirm if your HVM guest uses emulated or PV frontends?
Ian.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-20 13:30 ` Ian Campbell
@ 2011-07-20 14:27 ` Christoph Egger
2011-07-21 8:39 ` Roger Pau Monné
0 siblings, 1 reply; 12+ messages in thread
From: Christoph Egger @ 2011-07-20 14:27 UTC (permalink / raw)
To: Ian Campbell; +Cc: Roger Pau Monné, xen-devel, Stabellini
On 07/20/11 15:30, Ian Campbell wrote:
> On Wed, 2011-07-20 at 14:10 +0100, Roger Pau Monné wrote:
>> 2011/7/20 Ian Campbell<Ian.Campbell@citrix.com>:
>>> On Wed, 2011-07-20 at 12:37 +0100, Stefano Stabellini wrote:
>>>> On Wed, 20 Jul 2011, Roger Pau Monné wrote:
>>>>> Hello,
>>>>>
>>>>> I'm trying to run PV machines using the new libxenlight toolstack on
>>>>> NetBSD. So far, I've been able to connect to the domu using the
>>>>> console (I was unable to do so before). I'm attaching a little patch
>>>>> that removes setting the consback to IOEMU when detecting a qdisk
>>>>> (that was preventing the domu from even booting). With the
>>>>> introduction of libxenlight, Xen doesn't use vbd for disk backend
>>>>> anymore, it uses qdisk, which I assume Qemu automatically attaches to
>>>>> running guests. It works fine with HVM guests, but it seems to fail
>>>>> with PV guests. The config file I'm using is:
>>>>>
>>>>
>>>> Xen uses qdisk only when blktap is not available.
>>>> I suggest you install blktap if you can because it is significantly
>>>> faster than qdisk at the moment.
>>>
>>> This is a NetBSD host so I don't think blktap is an option.
>>
>> NetBSD has the vnd driver, which provides a disk interface to a file
>> (it's basically a loop device):
>>
>> http://netbsd.gw.com/cgi-bin/man-cgi?vnd+4+NetBSD-current
>>
>> It is used with xend and xenbackendd to mount raw disks.
>
> If I understand correctly (which is remotish possibility):
>
> xenbackendd watches the backend/vbd directory and creates disks based on
> it. A "vbd" directory roughly corresponds to a "phy:" type device
> configuration in libxl.
Correct.
> xend, at least on Linux, internally handles "file:" by setting up a loop
> device and translating the config into a "phy:"==vbd refering to
> the /dev/loopN device. I suspect that on NetBSD xend used to just create
> the vbd referring to the file directly and let xenbackendd take care of
> any necessary loop back stuff.
More precisely, xenbackendd invokes the hotplug scripts and those take
care of choosing a spare vnd device.
> libxl on the other hand does not do this loop device magic but instead
> palms "file:" off onto either qdisk or blktap and furthermore libxl
> requires that a "phy:" disk configuration refers to a block device and
> not a file.
>
> The upshot is that the backend selection logic in libxl is apparently
> not correct for NetBSD which _can_ handle files passed as "phy:" due to
> xenbackendd doing the right thing via the vnd driver.
Right.
> Fortunately Ian Jackson recently cleaned up the backend selection logic
> in libxl and it should be much cleaner and easier to express these sorts
> of system specific things, presumably by patching disk_try_backend to
> allow files as well as block devices for LIBXL_DISK_BACKEND_PHY on
> NetBSD.
Roger: In libxl/libxl_device.c:disk_try_backend() can you try disabling
the second check in the LIBXL_DISK_BACKEND_PHY case, please?
Or surrounding it with #ifdef __Linux__ ... #endif should do it.
Christoph
>>>> Otherwise if you use a physical partition and specify phy: in the config
>>>> file you should get the kernel based blkback that is the fastest option
>>>> available.
>>>
>>> phy: would be worth a go since it should tie into NetBSD's equivalent of
>>> blkback, whereas file: presumably goes to qdisk in the absence of
>>> blktap.
>>
>> Haven't tried phy:/ with unstable, but I supose it works, since it
>> worked in previous versions.
>
> Since you are passing a file name a block device it is possible that
> libxl today might reject it when used with phy:/...
>
> [...]
>> What I don't get is why qdisk work with HVM machines but not with PV
>> machines, shouldn't it be the same?
>
> I suspect that in the HVM case you aren't really getting a qdisk (a PV
> backend) but actually an emulated disk device. Or rather you are
> probably getting both but the guest only tries to use the emulated one
> and hence the PV backend never tries to initialise and so never falls
> over due to lack of gntdev.
>
> Can you confirm if your HVM guest uses emulated or PV frontends?
>
> Ian.
--
---to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Einsteinring 24, 85689 Dornach b. Muenchen
Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-20 14:27 ` Christoph Egger
@ 2011-07-21 8:39 ` Roger Pau Monné
2011-07-21 9:03 ` Ian Campbell
0 siblings, 1 reply; 12+ messages in thread
From: Roger Pau Monné @ 2011-07-21 8:39 UTC (permalink / raw)
To: Christoph Egger; +Cc: Ian Campbell, xen-devel, Stefano, Stabellini
[-- Attachment #1: Type: text/plain, Size: 2455 bytes --]
Hello,
Thanks for the help, I have a preliminary patch that allows booting PV
machines with xl on NetBSD, but there seems to be some issues with PCI
devices, the guest takes a very long time to start, and I think it's
because it's waiting for PCI devices to initialize.
[ 5.280068] XENBUS: Waiting for devices to initialise:
295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s...25s...20s...15s...10s...5s...0s...
[ 300.280078] XENBUS: Device with no driver: device/vbd/51714
[ 300.280086] XENBUS: Device with no driver: device/vbd/51713
[ 300.280495] XENBUS: Timeout connecting to device: device/pci/0
(local state 3, remote state 1)
xm didn't add pci entries to xenstore, and I don't know what should be
the status of those entries, or if it's best to disable the adding of
pci entries to xenstore in libxl if no devices are configured?
I've looked at http://wiki.xensource.com/xenwiki/XenStoreReference,
but the wiki doesn't have any information regarding PCI xenstore
entries.
The status of those entries in xenstore is the following:
/local/domain/0/backend/pci = "" (n0)
/local/domain/0/backend/pci/1 = "" (n0)
/local/domain/0/backend/pci/1/0 = "" (n0,r1)
/local/domain/0/backend/pci/1/0/frontend =
"/local/domain/1/device/pci/0" (n0,r1)
/local/domain/0/backend/pci/1/0/frontend-id = "1" (n0,r1)
/local/domain/0/backend/pci/1/0/online = "1" (n0,r1)
/local/domain/0/backend/pci/1/0/state = "1" (n0,r1)
/local/domain/0/backend/pci/1/0/domain = "debian" (n0,r1)
/local/domain/0/backend/pci/1/0/num_devs = "0" (n0,r1)
/local/domain/1/device/pci = "" (n0,r1)
/local/domain/1/device/pci/0 = "" (n1,r0)
/local/domain/1/device/pci/0/backend =
"/local/domain/0/backend/pci/1/0" (n1,r0)
/local/domain/1/device/pci/0/backend-id = "0" (n1,r0)
/local/domain/1/device/pci/0/state = "3" (n1,r0)
/local/domain/1/device/pci/0/pci-op-ref = "8" (n1,r0)
/local/domain/1/device/pci/0/event-channel = "8" (n1,r0)
/local/domain/1/device/pci/0/magic = "7" (n1,r0)
The patch attached breaks linux support, it is only attached for
informative purposes.
Regards, Roger.
[-- Attachment #2: patch-libxl --]
[-- Type: application/octet-stream, Size: 2533 bytes --]
diff -r e298ce67777e tools/hotplug/NetBSD/block
--- a/tools/hotplug/NetBSD/block Mon Jul 18 14:38:31 2011 +0100
+++ b/tools/hotplug/NetBSD/block Thu Jul 21 12:36:05 2011 +0200
@@ -19,7 +19,8 @@
xpath=$1
xstatus=$2
-xtype=$(xenstore-read "$xpath/type")
+xtype=$3
+#xtype=$(xenstore-read "$xpath/type")
xparams=$(xenstore-read "$xpath/params")
case $xstatus in
diff -r e298ce67777e tools/libxl/libxl_device.c
--- a/tools/libxl/libxl_device.c Mon Jul 18 14:38:31 2011 +0100
+++ b/tools/libxl/libxl_device.c Thu Jul 21 12:36:05 2011 +0200
@@ -136,13 +136,13 @@
a->disk->format == LIBXL_DISK_FORMAT_EMPTY)) {
goto bad_format;
}
- if (a->disk->format != LIBXL_DISK_FORMAT_EMPTY &&
+ /*if (a->disk->format != LIBXL_DISK_FORMAT_EMPTY &&
!S_ISBLK(a->stab.st_mode)) {
LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "Disk vdev=%s, backend phy"
" unsuitable as phys path not a block device",
a->disk->vdev);
return 0;
- }
+ }*/
return backend;
diff -r e298ce67777e tools/xenbackendd/xenbackendd.c
--- a/tools/xenbackendd/xenbackendd.c Mon Jul 18 14:38:31 2011 +0100
+++ b/tools/xenbackendd/xenbackendd.c Thu Jul 21 12:36:05 2011 +0200
@@ -89,15 +89,15 @@
}
static void
-doexec(const char *cmd, const char *arg1, const char *arg2)
+doexec(const char *cmd, const char *arg1, const char *arg2, const char *arg3)
{
- dodebug("exec %s %s %s", cmd, arg1, arg2);
+ dodebug("exec %s %s %s %s", cmd, arg1, arg2, arg3);
switch(vfork()) {
case -1:
dolog(LOG_ERR, "can't vfork: %s", strerror(errno));
break;
case 0:
- execl(cmd, cmd, arg1, arg2, NULL);
+ execl(cmd, cmd, arg1, arg2, arg3, NULL);
dolog(LOG_ERR, "can't exec %s: %s", cmd, strerror(errno));
exit(EXIT_FAILURE);
/* NOTREACHED */
@@ -150,6 +150,8 @@
char *s;
int state;
char *sstate;
+ char *params;
+ char *stype;
char *p;
char buf[80];
int type;
@@ -297,11 +299,18 @@
strerror(errno));
goto next2;
}
- doexec(s, vec[XS_WATCH_PATH], sstate);
+ doexec(s, vec[XS_WATCH_PATH], sstate, NULL);
break;
case DEVTYPE_VBD:
- doexec(vbd_script, vec[XS_WATCH_PATH], sstate);
+ snprintf(buf, sizeof(buf), "%s/params", vec[XS_WATCH_PATH]);
+ params = xs_read(xs, XBT_NULL, buf, 0);
+ if (strncmp(params,
+ "/dev",
+ strlen(params)) == 0)
+ stype = "phy";
+ stype = "file";
+ doexec(vbd_script, vec[XS_WATCH_PATH], sstate, stype);
break;
default:
[-- Attachment #3: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-21 8:39 ` Roger Pau Monné
@ 2011-07-21 9:03 ` Ian Campbell
2011-07-21 9:19 ` Roger Pau Monné
0 siblings, 1 reply; 12+ messages in thread
From: Ian Campbell @ 2011-07-21 9:03 UTC (permalink / raw)
To: Roger Pau Monné
Cc: Christoph Egger, xen-devel, Stefano, Stefano Stabellini
On Thu, 2011-07-21 at 09:39 +0100, Roger Pau Monné wrote:
> Hello,
>
> Thanks for the help, I have a preliminary patch that allows booting PV
> machines with xl on NetBSD, but there seems to be some issues with PCI
> devices, the guest takes a very long time to start, and I think it's
> because it's waiting for PCI devices to initialize.
>
> [ 5.280068] XENBUS: Waiting for devices to initialise:
> 295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s...25s...20s...15s...10s...5s...0s...
> [ 300.280078] XENBUS: Device with no driver: device/vbd/51714
> [ 300.280086] XENBUS: Device with no driver: device/vbd/51713
> [ 300.280495] XENBUS: Timeout connecting to device: device/pci/0
> (local state 3, remote state 1)
>
> xm didn't add pci entries to xenstore, and I don't know what should be
> the status of those entries, or if it's best to disable the adding of
> pci entries to xenstore in libxl if no devices are configured?
>
> I've looked at http://wiki.xensource.com/xenwiki/XenStoreReference,
> but the wiki doesn't have any information regarding PCI xenstore
> entries.
>
> The status of those entries in xenstore is the following:
>
> /local/domain/0/backend/pci = "" (n0)
> /local/domain/0/backend/pci/1 = "" (n0)
> /local/domain/0/backend/pci/1/0 = "" (n0,r1)
> /local/domain/0/backend/pci/1/0/frontend =
> "/local/domain/1/device/pci/0" (n0,r1)
> /local/domain/0/backend/pci/1/0/frontend-id = "1" (n0,r1)
> /local/domain/0/backend/pci/1/0/online = "1" (n0,r1)
> /local/domain/0/backend/pci/1/0/state = "1" (n0,r1)
> /local/domain/0/backend/pci/1/0/domain = "debian" (n0,r1)
> /local/domain/0/backend/pci/1/0/num_devs = "0" (n0,r1)
num_devs = 0, is it the case that you don't actually have any pci
devices in your configuration?
I wonder if perhaps the netbsd pciback doesn't like seeing a backend
directory with no actual devices in it.
Could you try experimentally commenting out the call to
libxl__create_pci_backend in do_domain_create, or gating it on "if
(d_config->num_pcidevs)".
That call was added in 23565:72eafe80ebc1 but I don't think adding the
if would invalidate it.
> /local/domain/1/device/pci = "" (n0,r1)
> /local/domain/1/device/pci/0 = "" (n1,r0)
> /local/domain/1/device/pci/0/backend =
> "/local/domain/0/backend/pci/1/0" (n1,r0)
> /local/domain/1/device/pci/0/backend-id = "0" (n1,r0)
> /local/domain/1/device/pci/0/state = "3" (n1,r0)
> /local/domain/1/device/pci/0/pci-op-ref = "8" (n1,r0)
> /local/domain/1/device/pci/0/event-channel = "8" (n1,r0)
> /local/domain/1/device/pci/0/magic = "7" (n1,r0)
>
> The patch attached breaks linux support, it is only attached for
> informative purposes.
>
> Regards, Roger.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-21 9:03 ` Ian Campbell
@ 2011-07-21 9:19 ` Roger Pau Monné
2011-07-21 9:39 ` Roger Pau Monné
2011-07-21 9:42 ` Ian Campbell
0 siblings, 2 replies; 12+ messages in thread
From: Roger Pau Monné @ 2011-07-21 9:19 UTC (permalink / raw)
To: Ian Campbell; +Cc: Christoph Egger, xen-devel, Stefano, Stefano Stabellini
2011/7/21 Ian Campbell <Ian.Campbell@eu.citrix.com>:
> On Thu, 2011-07-21 at 09:39 +0100, Roger Pau Monné wrote:
>> Hello,
>>
>> Thanks for the help, I have a preliminary patch that allows booting PV
>> machines with xl on NetBSD, but there seems to be some issues with PCI
>> devices, the guest takes a very long time to start, and I think it's
>> because it's waiting for PCI devices to initialize.
>>
>> [ 5.280068] XENBUS: Waiting for devices to initialise:
>> 295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s...25s...20s...15s...10s...5s...0s...
>> [ 300.280078] XENBUS: Device with no driver: device/vbd/51714
>> [ 300.280086] XENBUS: Device with no driver: device/vbd/51713
>> [ 300.280495] XENBUS: Timeout connecting to device: device/pci/0
>> (local state 3, remote state 1)
>>
>> xm didn't add pci entries to xenstore, and I don't know what should be
>> the status of those entries, or if it's best to disable the adding of
>> pci entries to xenstore in libxl if no devices are configured?
>>
>> I've looked at http://wiki.xensource.com/xenwiki/XenStoreReference,
>> but the wiki doesn't have any information regarding PCI xenstore
>> entries.
>>
>> The status of those entries in xenstore is the following:
>>
>> /local/domain/0/backend/pci = "" (n0)
>> /local/domain/0/backend/pci/1 = "" (n0)
>> /local/domain/0/backend/pci/1/0 = "" (n0,r1)
>> /local/domain/0/backend/pci/1/0/frontend =
>> "/local/domain/1/device/pci/0" (n0,r1)
>> /local/domain/0/backend/pci/1/0/frontend-id = "1" (n0,r1)
>> /local/domain/0/backend/pci/1/0/online = "1" (n0,r1)
>> /local/domain/0/backend/pci/1/0/state = "1" (n0,r1)
>> /local/domain/0/backend/pci/1/0/domain = "debian" (n0,r1)
>> /local/domain/0/backend/pci/1/0/num_devs = "0" (n0,r1)
>
> num_devs = 0, is it the case that you don't actually have any pci
> devices in your configuration?
No, I didn't have any PCI devices set.
>
> I wonder if perhaps the netbsd pciback doesn't like seeing a backend
> directory with no actual devices in it.
>
> Could you try experimentally commenting out the call to
> libxl__create_pci_backend in do_domain_create, or gating it on "if
> (d_config->num_pcidevs)".
Disabling or setting it with a if condition makes the guest boot, I
through that setting the appropriate values in the status of xenstore
should also work, but it's useless to have references to PCI devices
in xenstore if no devices are actually configured.
>
> That call was added in 23565:72eafe80ebc1 but I don't think adding the
> if would invalidate it.
>
>> /local/domain/1/device/pci = "" (n0,r1)
>> /local/domain/1/device/pci/0 = "" (n1,r0)
>> /local/domain/1/device/pci/0/backend =
>> "/local/domain/0/backend/pci/1/0" (n1,r0)
>> /local/domain/1/device/pci/0/backend-id = "0" (n1,r0)
>> /local/domain/1/device/pci/0/state = "3" (n1,r0)
>> /local/domain/1/device/pci/0/pci-op-ref = "8" (n1,r0)
>> /local/domain/1/device/pci/0/event-channel = "8" (n1,r0)
>> /local/domain/1/device/pci/0/magic = "7" (n1,r0)
>>
>> The patch attached breaks linux support, it is only attached for
>> informative purposes.
>>
>> Regards, Roger.
Thanks again for all the help, I would prepare a new proper patch to
be added to mainline Xen.
Regards, Roger.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-21 9:19 ` Roger Pau Monné
@ 2011-07-21 9:39 ` Roger Pau Monné
2011-07-21 10:36 ` Ian Campbell
2011-07-21 9:42 ` Ian Campbell
1 sibling, 1 reply; 12+ messages in thread
From: Roger Pau Monné @ 2011-07-21 9:39 UTC (permalink / raw)
To: Ian Campbell; +Cc: Christoph Egger, xen-devel, Stefano, Stefano Stabellini
BTW, would it be ok to create a new disk backend, like LOOP or VND
(LIBXL_DISK_BACKEND_VND)?
Now it's a bit hackish, since type is always set to phy, I have to
guess the real type by checking if 'params' starts with '/dev' on
xenbackend. I think it would be clearer to set the type to
LIBXL_DISK_BACKEND_VND if passed file is not a block device and OS is
NetBSD in disk_try_backend.
Regards, Roger.
2011/7/21 Roger Pau Monné <roger.pau@entel.upc.edu>:
> 2011/7/21 Ian Campbell <Ian.Campbell@eu.citrix.com>:
>> On Thu, 2011-07-21 at 09:39 +0100, Roger Pau Monné wrote:
>>> Hello,
>>>
>>> Thanks for the help, I have a preliminary patch that allows booting PV
>>> machines with xl on NetBSD, but there seems to be some issues with PCI
>>> devices, the guest takes a very long time to start, and I think it's
>>> because it's waiting for PCI devices to initialize.
>>>
>>> [ 5.280068] XENBUS: Waiting for devices to initialise:
>>> 295s...290s...285s...280s...275s...270s...265s...260s...255s...250s...245s...240s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s...25s...20s...15s...10s...5s...0s...
>>> [ 300.280078] XENBUS: Device with no driver: device/vbd/51714
>>> [ 300.280086] XENBUS: Device with no driver: device/vbd/51713
>>> [ 300.280495] XENBUS: Timeout connecting to device: device/pci/0
>>> (local state 3, remote state 1)
>>>
>>> xm didn't add pci entries to xenstore, and I don't know what should be
>>> the status of those entries, or if it's best to disable the adding of
>>> pci entries to xenstore in libxl if no devices are configured?
>>>
>>> I've looked at http://wiki.xensource.com/xenwiki/XenStoreReference,
>>> but the wiki doesn't have any information regarding PCI xenstore
>>> entries.
>>>
>>> The status of those entries in xenstore is the following:
>>>
>>> /local/domain/0/backend/pci = "" (n0)
>>> /local/domain/0/backend/pci/1 = "" (n0)
>>> /local/domain/0/backend/pci/1/0 = "" (n0,r1)
>>> /local/domain/0/backend/pci/1/0/frontend =
>>> "/local/domain/1/device/pci/0" (n0,r1)
>>> /local/domain/0/backend/pci/1/0/frontend-id = "1" (n0,r1)
>>> /local/domain/0/backend/pci/1/0/online = "1" (n0,r1)
>>> /local/domain/0/backend/pci/1/0/state = "1" (n0,r1)
>>> /local/domain/0/backend/pci/1/0/domain = "debian" (n0,r1)
>>> /local/domain/0/backend/pci/1/0/num_devs = "0" (n0,r1)
>>
>> num_devs = 0, is it the case that you don't actually have any pci
>> devices in your configuration?
>
> No, I didn't have any PCI devices set.
>
>>
>> I wonder if perhaps the netbsd pciback doesn't like seeing a backend
>> directory with no actual devices in it.
>>
>> Could you try experimentally commenting out the call to
>> libxl__create_pci_backend in do_domain_create, or gating it on "if
>> (d_config->num_pcidevs)".
>
> Disabling or setting it with a if condition makes the guest boot, I
> through that setting the appropriate values in the status of xenstore
> should also work, but it's useless to have references to PCI devices
> in xenstore if no devices are actually configured.
>
>>
>> That call was added in 23565:72eafe80ebc1 but I don't think adding the
>> if would invalidate it.
>>
>>> /local/domain/1/device/pci = "" (n0,r1)
>>> /local/domain/1/device/pci/0 = "" (n1,r0)
>>> /local/domain/1/device/pci/0/backend =
>>> "/local/domain/0/backend/pci/1/0" (n1,r0)
>>> /local/domain/1/device/pci/0/backend-id = "0" (n1,r0)
>>> /local/domain/1/device/pci/0/state = "3" (n1,r0)
>>> /local/domain/1/device/pci/0/pci-op-ref = "8" (n1,r0)
>>> /local/domain/1/device/pci/0/event-channel = "8" (n1,r0)
>>> /local/domain/1/device/pci/0/magic = "7" (n1,r0)
>>>
>>> The patch attached breaks linux support, it is only attached for
>>> informative purposes.
>>>
>>> Regards, Roger.
>
> Thanks again for all the help, I would prepare a new proper patch to
> be added to mainline Xen.
>
> Regards, Roger.
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-21 9:19 ` Roger Pau Monné
2011-07-21 9:39 ` Roger Pau Monné
@ 2011-07-21 9:42 ` Ian Campbell
1 sibling, 0 replies; 12+ messages in thread
From: Ian Campbell @ 2011-07-21 9:42 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: Christoph Egger, xen-devel, Stefano Stabellini
On Thu, 2011-07-21 at 10:19 +0100, Roger Pau Monné wrote:
> 2011/7/21 Ian Campbell <Ian.Campbell@eu.citrix.com>:
> > On Thu, 2011-07-21 at 09:39 +0100, Roger Pau Monné wrote:
> >
> > I wonder if perhaps the netbsd pciback doesn't like seeing a backend
> > directory with no actual devices in it.
> >
> > Could you try experimentally commenting out the call to
> > libxl__create_pci_backend in do_domain_create, or gating it on "if
> > (d_config->num_pcidevs)".
>
> Disabling or setting it with a if condition makes the guest boot, I
> through that setting the appropriate values in the status of xenstore
> should also work, but it's useless to have references to PCI devices
> in xenstore if no devices are actually configured.
There was a tricky interaction between the support for hotplugging PCI
devices and adding them one-by-one to a non-running guest during domain
build (the latter confuses the Linux pciback because it waits for the
frontend to respond to the reconfiguration requests for the second and
subsequent devices added and the guest ain't running). At the time we
made the creation of the pcibackend xenstore directory unconditional
because it simplified things
However I think enough fixes have accumulated since then that we can
switch to only creating the backend if the guest has any pci devices
configured and hotplug will still work once the guest is unpaused. (IOW
I think adding the if (...) is correct and safe but we should be mindful
of breaking hotplug)
> >
> > That call was added in 23565:72eafe80ebc1 but I don't think adding the
> > if would invalidate it.
> >
> >> /local/domain/1/device/pci = "" (n0,r1)
> >> /local/domain/1/device/pci/0 = "" (n1,r0)
> >> /local/domain/1/device/pci/0/backend =
> >> "/local/domain/0/backend/pci/1/0" (n1,r0)
> >> /local/domain/1/device/pci/0/backend-id = "0" (n1,r0)
> >> /local/domain/1/device/pci/0/state = "3" (n1,r0)
> >> /local/domain/1/device/pci/0/pci-op-ref = "8" (n1,r0)
> >> /local/domain/1/device/pci/0/event-channel = "8" (n1,r0)
> >> /local/domain/1/device/pci/0/magic = "7" (n1,r0)
> >>
> >> The patch attached breaks linux support, it is only attached for
> >> informative purposes.
> >>
> >> Regards, Roger.
>
> Thanks again for all the help, I would prepare a new proper patch to
> be added to mainline Xen.
That would be great, thanks.
WRT the proper fix I think rather than sprinkling ifdef __NetBSD__ and
__Linux__ about it would preferable to collect that logic together in
one place (maybe in libxl_osdeps.h?) and define meaningful tags
(HAVE_DISK_PHY_FILE_SUPPORT and HAVE_DISK_BLKTAP_SUPPORT or something
better) to use in the actual code.
Ian
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: libxl: problem with devices in PV
2011-07-21 9:39 ` Roger Pau Monné
@ 2011-07-21 10:36 ` Ian Campbell
0 siblings, 0 replies; 12+ messages in thread
From: Ian Campbell @ 2011-07-21 10:36 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: Christoph Egger, xen-devel, Stefano Stabellini
On Thu, 2011-07-21 at 10:39 +0100, Roger Pau Monné wrote:
> BTW, would it be ok to create a new disk backend, like LOOP or VND
> (LIBXL_DISK_BACKEND_VND)?
>
> Now it's a bit hackish, since type is always set to phy, I have to
> guess the real type by checking if 'params' starts with '/dev' on
> xenbackend. I think it would be clearer to set the type to
> LIBXL_DISK_BACKEND_VND if passed file is not a block device and OS is
> NetBSD in disk_try_backend.
The VND stuff is transparent to the toolstack though, it's internal to
xenbackendd and the hotplug scripts, all the toolstack sees is a vbd
directory in xenstore. So I think it would be better to stick with the
phy/vbd types within the toolstack.
Can the code in disk_try_backend become:
[in some suitable central location:]
#if __NetBSD__
#define HAVE_PHY_BACKNED_FILE_SUPPORT
#endif
[...]
case LIBXL_DISK_BACKEND_PHY:
if (!(a->disk->format == LIBXL_DISK_FORMAT_RAW ||
a->disk->format == LIBXL_DISK_FORMAT_EMPTY)) {
goto bad_format;
}
if (a->disk->format == LIBXL_DISK_FORMAT_EMPTY)
return backend;
#ifdef HAVE_PHY_BACKEND_FILE_SUPPORT
if (S_ISREG(a->stab.st_mode)) /* S_ISLNK? */
return backend;
#endif
if (S_ISBLK(a->stab.st_mode))
return backend;
LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "Disk vdev=%s, backend phy"
" unsuitable as phys path not a block device"
#ifdef HAVE_PHY_...
" nor a regular file"
#endif
, a->disk->vdev);
return 0;
? (perhaps the log message construction could be done cleaner)
Ian.
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2011-07-21 10:36 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-07-20 11:01 libxl: problem with devices in PV Roger Pau Monné
2011-07-20 11:37 ` Stefano Stabellini
2011-07-20 12:54 ` Ian Campbell
2011-07-20 13:10 ` Roger Pau Monné
2011-07-20 13:30 ` Ian Campbell
2011-07-20 14:27 ` Christoph Egger
2011-07-21 8:39 ` Roger Pau Monné
2011-07-21 9:03 ` Ian Campbell
2011-07-21 9:19 ` Roger Pau Monné
2011-07-21 9:39 ` Roger Pau Monné
2011-07-21 10:36 ` Ian Campbell
2011-07-21 9:42 ` Ian Campbell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.