[Qemu-devel] qemu.git hangs booting Linux after insmod virtio_blk.ko

[Qemu-devel] qemu.git hangs booting Linux after insmod virtio_blk.ko

[Richard W.M. Jones]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]


I've not looked into this at all, it's just a report that something
seems to be "up".  I will try to git bisect this later if no one spots
anything obvious.

The next operation after insmod virtio_blk would be insmod_virtio_net.

Guest kernel is a Fedora kernel, version 3.1.0-0.rc6.git0.3.fc16.x86_64.

The ImExPS/2 message may or may not be coincidental.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

[-- Attachment #2: log --]
[-- Type: text/plain, Size: 15371 bytes --]

libguestfs: [00000ms] febootstrap-supermin-helper --verbose -f checksum '/home/rjones/d/libguestfs/appliance/supermin.d' x86_64
supermin helper [00000ms] whitelist = (not specified), host_cpu = x86_64, kernel = (null), initrd = (null), appliance = (null)
supermin helper [00000ms] inputs[0] = /home/rjones/d/libguestfs/appliance/supermin.d
checking modpath /lib/modules/3.1.0-0.rc6.git0.0.fc16.x86_64 is a directory
picked vmlinuz-3.1.0-0.rc6.git0.0.fc16.x86_64 because modpath /lib/modules/3.1.0-0.rc6.git0.0.fc16.x86_64 exists
checking modpath /lib/modules/3.1.0-0.rc3.git0.0.fc16.x86_64 is a directory
picked vmlinuz-3.1.0-0.rc3.git0.0.fc16.x86_64 because modpath /lib/modules/3.1.0-0.rc3.git0.0.fc16.x86_64 exists
checking modpath /lib/modules/3.1.0-0.rc6.git0.3.fc16.x86_64 is a directory
picked vmlinuz-3.1.0-0.rc6.git0.3.fc16.x86_64 because modpath /lib/modules/3.1.0-0.rc6.git0.3.fc16.x86_64 exists
supermin helper [00000ms] finished creating kernel
supermin helper [00000ms] visiting /home/rjones/d/libguestfs/appliance/supermin.d
supermin helper [00000ms] visiting /home/rjones/d/libguestfs/appliance/supermin.d/base.img
supermin helper [00000ms] visiting /home/rjones/d/libguestfs/appliance/supermin.d/daemon.img
supermin helper [00000ms] visiting /home/rjones/d/libguestfs/appliance/supermin.d/hostfiles
supermin helper [00028ms] visiting /home/rjones/d/libguestfs/appliance/supermin.d/init.img
supermin helper [00112ms] finished creating appliance
libguestfs: [00118ms] begin testing qemu features
libguestfs: [00150ms] finished testing qemu features
libguestfs: accept_from_daemon: 0x1b6ad20 g->state = 1
[00151ms] /home/rjones/d/qemu/qemu.wrapper \
    -drive file=/dev/null,if=virtio \
    -nodefconfig \
    -machine accel=kvm:tcg \
    -nodefaults \
    -nographic \
    -m 500 \
    -no-reboot \
    -no-hpet \
    -device virtio-serial \
    -serial stdio \
    -chardev socket,path=/home/rjones/d/libguestfs/libguestfsunnq4X/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -kernel /home/rjones/d/libguestfs/.guestfs-500/kernel.26648 \
    -initrd /home/rjones/d/libguestfs/.guestfs-500/initrd.26648 \
    -append 'panic=1 console=ttyS0 udevtimeout=300 no_timer_check acpi=off printk.time=1 cgroup_disable=memory selinux=0 guestfs_verbose=1 TERM=xterm ' \
    -drive file=/home/rjones/d/libguestfs/.guestfs-500/root.26648,snapshot=on,if=virtio,cache=unsafeCould not open option rom 'sgabios.bin': No such file or directory
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 3.1.0-0.rc6.git0.3.fc16.x86_64 (mockbuild@x86-02.phx2.fedoraproject.org) (gcc version 4.6.1 20110824 (Red Hat 4.6.1-8) (GCC) ) #1 SMP Fri Sep 16 12:26:22 UTC 2011
[    0.000000] Command line: panic=1 console=ttyS0 udevtimeout=300 no_timer_check acpi=off printk.time=1 cgroup_disable=memory selinux=0 guestfs_verbose=1 TERM=xterm 
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009bc00 (usable)
[    0.000000]  BIOS-e820: 000000000009bc00 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 000000001f3fd000 (usable)
[    0.000000]  BIOS-e820: 000000001f3fd000 - 000000001f400000 (reserved)
[    0.000000]  BIOS-e820: 00000000feffc000 - 00000000ff000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.4 present.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x1f3fd max_arch_pfn = 0x400000000
[    0.000000] PAT not supported by CPU.
[    0.000000] found SMP MP-table at [ffff8800000fda70] fda70
[    0.000000] init_memory_mapping: 0000000000000000-000000001f3fd000
[    0.000000] RAMDISK: 1f2ed000 - 1f3f0000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000001f3fd000
[    0.000000] Initmem setup node 0 0000000000000000-000000001f3fd000
[    0.000000]   NODE_DATA [000000001f2d9000 - 000000001f2ecfff]
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 0:1b74f01, boot clock
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009b
[    0.000000]     0: 0x00000100 -> 0x0001f3fd
[    0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org
[    0.000000] Intel MultiProcessor Specification v1.4
[    0.000000] MPTABLE: OEM ID: BOCHSCPU
[    0.000000] MPTABLE: Product ID: 0.1         
[    0.000000] MPTABLE: APIC at: 0xFEE00000
[    0.000000] Processor #0 (Bootup-CPU)
[    0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
[    0.000000] Processors: 1
[    0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] PM: Registered nosave memory: 000000000009b000 - 000000000009c000
[    0.000000] PM: Registered nosave memory: 000000000009c000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
[    0.000000] PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 1f400000 (gap: 1f400000:dfbfc000)
[    0.000000] Booting paravirtualized kernel on KVM
[    0.000000] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:1 nr_node_ids:1
[    0.000000] PERCPU: Embedded 27 pages/cpu @ffff88001f000000 s81024 r8192 d21376 u2097152
[    0.000000] kvm-clock: cpu 0, msr 0:1f012f01, primary cpu clock
[    0.000000] KVM setup async PF for cpu 0
[    0.000000] kvm-stealtime: cpu 0, msr 1f00d700
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 125875
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line: panic=1 console=ttyS0 udevtimeout=300 no_timer_check acpi=off printk.time=1 cgroup_disable=memory selinux=0 guestfs_verbose=1 TERM=xterm 
[    0.000000] Disabling memory control group subsystem
[    0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 487824k/511988k available (4866k kernel code, 468k absent, 23696k reserved, 6783k data, 940k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] \tRCU dyntick-idle grace-period acceleration is enabled.
[    0.000000] NR_IRQS:16640 nr_irqs:256 16
[    0.000000] Console: colour *CGA 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] Detected 2659.998 MHz processor.
[    0.000999] Calibrating delay loop (skipped) preset value.. 5319.99 BogoMIPS (lpj=2659998)
[    0.001023] pid_max: default: 32768 minimum: 301
[    0.001564] Security Framework initialized
[    0.002031] SELinux:  Disabled at boot.
[    0.002565] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.003118] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.004051] Mount-cache hash table entries: 256
[    0.004500] Initializing cgroup subsys cpuacct
[    0.004812] Initializing cgroup subsys memory
[    0.005016] Initializing cgroup subsys devices
[    0.005296] Initializing cgroup subsys freezer
[    0.005612] Initializing cgroup subsys net_cls
[    0.006006] Initializing cgroup subsys blkio
[    0.006279] Initializing cgroup subsys perf_event
[    0.006649] mce: CPU supports 10 MCE banks
[    0.007139] SMP alternatives: switching to UP code
[    0.013494] Freeing SMP alternatives: 12k freed
[    0.013847] ftrace: allocating 25203 entries in 99 pages
[    0.016806] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.017002] CPU0: Intel QEMU Virtual CPU version 0.15.50 stepping 03
[    0.119301] Performance Events: unsupported p6 CPU model 2 no PMU driver, software events only.
[    0.120132] NMI watchdog disabled (cpu0): hardware events not enabled
[    0.120619] Brought up 1 CPUs
[    0.120828] Total of 1 processors activated (5319.99 BogoMIPS).
[    0.121469] devtmpfs: initialized
[    0.125600] atomic64 test passed for x86-64 platform with CX8 and with SSE
[    0.126036] RTC time: 12:25:30, date: 09/30/11
[    0.126393] NET: Registered protocol family 16
[    0.127146] PCI: Using configuration type 1 for base access
[    0.128325] bio: create slab <bio-0> at 0
[    0.128667] ACPI: Interpreter disabled.
[    0.129006] vgaarb: loaded
[    0.129246] SCSI subsystem initialized
[    0.129580] usbcore: registered new interface driver usbfs
[    0.130003] usbcore: registered new interface driver hub
[    0.130421] usbcore: registered new device driver usb
[    0.130778] PCI: Probing PCI hardware
[    0.132369] pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
[    0.132838] pci 0000:00:01.3: quirk: [io  0xb100-0xb10f] claimed by PIIX4 SMB
[    0.140668] pci 0000:00:01.0: PIIX/ICH IRQ router [8086:7000]
[    0.141198] NetLabel: Initializing
[    0.141434] NetLabel:  domain hash size = 128
[    0.141715] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.142003] NetLabel:  unlabeled traffic allowed by default
[    0.142373] Switching to clocksource kvm-clock
[    0.143983] Switched to NOHz mode on CPU #0
[    0.149215] pnp: PnP ACPI: disabled
[    0.150955] NET: Registered protocol family 2
[    0.151449] IP route cache hash table entries: 4096 (order: 3, 32768 bytes)
[    0.151970] TCP established hash table entries: 16384 (order: 6, 262144 bytes)
[    0.152961] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
[    0.153799] TCP: Hash tables configured (established 16384 bind 16384)
[    0.154506] TCP reno registered
[    0.154710] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    0.155250] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    0.155960] NET: Registered protocol family 1
[    0.156409] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    0.156785] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    0.157390] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    0.158081] Unpacking initramfs...
[    0.159554] Freeing initrd memory: 1036k freed
[    0.160195] platform rtc_cmos: registered platform RTC device (no PNP device found)
[    0.161020] Intel AES-NI instructions are not detected.
[    0.161704] audit: initializing netlink socket (disabled)
[    0.162180] type=2000 audit(1317385531.161:1): initialized
[    0.181823] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.183870] VFS: Disk quotas dquot_6.5.2
[    0.184253] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.185201] msgmni has been set to 954
[    0.185770] alg: No test for stdrng (krng)
[    0.186091] NET: Registered protocol family 38
[    0.186454] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    0.187199] io scheduler noop registered
[    0.187488] io scheduler deadline registered
[    0.187784] io scheduler cfq registered (default)
[    0.188355] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.188884] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    0.189412] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.190203] virtio-pci 0000:00:02.0: PCI->APIC IRQ transform: INT A -> IRQ 34
[    0.190963] virtio-pci 0000:00:03.0: PCI->APIC IRQ transform: INT A -> IRQ 35
[    0.191869] virtio-pci 0000:00:04.0: PCI->APIC IRQ transform: INT A -> IRQ 35
[    0.192789] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.214514] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    0.220269] Non-volatile memory driver v1.3
[    0.220660] Linux agpgart interface v0.103
[    0.221586] loop: module loaded
[    0.222646] scsi0 : ata_piix
[    0.222929] scsi1 : ata_piix
[    0.223282] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
[    0.223910] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
[    0.224703] Fixed MDIO Bus: probed
[    0.224979] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.225876] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.226506] uhci_hcd: USB Universal Host Controller Interface driver
[    0.227139] usbcore: registered new interface driver usbserial
[    0.227846] USB Serial support registered for generic
[    0.228350] usbcore: registered new interface driver usbserial_generic
[    0.228993] usbserial: USB Serial Driver core
[    0.229515] i8042: PNP: No PS/2 controller found. Probing ports directly.
[    0.230885] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.231460] serio: i8042 AUX port at 0x60,0x64 irq 12
[    0.231858] mousedev: PS/2 mouse device common for all mice
[    0.232758] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0
[    0.236537] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.237132] rtc0: alarms up to one day, 114 bytes nvram
[    0.237616] device-mapper: uevent: version 1.0.3
[    0.238322] device-mapper: ioctl: 4.21.0-ioctl (2011-07-06) initialised: dm-devel@redhat.com
[    0.239153] cpuidle: using governor ladder
[    0.239651] cpuidle: using governor menu
[    0.240181] EFI Variables Facility v0.08 2004-May-17
[    0.240672] usbcore: registered new interface driver usbhid
[    0.241133] usbhid: USB HID core driver
[    0.241759] ip_tables: (C) 2000-2006 Netfilter Core Team
[    0.242255] TCP cubic registered
[    0.242667] Initializing XFRM netlink socket
[    0.243254] NET: Registered protocol family 10
[    0.243805] Mobile IPv6
[    0.243981] NET: Registered protocol family 17
[    0.244998] Registering the dns_resolver key type
[    0.245591] registered taskstats version 1
[    0.245998] IMA: No TPM chip found, activating TPM-bypass!
[    0.246787]   Magic number: 11:918:431
[    0.247225] rtc_cmos rtc_cmos: setting system clock to 2011-09-30 12:25:30 UTC (1317385530)
[    0.248188] Initializing network drop monitor service
[    0.402221] Freeing unused kernel memory: 940k freed
[    0.403955] Write protecting the kernel read-only data: 10240k
[    0.415286] Freeing unused kernel memory: 1260k freed
[    0.421039] Freeing unused kernel memory: 1584k freed
febootstrap: mounting /proc
febootstrap: uptime: 0.42 0.16
febootstrap: ext2 mini initrd starting up
febootstrap: mounting /sys
febootstrap: internal insmod libcrc32c.ko
febootstrap: internal insmod crc32c-intel.ko
insmod: init_module: crc32c-intel.ko: No such device
febootstrap: internal insmod crc-itu-t.ko
febootstrap: internal insmod crc-ccitt.ko
febootstrap: internal insmod crc7.ko
febootstrap: internal insmod crc8.ko
febootstrap: internal insmod scsi_transport_spi.ko
febootstrap: internal insmod sym53c8xx.ko
febootstrap: internal insmod rfkill.ko
febootstrap: internal insmod sparse-keymap.ko
febootstrap: internal insmod ideapad-laptop.ko
insmod: init_module: ideapad-laptop.ko: No such device
febootstrap: internal insmod virtio_balloon.ko
febootstrap: internal insmod virtio-rng.ko
febootstrap: internal insmod virtio_blk.ko
[    0.669479] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input1

[Qemu-devel] git-bisect results (was: Re: qemu.git hangs booting Linux after insmod virtio_blk.ko)

[Richard W.M. Jones]

On Fri, Sep 30, 2011 at 02:11:48PM +0100, Richard W.M. Jones wrote:
> 
> I've not looked into this at all, it's just a report that something
> seems to be "up".  I will try to git bisect this later if no one spots
> anything obvious.
> 
> The next operation after insmod virtio_blk would be insmod_virtio_net.
> 
> Guest kernel is a Fedora kernel, version 3.1.0-0.rc6.git0.3.fc16.x86_64.
> 
> The ImExPS/2 message may or may not be coincidental.

git-bisect points to:

d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09 is the first bad commit
commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
Author: Gerd Hoffmann <kraxel@redhat.com>
Date:   Wed Aug 10 17:34:13 2011 +0200

    seabios: update to master
    
    commit 8e301472e324b6d6496d8b4ffc66863e99d7a505
    
    user visible changes in seabios:
      * ahci is enabled by default (and thus in this build).
      * bootorder support for ahci.
      * two-pass pci allocator (orders bars by size for better packing).
    
    Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

:040000 040000 76eb0c81b76563b55cb2bb5c484ccd48b8cfcded 5ec0d65d3a763a5566fe1f4c86269cad6d671020 M	pc-bios
:040000 040000 a5a7ea6e297c1e7490b0a2c28a06ce56e5be9449 78adb664d3ea82f1a4dd5ec239887ac5b0168a7f M	roms
bisect run success

Rich.

[...]
> [    0.248188] Initializing network drop monitor service
> [    0.402221] Freeing unused kernel memory: 940k freed
> [    0.403955] Write protecting the kernel read-only data: 10240k
> [    0.415286] Freeing unused kernel memory: 1260k freed
> [    0.421039] Freeing unused kernel memory: 1584k freed
> febootstrap: mounting /proc
> febootstrap: uptime: 0.42 0.16
> febootstrap: ext2 mini initrd starting up
> febootstrap: mounting /sys
> febootstrap: internal insmod libcrc32c.ko
> febootstrap: internal insmod crc32c-intel.ko
> insmod: init_module: crc32c-intel.ko: No such device
> febootstrap: internal insmod crc-itu-t.ko
> febootstrap: internal insmod crc-ccitt.ko
> febootstrap: internal insmod crc7.ko
> febootstrap: internal insmod crc8.ko
> febootstrap: internal insmod scsi_transport_spi.ko
> febootstrap: internal insmod sym53c8xx.ko
> febootstrap: internal insmod rfkill.ko
> febootstrap: internal insmod sparse-keymap.ko
> febootstrap: internal insmod ideapad-laptop.ko
> insmod: init_module: ideapad-laptop.ko: No such device
> febootstrap: internal insmod virtio_balloon.ko
> febootstrap: internal insmod virtio-rng.ko
> febootstrap: internal insmod virtio_blk.ko
> [    0.669479] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input1

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

Re: [Qemu-devel] git-bisect results (was: Re: qemu.git hangs booting Linux after insmod virtio_blk.ko)

[Richard W.M. Jones]

On Fri, Sep 30, 2011 at 05:51:52PM +0100, Richard W.M. Jones wrote:
> On Fri, Sep 30, 2011 at 02:11:48PM +0100, Richard W.M. Jones wrote:
> > 
> > I've not looked into this at all, it's just a report that something
> > seems to be "up".  I will try to git bisect this later if no one spots
> > anything obvious.
> > 
> > The next operation after insmod virtio_blk would be insmod_virtio_net.
> > 
> > Guest kernel is a Fedora kernel, version 3.1.0-0.rc6.git0.3.fc16.x86_64.
> > 
> > The ImExPS/2 message may or may not be coincidental.
> 
> git-bisect points to:
> 
> d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09 is the first bad commit
> commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
> Author: Gerd Hoffmann <kraxel@redhat.com>
> Date:   Wed Aug 10 17:34:13 2011 +0200
> 
>     seabios: update to master
>     
>     commit 8e301472e324b6d6496d8b4ffc66863e99d7a505
>     
>     user visible changes in seabios:
>       * ahci is enabled by default (and thus in this build).
>       * bootorder support for ahci.
>       * two-pass pci allocator (orders bars by size for better packing).
>     
>     Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> 
> :040000 040000 76eb0c81b76563b55cb2bb5c484ccd48b8cfcded 5ec0d65d3a763a5566fe1f4c86269cad6d671020 M	pc-bios
> :040000 040000 a5a7ea6e297c1e7490b0a2c28a06ce56e5be9449 78adb664d3ea82f1a4dd5ec239887ac5b0168a7f M	roms
> bisect run success
> 
> Rich.
> 
> [...]
> > [    0.248188] Initializing network drop monitor service
> > [    0.402221] Freeing unused kernel memory: 940k freed
> > [    0.403955] Write protecting the kernel read-only data: 10240k
> > [    0.415286] Freeing unused kernel memory: 1260k freed
> > [    0.421039] Freeing unused kernel memory: 1584k freed
> > febootstrap: mounting /proc
> > febootstrap: uptime: 0.42 0.16
> > febootstrap: ext2 mini initrd starting up
> > febootstrap: mounting /sys
> > febootstrap: internal insmod libcrc32c.ko
> > febootstrap: internal insmod crc32c-intel.ko
> > insmod: init_module: crc32c-intel.ko: No such device
> > febootstrap: internal insmod crc-itu-t.ko
> > febootstrap: internal insmod crc-ccitt.ko
> > febootstrap: internal insmod crc7.ko
> > febootstrap: internal insmod crc8.ko
> > febootstrap: internal insmod scsi_transport_spi.ko
> > febootstrap: internal insmod sym53c8xx.ko
> > febootstrap: internal insmod rfkill.ko
> > febootstrap: internal insmod sparse-keymap.ko
> > febootstrap: internal insmod ideapad-laptop.ko
> > insmod: init_module: ideapad-laptop.ko: No such device
> > febootstrap: internal insmod virtio_balloon.ko
> > febootstrap: internal insmod virtio-rng.ko
> > febootstrap: internal insmod virtio_blk.ko
> > [    0.669479] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input1

This bug, or something that looks very much like it, seems to have
turned up again in qemu 1.0.  I've had several reports and have opened
a bug:

https://bugzilla.redhat.com/show_bug.cgi?id=768508

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

[Qemu-devel] insmod virtio-blk is broken in qemu 1.0 (was: Re: git-bisect results (was: Re: qemu.git hangs booting Linux after insmod virtio_blk.ko))

[Richard W.M. Jones]

git bisect says this.  I didn't believe it first time, so I ran it
twice with a few modifications, and it pointed to the same commit both
times ...

67882fd177389527510eb36b3f7712011a835545 is the first bad commit
commit 67882fd177389527510eb36b3f7712011a835545
Author: Max Filippov <jcmvbkbc@gmail.com>
Date:   Tue Sep 6 03:55:28 2011 +0400

    target-xtensa: implement narrow instructions
    
    Instructions with op0 >= 8 are 2 bytes long, others are 3 bytes long.
    
    Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
    Signed-off-by: Blue Swirl <blauwirbel@gmail.com>

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0 (was: Re: git-bisect results (was: Re: qemu.git hangs booting Linux after insmod virtio_blk.ko))

[Max Filippov]

> git bisect says this.  I didn't believe it first time, so I ran it
> twice with a few modifications, and it pointed to the same commit both
> times ...

Richard,
could you please elaborate on your testcase and configuration
(host/target architecture, command lines, etc).

> 67882fd177389527510eb36b3f7712011a835545 is the first bad commit
> commit 67882fd177389527510eb36b3f7712011a835545
> Author: Max Filippov <jcmvbkbc@gmail.com>
> Date:   Tue Sep 6 03:55:28 2011 +0400
>
>    target-xtensa: implement narrow instructions
>
>    Instructions with op0 >= 8 are 2 bytes long, others are 3 bytes long.
>
>    Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
>    Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
>
> Rich.
--
Thanks.
-- Max

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0 (was: Re: git-bisect results (was: Re: qemu.git hangs booting Linux after insmod virtio_blk.ko))

[Max Filippov]

>> git bisect says this.  I didn't believe it first time, so I ran it
>> twice with a few modifications, and it pointed to the same commit both
>> times ...
>
> Richard,
> could you please elaborate on your testcase and configuration
> (host/target architecture, command lines, etc).

Ok, I've found most of details, what's not clear to me is how you
decide whether the build is good or bad.

I mean, you need to rebuild qemu on every bisection step, but neither
this commit nor the previous or the next one
change anything that would compile for x86 targets.

>> 67882fd177389527510eb36b3f7712011a835545 is the first bad commit
>> commit 67882fd177389527510eb36b3f7712011a835545
>> Author: Max Filippov <jcmvbkbc@gmail.com>
>> Date:   Tue Sep 6 03:55:28 2011 +0400
>>
>>    target-xtensa: implement narrow instructions
>>
>>    Instructions with op0 >= 8 are 2 bytes long, others are 3 bytes long.
>>
>>    Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
>>    Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
>>
>> Rich.

-- 
Thanks.
-- Max

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0 (was: Re: git-bisect results

[Anthony Liguori]

On 12/16/2011 06:07 PM, Richard W.M. Jones wrote:
>
> git bisect says this.  I didn't believe it first time, so I ran it
> twice with a few modifications, and it pointed to the same commit both
> times ...

Need more details because it doesn't appear to be broken to me.

What guest, what's your command line, what guest kernel version, how does it 
break, etc..

Regards,

Anthony Liguori

>
> 67882fd177389527510eb36b3f7712011a835545 is the first bad commit
> commit 67882fd177389527510eb36b3f7712011a835545
> Author: Max Filippov<jcmvbkbc@gmail.com>
> Date:   Tue Sep 6 03:55:28 2011 +0400
>
>      target-xtensa: implement narrow instructions
>
>      Instructions with op0>= 8 are 2 bytes long, others are 3 bytes long.
>
>      Signed-off-by: Max Filippov<jcmvbkbc@gmail.com>
>      Signed-off-by: Blue Swirl<blauwirbel@gmail.com>
>
> Rich.
>

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0 (was: Re: git-bisect results

[Anthony Liguori]

On 12/16/2011 06:07 PM, Richard W.M. Jones wrote:
>
> git bisect says this.  I didn't believe it first time, so I ran it
> twice with a few modifications, and it pointed to the same commit both
> times ...
>
> 67882fd177389527510eb36b3f7712011a835545 is the first bad commit
> commit 67882fd177389527510eb36b3f7712011a835545
> Author: Max Filippov<jcmvbkbc@gmail.com>
> Date:   Tue Sep 6 03:55:28 2011 +0400
>
>      target-xtensa: implement narrow instructions
>
>      Instructions with op0>= 8 are 2 bytes long, others are 3 bytes long.
>
>      Signed-off-by: Max Filippov<jcmvbkbc@gmail.com>
>      Signed-off-by: Blue Swirl<blauwirbel@gmail.com>
>
> Rich.

If you disable virtio-serial on the QEMU command line, does the guest boot 
successfully?

I believe this is a guest kernel regression in virtio-serial.

Regards,

Anthony Liguori

>

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0 (was: Re: git-bisect results

[Anthony Liguori]

On 12/16/2011 06:53 PM, Max Filippov wrote:
>>> git bisect says this.  I didn't believe it first time, so I ran it
>>> twice with a few modifications, and it pointed to the same commit both
>>> times ...
>>
>> Richard,
>> could you please elaborate on your testcase and configuration
>> (host/target architecture, command lines, etc).
>
> Ok, I've found most of details, what's not clear to me is how you
> decide whether the build is good or bad.
>
> I mean, you need to rebuild qemu on every bisection step, but neither
> this commit nor the previous or the next one
> change anything that would compile for x86 targets.

Fairly certain this bisect is a red herring.

tglx reported this the other day in IRC.  He narrowed it down to virtio-serial. 
  He was able to reproduce it both with kvm tools and QEMU.

Regards,

Anthony Liguori

>
>>> 67882fd177389527510eb36b3f7712011a835545 is the first bad commit
>>> commit 67882fd177389527510eb36b3f7712011a835545
>>> Author: Max Filippov<jcmvbkbc@gmail.com>
>>> Date:   Tue Sep 6 03:55:28 2011 +0400
>>>
>>>     target-xtensa: implement narrow instructions
>>>
>>>     Instructions with op0>= 8 are 2 bytes long, others are 3 bytes long.
>>>
>>>     Signed-off-by: Max Filippov<jcmvbkbc@gmail.com>
>>>     Signed-off-by: Blue Swirl<blauwirbel@gmail.com>
>>>
>>> Rich.
>

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0 (was: Re: git-bisect results

[Richard W.M. Jones]

On Fri, Dec 16, 2011 at 07:44:10PM -0600, Anthony Liguori wrote:
> On 12/16/2011 06:53 PM, Max Filippov wrote:
> >>>git bisect says this.  I didn't believe it first time, so I ran it
> >>>twice with a few modifications, and it pointed to the same commit both
> >>>times ...
> >>
> >>Richard,
> >>could you please elaborate on your testcase and configuration
> >>(host/target architecture, command lines, etc).
> >
> >Ok, I've found most of details, what's not clear to me is how you
> >decide whether the build is good or bad.
> >
> >I mean, you need to rebuild qemu on every bisection step, but neither
> >this commit nor the previous or the next one
> >change anything that would compile for x86 targets.
> 
> Fairly certain this bisect is a red herring.
> 
> tglx reported this the other day in IRC.  He narrowed it down to
> virtio-serial.  He was able to reproduce it both with kvm tools and
> QEMU.

Yes, we do use virtio-serial.  The command line is:

/home/rjones/d/qemu/qemu.wrapper \
    -drive file=/tmp/libguestfs-test-tool-sda-b4hesH,cache=off,format=raw,if=virtio \
    -nodefconfig \
    -machine accel=kvm:tcg \
    -nodefaults \
    -nographic \
    -m 500 \
    -no-reboot \
    -no-hpet \
    -device virtio-serial \
    -serial stdio \
    -chardev socket,path=/tmp/libguestfsQQ187c/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -kernel /var/tmp/.guestfs-500/kernel.24171 \
    -initrd /var/tmp/.guestfs-500/initrd.24171 \
    -append 'panic=1 console=ttyS0 udevtimeout=300 no_timer_check acpi=off printk.time=1 cgroup_disable=memory selinux=0 guestfs_verbose=1 TERM=xterm ' \
    -drive file=/var/tmp/.guestfs-500/root.24171,snapshot=on,if=virtio,cache=unsafe

which comes from this libguestfs test case:

LIBGUESTFS_QEMU=~/d/qemu/qemu.wrapper libguestfs-test-tool -t 60

where qemu.wrapper is:

#!/bin/sh -
qemudir=/home/rjones/d/qemu
exec $qemudir/x86_64-softmmu/qemu-system-x86_64 -L $qemudir/pc-bios "$@"

I'll try it with/without virtio-serial.

git bisect red herring is pretty strange?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0 (was: Re: git-bisect results

[Richard W.M. Jones]

On Fri, Dec 16, 2011 at 07:44:10PM -0600, Anthony Liguori wrote:
> Fairly certain this bisect is a red herring.
> 
> tglx reported this the other day in IRC.  He narrowed it down to
> virtio-serial.  He was able to reproduce it both with kvm tools and
> QEMU.

I looked at this a bit more closely, and what fails is the loading in
the guest of the virtio_blk.ko driver.  virtio_console is not loaded
when the failure happens.

Disabling virtio-serial on the qemu side makes no difference.

Switching to IDE works, but only if I prevent virtio_blk from being
loaded during boot.  In other words, it's the act of loading
virtio_blk which causes the strange hang, even if there are no virtio
block devices exported to the guest.

Not sure what any of this means, since I can't honestly believe that
no one has tried a virtio guest on qemu 1.0.

Guest kernel is vmlinuz-3.2.0-0.rc1.git2.1.fc17.x86_64 (from Fedora).

If you want me to try patches or take a look at any particular qemu
git commits, I'm all ears.  I don't know where to go from here ...

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Anthony Liguori]

On 12/17/2011 02:33 AM, Richard W.M. Jones wrote:
> On Fri, Dec 16, 2011 at 07:44:10PM -0600, Anthony Liguori wrote:
>> On 12/16/2011 06:53 PM, Max Filippov wrote:
>>>>> git bisect says this.  I didn't believe it first time, so I ran it
>>>>> twice with a few modifications, and it pointed to the same commit both
>>>>> times ...
>>>>
>>>> Richard,
>>>> could you please elaborate on your testcase and configuration
>>>> (host/target architecture, command lines, etc).
>>>
>>> Ok, I've found most of details, what's not clear to me is how you
>>> decide whether the build is good or bad.
>>>
>>> I mean, you need to rebuild qemu on every bisection step, but neither
>>> this commit nor the previous or the next one
>>> change anything that would compile for x86 targets.
>>
>> Fairly certain this bisect is a red herring.
>>
>> tglx reported this the other day in IRC.  He narrowed it down to
>> virtio-serial.  He was able to reproduce it both with kvm tools and
>> QEMU.
>
> Yes, we do use virtio-serial.  The command line is:
>
> /home/rjones/d/qemu/qemu.wrapper \
>      -drive file=/tmp/libguestfs-test-tool-sda-b4hesH,cache=off,format=raw,if=virtio \
>      -nodefconfig \
>      -machine accel=kvm:tcg \
>      -nodefaults \
>      -nographic \
>      -m 500 \
>      -no-reboot \
>      -no-hpet \
>      -device virtio-serial \
>      -serial stdio \

Okay, I can reproduce this now with a F15 guest.

I've narrowed it down to '-nodefaults -serial stdio'.  Can you confirm that if 
you remove those options it works for you?

Regards,

Anthony Liguori


>      -chardev socket,path=/tmp/libguestfsQQ187c/guestfsd.sock,id=channel0 \
>      -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
>      -kernel /var/tmp/.guestfs-500/kernel.24171 \
>      -initrd /var/tmp/.guestfs-500/initrd.24171 \
>      -append 'panic=1 console=ttyS0 udevtimeout=300 no_timer_check acpi=off printk.time=1 cgroup_disable=memory selinux=0 guestfs_verbose=1 TERM=xterm ' \
>      -drive file=/var/tmp/.guestfs-500/root.24171,snapshot=on,if=virtio,cache=unsafe
>
> which comes from this libguestfs test case:
>
> LIBGUESTFS_QEMU=~/d/qemu/qemu.wrapper libguestfs-test-tool -t 60
>
> where qemu.wrapper is:
>
> #!/bin/sh -
> qemudir=/home/rjones/d/qemu
> exec $qemudir/x86_64-softmmu/qemu-system-x86_64 -L $qemudir/pc-bios "$@"
>
> I'll try it with/without virtio-serial.
>
> git bisect red herring is pretty strange?
>
> Rich.
>

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Richard W.M. Jones]

On Sat, Dec 17, 2011 at 09:13:52AM -0600, Anthony Liguori wrote:
> Okay, I can reproduce this now with a F15 guest.
> 
> I've narrowed it down to '-nodefaults -serial stdio'.  Can you
> confirm that if you remove those options it works for you?

Confirmed: removing those options allows it to boot normally.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Anthony Liguori]

On 12/17/2011 09:13 AM, Anthony Liguori wrote:
> On 12/17/2011 02:33 AM, Richard W.M. Jones wrote:
>> On Fri, Dec 16, 2011 at 07:44:10PM -0600, Anthony Liguori wrote:
>>> On 12/16/2011 06:53 PM, Max Filippov wrote:
>>>>>> git bisect says this. I didn't believe it first time, so I ran it
>>>>>> twice with a few modifications, and it pointed to the same commit both
>>>>>> times ...
>>>>>
>>>>> Richard,
>>>>> could you please elaborate on your testcase and configuration
>>>>> (host/target architecture, command lines, etc).
>>>>
>>>> Ok, I've found most of details, what's not clear to me is how you
>>>> decide whether the build is good or bad.
>>>>
>>>> I mean, you need to rebuild qemu on every bisection step, but neither
>>>> this commit nor the previous or the next one
>>>> change anything that would compile for x86 targets.
>>>
>>> Fairly certain this bisect is a red herring.
>>>
>>> tglx reported this the other day in IRC. He narrowed it down to
>>> virtio-serial. He was able to reproduce it both with kvm tools and
>>> QEMU.
>>
>> Yes, we do use virtio-serial. The command line is:
>>
>> /home/rjones/d/qemu/qemu.wrapper \
>> -drive file=/tmp/libguestfs-test-tool-sda-b4hesH,cache=off,format=raw,if=virtio \
>> -nodefconfig \
>> -machine accel=kvm:tcg \
>> -nodefaults \
>> -nographic \
>> -m 500 \
>> -no-reboot \
>> -no-hpet \
>> -device virtio-serial \
>> -serial stdio \
>
> Okay, I can reproduce this now with a F15 guest.
>
> I've narrowed it down to '-nodefaults -serial stdio'. Can you confirm that if
> you remove those options it works for you?

I've even further narrowed it down to the presents or lack of '-vga cirrus'.  If 
you add '-vga cirrus' to the above command line, the guest will boot successfully.

Regards,

Anthony Liguori

>
> Regards,
>
> Anthony Liguori
>
>
>> -chardev socket,path=/tmp/libguestfsQQ187c/guestfsd.sock,id=channel0 \
>> -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
>> -kernel /var/tmp/.guestfs-500/kernel.24171 \
>> -initrd /var/tmp/.guestfs-500/initrd.24171 \
>> -append 'panic=1 console=ttyS0 udevtimeout=300 no_timer_check acpi=off
>> printk.time=1 cgroup_disable=memory selinux=0 guestfs_verbose=1 TERM=xterm ' \
>> -drive file=/var/tmp/.guestfs-500/root.24171,snapshot=on,if=virtio,cache=unsafe
>>
>> which comes from this libguestfs test case:
>>
>> LIBGUESTFS_QEMU=~/d/qemu/qemu.wrapper libguestfs-test-tool -t 60
>>
>> where qemu.wrapper is:
>>
>> #!/bin/sh -
>> qemudir=/home/rjones/d/qemu
>> exec $qemudir/x86_64-softmmu/qemu-system-x86_64 -L $qemudir/pc-bios "$@"
>>
>> I'll try it with/without virtio-serial.
>>
>> git bisect red herring is pretty strange?
>>
>> Rich.
>>
>

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Richard W.M. Jones]

On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
> I've even further narrowed it down to the presents or lack of '-vga
> cirrus'.  If you add '-vga cirrus' to the above command line, the
> guest will boot successfully.

Confirmed: Adding -vga cirrus to the command line cures it too.

That's a strange one :-)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Anthony Liguori]

On 12/17/2011 09:25 AM, Richard W.M. Jones wrote:
> On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
>> I've even further narrowed it down to the presents or lack of '-vga
>> cirrus'.  If you add '-vga cirrus' to the above command line, the
>> guest will boot successfully.
>
> Confirmed: Adding -vga cirrus to the command line cures it too.
>
> That's a strange one :-)

vga sticks out a bit because it's one of the few places where we treat device 
memory as ram as a performance optimization.

The only time vga has been touched in between v0.15 and v1.0 was during the 
introduction of the memory API.

It's this commit:

commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
Author: Gerd Hoffmann <kraxel@redhat.com>
Date:   Wed Aug 10 17:34:13 2011 +0200

     seabios: update to master

     commit 8e301472e324b6d6496d8b4ffc66863e99d7a505

     user visible changes in seabios:
       * ahci is enabled by default (and thus in this build).
       * bootorder support for ahci.
       * two-pass pci allocator (orders bars by size for better packing).

     Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

:040000 040000 76eb0c81b76563b55cb2bb5c484ccd48b8cfcded 
5ec0d65d3a763a5566fe1f4c86269cad6d671020 M	pc-bios
:040000 040000 a5a7ea6e297c1e7490b0a2c28a06ce56e5be9449 
78adb664d3ea82f1a4dd5ec239887ac5b0168a7f M	roms

It can be reproduced by using virtio and -vga none with a number of PCI devices. 
  The line below is what I used to bisect and reproduce 100% of the time.  It's 
a 64-bit Fedora 15 guest.

$ qemu-system-x86_64 -drive 
file=/home/anthony/images/fedora.img,if=none,snapshot=on,id=hd0 -device 
virtio-balloon-pci,addr=03.0 -device virtio-blk-pci,addr=04.0,drive=hd0 -kernel 
~/vmlinuz-2.6.38.6-26.rc1.fc15.x86_64 -initrd 
~/initramfs-2.6.38.6-26.rc1.fc15.x86_64.img -append 
"root=/dev/mapper/VolGroup-lv_root rd_LVM_LV=VolGroup/lv_root 
rd_LVM_LV=VolGroup/lv_swap ro console=ttyS0 selinux=0" -nographic -nodefconfig 
-m 1G -no-reboot -no-hpet -device virtio-serial -chardev 
socket,path=/tmp/foo.sock,id=channel0,server,nowait -device 
virtserialport,chardev=channel0,name=org.libguestfs.channel.0 -nodefaults 
-serial stdio -enable-kvm

My guess it that it has something to do with the changes to the PCI allocator. 
I've confirmed reverting this commit fixes the problem.

Regards,

Anthony Liguori

>
> Rich.
>

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Richard W.M. Jones]

On Sat, Dec 17, 2011 at 10:24:07AM -0600, Anthony Liguori wrote:
> On 12/17/2011 09:25 AM, Richard W.M. Jones wrote:
> >On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
> >>I've even further narrowed it down to the presents or lack of '-vga
> >>cirrus'.  If you add '-vga cirrus' to the above command line, the
> >>guest will boot successfully.
> >
> >Confirmed: Adding -vga cirrus to the command line cures it too.
> >
> >That's a strange one :-)
> 
> vga sticks out a bit because it's one of the few places where we
> treat device memory as ram as a performance optimization.
> 
> The only time vga has been touched in between v0.15 and v1.0 was
> during the introduction of the memory API.
> 
> It's this commit:
> 
> commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
> Author: Gerd Hoffmann <kraxel@redhat.com>
> Date:   Wed Aug 10 17:34:13 2011 +0200
> 
>     seabios: update to master
> 
>     commit 8e301472e324b6d6496d8b4ffc66863e99d7a505
> 
>     user visible changes in seabios:
>       * ahci is enabled by default (and thus in this build).
>       * bootorder support for ahci.
>       * two-pass pci allocator (orders bars by size for better packing).
> 
>     Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> 
> :040000 040000 76eb0c81b76563b55cb2bb5c484ccd48b8cfcded
> 5ec0d65d3a763a5566fe1f4c86269cad6d671020 M	pc-bios
> :040000 040000 a5a7ea6e297c1e7490b0a2c28a06ce56e5be9449
> 78adb664d3ea82f1a4dd5ec239887ac5b0168a7f M	roms
> 
> It can be reproduced by using virtio and -vga none with a number of
> PCI devices.  The line below is what I used to bisect and reproduce
> 100% of the time.  It's a 64-bit Fedora 15 guest.
> 
> $ qemu-system-x86_64 -drive
> file=/home/anthony/images/fedora.img,if=none,snapshot=on,id=hd0
> -device virtio-balloon-pci,addr=03.0 -device
> virtio-blk-pci,addr=04.0,drive=hd0 -kernel
> ~/vmlinuz-2.6.38.6-26.rc1.fc15.x86_64 -initrd
> ~/initramfs-2.6.38.6-26.rc1.fc15.x86_64.img -append
> "root=/dev/mapper/VolGroup-lv_root rd_LVM_LV=VolGroup/lv_root
> rd_LVM_LV=VolGroup/lv_swap ro console=ttyS0 selinux=0" -nographic
> -nodefconfig -m 1G -no-reboot -no-hpet -device virtio-serial
> -chardev socket,path=/tmp/foo.sock,id=channel0,server,nowait -device
> virtserialport,chardev=channel0,name=org.libguestfs.channel.0
> -nodefaults -serial stdio -enable-kvm
> 
> My guess it that it has something to do with the changes to the PCI
> allocator. I've confirmed reverting this commit fixes the problem.

Confirmed: reverting this patch fixes it for me.

Note that we suspected this patch before, way back in September:
https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg03830.html

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Kevin O'Connor]

On Sat, Dec 17, 2011 at 10:24:07AM -0600, Anthony Liguori wrote:
> On 12/17/2011 09:25 AM, Richard W.M. Jones wrote:
> >On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
> >>I've even further narrowed it down to the presents or lack of '-vga
> >>cirrus'.  If you add '-vga cirrus' to the above command line, the
> >>guest will boot successfully.
> >
> >Confirmed: Adding -vga cirrus to the command line cures it too.
> >
> >That's a strange one :-)
> 
> vga sticks out a bit because it's one of the few places where we
> treat device memory as ram as a performance optimization.
> 
> The only time vga has been touched in between v0.15 and v1.0 was
> during the introduction of the memory API.
> 
> It's this commit:
> 
> commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
> Author: Gerd Hoffmann <kraxel@redhat.com>
> Date:   Wed Aug 10 17:34:13 2011 +0200
> 
>     seabios: update to master

This looks like the same issue reported at:

http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00029.html

The SeaBIOS fix for this was in rel-1.6.3.1 - but that didn't make
QEmu 1.0.  Does the problem go away if you upgrade to the newer
SeaBIOS version?

-Kevin

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Anthony Liguori]

On 12/17/2011 10:49 AM, Kevin O'Connor wrote:
> On Sat, Dec 17, 2011 at 10:24:07AM -0600, Anthony Liguori wrote:
>> On 12/17/2011 09:25 AM, Richard W.M. Jones wrote:
>>> On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
>>>> I've even further narrowed it down to the presents or lack of '-vga
>>>> cirrus'.  If you add '-vga cirrus' to the above command line, the
>>>> guest will boot successfully.
>>>
>>> Confirmed: Adding -vga cirrus to the command line cures it too.
>>>
>>> That's a strange one :-)
>>
>> vga sticks out a bit because it's one of the few places where we
>> treat device memory as ram as a performance optimization.
>>
>> The only time vga has been touched in between v0.15 and v1.0 was
>> during the introduction of the memory API.
>>
>> It's this commit:
>>
>> commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
>> Author: Gerd Hoffmann<kraxel@redhat.com>
>> Date:   Wed Aug 10 17:34:13 2011 +0200
>>
>>      seabios: update to master
>
> This looks like the same issue reported at:
>
> http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00029.html
>
> The SeaBIOS fix for this was in rel-1.6.3.1 - but that didn't make
> QEmu 1.0.  Does the problem go away if you upgrade to the newer
> SeaBIOS version?

Er, I can't actually build SeaBIOS anymore...

The version of LD on this system does not properly handle
alignments.  As a result, this project can not be built.

The problem may be the result of this LD bug report:
  http://sourceware.org/bugzilla/show_bug.cgi?id=12726

Please update to a working version of binutils and retry.
Makefile:75: *** "Please upgrade GCC and/or binutils".  Stop.

Let me find a box with a newer binutils...

Regards,

Anthony Liguori

>
> -Kevin
>

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Richard W.M. Jones]

On Sat, Dec 17, 2011 at 11:49:56AM -0500, Kevin O'Connor wrote:
> On Sat, Dec 17, 2011 at 10:24:07AM -0600, Anthony Liguori wrote:
> > On 12/17/2011 09:25 AM, Richard W.M. Jones wrote:
> > >On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
> > >>I've even further narrowed it down to the presents or lack of '-vga
> > >>cirrus'.  If you add '-vga cirrus' to the above command line, the
> > >>guest will boot successfully.
> > >
> > >Confirmed: Adding -vga cirrus to the command line cures it too.
> > >
> > >That's a strange one :-)
> > 
> > vga sticks out a bit because it's one of the few places where we
> > treat device memory as ram as a performance optimization.
> > 
> > The only time vga has been touched in between v0.15 and v1.0 was
> > during the introduction of the memory API.
> > 
> > It's this commit:
> > 
> > commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
> > Author: Gerd Hoffmann <kraxel@redhat.com>
> > Date:   Wed Aug 10 17:34:13 2011 +0200
> > 
> >     seabios: update to master
> 
> This looks like the same issue reported at:
> 
> http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00029.html
> 
> The SeaBIOS fix for this was in rel-1.6.3.1 - but that didn't make
> QEmu 1.0.  Does the problem go away if you upgrade to the newer
> SeaBIOS version?

Yes, SeaBIOS upstream + qemu 1.0 does fix the problem.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Daniel P. Berrange]

On Sat, Dec 17, 2011 at 11:49:56AM -0500, Kevin O'Connor wrote:
> On Sat, Dec 17, 2011 at 10:24:07AM -0600, Anthony Liguori wrote:
> > On 12/17/2011 09:25 AM, Richard W.M. Jones wrote:
> > >On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
> > >>I've even further narrowed it down to the presents or lack of '-vga
> > >>cirrus'.  If you add '-vga cirrus' to the above command line, the
> > >>guest will boot successfully.
> > >
> > >Confirmed: Adding -vga cirrus to the command line cures it too.
> > >
> > >That's a strange one :-)
> > 
> > vga sticks out a bit because it's one of the few places where we
> > treat device memory as ram as a performance optimization.
> > 
> > The only time vga has been touched in between v0.15 and v1.0 was
> > during the introduction of the memory API.
> > 
> > It's this commit:
> > 
> > commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
> > Author: Gerd Hoffmann <kraxel@redhat.com>
> > Date:   Wed Aug 10 17:34:13 2011 +0200
> > 
> >     seabios: update to master
> 
> This looks like the same issue reported at:
> 
> http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00029.html
> 
> The SeaBIOS fix for this was in rel-1.6.3.1 - but that didn't make
> QEmu 1.0.  Does the problem go away if you upgrade to the newer
> SeaBIOS version?

Sigh, we really need to be better about updating SeaBIOS in QEMU before
release. We had plenty of time to pull in a newer SeaBIOS before 1.0
that would have fixed this :-( We've had multiple releases now where
functionality is broken due to QEMU shipping with an older SeaBIOS
release than is available upstream.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [SeaBIOS] insmod virtio-blk is broken in qemu 1.0

[Fred .]

On Mon, Dec 19, 2011 at 11:31 AM, Daniel P. Berrange
<berrange@redhat.com> wrote:
> Sigh, we really need to be better about updating SeaBIOS in QEMU before
> release. We had plenty of time to pull in a newer SeaBIOS before 1.0
> that would have fixed this :-( We've had multiple releases now where
> functionality is broken due to QEMU shipping with an older SeaBIOS
> release than is available upstream.
>
> Regards,
> Daniel

Yes.

SeaBIOS announcements should be made on the QEMU, KVM and Bochs
mailing lists to raise awareness.

Before the merge window of QEMU (and other software that bundle
SeaBIOS) close, SeaBIOS developers or contributors should verify if
the latest version of SeaBIOS is shipped, if not then request or
commit the latest SeaBIOS to be integrated.

Re: [Qemu-devel] [SeaBIOS] insmod virtio-blk is broken in qemu 1.0

[Fred .]

On Mon, Dec 19, 2011 at 11:31 AM, Daniel P. Berrange
<berrange@redhat.com> wrote:
> Sigh, we really need to be better about updating SeaBIOS in QEMU before
> release. We had plenty of time to pull in a newer SeaBIOS before 1.0
> that would have fixed this :-( We've had multiple releases now where
> functionality is broken due to QEMU shipping with an older SeaBIOS
> release than is available upstream.
>
> Regards,
> Daniel

New releases of SeaBIOS needs to be more prominently announced and
displayed on the front page of the SeaBIOS website.

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Anthony Liguori]

On 12/19/2011 04:31 AM, Daniel P. Berrange wrote:
> On Sat, Dec 17, 2011 at 11:49:56AM -0500, Kevin O'Connor wrote:
>> On Sat, Dec 17, 2011 at 10:24:07AM -0600, Anthony Liguori wrote:
>>> On 12/17/2011 09:25 AM, Richard W.M. Jones wrote:
>>>> On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
>>>>> I've even further narrowed it down to the presents or lack of '-vga
>>>>> cirrus'.  If you add '-vga cirrus' to the above command line, the
>>>>> guest will boot successfully.
>>>>
>>>> Confirmed: Adding -vga cirrus to the command line cures it too.
>>>>
>>>> That's a strange one :-)
>>>
>>> vga sticks out a bit because it's one of the few places where we
>>> treat device memory as ram as a performance optimization.
>>>
>>> The only time vga has been touched in between v0.15 and v1.0 was
>>> during the introduction of the memory API.
>>>
>>> It's this commit:
>>>
>>> commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
>>> Author: Gerd Hoffmann<kraxel@redhat.com>
>>> Date:   Wed Aug 10 17:34:13 2011 +0200
>>>
>>>      seabios: update to master
>>
>> This looks like the same issue reported at:
>>
>> http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00029.html
>>
>> The SeaBIOS fix for this was in rel-1.6.3.1 - but that didn't make
>> QEmu 1.0.  Does the problem go away if you upgrade to the newer
>> SeaBIOS version?
>
> Sigh, we really need to be better about updating SeaBIOS in QEMU before
> release. We had plenty of time to pull in a newer SeaBIOS before 1.0
> that would have fixed this :-(

1.6.3.1 was released on Nov 24th, which was actually after the soft feature 
freeze.  We could have pulled 1.6.3 which was Oct 4th but updating the BIOS 
always results in some interesting things happening so it's not something I like 
to do unless we have to.

I'd rather have known that this functionality broken before that commit event 
went in to begin with than allowing it to remain broken until we happened to 
update past the bug.

> We've had multiple releases now where
> functionality is broken due to QEMU shipping with an older SeaBIOS
> release than is available upstream.

I think the real issue here is testing.  -nodefconfig -nodefaults is used by 
both libguestfs and libvirt but I'd wager to say that almost noone tests it in QEMU.

I thought about it quite a bit and what I came to was that we need to do a 
better job of making it easy to test these things, hence qemu-test that I just 
announced.

Regards,

Anthony Liguori

> Regards,
> Daniel

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Daniel P. Berrange]

On Mon, Dec 19, 2011 at 11:34:13AM -0600, Anthony Liguori wrote:
> On 12/19/2011 04:31 AM, Daniel P. Berrange wrote:
> >On Sat, Dec 17, 2011 at 11:49:56AM -0500, Kevin O'Connor wrote:
> >>On Sat, Dec 17, 2011 at 10:24:07AM -0600, Anthony Liguori wrote:
> >>>On 12/17/2011 09:25 AM, Richard W.M. Jones wrote:
> >>>>On Sat, Dec 17, 2011 at 09:22:45AM -0600, Anthony Liguori wrote:
> >>>>>I've even further narrowed it down to the presents or lack of '-vga
> >>>>>cirrus'.  If you add '-vga cirrus' to the above command line, the
> >>>>>guest will boot successfully.
> >>>>
> >>>>Confirmed: Adding -vga cirrus to the command line cures it too.
> >>>>
> >>>>That's a strange one :-)
> >>>
> >>>vga sticks out a bit because it's one of the few places where we
> >>>treat device memory as ram as a performance optimization.
> >>>
> >>>The only time vga has been touched in between v0.15 and v1.0 was
> >>>during the introduction of the memory API.
> >>>
> >>>It's this commit:
> >>>
> >>>commit d67c3f2cd92aed2247bfa8a9da61a902b7b2ff09
> >>>Author: Gerd Hoffmann<kraxel@redhat.com>
> >>>Date:   Wed Aug 10 17:34:13 2011 +0200
> >>>
> >>>     seabios: update to master
> >>
> >>This looks like the same issue reported at:
> >>
> >>http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00029.html
> >>
> >>The SeaBIOS fix for this was in rel-1.6.3.1 - but that didn't make
> >>QEmu 1.0.  Does the problem go away if you upgrade to the newer
> >>SeaBIOS version?
> >
> >Sigh, we really need to be better about updating SeaBIOS in QEMU before
> >release. We had plenty of time to pull in a newer SeaBIOS before 1.0
> >that would have fixed this :-(
> 
> 1.6.3.1 was released on Nov 24th, which was actually after the soft
> feature freeze.  We could have pulled 1.6.3 which was Oct 4th but
> updating the BIOS always results in some interesting things
> happening so it's not something I like to do unless we have to.
> 
> I'd rather have known that this functionality broken before that
> commit event went in to begin with than allowing it to remain broken
> until we happened to update past the bug.
> 
> >We've had multiple releases now where
> >functionality is broken due to QEMU shipping with an older SeaBIOS
> >release than is available upstream.
> 
> I think the real issue here is testing.  -nodefconfig -nodefaults is
> used by both libguestfs and libvirt but I'd wager to say that almost
> noone tests it in QEMU.

I had actually discovered & pointed out this flaw on qemu-devel back
in September, and Kevin had the seabios fix by Oct

http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00029.html

I hadn't raised it again, because I had mistakenly assumed QEMU
will automatically pull in the newer SeaBios release before 1.0
came out. I could have more aggresively bugged people on qemu-devel
to update SeaBios, but given your point above about not wanting to
rebase Seabios its not clear that would have helped sort this out
before 1.0

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Anthony Liguori]

On 12/19/2011 11:43 AM, Daniel P. Berrange wrote:
> On Mon, Dec 19, 2011 at 11:34:13AM -0600, Anthony Liguori wrote:
>> On 12/19/2011 04:31 AM, Daniel P. Berrange wrote:
>>> Sigh, we really need to be better about updating SeaBIOS in QEMU before
>>> release. We had plenty of time to pull in a newer SeaBIOS before 1.0
>>> that would have fixed this :-(
>>
>> 1.6.3.1 was released on Nov 24th, which was actually after the soft
>> feature freeze.  We could have pulled 1.6.3 which was Oct 4th but
>> updating the BIOS always results in some interesting things
>> happening so it's not something I like to do unless we have to.
>>
>> I'd rather have known that this functionality broken before that
>> commit event went in to begin with than allowing it to remain broken
>> until we happened to update past the bug.
>>
>>> We've had multiple releases now where
>>> functionality is broken due to QEMU shipping with an older SeaBIOS
>>> release than is available upstream.
>>
>> I think the real issue here is testing.  -nodefconfig -nodefaults is
>> used by both libguestfs and libvirt but I'd wager to say that almost
>> noone tests it in QEMU.
>
> I had actually discovered&  pointed out this flaw on qemu-devel back
> in September, and Kevin had the seabios fix by Oct
>
> http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00029.html
>
> I hadn't raised it again, because I had mistakenly assumed QEMU
> will automatically pull in the newer SeaBios release before 1.0
> came out. I could have more aggresively bugged people on qemu-devel
> to update SeaBios, but given your point above about not wanting to
> rebase Seabios its not clear that would have helped sort this out
> before 1.0

We really need to update SeaBIOS whenever there is a bug that we know requires 
an update.  Things breakdown because of one or more of the following reasons:

1) User submits a patch to seabios@, Kevin applies it.  But that doesn't 
necessarily trigger anything happening in QEMU.

Ideally, the above mentioned user would submit a submodule update once (1) happens.

2) Kevin fixes something on his own or someone else changes something in the 
broader SeaBIOS community.  That may not even be visible in QEMU.

Syncing right before release isn't a good strategy either because that means 
we're pulling in something that hasn't been tested extensively at the very tail 
end of our release cycle.

I would like to point out that August -> October is a pretty long time period 
for a regression like this to exist.  I think that really indicates that the 
primary problem is testing, not frequency of SeaBIOS updates.

Regards,

Anthony Liguori

> Regards,
> Daniel

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Richard W.M. Jones]

On Mon, Dec 19, 2011 at 12:02:59PM -0600, Anthony Liguori wrote:
> I would like to point out that August -> October is a pretty long
> time period for a regression like this to exist.  I think that
> really indicates that the primary problem is testing, not frequency
> of SeaBIOS updates.

Fair point.

My understanding is we're going to switch to having qemu.git in Fedora
Rawhide, which means that libguestfs will always be testing the
'perfect storm' of qemu + kernel + glibc from git (once glibc get
their act together anyhow, just qemu + kernel at first).

We usually do a build and a comprehensive test at least once a week,
often a few times a week, so we would have picked this up much sooner.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Daniel P. Berrange]

On Mon, Dec 19, 2011 at 07:04:54PM +0000, Richard W.M. Jones wrote:
> On Mon, Dec 19, 2011 at 12:02:59PM -0600, Anthony Liguori wrote:
> > I would like to point out that August -> October is a pretty long
> > time period for a regression like this to exist.  I think that
> > really indicates that the primary problem is testing, not frequency
> > of SeaBIOS updates.
> 
> Fair point.
> 
> My understanding is we're going to switch to having qemu.git in Fedora
> Rawhide, which means that libguestfs will always be testing the
> 'perfect storm' of qemu + kernel + glibc from git (once glibc get
> their act together anyhow, just qemu + kernel at first).
> 
> We usually do a build and a comprehensive test at least once a week,
> often a few times a week, so we would have picked this up much sooner.

That wouldn't actually catch this problem, because when we build
QEMU in Fedora, we never use the SeaBIOS that QEMU includes in
GIT. Fedora always ships the newest SeaBIOS release available
from upstream, regardless of what QEMU includes.


Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Daniel P. Berrange]

On Mon, Dec 19, 2011 at 12:02:59PM -0600, Anthony Liguori wrote:
> On 12/19/2011 11:43 AM, Daniel P. Berrange wrote:
> >I hadn't raised it again, because I had mistakenly assumed QEMU
> >will automatically pull in the newer SeaBios release before 1.0
> >came out. I could have more aggresively bugged people on qemu-devel
> >to update SeaBios, but given your point above about not wanting to
> >rebase Seabios its not clear that would have helped sort this out
> >before 1.0
> 
> We really need to update SeaBIOS whenever there is a bug that we
> know requires an update.  Things breakdown because of one or more of
> the following reasons:
> 
> 1) User submits a patch to seabios@, Kevin applies it.  But that
> doesn't necessarily trigger anything happening in QEMU.
> 
> Ideally, the above mentioned user would submit a submodule update once (1) happens.
> 
> 2) Kevin fixes something on his own or someone else changes
> something in the broader SeaBIOS community.  That may not even be
> visible in QEMU.
> 
> Syncing right before release isn't a good strategy either because
> that means we're pulling in something that hasn't been tested
> extensively at the very tail end of our release cycle.
> 
> I would like to point out that August -> October is a pretty long
> time period for a regression like this to exist.  I think that
> really indicates that the primary problem is testing, not frequency
> of SeaBIOS updates.

One complication is that alot of us are not necessarily testing the
SeaBIOS that is in QEMU GIT. Fedora rawhide includes qemu-kvm.git
snapshots which are updated fairly frequently, but we don't use
the SeaBIOS QEMU includes. Instead Fedora includes the latest
SeaBIOS upstream release.

So Fedora 16/rawhide users would never have seen this particular
bug for longer than a couple of weeks until the fixed SeaBIOS
arrived.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Anthony Liguori]

On 12/19/2011 01:19 PM, Daniel P. Berrange wrote:
> On Mon, Dec 19, 2011 at 12:02:59PM -0600, Anthony Liguori wrote:
>> I would like to point out that August ->  October is a pretty long
>> time period for a regression like this to exist.  I think that
>> really indicates that the primary problem is testing, not frequency
>> of SeaBIOS updates.
>
> One complication is that alot of us are not necessarily testing the
> SeaBIOS that is in QEMU GIT. Fedora rawhide includes qemu-kvm.git
> snapshots which are updated fairly frequently, but we don't use
> the SeaBIOS QEMU includes. Instead Fedora includes the latest
> SeaBIOS upstream release.

I understand why Debian and Fedora do this but it is unfortunate from a QA 
perspective.

Fedora is on a different release schedule than QEMU, so it's easy for it to do 
an "upstream freeze", and grab the latest SeaBIOS release and QEMU release.

But we don't treat SeaBIOS as a separate package supporting arbitrary 
combinations of SeaBIOS and QEMU.

Regards,

Anthony Liguori

>
> So Fedora 16/rawhide users would never have seen this particular
> bug for longer than a couple of weeks until the fixed SeaBIOS
> arrived.
>
> Regards,
> Daniel

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Richard W.M. Jones]

On Mon, Dec 19, 2011 at 07:16:02PM +0000, Daniel P. Berrange wrote:
> On Mon, Dec 19, 2011 at 07:04:54PM +0000, Richard W.M. Jones wrote:
> > On Mon, Dec 19, 2011 at 12:02:59PM -0600, Anthony Liguori wrote:
> > > I would like to point out that August -> October is a pretty long
> > > time period for a regression like this to exist.  I think that
> > > really indicates that the primary problem is testing, not frequency
> > > of SeaBIOS updates.
> > 
> > Fair point.
> > 
> > My understanding is we're going to switch to having qemu.git in Fedora
> > Rawhide, which means that libguestfs will always be testing the
> > 'perfect storm' of qemu + kernel + glibc from git (once glibc get
> > their act together anyhow, just qemu + kernel at first).
> > 
> > We usually do a build and a comprehensive test at least once a week,
> > often a few times a week, so we would have picked this up much sooner.
> 
> That wouldn't actually catch this problem, because when we build
> QEMU in Fedora, we never use the SeaBIOS that QEMU includes in
> GIT. Fedora always ships the newest SeaBIOS release available
> from upstream, regardless of what QEMU includes.

Ah yes indeed, I forgot about this.

Nevertheless, it'll at least improve other aspects of our
qemu testing :-)

In reply to Anthony: the reason Fedora does this is because
binary blobs aren't permitted, no matter what the origin.  We
have to build SeaBIOS from source, and the choice is made to
build from the upstream SeaBIOS source, not from the source
release indirectly pointed to by qemu.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Richard W.M. Jones]

On Mon, Dec 19, 2011 at 07:40:05PM +0000, Richard W.M. Jones wrote:
> In reply to Anthony: the reason Fedora does this is because
> binary blobs aren't permitted, no matter what the origin.  We
> have to build SeaBIOS from source, and the choice is made to
> build from the upstream SeaBIOS source, not from the source
> release indirectly pointed to by qemu.

... as you know already.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Anthony Liguori]

On 12/19/2011 01:40 PM, Richard W.M. Jones wrote:
> On Mon, Dec 19, 2011 at 07:16:02PM +0000, Daniel P. Berrange wrote:
>> On Mon, Dec 19, 2011 at 07:04:54PM +0000, Richard W.M. Jones wrote:
>>> On Mon, Dec 19, 2011 at 12:02:59PM -0600, Anthony Liguori wrote:
>>>> I would like to point out that August ->  October is a pretty long
>>>> time period for a regression like this to exist.  I think that
>>>> really indicates that the primary problem is testing, not frequency
>>>> of SeaBIOS updates.
>>>
>>> Fair point.
>>>
>>> My understanding is we're going to switch to having qemu.git in Fedora
>>> Rawhide, which means that libguestfs will always be testing the
>>> 'perfect storm' of qemu + kernel + glibc from git (once glibc get
>>> their act together anyhow, just qemu + kernel at first).
>>>
>>> We usually do a build and a comprehensive test at least once a week,
>>> often a few times a week, so we would have picked this up much sooner.
>>
>> That wouldn't actually catch this problem, because when we build
>> QEMU in Fedora, we never use the SeaBIOS that QEMU includes in
>> GIT. Fedora always ships the newest SeaBIOS release available
>> from upstream, regardless of what QEMU includes.
>
> Ah yes indeed, I forgot about this.
>
> Nevertheless, it'll at least improve other aspects of our
> qemu testing :-)
>
> In reply to Anthony: the reason Fedora does this is because
> binary blobs aren't permitted, no matter what the origin.  We
> have to build SeaBIOS from source, and the choice is made to
> build from the upstream SeaBIOS source, not from the source
> release indirectly pointed to by qemu.

FWIW, we ship SeaBIOS source directly in our release tarballs.  There's nothing 
indirect about it.

Fedora could have a seabios-qemu RPM that was built from the qemu SRPM.  Since 
it ultimately is going to live in /usr/share/qemu, that seems like a nicer thing 
to do AFAICT.

You could have an alternatives mechanism if people really wanted to use upstream 
SeaBIOS...

Regards,

Anthony Liguori

>
> Rich.
>

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Kevin O'Connor]

On Mon, Dec 19, 2011 at 12:02:59PM -0600, Anthony Liguori wrote:
> We really need to update SeaBIOS whenever there is a bug that we
> know requires an update.  Things breakdown because of one or more of
> the following reasons:
> 
> 1) User submits a patch to seabios@, Kevin applies it.  But that
> doesn't necessarily trigger anything happening in QEMU.
> 
> Ideally, the above mentioned user would submit a submodule update once (1) happens.
> 
> 2) Kevin fixes something on his own or someone else changes
> something in the broader SeaBIOS community.  That may not even be
> visible in QEMU.

There is another complexity here - it's not always clear to me when a
group pulls a particular revision of SeaBIOS.  So, knowing who to
notify is harder.

> Syncing right before release isn't a good strategy either because
> that means we're pulling in something that hasn't been tested
> extensively at the very tail end of our release cycle.

Agreed.  There has to be a balance here.

There are some USB drive booting fixes along with some ACPI and
MPTable changes in SeaBIOS post v1.6.3.1.  These changes are a bit
large though, so I'm not sure QEMU would be best served by pulling
them in if a release is pending.

That said, I'm glad to see users testing recent SeaBIOS revs as it
helps greatly with shaking out issues.  For example, had QEMU not
pulled a revision of SeaBIOS in August, there's a good chance this
particular bug would not have been found before the v1.6.3 release and
we might still have ended up in the same situation.

> I would like to point out that August -> October is a pretty long
> time period for a regression like this to exist.  I think that
> really indicates that the primary problem is testing, not frequency
> of SeaBIOS updates.

If we can catch these types of things in test cases, that would be
great.  This particular bug had a complex set of triggers - it was in
SeaBIOS code specific to QEMU (so non-QEMU/KVM users wouldn't find
it), using QEMU's default Cirrus VGA driver masks the bug (it happens
to have PCI prefmem), and it was an off-by-one in low-level alignment
code (a code review wouldn't catch it).

-Kevin

Re: [Qemu-devel] insmod virtio-blk is broken in qemu 1.0

[Gleb Natapov]

On Mon, Dec 19, 2011 at 10:38:02PM -0500, Kevin O'Connor wrote:
> On Mon, Dec 19, 2011 at 12:02:59PM -0600, Anthony Liguori wrote:
> > We really need to update SeaBIOS whenever there is a bug that we
> > know requires an update.  Things breakdown because of one or more of
> > the following reasons:
> > 
> > 1) User submits a patch to seabios@, Kevin applies it.  But that
> > doesn't necessarily trigger anything happening in QEMU.
> > 
> > Ideally, the above mentioned user would submit a submodule update once (1) happens.
> > 
> > 2) Kevin fixes something on his own or someone else changes
> > something in the broader SeaBIOS community.  That may not even be
> > visible in QEMU.
> 
> There is another complexity here - it's not always clear to me when a
> group pulls a particular revision of SeaBIOS.  So, knowing who to
> notify is harder.
> 
Why not just pull recent Seabios into QEMU weekly (or as fast as Seabios
HEAD moves). Before going into qemu rc stage ask Seabios for formal
release.

--
			Gleb.