* KVM git hangs with if=virtio (works under kvm 0.12.3) @ 2010-07-02 3:31 ewheeler 2010-07-02 8:07 ` Stefan Hajnoczi 0 siblings, 1 reply; 11+ messages in thread From: ewheeler @ 2010-07-02 3:31 UTC (permalink / raw) To: kvm Hello all, I'm booting a CentOS kernel under today's KVM git and it hangs after initializing the serial port when the drive if=virtio, but not when drive if=ide. Look close---this is not a "forgot to add virtio_blk" problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. Reproduction: Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms +0ubuntu9) it will work properly: qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus As expected, the kernel panics unable to mount root (good-boot.png). This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. ---However---if I use today's git (2010-07-01) of kvm: /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus This hangs just after initializing the Serial device (obtained by adding -serial stdio -append console=ttyS0): Note that this only happens with the disk interface set to virtio (if=virtio). It works fine for ide (if=ide). Am I doing something wrong here? Is anyone else having this problem? The pictures, disk and kernel image is available here: http://www.portlandlinuxsupport.com/src/kvm-20100701-regression/ -Eric More detail: ==== snip ==== (see also bad-boot.png) PCI: PIIX3: Enabling Passive Release on 0000:00:01.0 Activating ISA DMA hang workarounds. pci_hotplug: PCI Hot Plug PCI Core version: 0.5 Real Time Clock Driver v1.12ac Non-volatile memory driver v1.2 Linux agpgart interface v0.101 (c) Dave Jones Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled [[ hangs here and spins at >100% cpu ]] ==== snip ==== And the qemu-system-x86_64 process spins at >100% cpu: == snip from top == PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 8895 root 20 0 287m 28m 3124 S 102 0.5 0:56.82 qemu-system-x86 == snip from top == ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-02 3:31 KVM git hangs with if=virtio (works under kvm 0.12.3) ewheeler @ 2010-07-02 8:07 ` Stefan Hajnoczi 2010-07-05 12:11 ` Stefan Hajnoczi 0 siblings, 1 reply; 11+ messages in thread From: Stefan Hajnoczi @ 2010-07-02 8:07 UTC (permalink / raw) To: ewheeler; +Cc: kvm On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: > Hello all, > > I'm booting a CentOS kernel under today's KVM git and it hangs after > initializing the serial port when the drive if=virtio, but not when > drive if=ide. Look close---this is not a "forgot to add virtio_blk" > problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. > > Reproduction: > > Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms > +0ubuntu9) it will work properly: > > qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > > As expected, the kernel panics unable to mount root (good-boot.png). > This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. > > ---However---if I use today's git (2010-07-01) of kvm: > > /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > > This hangs just after initializing the Serial device (obtained by adding > -serial stdio -append console=ttyS0): > > Note that this only happens with the disk interface set to virtio > (if=virtio). It works fine for ide (if=ide). > > > Am I doing something wrong here? > Is anyone else having this problem? I have seen this issue with a RHEL 5.5 guest running under qemu-kvm.git. It boots a new guest fine but hangs as you described with the RHEL 5.5 kernel. I have not investigated. Stefan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-02 8:07 ` Stefan Hajnoczi @ 2010-07-05 12:11 ` Stefan Hajnoczi 2010-07-05 12:17 ` Gleb Natapov 0 siblings, 1 reply; 11+ messages in thread From: Stefan Hajnoczi @ 2010-07-05 12:11 UTC (permalink / raw) To: ewheeler; +Cc: kvm On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: >> Hello all, >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after >> initializing the serial port when the drive if=virtio, but not when >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. >> >> Reproduction: >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms >> +0ubuntu9) it will work properly: >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> As expected, the kernel panics unable to mount root (good-boot.png). >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. >> >> ---However---if I use today's git (2010-07-01) of kvm: >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> This hangs just after initializing the Serial device (obtained by adding >> -serial stdio -append console=ttyS0): >> >> Note that this only happens with the disk interface set to virtio >> (if=virtio). It works fine for ide (if=ide). >> >> >> Am I doing something wrong here? >> Is anyone else having this problem? > > I have seen this issue with a RHEL 5.5 guest running under > qemu-kvm.git. It boots a new guest fine but hangs as you described > with the RHEL 5.5 kernel. I have not investigated. This issue is affected by extboot, a feature that enables booting from virtio-blk devices. I have just sent a patch to the KVM mailing list to restore extboot functionality which has been broken in qemu-kvm.git. That patch can be used to work around this issue by using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 kernel hangs during serial initialization when extboot is not present. Stefan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-05 12:11 ` Stefan Hajnoczi @ 2010-07-05 12:17 ` Gleb Natapov 2010-07-05 12:36 ` Stefan Hajnoczi 0 siblings, 1 reply; 11+ messages in thread From: Gleb Natapov @ 2010-07-05 12:17 UTC (permalink / raw) To: Stefan Hajnoczi; +Cc: ewheeler, kvm On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: > On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: > > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: > >> Hello all, > >> > >> I'm booting a CentOS kernel under today's KVM git and it hangs after > >> initializing the serial port when the drive if=virtio, but not when > >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" > >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. > >> > >> Reproduction: > >> > >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms > >> +0ubuntu9) it will work properly: > >> > >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> > >> As expected, the kernel panics unable to mount root (good-boot.png). > >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. > >> > >> ---However---if I use today's git (2010-07-01) of kvm: > >> > >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> > >> This hangs just after initializing the Serial device (obtained by adding > >> -serial stdio -append console=ttyS0): > >> > >> Note that this only happens with the disk interface set to virtio > >> (if=virtio). It works fine for ide (if=ide). > >> > >> > >> Am I doing something wrong here? > >> Is anyone else having this problem? > > > > I have seen this issue with a RHEL 5.5 guest running under > > qemu-kvm.git. It boots a new guest fine but hangs as you described > > with the RHEL 5.5 kernel. I have not investigated. > > This issue is affected by extboot, a feature that enables booting from > virtio-blk devices. I have just sent a patch to the KVM mailing list > to restore extboot functionality which has been broken in > qemu-kvm.git. That patch can be used to work around this issue by > using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 > kernel hangs during serial initialization when extboot is not present. > Hang that happens during guest boot (after bootloader started the kernel) cannot be worked around by extboot. extboot is also not needed with latest qemu git to boot from virtio disks since the support for that is in the bios now. -- Gleb. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-05 12:17 ` Gleb Natapov @ 2010-07-05 12:36 ` Stefan Hajnoczi 2010-07-05 12:41 ` Gleb Natapov 0 siblings, 1 reply; 11+ messages in thread From: Stefan Hajnoczi @ 2010-07-05 12:36 UTC (permalink / raw) To: Gleb Natapov; +Cc: ewheeler, kvm 2010/7/5 Gleb Natapov <gleb@redhat.com>: > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: >> >> Hello all, >> >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after >> >> initializing the serial port when the drive if=virtio, but not when >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. >> >> >> >> Reproduction: >> >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms >> >> +0ubuntu9) it will work properly: >> >> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> >> >> As expected, the kernel panics unable to mount root (good-boot.png). >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. >> >> >> >> ---However---if I use today's git (2010-07-01) of kvm: >> >> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> >> >> This hangs just after initializing the Serial device (obtained by adding >> >> -serial stdio -append console=ttyS0): >> >> >> >> Note that this only happens with the disk interface set to virtio >> >> (if=virtio). It works fine for ide (if=ide). >> >> >> >> >> >> Am I doing something wrong here? >> >> Is anyone else having this problem? >> > >> > I have seen this issue with a RHEL 5.5 guest running under >> > qemu-kvm.git. It boots a new guest fine but hangs as you described >> > with the RHEL 5.5 kernel. I have not investigated. >> >> This issue is affected by extboot, a feature that enables booting from >> virtio-blk devices. I have just sent a patch to the KVM mailing list >> to restore extboot functionality which has been broken in >> qemu-kvm.git. That patch can be used to work around this issue by >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 >> kernel hangs during serial initialization when extboot is not present. >> > Hang that happens during guest boot (after bootloader started the > kernel) cannot be worked around by extboot. extboot is also not needed > with latest qemu git to boot from virtio disks since the support for > that is in the bios now. I agree that something else is going on here and needs to be investigated, but I do think that extboot can indirectly affect the guest boot. With extboot the virtio-blk PCI adapter is not touched by the firmware/bootloader. Is it possible that a virtio-blk interrupt is raised and not acknowledged before entering Linux. When Linux brings up the serial port it gets swamped with interrupts? That's just a guess. Newer (non-RHEL 5.5) kernels boot fine without extboot, so another path to solving this bug is to bisect guest kernels. Stefan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-05 12:36 ` Stefan Hajnoczi @ 2010-07-05 12:41 ` Gleb Natapov 2010-07-07 9:03 ` Stefan Hajnoczi 0 siblings, 1 reply; 11+ messages in thread From: Gleb Natapov @ 2010-07-05 12:41 UTC (permalink / raw) To: Stefan Hajnoczi; +Cc: ewheeler, kvm On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote: > 2010/7/5 Gleb Natapov <gleb@redhat.com>: > > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: > >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: > >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: > >> >> Hello all, > >> >> > >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after > >> >> initializing the serial port when the drive if=virtio, but not when > >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" > >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. > >> >> > >> >> Reproduction: > >> >> > >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms > >> >> +0ubuntu9) it will work properly: > >> >> > >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> > >> >> As expected, the kernel panics unable to mount root (good-boot.png). > >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. > >> >> > >> >> ---However---if I use today's git (2010-07-01) of kvm: > >> >> > >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> > >> >> This hangs just after initializing the Serial device (obtained by adding > >> >> -serial stdio -append console=ttyS0): > >> >> > >> >> Note that this only happens with the disk interface set to virtio > >> >> (if=virtio). It works fine for ide (if=ide). > >> >> > >> >> > >> >> Am I doing something wrong here? > >> >> Is anyone else having this problem? > >> > > >> > I have seen this issue with a RHEL 5.5 guest running under > >> > qemu-kvm.git. It boots a new guest fine but hangs as you described > >> > with the RHEL 5.5 kernel. I have not investigated. > >> > >> This issue is affected by extboot, a feature that enables booting from > >> virtio-blk devices. I have just sent a patch to the KVM mailing list > >> to restore extboot functionality which has been broken in > >> qemu-kvm.git. That patch can be used to work around this issue by > >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 > >> kernel hangs during serial initialization when extboot is not present. > >> > > Hang that happens during guest boot (after bootloader started the > > kernel) cannot be worked around by extboot. extboot is also not needed > > with latest qemu git to boot from virtio disks since the support for > > that is in the bios now. > > I agree that something else is going on here and needs to be > investigated, but I do think that extboot can indirectly affect the > guest boot. > > With extboot the virtio-blk PCI adapter is not touched by the > firmware/bootloader. Is it possible that a virtio-blk interrupt is > raised and not acknowledged before entering Linux. When Linux brings > up the serial port it gets swamped with interrupts? That's just a > guess. > That is possible when bios is actually used to boot the guest, but bug reporter uses -kernel option so no bios boot code should run at all. Virtio is initialized anyway, but this will happen with boot=on too. > Newer (non-RHEL 5.5) kernels boot fine without extboot, so another > path to solving this bug is to bisect guest kernels. > > Stefan -- Gleb. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-05 12:41 ` Gleb Natapov @ 2010-07-07 9:03 ` Stefan Hajnoczi 2010-07-07 9:13 ` Gleb Natapov 2010-07-07 20:28 ` ewheeler 0 siblings, 2 replies; 11+ messages in thread From: Stefan Hajnoczi @ 2010-07-07 9:03 UTC (permalink / raw) To: Gleb Natapov; +Cc: ewheeler, kvm, Anthony Liguori, Avi Kivity 2010/7/5 Gleb Natapov <gleb@redhat.com>: > On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote: >> 2010/7/5 Gleb Natapov <gleb@redhat.com>: >> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: >> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: >> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: >> >> >> Hello all, >> >> >> >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after >> >> >> initializing the serial port when the drive if=virtio, but not when >> >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" >> >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. >> >> >> >> >> >> Reproduction: >> >> >> >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms >> >> >> +0ubuntu9) it will work properly: >> >> >> >> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> >> >> >> >> As expected, the kernel panics unable to mount root (good-boot.png). >> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. >> >> >> >> >> >> ---However---if I use today's git (2010-07-01) of kvm: >> >> >> >> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> >> >> >> >> This hangs just after initializing the Serial device (obtained by adding >> >> >> -serial stdio -append console=ttyS0): >> >> >> >> >> >> Note that this only happens with the disk interface set to virtio >> >> >> (if=virtio). It works fine for ide (if=ide). >> >> >> >> >> >> >> >> >> Am I doing something wrong here? >> >> >> Is anyone else having this problem? >> >> > >> >> > I have seen this issue with a RHEL 5.5 guest running under >> >> > qemu-kvm.git. It boots a new guest fine but hangs as you described >> >> > with the RHEL 5.5 kernel. I have not investigated. >> >> >> >> This issue is affected by extboot, a feature that enables booting from >> >> virtio-blk devices. I have just sent a patch to the KVM mailing list >> >> to restore extboot functionality which has been broken in >> >> qemu-kvm.git. That patch can be used to work around this issue by >> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 >> >> kernel hangs during serial initialization when extboot is not present. >> >> >> > Hang that happens during guest boot (after bootloader started the >> > kernel) cannot be worked around by extboot. extboot is also not needed >> > with latest qemu git to boot from virtio disks since the support for >> > that is in the bios now. >> >> I agree that something else is going on here and needs to be >> investigated, but I do think that extboot can indirectly affect the >> guest boot. >> >> With extboot the virtio-blk PCI adapter is not touched by the >> firmware/bootloader. Is it possible that a virtio-blk interrupt is >> raised and not acknowledged before entering Linux. When Linux brings >> up the serial port it gets swamped with interrupts? That's just a >> guess. >> > That is possible when bios is actually used to boot the guest, but bug > reporter uses -kernel option so no bios boot code should run at all. > Virtio is initialized anyway, but this will happen with boot=on too. Okay, I got to the bottom of this. Here's the story, see bottom of the mail for the solution and workarounds: It turns out that -kernel does involve the BIOS. KVM pulls apart the bzImage and makes it available via the fw_cfg interface. The linuxboot.bin option ROM is executed by the BIOS inside the VM to actually jump into the kernel. This means the BIOS does POST and sets itself up; the kernel's real-mode boot code is going to use BIOS interrupts. So now the VM has booted, BIOS finished POST and executed linuxboot.bin, linuxboot.bin transferred control to Linux. Then, in Linux arch/x86/boot/edd.c a disk read to sector=0 bytes=512 is made using INT 13h. Since this disk read comes from the Linux kernel, it happens regardless of -kernel or not. The disk read is serviced by the BIOS. Older versions of SeaBIOS leave the interrupt raised here, so then the kernel hangs in serial initialization later. However, there is a simple workaround: x86_64-softmmu/qemu-system-x86_64 -m 512 -drive file=~/rhel5u5.img,if=virtio -kernel /boot/vmlinuz-2.6.32-5-amd64 -append edd=skipmbr When the edd=skipmbr kernel parameter is used, the kernel will not perform the disk read and the interrupt will never get stuck. The VM boots successfully. The edd=skipmbr workaround is not enough when booting from disk and not -kernel. The BIOS and bootloader will invoke INT 13h and the only way around that is to use extboot.bin with boot=on as I originally described. The root cause was fixed in SeaBIOS commit: commit 4030db0d2c5a79de2a1b5c31514503e4ff2a3cd1 Author: Gleb Natapov <gleb@redhat.com> Date: Mon May 17 16:27:27 2010 +0300 fix two issues with virtio-blk 1. Check if blk_size is valid in virtio_blk config. 2. Disable interrupt otherwise interrupt may stuck with some guests. If you build SeaBIOS from source and launch KVM with -bios path/to/seabios/out/bios.bin, RHEL5.5 will boot without hanging at serial initialization time. QEMU and KVM need to grab a later SeaBIOS build that contains 4030db0. Stefan PS: If you test this on qemu.git, you may find that the kernel doesn't hang at serial initialization, despite the bug being present in the BIOS. I'm not sure how QEMU gets away with it but I wonder if the interrupt configuration is different. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-07 9:03 ` Stefan Hajnoczi @ 2010-07-07 9:13 ` Gleb Natapov 2010-07-07 9:36 ` Stefan Hajnoczi 2010-07-07 20:28 ` ewheeler 1 sibling, 1 reply; 11+ messages in thread From: Gleb Natapov @ 2010-07-07 9:13 UTC (permalink / raw) To: Stefan Hajnoczi; +Cc: ewheeler, kvm, Anthony Liguori, Avi Kivity On Wed, Jul 07, 2010 at 10:03:31AM +0100, Stefan Hajnoczi wrote: > 2010/7/5 Gleb Natapov <gleb@redhat.com>: > > On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote: > >> 2010/7/5 Gleb Natapov <gleb@redhat.com>: > >> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: > >> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: > >> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: > >> >> >> Hello all, > >> >> >> > >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after > >> >> >> initializing the serial port when the drive if=virtio, but not when > >> >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" > >> >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. > >> >> >> > >> >> >> Reproduction: > >> >> >> > >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms > >> >> >> +0ubuntu9) it will work properly: > >> >> >> > >> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> >> > >> >> >> As expected, the kernel panics unable to mount root (good-boot.png). > >> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. > >> >> >> > >> >> >> ---However---if I use today's git (2010-07-01) of kvm: > >> >> >> > >> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> >> > >> >> >> This hangs just after initializing the Serial device (obtained by adding > >> >> >> -serial stdio -append console=ttyS0): > >> >> >> > >> >> >> Note that this only happens with the disk interface set to virtio > >> >> >> (if=virtio). It works fine for ide (if=ide). > >> >> >> > >> >> >> > >> >> >> Am I doing something wrong here? > >> >> >> Is anyone else having this problem? > >> >> > > >> >> > I have seen this issue with a RHEL 5.5 guest running under > >> >> > qemu-kvm.git. It boots a new guest fine but hangs as you described > >> >> > with the RHEL 5.5 kernel. I have not investigated. > >> >> > >> >> This issue is affected by extboot, a feature that enables booting from > >> >> virtio-blk devices. I have just sent a patch to the KVM mailing list > >> >> to restore extboot functionality which has been broken in > >> >> qemu-kvm.git. That patch can be used to work around this issue by > >> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 > >> >> kernel hangs during serial initialization when extboot is not present. > >> >> > >> > Hang that happens during guest boot (after bootloader started the > >> > kernel) cannot be worked around by extboot. extboot is also not needed > >> > with latest qemu git to boot from virtio disks since the support for > >> > that is in the bios now. > >> > >> I agree that something else is going on here and needs to be > >> investigated, but I do think that extboot can indirectly affect the > >> guest boot. > >> > >> With extboot the virtio-blk PCI adapter is not touched by the > >> firmware/bootloader. Is it possible that a virtio-blk interrupt is > >> raised and not acknowledged before entering Linux. When Linux brings > >> up the serial port it gets swamped with interrupts? That's just a > >> guess. > >> > > That is possible when bios is actually used to boot the guest, but bug > > reporter uses -kernel option so no bios boot code should run at all. > > Virtio is initialized anyway, but this will happen with boot=on too. > > Okay, I got to the bottom of this. Here's the story, see bottom of the mail for > the solution and workarounds: > Great. Thanks Stefan. I was sure that fix you found is in qemu.git already. It is good to know that Linux actually uses int13 once. Didn't know that. To be absolutely sure no interrupt will stuck we should probably clear interrupt after each virtio read in bios by reading status register. > It turns out that -kernel does involve the BIOS. KVM pulls apart the bzImage > and makes it available via the fw_cfg interface. The linuxboot.bin option ROM > is executed by the BIOS inside the VM to actually jump into the kernel. This > means the BIOS does POST and sets itself up; the kernel's real-mode boot > code is going to use BIOS interrupts. > > So now the VM has booted, BIOS finished POST and executed linuxboot.bin, > linuxboot.bin transferred control to Linux. Then, in Linux arch/x86/boot/edd.c > a disk read to sector=0 bytes=512 is made using INT 13h. Since this disk > read comes from the Linux kernel, it happens regardless of -kernel or not. > > The disk read is serviced by the BIOS. Older versions of SeaBIOS leave the > interrupt raised here, so then the kernel hangs in serial initialization later. > > However, there is a simple workaround: > > x86_64-softmmu/qemu-system-x86_64 -m 512 -drive > file=~/rhel5u5.img,if=virtio -kernel /boot/vmlinuz-2.6.32-5-amd64 > -append edd=skipmbr > > When the edd=skipmbr kernel parameter is used, the kernel will not perform > the disk read and the interrupt will never get stuck. The VM boots > successfully. > > The edd=skipmbr workaround is not enough when booting from disk and not > -kernel. The BIOS and bootloader will invoke INT 13h and the only way > around that is to use extboot.bin with boot=on as I originally described. > > The root cause was fixed in SeaBIOS commit: > > commit 4030db0d2c5a79de2a1b5c31514503e4ff2a3cd1 > Author: Gleb Natapov <gleb@redhat.com> > Date: Mon May 17 16:27:27 2010 +0300 > > fix two issues with virtio-blk > > 1. Check if blk_size is valid in virtio_blk config. > 2. Disable interrupt otherwise interrupt may stuck > with some guests. > > If you build SeaBIOS from source and launch KVM with -bios > path/to/seabios/out/bios.bin, RHEL5.5 will boot without hanging at serial > initialization time. > > QEMU and KVM need to grab a later SeaBIOS build that contains 4030db0. > > Stefan > > PS: If you test this on qemu.git, you may find that the kernel doesn't hang > at serial initialization, despite the bug being present in the BIOS. I'm not > sure how QEMU gets away with it but I wonder if the interrupt configuration > is different. -- Gleb. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-07 9:13 ` Gleb Natapov @ 2010-07-07 9:36 ` Stefan Hajnoczi 2010-07-07 9:38 ` Gleb Natapov 0 siblings, 1 reply; 11+ messages in thread From: Stefan Hajnoczi @ 2010-07-07 9:36 UTC (permalink / raw) To: Gleb Natapov; +Cc: ewheeler, kvm, Anthony Liguori, Avi Kivity 2010/7/7 Gleb Natapov <gleb@redhat.com>: > On Wed, Jul 07, 2010 at 10:03:31AM +0100, Stefan Hajnoczi wrote: >> 2010/7/5 Gleb Natapov <gleb@redhat.com>: >> > On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote: >> >> 2010/7/5 Gleb Natapov <gleb@redhat.com>: >> >> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: >> >> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: >> >> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: >> >> >> >> Hello all, >> >> >> >> >> >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after >> >> >> >> initializing the serial port when the drive if=virtio, but not when >> >> >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" >> >> >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. >> >> >> >> >> >> >> >> Reproduction: >> >> >> >> >> >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms >> >> >> >> +0ubuntu9) it will work properly: >> >> >> >> >> >> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> >> >> >> >> >> >> As expected, the kernel panics unable to mount root (good-boot.png). >> >> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. >> >> >> >> >> >> >> >> ---However---if I use today's git (2010-07-01) of kvm: >> >> >> >> >> >> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ >> >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus >> >> >> >> >> >> >> >> This hangs just after initializing the Serial device (obtained by adding >> >> >> >> -serial stdio -append console=ttyS0): >> >> >> >> >> >> >> >> Note that this only happens with the disk interface set to virtio >> >> >> >> (if=virtio). It works fine for ide (if=ide). >> >> >> >> >> >> >> >> >> >> >> >> Am I doing something wrong here? >> >> >> >> Is anyone else having this problem? >> >> >> > >> >> >> > I have seen this issue with a RHEL 5.5 guest running under >> >> >> > qemu-kvm.git. It boots a new guest fine but hangs as you described >> >> >> > with the RHEL 5.5 kernel. I have not investigated. >> >> >> >> >> >> This issue is affected by extboot, a feature that enables booting from >> >> >> virtio-blk devices. I have just sent a patch to the KVM mailing list >> >> >> to restore extboot functionality which has been broken in >> >> >> qemu-kvm.git. That patch can be used to work around this issue by >> >> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 >> >> >> kernel hangs during serial initialization when extboot is not present. >> >> >> >> >> > Hang that happens during guest boot (after bootloader started the >> >> > kernel) cannot be worked around by extboot. extboot is also not needed >> >> > with latest qemu git to boot from virtio disks since the support for >> >> > that is in the bios now. >> >> >> >> I agree that something else is going on here and needs to be >> >> investigated, but I do think that extboot can indirectly affect the >> >> guest boot. >> >> >> >> With extboot the virtio-blk PCI adapter is not touched by the >> >> firmware/bootloader. Is it possible that a virtio-blk interrupt is >> >> raised and not acknowledged before entering Linux. When Linux brings >> >> up the serial port it gets swamped with interrupts? That's just a >> >> guess. >> >> >> > That is possible when bios is actually used to boot the guest, but bug >> > reporter uses -kernel option so no bios boot code should run at all. >> > Virtio is initialized anyway, but this will happen with boot=on too. >> >> Okay, I got to the bottom of this. Here's the story, see bottom of the mail for >> the solution and workarounds: >> > Great. Thanks Stefan. I was sure that fix you found is in qemu.git > already. It is good to know that Linux actually uses int13 once. Didn't > know that. To be absolutely sure no interrupt will stuck we should probably > clear interrupt after each virtio read in bios by reading status register. Yes, this is the approach I took when rewriting gPXE's virtio-net driver recently: http://git.etherboot.org/?p=people/stefanha/gpxe.git;a=blob;f=src/drivers/net/virtio-net.c;h=50c580049c3663f0d1332d51e30c8a59b5663b68;hb=d58bf6fc8db4a55075e370c7e1d436a570400f55#l299 Are you spinning a patch or should I submit one? >> It turns out that -kernel does involve the BIOS. KVM pulls apart the bzImage >> and makes it available via the fw_cfg interface. The linuxboot.bin option ROM >> is executed by the BIOS inside the VM to actually jump into the kernel. This >> means the BIOS does POST and sets itself up; the kernel's real-mode boot >> code is going to use BIOS interrupts. >> >> So now the VM has booted, BIOS finished POST and executed linuxboot.bin, >> linuxboot.bin transferred control to Linux. Then, in Linux arch/x86/boot/edd.c >> a disk read to sector=0 bytes=512 is made using INT 13h. Since this disk >> read comes from the Linux kernel, it happens regardless of -kernel or not. >> >> The disk read is serviced by the BIOS. Older versions of SeaBIOS leave the >> interrupt raised here, so then the kernel hangs in serial initialization later. >> >> However, there is a simple workaround: >> >> x86_64-softmmu/qemu-system-x86_64 -m 512 -drive >> file=~/rhel5u5.img,if=virtio -kernel /boot/vmlinuz-2.6.32-5-amd64 >> -append edd=skipmbr >> >> When the edd=skipmbr kernel parameter is used, the kernel will not perform >> the disk read and the interrupt will never get stuck. The VM boots >> successfully. >> >> The edd=skipmbr workaround is not enough when booting from disk and not >> -kernel. The BIOS and bootloader will invoke INT 13h and the only way >> around that is to use extboot.bin with boot=on as I originally described. >> >> The root cause was fixed in SeaBIOS commit: >> >> commit 4030db0d2c5a79de2a1b5c31514503e4ff2a3cd1 >> Author: Gleb Natapov <gleb@redhat.com> >> Date: Mon May 17 16:27:27 2010 +0300 >> >> fix two issues with virtio-blk >> >> 1. Check if blk_size is valid in virtio_blk config. >> 2. Disable interrupt otherwise interrupt may stuck >> with some guests. >> >> If you build SeaBIOS from source and launch KVM with -bios >> path/to/seabios/out/bios.bin, RHEL5.5 will boot without hanging at serial >> initialization time. >> >> QEMU and KVM need to grab a later SeaBIOS build that contains 4030db0. >> >> Stefan >> >> PS: If you test this on qemu.git, you may find that the kernel doesn't hang >> at serial initialization, despite the bug being present in the BIOS. I'm not >> sure how QEMU gets away with it but I wonder if the interrupt configuration >> is different. > > -- > Gleb. > Stefan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-07 9:36 ` Stefan Hajnoczi @ 2010-07-07 9:38 ` Gleb Natapov 0 siblings, 0 replies; 11+ messages in thread From: Gleb Natapov @ 2010-07-07 9:38 UTC (permalink / raw) To: Stefan Hajnoczi; +Cc: ewheeler, kvm, Anthony Liguori, Avi Kivity On Wed, Jul 07, 2010 at 10:36:17AM +0100, Stefan Hajnoczi wrote: > 2010/7/7 Gleb Natapov <gleb@redhat.com>: > > On Wed, Jul 07, 2010 at 10:03:31AM +0100, Stefan Hajnoczi wrote: > >> 2010/7/5 Gleb Natapov <gleb@redhat.com>: > >> > On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote: > >> >> 2010/7/5 Gleb Natapov <gleb@redhat.com>: > >> >> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: > >> >> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: > >> >> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: > >> >> >> >> Hello all, > >> >> >> >> > >> >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after > >> >> >> >> initializing the serial port when the drive if=virtio, but not when > >> >> >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" > >> >> >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. > >> >> >> >> > >> >> >> >> Reproduction: > >> >> >> >> > >> >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms > >> >> >> >> +0ubuntu9) it will work properly: > >> >> >> >> > >> >> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> >> >> > >> >> >> >> As expected, the kernel panics unable to mount root (good-boot.png). > >> >> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. > >> >> >> >> > >> >> >> >> ---However---if I use today's git (2010-07-01) of kvm: > >> >> >> >> > >> >> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> >> >> > >> >> >> >> This hangs just after initializing the Serial device (obtained by adding > >> >> >> >> -serial stdio -append console=ttyS0): > >> >> >> >> > >> >> >> >> Note that this only happens with the disk interface set to virtio > >> >> >> >> (if=virtio). It works fine for ide (if=ide). > >> >> >> >> > >> >> >> >> > >> >> >> >> Am I doing something wrong here? > >> >> >> >> Is anyone else having this problem? > >> >> >> > > >> >> >> > I have seen this issue with a RHEL 5.5 guest running under > >> >> >> > qemu-kvm.git. It boots a new guest fine but hangs as you described > >> >> >> > with the RHEL 5.5 kernel. I have not investigated. > >> >> >> > >> >> >> This issue is affected by extboot, a feature that enables booting from > >> >> >> virtio-blk devices. I have just sent a patch to the KVM mailing list > >> >> >> to restore extboot functionality which has been broken in > >> >> >> qemu-kvm.git. That patch can be used to work around this issue by > >> >> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 > >> >> >> kernel hangs during serial initialization when extboot is not present. > >> >> >> > >> >> > Hang that happens during guest boot (after bootloader started the > >> >> > kernel) cannot be worked around by extboot. extboot is also not needed > >> >> > with latest qemu git to boot from virtio disks since the support for > >> >> > that is in the bios now. > >> >> > >> >> I agree that something else is going on here and needs to be > >> >> investigated, but I do think that extboot can indirectly affect the > >> >> guest boot. > >> >> > >> >> With extboot the virtio-blk PCI adapter is not touched by the > >> >> firmware/bootloader. Is it possible that a virtio-blk interrupt is > >> >> raised and not acknowledged before entering Linux. When Linux brings > >> >> up the serial port it gets swamped with interrupts? That's just a > >> >> guess. > >> >> > >> > That is possible when bios is actually used to boot the guest, but bug > >> > reporter uses -kernel option so no bios boot code should run at all. > >> > Virtio is initialized anyway, but this will happen with boot=on too. > >> > >> Okay, I got to the bottom of this. Here's the story, see bottom of the mail for > >> the solution and workarounds: > >> > > Great. Thanks Stefan. I was sure that fix you found is in qemu.git > > already. It is good to know that Linux actually uses int13 once. Didn't > > know that. To be absolutely sure no interrupt will stuck we should probably > > clear interrupt after each virtio read in bios by reading status register. > > Yes, this is the approach I took when rewriting gPXE's virtio-net > driver recently: > > http://git.etherboot.org/?p=people/stefanha/gpxe.git;a=blob;f=src/drivers/net/virtio-net.c;h=50c580049c3663f0d1332d51e30c8a59b5663b68;hb=d58bf6fc8db4a55075e370c7e1d436a570400f55#l299 > > Are you spinning a patch or should I submit one? > If you have time do it please. I am a little bit busy right now :( -- Gleb. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: KVM git hangs with if=virtio (works under kvm 0.12.3) 2010-07-07 9:03 ` Stefan Hajnoczi 2010-07-07 9:13 ` Gleb Natapov @ 2010-07-07 20:28 ` ewheeler 1 sibling, 0 replies; 11+ messages in thread From: ewheeler @ 2010-07-07 20:28 UTC (permalink / raw) To: Stefan Hajnoczi; +Cc: Gleb Natapov, kvm, Anthony Liguori, Avi Kivity On Wed, 2010-07-07 at 10:03 +0100, Stefan Hajnoczi wrote: > 2010/7/5 Gleb Natapov <gleb@redhat.com>: > > On Mon, Jul 05, 2010 at 01:36:08PM +0100, Stefan Hajnoczi wrote: > >> 2010/7/5 Gleb Natapov <gleb@redhat.com>: > >> > On Mon, Jul 05, 2010 at 01:11:25PM +0100, Stefan Hajnoczi wrote: > >> >> On Fri, Jul 2, 2010 at 9:07 AM, Stefan Hajnoczi <stefanha@gmail.com> wrote: > >> >> > On Fri, Jul 2, 2010 at 4:31 AM, ewheeler <kvm@ew.ewheeler.org> wrote: > >> >> >> Hello all, > >> >> >> > >> >> >> I'm booting a CentOS kernel under today's KVM git and it hangs after > >> >> >> initializing the serial port when the drive if=virtio, but not when > >> >> >> drive if=ide. Look close---this is not a "forgot to add virtio_blk" > >> >> >> problem. If I use 0.12.3 from Ubuntu 10.04 it works properly. > >> >> >> > >> >> >> Reproduction: > >> >> >> > >> >> >> Using kvm 0.12.3 on ubuntu 10.04 (1:84+dfsg-0ubuntu16+0.12.3+noroms > >> >> >> +0ubuntu9) it will work properly: > >> >> >> > >> >> >> qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> >> > >> >> >> As expected, the kernel panics unable to mount root (good-boot.png). > >> >> >> This makes sense, as "dummy-disk-image" is 1MB of 0x00 bytes. > >> >> >> > >> >> >> ---However---if I use today's git (2010-07-01) of kvm: > >> >> >> > >> >> >> /usr/local/kvm-git/bin/qemu-system-x86_64 -drive file=dummy-disk-image,if=virtio \ > >> >> >> -kernel vmlinuz-2.6.18-194.3.1.el5.centos.plus > >> >> >> > >> >> >> This hangs just after initializing the Serial device (obtained by adding > >> >> >> -serial stdio -append console=ttyS0): > >> >> >> > >> >> >> Note that this only happens with the disk interface set to virtio > >> >> >> (if=virtio). It works fine for ide (if=ide). > >> >> >> > >> >> >> > >> >> >> Am I doing something wrong here? > >> >> >> Is anyone else having this problem? > >> >> > > >> >> > I have seen this issue with a RHEL 5.5 guest running under > >> >> > qemu-kvm.git. It boots a new guest fine but hangs as you described > >> >> > with the RHEL 5.5 kernel. I have not investigated. > >> >> > >> >> This issue is affected by extboot, a feature that enables booting from > >> >> virtio-blk devices. I have just sent a patch to the KVM mailing list > >> >> to restore extboot functionality which has been broken in > >> >> qemu-kvm.git. That patch can be used to work around this issue by > >> >> using "-drive ...,boot=on" but it doesn't explain why the RHEL 5.5 > >> >> kernel hangs during serial initialization when extboot is not present. > >> >> > >> > Hang that happens during guest boot (after bootloader started the > >> > kernel) cannot be worked around by extboot. extboot is also not needed > >> > with latest qemu git to boot from virtio disks since the support for > >> > that is in the bios now. > >> > >> I agree that something else is going on here and needs to be > >> investigated, but I do think that extboot can indirectly affect the > >> guest boot. > >> > >> With extboot the virtio-blk PCI adapter is not touched by the > >> firmware/bootloader. Is it possible that a virtio-blk interrupt is > >> raised and not acknowledged before entering Linux. When Linux brings > >> up the serial port it gets swamped with interrupts? That's just a > >> guess. > >> > > That is possible when bios is actually used to boot the guest, but bug > > reporter uses -kernel option so no bios boot code should run at all. > > Virtio is initialized anyway, but this will happen with boot=on too. > > Okay, I got to the bottom of this. Here's the story, see bottom of the mail for > the solution and workarounds: > > It turns out that -kernel does involve the BIOS. KVM pulls apart the bzImage > and makes it available via the fw_cfg interface. The linuxboot.bin option ROM > is executed by the BIOS inside the VM to actually jump into the kernel. This > means the BIOS does POST and sets itself up; the kernel's real-mode boot > code is going to use BIOS interrupts. > > So now the VM has booted, BIOS finished POST and executed linuxboot.bin, > linuxboot.bin transferred control to Linux. Then, in Linux arch/x86/boot/edd.c > a disk read to sector=0 bytes=512 is made using INT 13h. Since this disk > read comes from the Linux kernel, it happens regardless of -kernel or not. > > The disk read is serviced by the BIOS. Older versions of SeaBIOS leave the > interrupt raised here, so then the kernel hangs in serial initialization later. > > However, there is a simple workaround: > > x86_64-softmmu/qemu-system-x86_64 -m 512 -drive > file=~/rhel5u5.img,if=virtio -kernel /boot/vmlinuz-2.6.32-5-amd64 > -append edd=skipmbr > > When the edd=skipmbr kernel parameter is used, the kernel will not perform > the disk read and the interrupt will never get stuck. The VM boots > successfully. Wow, brilliant. Thank you for jumping into this so quickly! -Eric ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2010-07-07 20:28 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-07-02 3:31 KVM git hangs with if=virtio (works under kvm 0.12.3) ewheeler 2010-07-02 8:07 ` Stefan Hajnoczi 2010-07-05 12:11 ` Stefan Hajnoczi 2010-07-05 12:17 ` Gleb Natapov 2010-07-05 12:36 ` Stefan Hajnoczi 2010-07-05 12:41 ` Gleb Natapov 2010-07-07 9:03 ` Stefan Hajnoczi 2010-07-07 9:13 ` Gleb Natapov 2010-07-07 9:36 ` Stefan Hajnoczi 2010-07-07 9:38 ` Gleb Natapov 2010-07-07 20:28 ` ewheeler
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.