[Qemu-devel] [Bug 688085] Re: Guest kernel hang during boot when KVM is active on i386 host

[Bug Watch Updater <688085@bugs.launchpad.net>]

Launchpad has imported 7 comments from the remote bug at
http://bugs.meego.com/show_bug.cgi?id=10075.

If you reply to an imported comment from within Launchpad, your comment
will be sent to the remote bug automatically. Read more about
Launchpad's inter-bugtracker facilities at
https://help.launchpad.net/InterBugTracking.

------------------------------------------------------------------------
On 2010-11-16T01:36:48+00:00 Muralidharan-jayavelu wrote:

BUILD IMAGE(meego-handset-ia32-qemu-1.1.20101031.2201-sda-runtime):

HARDWARE MODEL (on what HW this bug is uncovered):

BUG DETAILED DESCRIPTIONS
===========================================================
Trying to install and launch QEMU for handset by following instructions using the below link.
http://wiki.meego.com/SDK/Docs1.1 Configuring_QEMU_runtimes

After installation, trying to poweron the emulator using 'mad remote -r
meego-handset-ia32-qemu-1.1.20101031.2201-sda-runtime poweron'. QEMU
launches and stuck up in the boot mode.


EXPECTED OUTCOME:
===================
QEMU should launch successfully and should show the home screen of meego handset.

ACTUAL OUTCOME:
===================
QEMU is stuck up in the boot mode, showing messages like,
" Starting SeaBIOS (version .... )
 gPXE .....
 
 Booting from hard disk ....
 "

USER IMPACT:
===================
Unable to use QEMU emulator for development.

OTHER COMMENTS:
===================
Using Meego SDK in Ubuntu 10.10.

Reply at: https://bugs.launchpad.net/meego/+bug/688085/comments/0

------------------------------------------------------------------------
On 2010-11-20T04:54:03+00:00 Fathi-boudra wrote:

Please, specify target build and set the status to "accepted" if you
work on the issue.

Reply at: https://bugs.launchpad.net/meego/+bug/688085/comments/1

------------------------------------------------------------------------
On 2010-12-16T03:50:08+00:00 Zhiyuan-lv wrote:

*** Bug 11378 has been marked as a duplicate of this bug. ***

Reply at: https://bugs.launchpad.net/meego/+bug/688085/comments/4

------------------------------------------------------------------------
On 2010-12-16T03:51:39+00:00 Zhiyuan-lv wrote:

Copied from #11271 and #11378

Some findings of the QEMU-KVM issue on Ubuntu 10.10.

The QEMU-KVM hung during booting up meego-netbook kernel. The netbook kernel is
2.6.35 with some Intel patches. I did below test:

QEMU:
1, qemugl in MeeGo 1.1 SDK, which version is 0.12.4
2, qemu installed from Ubuntu 10.10 repo, which version is 0.12.5

Kernel:
a, handset netbook kernel in MeeGo 1.1 release, major version is 2.6.35
b, Ubuntu 10.10 kernel, major version is 2.6.35

                    Result       
1 to boot a         FAIL
1 to boot b         SUCCESS
2 to boot a         FAIL
2 to boot b         SUCCESS

So the problem is quite like that MeeGo kernel has something special which
cannot work well with the KVM in Ubuntu 10.10 host kernel.

Reply at: https://bugs.launchpad.net/meego/+bug/688085/comments/5

------------------------------------------------------------------------
On 2010-12-20T21:30:01+00:00 Zhiyuan-lv wrote:

With gdb to debug kernel running inside QEMU, it could be seen that QEMU
hangs at below inline asm code:


kernel-netbook-2.6.35.3/linux-2.6.35/arch/x86/include/asm/cmpxchg_32.h

static inline void set_64bit(volatile u64 *ptr, u64 value)
{
        u32 low  = value;
        u32 high = value >> 32;
        u64 prev = *ptr;

        asm volatile("\n1:\t"
                     LOCK_PREFIX "cmpxchg8b %0\n\t"
                     "jnz 1b"
                     : "=m" (*ptr), "+A" (prev)
                     : "b" (low), "c" (high)
                     : "memory");
......
}

The trace is as below:
#0  0xc1742bd9 in set_64bit () at /home/abuild/rpmbuild/BUILD/kernel-netbook-2.6.35.3/linux-2.6.35/arch/x86/include/asm/cmpxchg_32.h:74
#1  native_set_pmd () at /home/abuild/rpmbuild/BUILD/kernel-netbook-2.6.35.3/linux-2.6.35/arch/x86/include/asm/pgtable-3level.h:41
#2  pmd_populate_kernel () at /home/abuild/rpmbuild/BUILD/kernel-netbook-2.6.35.3/linux-2.6.35/arch/x86/include/asm/pgalloc.h:66
#3  early_ioremap_init () at arch/x86/mm/ioremap.c:382
#4  0xc173518c in ?? ()
#5  0xc1733545 in start_kernel () at init/main.c:573
#6  0xc17330ca in i386_start_kernel () at arch/x86/kernel/head32.c:72
#7  0x00000000 in ?? ()

Reply at: https://bugs.launchpad.net/meego/+bug/688085/comments/6

------------------------------------------------------------------------
On 2010-12-26T05:05:07+00:00 Zhiyuan-lv wrote:

Update some more investigation result.

The KVM (in 2.6.35.23 linux kernel on Ubuntu 10.10) seems to hang in
__vcpu_run() in arch/x86/kvm/x86.c. When the client OS is running above
instruction, KVM goes into dead loop in __vcpu_run(), because the
"vcpu_enter_guest ()" always returns "1".

The scenario seems to be as follow: The instruction triggered page fault
to exit kvm guest, and then "kvm_mmu_page_fault" was called to handle
the exception. The function returns non-zero value to let instruction be
run again, and suppose that the page fault has been resolved. But for
some reason, the page-fault exception was still triggered, and code run
infinitely in the loop.

Reply at: https://bugs.launchpad.net/meego/+bug/688085/comments/7

------------------------------------------------------------------------
On 2010-12-27T01:15:13+00:00 Zhiyuan-lv wrote:

Update some findings through GooGle:

1, Similar KVM hanging issues were reported, like:
https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/688085

According to the fix from Avi Kivity, it looks like a bug in KVM
handling 64-bit operand of "cmpxchg8b". But it is interesting that KVM
on Ubuntu 10.04 and ubuntu 9.10 did not encounter the hanging issue
running the 2.6.35 guest kernel.

http://www.spinics.net/lists/kvm/msg40800.html

Will try the kernel built from latest git tree to see whether the
problem could be resolved.

2, Below one seems to be discussing the similar problem but for
different root cause. Just a note here. The patched mentioned in below
link is still not found in the kernel gitorious tree.

http://kerneltrap.org/mailarchive/linux-kernel/2010/8/3/4601781

Reply at: https://bugs.launchpad.net/meego/+bug/688085/comments/8


** Changed in: meego
       Status: Unknown => In Progress

** Changed in: meego
   Importance: Unknown => Critical

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/688085

Title:
  Guest kernel hang during boot when KVM is active on i386 host

Status in meego project:
  In Progress
Status in QEMU:
  Fix Released
Status in qemu-kvm:
  Fix Released
Status in “kvm” package in Ubuntu:
  Invalid
Status in “linux” package in Ubuntu:
  In Progress
Status in “qemu” package in Ubuntu:
  Invalid
Status in “qemu-kvm” package in Ubuntu:
  Invalid
Status in “kvm” source package in Maverick:
  New
Status in “linux” source package in Maverick:
  New
Status in “qemu” source package in Maverick:
  New
Status in “qemu-kvm” source package in Maverick:
  New

Bug description:
  Binary package hint: qemu

Guest kernel hang during boot when KVM is active on i386 host

See the patch.
http://www.spinics.net/lists/kvm/msg40800.html

How to reproduce:
1. install Maversick x86 (not amd64)
2. ensure you have  kvm support in processor
3. kvm -kernel /boot/initrd.img-2.6.35-24-generic-pae
4. kvm -no-kvm -kernel /boot/initrd.img-2.6.35-24-generic-pae works OK.

SRU Justification:
Impact: Users cannot boot KVM guests on i386 hosts
2. How bug addressed:  The upstream commit at http://www.spinics.net/lists/kvm/msg40800.html fixed it
3. Patch:  A kernel patch is attached to this bug.
4. Reproduce: boot an i386 kernel on a kvm-capable host.  Try to boot a kvm guest.
5. Regression potential: since this is cherrypicking a commit from a future upstream which had already been changed, regression is possible.  However if there is a regression, it should only affect users of KVM on i386 hosts, which currently fail anyway.