[ kvm-Bugs-2915201 ] Nested kvm (SVM)

[ kvm-Bugs-2915201 ] Nested kvm (SVM)

[SourceForge.net]

Bugs item #2915201, was opened at 2009-12-15 16:35
Message generated for change (Tracker Item Submitted) made by jbl001
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: amd
Group: v1.0 (example)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: jbl001 (jbl001)
Assigned to: Nobody/Anonymous (nobody)
Summary: Nested kvm (SVM)

Initial Comment:
I have seen a couple messages where people have stated that nested SVM works properly, but I cannot replicate it. I first attempted to use the following configurations:

Hardware:
desktop system: Gigabyte MA785GM board with Athlon X2 4400
server system: Tyan h2000M board with Opteron 2354

Software:
Host OS: Ubuntu 9.10 with production kernel
Host KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
Guest VMM Host OS: Ubuntu 9.10
Guest VMM KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
True guest: tried Slackware 10.2, 64-bit Ubuntu 8.10, 64-bit Ubuntu 9.10, and 32-bit XP

All configurations result in the true guest not booting, but the Slackware 10.2 true guest is the easiest to analyze. It hangs at various places during boot with the most common being the "calibrating delay loop", "testing HLT instruction", mounting the hard disks, or starting the INIT processes. It seems it is losing interrupts. 

I also tried an older host (64-bit Ubuntu 8.10) and guest VMM (64-bit Ubuntu 8.10) with the KVM-88 release. With this configuration, the Slackware 10.2 true guest will usually boot, but will then get a constant flow of "hda: lost interrupt" and "hda: dma_timer_expiry: dma status == 0x24". Again, it seems to be losing interrupts.

I have ensured that the nested=1 is passed to the module and that enable-nesting is passed to the qemu. It obviously works for some time and I've tried printing out exit reasons in the handle_exit() function of the guest VMM, but it consistently fails in some form or another across all the hardware and software I have to try it on.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

[ kvm-Bugs-2915201 ] Nested kvm (SVM)

[SourceForge.net]

Bugs item #2915201, was opened at 2009-12-16 01:35
Message generated for change (Comment added) made by jessorensen
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: amd
Group: v1.0 (example)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: jbl001 (jbl001)
Assigned to: Nobody/Anonymous (nobody)
Summary: Nested kvm (SVM)

Initial Comment:
I have seen a couple messages where people have stated that nested SVM works properly, but I cannot replicate it. I first attempted to use the following configurations:

Hardware:
desktop system: Gigabyte MA785GM board with Athlon X2 4400
server system: Tyan h2000M board with Opteron 2354

Software:
Host OS: Ubuntu 9.10 with production kernel
Host KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
Guest VMM Host OS: Ubuntu 9.10
Guest VMM KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
True guest: tried Slackware 10.2, 64-bit Ubuntu 8.10, 64-bit Ubuntu 9.10, and 32-bit XP

All configurations result in the true guest not booting, but the Slackware 10.2 true guest is the easiest to analyze. It hangs at various places during boot with the most common being the "calibrating delay loop", "testing HLT instruction", mounting the hard disks, or starting the INIT processes. It seems it is losing interrupts. 

I also tried an older host (64-bit Ubuntu 8.10) and guest VMM (64-bit Ubuntu 8.10) with the KVM-88 release. With this configuration, the Slackware 10.2 true guest will usually boot, but will then get a constant flow of "hda: lost interrupt" and "hda: dma_timer_expiry: dma status == 0x24". Again, it seems to be losing interrupts.

I have ensured that the nested=1 is passed to the module and that enable-nesting is passed to the qemu. It obviously works for some time and I've tried printing out exit reasons in the handle_exit() function of the guest VMM, but it consistently fails in some form or another across all the hardware and software I have to try it on.


----------------------------------------------------------------------

>Comment By: Jes Sorensen (jessorensen)
Date: 2010-11-26 13:05

Message:
Did this issue get resolved? Can we close the bug, there hasn't been any
updates for over 9 months?

----------------------------------------------------------------------

Comment By: Alex Williamson (alex_williamson)
Date: 2010-02-17 20:02

Message:
Try reverting cd3ff653ae0b45bac7a19208e9c75034fcacc85f from kvm-kmod
(kvm-svm).  I ran into trouble with nested kvm about a month ago and
bisected it back to this change.  I alerted Joerg, but he might need
another poke if this fixes nesting for you too.

----------------------------------------------------------------------

Comment By: jbl001 (jbl001)
Date: 2010-02-17 18:47

Message:
I tried this again with qemu-0.12.2 and kvm-kmod-2.6.32.3 while passing
no-kvmclock to both the host and guest VMM kernels. It did not help the
problem of lost interrupts in the true guest, however.


----------------------------------------------------------------------

Comment By: Brian Jackson (iggy_cav)
Date: 2010-02-09 20:07

Message:
Can you try disabling kvmclock in both guests?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

[ kvm-Bugs-2915201 ] Nested kvm (SVM)

[SourceForge.net]

Bugs item #2915201, was opened at 2009-12-15 17:35
Message generated for change (Comment added) made by alex_williamson
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: amd
Group: v1.0 (example)
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: jbl001 (jbl001)
Assigned to: Nobody/Anonymous (nobody)
Summary: Nested kvm (SVM)

Initial Comment:
I have seen a couple messages where people have stated that nested SVM works properly, but I cannot replicate it. I first attempted to use the following configurations:

Hardware:
desktop system: Gigabyte MA785GM board with Athlon X2 4400
server system: Tyan h2000M board with Opteron 2354

Software:
Host OS: Ubuntu 9.10 with production kernel
Host KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
Guest VMM Host OS: Ubuntu 9.10
Guest VMM KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
True guest: tried Slackware 10.2, 64-bit Ubuntu 8.10, 64-bit Ubuntu 9.10, and 32-bit XP

All configurations result in the true guest not booting, but the Slackware 10.2 true guest is the easiest to analyze. It hangs at various places during boot with the most common being the "calibrating delay loop", "testing HLT instruction", mounting the hard disks, or starting the INIT processes. It seems it is losing interrupts. 

I also tried an older host (64-bit Ubuntu 8.10) and guest VMM (64-bit Ubuntu 8.10) with the KVM-88 release. With this configuration, the Slackware 10.2 true guest will usually boot, but will then get a constant flow of "hda: lost interrupt" and "hda: dma_timer_expiry: dma status == 0x24". Again, it seems to be losing interrupts.

I have ensured that the nested=1 is passed to the module and that enable-nesting is passed to the qemu. It obviously works for some time and I've tried printing out exit reasons in the handle_exit() function of the guest VMM, but it consistently fails in some form or another across all the hardware and software I have to try it on.


----------------------------------------------------------------------

Comment By: Alex Williamson (alex_williamson)
Date: 2010-02-17 12:02

Message:
Try reverting cd3ff653ae0b45bac7a19208e9c75034fcacc85f from kvm-kmod
(kvm-svm).  I ran into trouble with nested kvm about a month ago and
bisected it back to this change.  I alerted Joerg, but he might need
another poke if this fixes nesting for you too.

----------------------------------------------------------------------

Comment By: jbl001 (jbl001)
Date: 2010-02-17 10:47

Message:
I tried this again with qemu-0.12.2 and kvm-kmod-2.6.32.3 while passing
no-kvmclock to both the host and guest VMM kernels. It did not help the
problem of lost interrupts in the true guest, however.


----------------------------------------------------------------------

Comment By: Brian Jackson (iggy_cav)
Date: 2010-02-09 12:07

Message:
Can you try disabling kvmclock in both guests?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

[ kvm-Bugs-2915201 ] Nested kvm (SVM)

[SourceForge.net]

Bugs item #2915201, was opened at 2009-12-15 16:35
Message generated for change (Comment added) made by jbl001
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: amd
Group: v1.0 (example)
>Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: jbl001 (jbl001)
Assigned to: Nobody/Anonymous (nobody)
Summary: Nested kvm (SVM)

Initial Comment:
I have seen a couple messages where people have stated that nested SVM works properly, but I cannot replicate it. I first attempted to use the following configurations:

Hardware:
desktop system: Gigabyte MA785GM board with Athlon X2 4400
server system: Tyan h2000M board with Opteron 2354

Software:
Host OS: Ubuntu 9.10 with production kernel
Host KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
Guest VMM Host OS: Ubuntu 9.10
Guest VMM KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
True guest: tried Slackware 10.2, 64-bit Ubuntu 8.10, 64-bit Ubuntu 9.10, and 32-bit XP

All configurations result in the true guest not booting, but the Slackware 10.2 true guest is the easiest to analyze. It hangs at various places during boot with the most common being the "calibrating delay loop", "testing HLT instruction", mounting the hard disks, or starting the INIT processes. It seems it is losing interrupts. 

I also tried an older host (64-bit Ubuntu 8.10) and guest VMM (64-bit Ubuntu 8.10) with the KVM-88 release. With this configuration, the Slackware 10.2 true guest will usually boot, but will then get a constant flow of "hda: lost interrupt" and "hda: dma_timer_expiry: dma status == 0x24". Again, it seems to be losing interrupts.

I have ensured that the nested=1 is passed to the module and that enable-nesting is passed to the qemu. It obviously works for some time and I've tried printing out exit reasons in the handle_exit() function of the guest VMM, but it consistently fails in some form or another across all the hardware and software I have to try it on.


----------------------------------------------------------------------

>Comment By: jbl001 (jbl001)
Date: 2010-02-17 09:47

Message:
I tried this again with qemu-0.12.2 and kvm-kmod-2.6.32.3 while passing
no-kvmclock to both the host and guest VMM kernels. It did not help the
problem of lost interrupts in the true guest, however.


----------------------------------------------------------------------

Comment By: Brian Jackson (iggy_cav)
Date: 2010-02-09 11:07

Message:
Can you try disabling kvmclock in both guests?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

[ kvm-Bugs-2915201 ] Nested kvm (SVM)

[SourceForge.net]

Bugs item #2915201, was opened at 2009-12-15 18:35
Message generated for change (Comment added) made by iggy_cav
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: amd
Group: v1.0 (example)
>Status: Pending
Resolution: None
Priority: 5
Private: No
Submitted By: jbl001 (jbl001)
Assigned to: Nobody/Anonymous (nobody)
Summary: Nested kvm (SVM)

Initial Comment:
I have seen a couple messages where people have stated that nested SVM works properly, but I cannot replicate it. I first attempted to use the following configurations:

Hardware:
desktop system: Gigabyte MA785GM board with Athlon X2 4400
server system: Tyan h2000M board with Opteron 2354

Software:
Host OS: Ubuntu 9.10 with production kernel
Host KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
Guest VMM Host OS: Ubuntu 9.10
Guest VMM KVM: tried kmod 2.6.32 with qemu 0.11.1 and qemu 0.12.0rc2, also tried git-tip
True guest: tried Slackware 10.2, 64-bit Ubuntu 8.10, 64-bit Ubuntu 9.10, and 32-bit XP

All configurations result in the true guest not booting, but the Slackware 10.2 true guest is the easiest to analyze. It hangs at various places during boot with the most common being the "calibrating delay loop", "testing HLT instruction", mounting the hard disks, or starting the INIT processes. It seems it is losing interrupts. 

I also tried an older host (64-bit Ubuntu 8.10) and guest VMM (64-bit Ubuntu 8.10) with the KVM-88 release. With this configuration, the Slackware 10.2 true guest will usually boot, but will then get a constant flow of "hda: lost interrupt" and "hda: dma_timer_expiry: dma status == 0x24". Again, it seems to be losing interrupts.

I have ensured that the nested=1 is passed to the module and that enable-nesting is passed to the qemu. It obviously works for some time and I've tried printing out exit reasons in the handle_exit() function of the guest VMM, but it consistently fails in some form or another across all the hardware and software I have to try it on.


----------------------------------------------------------------------

>Comment By: Brian Jackson (iggy_cav)
Date: 2010-02-09 13:07

Message:
Can you try disabling kvmclock in both guests?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=893831&aid=2915201&group_id=180599