* Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. @ 2014-09-30 15:55 Jeroen Groenewegen van der Weyden 2014-10-01 13:20 ` Jan Beulich 2014-10-07 7:58 ` Jan Beulich 0 siblings, 2 replies; 18+ messages in thread From: Jeroen Groenewegen van der Weyden @ 2014-09-30 15:55 UTC (permalink / raw) To: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 4318 bytes --] Hi, Recently I updated my openSuse box from 12.3 to 13.1. On this box I run xen with several guests. One of these guests is an appliance that has 4 kvm guests running. When I start this appliance with the nested vmx feature the appliance crashes either immediately or after a few minutes. This same guest was running without a problem on opensuse releases 11.4 until 12.3 ( remark that about 3 year ago some patches were written by tim degan to make this appliance work ) dom0 opensuse 13.1 x86 kernel 3.11.10-21-xen xen: 4.3..2_01-21.1 - domu (HVM) - sles11sp2 - mem: 8 GB - vcpu: 4 - kvm with 4 guest any body any thought on this? BR, Jeroen ==== outup xl demsg (XEN) vvmx.c:2459:d5 Unknown nested vmexit reason 80000021. (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid guest state (0). (XEN) ************* VMCS Area ************** (XEN) *** Guest State *** (XEN) CR0: actual=0x0000000080000033, shadow=0x0000000000000011, gh_mask=ffffffffffffffff (XEN) CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffffff (XEN) CR3: actual=0x00000000feffc000, target_count=0 (XEN) target0=0000000000000000, target1=0000000000000000 (XEN) target2=0000000000000000, target3=0000000000000000 (XEN) RSP = 0x0000000000000398 (0x0000000000000398) RIP = 0x000000000000045f (0x000000000000045f) (XEN) RFLAGS=0x0000000000000006 (0x0000000000000006) DR7 = 0x0000000000000400 (XEN) Sysenter RSP=0000000000000000 CS:RIP=0000:0000000000000000 (XEN) CS: sel=0x9dcc, attr=0x0009b, limit=0x00000ecb, base=0x0000000000689920 (XEN) DS: sel=0x9db4, attr=0x00093, limit=0x000038b9, base=0x00000000006778b0 (XEN) SS: sel=0x9e7c, attr=0x00093, limit=0x00000401, base=0x00000000006ba960 (XEN) ES: sel=0x3ec8, attr=0x00093, limit=0x00000099, base=0x000000000073c608 (XEN) FS: sel=0x0000, attr=0x1c000, limit=0xffffffff, base=0x0000000000000000 (XEN) GS: sel=0x0000, attr=0x1c000, limit=0xffffffff, base=0x0000000000000000 (XEN) GDTR: limit=0x0000ffff, base=0x0000000001000000 (XEN) LDTR: sel=0x0820, attr=0x00082, limit=0x0000ffff, base=0x0000000001010400 (XEN) IDTR: limit=0x000003ff, base=0x0000000001010000 (XEN) TR: sel=0x0690, attr=0x00083, limit=0x0000002b, base=0x0000000001213a30 (XEN) Guest PAT = 0x0000050100070406 (XEN) TSC Offset = fffd869460e8dc28 (XEN) DebugCtl=0000000000000000 DebugExceptions=0000000000000000 (XEN) Interruptibility=0008 ActivityState=0000 (XEN) *** Host State *** (XEN) RSP = 0xffff83083ff27f90 RIP = 0xffff82c4c01d9330 (XEN) CS=e008 DS=0000 ES=0000 FS=0000 GS=0000 SS=0000 TR=e040 (XEN) FSBase=0000000000000000 GSBase=0000000000000000 TRBase=ffff83083ff2dc80 (XEN) GDTBase=ffff8318035ee000 IDTBase=ffff8318035fa000 (XEN) CR0=0000000080050033 CR3=0000000eefcc0000 CR4=00000000000026f0 (XEN) Sysenter RSP=ffff83083ff27fc0 CS:RIP=e008:ffff82c4c0216c60 (XEN) Host PAT = 0x0000050100070406 (XEN) *** Control State *** (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb (XEN) EntryControls=000011ff ExitControls=000fefff (XEN) ExceptionBitmap=00040042 (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 (XEN) reason=80000021 qualification=00000000 (XEN) IDTVectoring: info=80000202 errcode=00000000 (XEN) TPR Threshold = 0x00 (XEN) EPT pointer = 0x0000000ef022501e (XEN) Virtual processor ID = 0xc0d0 (XEN) ************************************** (XEN) domain_crash called from vmx.c:2342 (XEN) Domain 5 (vcpu#3) crashed on cpu#20: (XEN) ----[ Xen-4.3.2_01-21.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 20 (XEN) RIP: 9dcc:[<000000000000045f>] (XEN) RFLAGS: 0000000000000006 CONTEXT: hvm guest (XEN) rax: 0000000000000001 rbx: 00000000000026db rcx: 000000009dd40003 (XEN) rdx: 00000000248816a0 rsi: 0000000000000000 rdi: 0000000026b010e4 (XEN) rbp: 00000000000003ae rsp: 0000000000000398 r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000080000033 cr4: 0000000000002010 (XEN) cr3: 00000000feffc000 cr2: 0000000000000000 (XEN) ds: 9db4 es: 3ec8 fs: 0000 gs: 0000 ss: 9e7c cs: 9dcc [-- Attachment #1.2: Type: text/html, Size: 5620 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-09-30 15:55 Crash of guest with nested vmx with Unknown nested vmexit reason 80000021 Jeroen Groenewegen van der Weyden @ 2014-10-01 13:20 ` Jan Beulich 2014-10-07 7:58 ` Jan Beulich 1 sibling, 0 replies; 18+ messages in thread From: Jan Beulich @ 2014-10-01 13:20 UTC (permalink / raw) To: Jeroen Groenewegen van der Weyden; +Cc: xen-devel >>> On 30.09.14 at 17:55, <groen692@grosc.com> wrote: > Recently I updated my openSuse box from 12.3 to 13.1. On this box I run > xen with several guests. One of these guests is an appliance that has 4 > kvm guests running. > When I start this appliance with the nested vmx feature the appliance > crashes either immediately or after a few minutes. I see you also sent this to opensuse-virtual, which is where I think it belongs at least initially. I'll take it up from there, but it may take some time until I can get to it. Jan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-09-30 15:55 Crash of guest with nested vmx with Unknown nested vmexit reason 80000021 Jeroen Groenewegen van der Weyden 2014-10-01 13:20 ` Jan Beulich @ 2014-10-07 7:58 ` Jan Beulich 2014-10-07 15:16 ` Jeroen Groenewegen van der Weyden 2014-10-07 19:56 ` Tian, Kevin 1 sibling, 2 replies; 18+ messages in thread From: Jan Beulich @ 2014-10-07 7:58 UTC (permalink / raw) To: Eddie Dong, Jun Nakajima, Kevin Tian Cc: Jeroen Groenewegen van der Weyden, xen-devel >>> On 30.09.14 at 17:55, <groen692@grosc.com> wrote: > Recently I updated my openSuse box from 12.3 to 13.1. On this box I run > xen with several guests. One of these guests is an appliance that has 4 > kvm guests running. > When I start this appliance with the nested vmx feature the appliance > crashes either immediately or after a few minutes. > > This same guest was running without a problem on opensuse releases 11.4 > until 12.3 >[...] > ==== outup xl demsg > (XEN) vvmx.c:2459:d5 Unknown nested vmexit reason 80000021. > (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid guest > state (0). > (XEN) ************* VMCS Area ************** >[...] So the problem here is that > (XEN) Interruptibility=0008 ActivityState=0000 VMX_INTR_SHADOW_NMI is set while > (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb PIN_BASED_VIRTUAL_NMIS is active and > (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 > (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 > (XEN) reason=80000021 qualification=00000000 > (XEN) IDTVectoring: info=80000202 errcode=00000000 an NMI is being injected. This case is explicitly mentioned in Vol 3 section 31.7.1.2 (Resuming Guest Software after Handling an Exception). Either there needs to be extra code in vvmx.c to clear VMX_INTR_SHADOW_NMI (as the second sub-bullet point of the last bullet point says), or the second half of vmx_idtv_reinject() needs to be done without regard to nestedhvm_vcpu_in_guestmode(v) (and maybe also without regard to EXIT_REASON_TASK_SWITCH). Speaking of SDM sections: There are quite a few references in the code that name just section numbers (in the case here, several references to sections 25.7.1.* exist). These numbers become stale quite quickly (here they're now 31.7.1.*), so in order to help digging through issues like the one here, can I please ask one of you to go through and replace (or at least amend) these numbers with the sections' titles (which I hope won't get altered that quickly)? Jan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-07 7:58 ` Jan Beulich @ 2014-10-07 15:16 ` Jeroen Groenewegen van der Weyden 2014-10-07 19:56 ` Tian, Kevin 1 sibling, 0 replies; 18+ messages in thread From: Jeroen Groenewegen van der Weyden @ 2014-10-07 15:16 UTC (permalink / raw) To: Jan Beulich, Eddie Dong, Jun Nakajima, Kevin Tian; +Cc: xen-devel Hi Jan, I installed xen-4.5.29823-1.xen_unstable.1.x86_64 from the http://download.opensuse.org/repositories/home:/olh:/xen-unstable/openSUSE_Factory/ After about 45 minutes the guests crashed again. mfg, Jeroen (XEN) vvmx.c:2476:d1v0 Unknown nested vmexit reason 80000021. (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid guest state (0). (XEN) ************* VMCS Area ************** (XEN) *** Guest State *** (XEN) CR0: actual=0x0000000000000033, shadow=0x0000000000000011, gh_mask=ffffffffffffffff (XEN) CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffffff (XEN) CR3: actual=0x00000000feffc000, target_count=0 (XEN) target0=0000000000000000, target1=0000000000000000 (XEN) target2=0000000000000000, target3=0000000000000000 (XEN) RSP = 0x00000000000006be (0x00000000000006be) RIP = 0x000000000000017d (0x000000000000017d) (XEN) RFLAGS=0x0000000000000002 (0x0000000000000002) DR7 = 0x0000000000000400 (XEN) Sysenter RSP=0000000000000000 CS:RIP=0000:0000000000000000 (XEN) CS: sel=0xbf70, attr=0x0009b, limit=0x0000075a, base=0x000000000194fec0 (XEN) DS: sel=0x3cec, attr=0x00093, limit=0x00000110, base=0x000000000bcbe276 (XEN) SS: sel=0x3cf4, attr=0x00093, limit=0x000007ff, base=0x000000000bcbe3aa (XEN) ES: sel=0x3cfc, attr=0x00093, limit=0x00000099, base=0x000000000bcbebac (XEN) FS: sel=0x0000, attr=0x1c000, limit=0xffffffff, base=0x0000000000000000 (XEN) GS: sel=0x0000, attr=0x1c000, limit=0xffffffff, base=0x0000000000000000 (XEN) GDTR: limit=0x0000ffff, base=0x0000000001000000 (XEN) LDTR: sel=0x0820, attr=0x00082, limit=0x0000ffff, base=0x0000000001010400 (XEN) IDTR: limit=0x000003ff, base=0x0000000001010000 (XEN) TR: sel=0x0690, attr=0x00083, limit=0x0000002b, base=0x0000000001213a30 (XEN) Guest PAT = 0x0000050100070406 (XEN) TSC Offset = fffffe8a52350fd4 (XEN) DebugCtl=0000000000000000 DebugExceptions=0000000000000000 (XEN) Interruptibility=0008 ActivityState=0000 (XEN) *** Host State *** (XEN) RSP = 0xffff830836b5ff90 RIP = 0xffff82d0801e3ed0 (XEN) CS=e008 DS=0000 ES=0000 FS=0000 GS=0000 SS=0000 TR=e040 (XEN) FSBase=0000000000000000 GSBase=0000000000000000 TRBase=ffff830836b64c80 (XEN) GDTBase=ffff830836b55000 IDTBase=ffff830836b61000 (XEN) CR0=0000000080050033 CR3=000000082ffdd000 CR4=00000000000026f0 (XEN) Sysenter RSP=ffff830836b5ffc0 CS:RIP=e008:ffff82d0802220a0 (XEN) Host PAT = 0x0000050100070406 (XEN) *** Control State *** (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb (XEN) EntryControls=000011ff ExitControls=000fefff (XEN) ExceptionBitmap=00040042 (XEN) VMEntry: intr_info=80000202 errcode=00000000 ilen=00000003 (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 (XEN) reason=80000021 qualification=00000000 (XEN) IDTVectoring: info=80000202 errcode=00000000 (XEN) TPR Threshold = 0x00 (XEN) EPT pointer = 0x000000082ffff01e (XEN) Virtual processor ID = 0xe6a3 (XEN) ************************************** (XEN) domain_crash called from vmx.c:2483 (XEN) Domain 1 (vcpu#0) crashed on cpu#9: (XEN) ----[ Xen-4.5.29823-1.xen_unstable.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 9 (XEN) RIP: bf70:[<000000000000017d>] (XEN) RFLAGS: 0000000000000002 CONTEXT: hvm guest (XEN) rax: 0000000000040002 rbx: 0000000000042701 rcx: 00000000bf480003 (XEN) rdx: 00000000248816a0 rsi: 0000000000000000 rdi: 0000000026b00096 (XEN) rbp: 00000000000006d4 rsp: 00000000000006be r8: 0000000000000000 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 (XEN) r15: 0000000000000000 cr0: 0000000000000033 cr4: 0000000000002010 (XEN) cr3: 00000000feffc000 cr2: 0000000000000000 (XEN) ds: 3cec es: 3cfc fs: 0000 gs: 0000 ss: 3cf4 cs: bf70 Jan Beulich schreef op 7-10-2014 om 09:58: >>>> On 30.09.14 at 17:55, <groen692@grosc.com> wrote: >> Recently I updated my openSuse box from 12.3 to 13.1. On this box I run >> xen with several guests. One of these guests is an appliance that has 4 >> kvm guests running. >> When I start this appliance with the nested vmx feature the appliance >> crashes either immediately or after a few minutes. >> >> This same guest was running without a problem on opensuse releases 11.4 >> until 12.3 >> [...] >> ==== outup xl demsg >> (XEN) vvmx.c:2459:d5 Unknown nested vmexit reason 80000021. >> (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid guest >> state (0). >> (XEN) ************* VMCS Area ************** >> [...] > So the problem here is that > >> (XEN) Interruptibility=0008 ActivityState=0000 > VMX_INTR_SHADOW_NMI is set while > >> (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb > PIN_BASED_VIRTUAL_NMIS is active and > >> (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 >> (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 >> (XEN) reason=80000021 qualification=00000000 >> (XEN) IDTVectoring: info=80000202 errcode=00000000 > an NMI is being injected. This case is explicitly mentioned in Vol 3 > section 31.7.1.2 (Resuming Guest Software after Handling an > Exception). Either there needs to be extra code in vvmx.c to clear > VMX_INTR_SHADOW_NMI (as the second sub-bullet point of the last > bullet point says), or the second half of vmx_idtv_reinject() needs > to be done without regard to nestedhvm_vcpu_in_guestmode(v) > (and maybe also without regard to EXIT_REASON_TASK_SWITCH). > > Speaking of SDM sections: There are quite a few references in the > code that name just section numbers (in the case here, several > references to sections 25.7.1.* exist). These numbers become stale > quite quickly (here they're now 31.7.1.*), so in order to help > digging through issues like the one here, can I please ask one of > you to go through and replace (or at least amend) these numbers > with the sections' titles (which I hope won't get altered that quickly)? > > Jan > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-07 7:58 ` Jan Beulich 2014-10-07 15:16 ` Jeroen Groenewegen van der Weyden @ 2014-10-07 19:56 ` Tian, Kevin 2014-10-10 10:31 ` Jeroen Groenewegen van der Weyden 1 sibling, 1 reply; 18+ messages in thread From: Tian, Kevin @ 2014-10-07 19:56 UTC (permalink / raw) To: Jan Beulich, Dong, Eddie, Nakajima, Jun, Zhang, Yang Z Cc: Jeroen Groenewegen van der Weyden, xen-devel Add Yang as the nested vmx owner. > From: Jan Beulich [mailto:JBeulich@suse.com] > Sent: Tuesday, October 07, 2014 12:59 AM > > >>> On 30.09.14 at 17:55, <groen692@grosc.com> wrote: > > Recently I updated my openSuse box from 12.3 to 13.1. On this box I run > > xen with several guests. One of these guests is an appliance that has 4 > > kvm guests running. > > When I start this appliance with the nested vmx feature the appliance > > crashes either immediately or after a few minutes. > > > > This same guest was running without a problem on opensuse releases 11.4 > > until 12.3 > >[...] > > ==== outup xl demsg > > (XEN) vvmx.c:2459:d5 Unknown nested vmexit reason 80000021. > > (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid guest > > state (0). > > (XEN) ************* VMCS Area ************** > >[...] > > So the problem here is that > > > (XEN) Interruptibility=0008 ActivityState=0000 > > VMX_INTR_SHADOW_NMI is set while > > > (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb > > PIN_BASED_VIRTUAL_NMIS is active and > > > (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 > > (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 > > (XEN) reason=80000021 qualification=00000000 > > (XEN) IDTVectoring: info=80000202 errcode=00000000 > > an NMI is being injected. This case is explicitly mentioned in Vol 3 > section 31.7.1.2 (Resuming Guest Software after Handling an > Exception). Either there needs to be extra code in vvmx.c to clear > VMX_INTR_SHADOW_NMI (as the second sub-bullet point of the last > bullet point says), or the second half of vmx_idtv_reinject() needs > to be done without regard to nestedhvm_vcpu_in_guestmode(v) > (and maybe also without regard to EXIT_REASON_TASK_SWITCH). > > Speaking of SDM sections: There are quite a few references in the > code that name just section numbers (in the case here, several > references to sections 25.7.1.* exist). These numbers become stale > quite quickly (here they're now 31.7.1.*), so in order to help > digging through issues like the one here, can I please ask one of > you to go through and replace (or at least amend) these numbers > with the sections' titles (which I hope won't get altered that quickly)? > > Jan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-07 19:56 ` Tian, Kevin @ 2014-10-10 10:31 ` Jeroen Groenewegen van der Weyden 2014-10-16 6:18 ` Zhang, Yang Z 2014-10-16 6:27 ` Zhang, Yang Z 0 siblings, 2 replies; 18+ messages in thread From: Jeroen Groenewegen van der Weyden @ 2014-10-10 10:31 UTC (permalink / raw) To: Tian, Kevin, Jan Beulich, Dong, Eddie, Nakajima, Jun, Zhang, Yang Z Cc: xen-devel Hi all, Any input from my side necessary to keep this rolling? BR, Jeroen. Tian, Kevin schreef op 7-10-2014 om 21:56: > Add Yang as the nested vmx owner. > >> From: Jan Beulich [mailto:JBeulich@suse.com] >> Sent: Tuesday, October 07, 2014 12:59 AM >> >>>>> On 30.09.14 at 17:55, <groen692@grosc.com> wrote: >>> Recently I updated my openSuse box from 12.3 to 13.1. On this box I run >>> xen with several guests. One of these guests is an appliance that has 4 >>> kvm guests running. >>> When I start this appliance with the nested vmx feature the appliance >>> crashes either immediately or after a few minutes. >>> >>> This same guest was running without a problem on opensuse releases 11.4 >>> until 12.3 >>> [...] >>> ==== outup xl demsg >>> (XEN) vvmx.c:2459:d5 Unknown nested vmexit reason 80000021. >>> (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid guest >>> state (0). >>> (XEN) ************* VMCS Area ************** >>> [...] >> So the problem here is that >> >>> (XEN) Interruptibility=0008 ActivityState=0000 >> VMX_INTR_SHADOW_NMI is set while >> >>> (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb >> PIN_BASED_VIRTUAL_NMIS is active and >> >>> (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 >>> (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 >>> (XEN) reason=80000021 qualification=00000000 >>> (XEN) IDTVectoring: info=80000202 errcode=00000000 >> an NMI is being injected. This case is explicitly mentioned in Vol 3 >> section 31.7.1.2 (Resuming Guest Software after Handling an >> Exception). Either there needs to be extra code in vvmx.c to clear >> VMX_INTR_SHADOW_NMI (as the second sub-bullet point of the last >> bullet point says), or the second half of vmx_idtv_reinject() needs >> to be done without regard to nestedhvm_vcpu_in_guestmode(v) >> (and maybe also without regard to EXIT_REASON_TASK_SWITCH). >> >> Speaking of SDM sections: There are quite a few references in the >> code that name just section numbers (in the case here, several >> references to sections 25.7.1.* exist). These numbers become stale >> quite quickly (here they're now 31.7.1.*), so in order to help >> digging through issues like the one here, can I please ask one of >> you to go through and replace (or at least amend) these numbers >> with the sections' titles (which I hope won't get altered that quickly)? >> >> Jan > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-10 10:31 ` Jeroen Groenewegen van der Weyden @ 2014-10-16 6:18 ` Zhang, Yang Z 2014-10-16 6:41 ` Jan Beulich 2014-10-16 6:27 ` Zhang, Yang Z 1 sibling, 1 reply; 18+ messages in thread From: Zhang, Yang Z @ 2014-10-16 6:18 UTC (permalink / raw) To: Jeroen Groenewegen van der Weyden, Tian, Kevin, Jan Beulich, Dong, Eddie, Nakajima, Jun Cc: xen-devel Jeroen Groenewegen van der Weyden wrote on 2014-10-10: > Hi all, > > Any input from my side necessary to keep this rolling? Sorry for the later reply. Yes, this is a known issue to me but I didn't have time to cook a patch fix it. As Jan pointed out, the NMI handling logic is wrong in current nested logic. But it is not a trivial task to fix them. I will do it once I have the time or if you are interesting in it, a patch from you is welcome. > > BR, > Jeroen. > > Tian, Kevin schreef op 7-10-2014 om 21:56: >> Add Yang as the nested vmx owner. >> >>> From: Jan Beulich [mailto:JBeulich@suse.com] >>> Sent: Tuesday, October 07, 2014 12:59 AM >>> >>>>>> On 30.09.14 at 17:55, <groen692@grosc.com> wrote: >>>> Recently I updated my openSuse box from 12.3 to 13.1. On this box >>>> I run xen with several guests. One of these guests is an appliance >>>> that has 4 kvm guests running. >>>> When I start this appliance with the nested vmx feature the >>>> appliance crashes either immediately or after a few minutes. >>>> >>>> This same guest was running without a problem on opensuse releases >>>> 11.4 until 12.3 [...] ==== outup xl demsg >>>> (XEN) vvmx.c:2459:d5 Unknown nested vmexit reason 80000021. >>>> (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid >>>> guest state (0). >>>> (XEN) ************* VMCS Area ************** [...] >>> So the problem here is that >>> >>>> (XEN) Interruptibility=0008 ActivityState=0000 >>> VMX_INTR_SHADOW_NMI is set while >>> >>>> (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb >>> PIN_BASED_VIRTUAL_NMIS is active and >>> >>>> (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 >>>> (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 >>>> (XEN) reason=80000021 qualification=00000000 >>>> (XEN) IDTVectoring: info=80000202 errcode=00000000 >>> an NMI is being injected. This case is explicitly mentioned in Vol >>> 3 section 31.7.1.2 (Resuming Guest Software after Handling an >>> Exception). Either there needs to be extra code in vvmx.c to clear >>> VMX_INTR_SHADOW_NMI (as the second sub-bullet point of the last >>> bullet point says), or the second half of vmx_idtv_reinject() needs >>> to be done without regard to nestedhvm_vcpu_in_guestmode(v) (and >>> maybe also without regard to EXIT_REASON_TASK_SWITCH). >>> >>> Speaking of SDM sections: There are quite a few references in the >>> code that name just section numbers (in the case here, several >>> references to sections 25.7.1.* exist). These numbers become stale >>> quite quickly (here they're now 31.7.1.*), so in order to help >>> digging through issues like the one here, can I please ask one of >>> you to go through and replace (or at least amend) these numbers >>> with the sections' titles (which I hope won't get altered that quickly)? >>> >>> Jan >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel >> Best regards, Yang ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-16 6:18 ` Zhang, Yang Z @ 2014-10-16 6:41 ` Jan Beulich 2014-10-29 14:17 ` Jeroen Groenewegen van der Weyden 0 siblings, 1 reply; 18+ messages in thread From: Jan Beulich @ 2014-10-16 6:41 UTC (permalink / raw) To: Yang Z Zhang Cc: Kevin Tian, Eddie Dong, Jeroen Groenewegen van der Weyden, Jun Nakajima, xen-devel >>> On 16.10.14 at 08:18, <yang.z.zhang@intel.com> wrote: > Sorry for the later reply. Yes, this is a known issue to me but I didn't > have time to cook a patch fix it. As Jan pointed out, the NMI handling logic > is wrong in current nested logic. But it is not a trivial task to fix them. I > will do it once I have the time or if you are interesting in it, a patch from > you is welcome. If you were to at least comment on the two possible routes I outlined (quoted below), I could see to get to this (considering that the issue - as you point out subsequently - got brought up more than half a year ago the first time, and is still unaddressed) at least if the vmx_idtv_reinject() related route would be a possible one. Jan >>>> From: Jan Beulich [mailto:JBeulich@suse.com] >>>> So the problem here is that >>>> >>>>> (XEN) Interruptibility=0008 ActivityState=0000 >>>> VMX_INTR_SHADOW_NMI is set while >>>> >>>>> (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb >>>> PIN_BASED_VIRTUAL_NMIS is active and >>>> >>>>> (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 >>>>> (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 >>>>> (XEN) reason=80000021 qualification=00000000 >>>>> (XEN) IDTVectoring: info=80000202 errcode=00000000 >>>> an NMI is being injected. This case is explicitly mentioned in Vol >>>> 3 section 31.7.1.2 (Resuming Guest Software after Handling an >>>> Exception). Either there needs to be extra code in vvmx.c to clear >>>> VMX_INTR_SHADOW_NMI (as the second sub-bullet point of the last >>>> bullet point says), or the second half of vmx_idtv_reinject() needs >>>> to be done without regard to nestedhvm_vcpu_in_guestmode(v) (and >>>> maybe also without regard to EXIT_REASON_TASK_SWITCH). >>>> >>>> Speaking of SDM sections: There are quite a few references in the >>>> code that name just section numbers (in the case here, several >>>> references to sections 25.7.1.* exist). These numbers become stale >>>> quite quickly (here they're now 31.7.1.*), so in order to help >>>> digging through issues like the one here, can I please ask one of >>>> you to go through and replace (or at least amend) these numbers >>>> with the sections' titles (which I hope won't get altered that quickly)? ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-16 6:41 ` Jan Beulich @ 2014-10-29 14:17 ` Jeroen Groenewegen van der Weyden 2014-10-29 16:42 ` Jan Beulich 0 siblings, 1 reply; 18+ messages in thread From: Jeroen Groenewegen van der Weyden @ 2014-10-29 14:17 UTC (permalink / raw) To: Jan Beulich, Yang Z Zhang; +Cc: Kevin Tian, Eddie Dong, Jun Nakajima, xen-devel Hi, Is there any test I can do on changed code? mvg, Jeroen. Jan Beulich schreef op 16-10-2014 om 08:41: >>>> On 16.10.14 at 08:18, <yang.z.zhang@intel.com> wrote: >> Sorry for the later reply. Yes, this is a known issue to me but I didn't >> have time to cook a patch fix it. As Jan pointed out, the NMI handling logic >> is wrong in current nested logic. But it is not a trivial task to fix them. I >> will do it once I have the time or if you are interesting in it, a patch from >> you is welcome. > If you were to at least comment on the two possible routes I > outlined (quoted below), I could see to get to this (considering > that the issue - as you point out subsequently - got brought up > more than half a year ago the first time, and is still unaddressed) > at least if the vmx_idtv_reinject() related route would be a > possible one. > > Jan > >>>>> From: Jan Beulich [mailto:JBeulich@suse.com] >>>>> So the problem here is that >>>>> >>>>>> (XEN) Interruptibility=0008 ActivityState=0000 >>>>> VMX_INTR_SHADOW_NMI is set while >>>>> >>>>>> (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb >>>>> PIN_BASED_VIRTUAL_NMIS is active and >>>>> >>>>>> (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 >>>>>> (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 >>>>>> (XEN) reason=80000021 qualification=00000000 >>>>>> (XEN) IDTVectoring: info=80000202 errcode=00000000 >>>>> an NMI is being injected. This case is explicitly mentioned in Vol >>>>> 3 section 31.7.1.2 (Resuming Guest Software after Handling an >>>>> Exception). Either there needs to be extra code in vvmx.c to clear >>>>> VMX_INTR_SHADOW_NMI (as the second sub-bullet point of the last >>>>> bullet point says), or the second half of vmx_idtv_reinject() needs >>>>> to be done without regard to nestedhvm_vcpu_in_guestmode(v) (and >>>>> maybe also without regard to EXIT_REASON_TASK_SWITCH). >>>>> >>>>> Speaking of SDM sections: There are quite a few references in the >>>>> code that name just section numbers (in the case here, several >>>>> references to sections 25.7.1.* exist). These numbers become stale >>>>> quite quickly (here they're now 31.7.1.*), so in order to help >>>>> digging through issues like the one here, can I please ask one of >>>>> you to go through and replace (or at least amend) these numbers >>>>> with the sections' titles (which I hope won't get altered that quickly)? > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-29 14:17 ` Jeroen Groenewegen van der Weyden @ 2014-10-29 16:42 ` Jan Beulich 2014-11-03 11:16 ` George Dunlap 0 siblings, 1 reply; 18+ messages in thread From: Jan Beulich @ 2014-10-29 16:42 UTC (permalink / raw) To: Jeroen Groenewegen van der Weyden Cc: Yang Z Zhang, Kevin Tian, Eddie Dong, Jun Nakajima, xen-devel >>> On 29.10.14 at 15:17, <groen692@grosc.com> wrote: > Is there any test I can do on changed code? Strange question with there not being any changed code yet. Jan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-29 16:42 ` Jan Beulich @ 2014-11-03 11:16 ` George Dunlap 2014-12-09 9:09 ` Jeroen Groenewegen van der Weyden 0 siblings, 1 reply; 18+ messages in thread From: George Dunlap @ 2014-11-03 11:16 UTC (permalink / raw) To: Jan Beulich Cc: Kevin Tian, Eddie Dong, Jeroen Groenewegen van der Weyden, xen-devel, Jun Nakajima, Yang Z Zhang On Wed, Oct 29, 2014 at 4:42 PM, Jan Beulich <JBeulich@suse.com> wrote: >>>> On 29.10.14 at 15:17, <groen692@grosc.com> wrote: >> Is there any test I can do on changed code? > > Strange question with there not being any changed code yet. I think what Jan means is, sorry, we haven't had a chance to actually work on this yet, but thanks for the ping. :-) -George ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-11-03 11:16 ` George Dunlap @ 2014-12-09 9:09 ` Jeroen Groenewegen van der Weyden 2014-12-09 9:17 ` Jan Beulich 0 siblings, 1 reply; 18+ messages in thread From: Jeroen Groenewegen van der Weyden @ 2014-12-09 9:09 UTC (permalink / raw) To: George Dunlap, Jan Beulich Cc: Yang Z Zhang, Kevin Tian, Eddie Dong, Jun Nakajima, xen-devel Hi All, Did anyone find the time yet? I'm still more then willing testing any patches. BR, Jeroen. George Dunlap schreef op 3-11-2014 om 12:16: > I think what Jan means is, sorry, we haven't had a chance to actually > work on this yet, but thanks for the ping.:-) > > -George ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-12-09 9:09 ` Jeroen Groenewegen van der Weyden @ 2014-12-09 9:17 ` Jan Beulich 2015-02-26 18:56 ` Jeroen Groenewegen van der Weyden 0 siblings, 1 reply; 18+ messages in thread From: Jan Beulich @ 2014-12-09 9:17 UTC (permalink / raw) To: Jeroen Groenewegen van der Weyden Cc: Kevin Tian, George Dunlap, Eddie Dong, xen-devel, Jun Nakajima, Yang Z Zhang >>> On 09.12.14 at 10:09, <groen692@grosc.com> wrote: > Did anyone find the time yet? I'm still more then willing testing any > patches. Just yesterday we were told by Intel that they still can't foresee when they will find time. Jan ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-12-09 9:17 ` Jan Beulich @ 2015-02-26 18:56 ` Jeroen Groenewegen van der Weyden 2015-02-27 8:08 ` Jan Beulich 0 siblings, 1 reply; 18+ messages in thread From: Jeroen Groenewegen van der Weyden @ 2015-02-26 18:56 UTC (permalink / raw) To: Jan Beulich Cc: Kevin Tian, George Dunlap, Eddie Dong, xen-devel, Jun Nakajima, Yang Z Zhang Hi Jan, Anything planned concerning this? BR, Jeroen. Jan Beulich schreef op 9-12-2014 om 10:17: >>>> On 09.12.14 at 10:09, <groen692@grosc.com> wrote: >> Did anyone find the time yet? I'm still more then willing testing any >> patches. > Just yesterday we were told by Intel that they still can't foresee when > they will find time. > > Jan > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2015-02-26 18:56 ` Jeroen Groenewegen van der Weyden @ 2015-02-27 8:08 ` Jan Beulich 2015-03-04 10:24 ` Jeroen Groenewegen van der Weyden 0 siblings, 1 reply; 18+ messages in thread From: Jan Beulich @ 2015-02-27 8:08 UTC (permalink / raw) To: Jeroen Groenewegen van der Weyden Cc: Kevin Tian, George Dunlap, Eddie Dong, xen-devel, Jun Nakajima, Yang Z Zhang >>> On 26.02.15 at 19:56, <groen692@grosc.com> wrote: > Anything planned concerning this? Thanks for asking, but to be honest I don't understand why you're (not the first time iirc) pinging me rather than the VMX maintainers who had promised to do that work. All I can say is that I'm being told it's still on their todo list, but no-one has found time yet. And please remember that all of nested virt is still marked preview, i.e. not fully supported - for reasons like the bug here. Jan > Jan Beulich schreef op 9-12-2014 om 10:17: >>>>> On 09.12.14 at 10:09, <groen692@grosc.com> wrote: >>> Did anyone find the time yet? I'm still more then willing testing any >>> patches. >> Just yesterday we were told by Intel that they still can't foresee when >> they will find time. >> >> Jan >> >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2015-02-27 8:08 ` Jan Beulich @ 2015-03-04 10:24 ` Jeroen Groenewegen van der Weyden 2015-04-07 7:18 ` Li, Liang Z 0 siblings, 1 reply; 18+ messages in thread From: Jeroen Groenewegen van der Weyden @ 2015-03-04 10:24 UTC (permalink / raw) To: Jan Beulich Cc: Kevin Tian, George Dunlap, Eddie Dong, xen-devel, Jun Nakajima, Yang Z Zhang Hi Jan, I am sorry, But had the impression you were somehow the owner of these feature. My mistake. I understand the feature is in preview, But are you feeling intense presure when I ask for some status every 2 months? :-)) BR, Jeroen Jan Beulich schreef op 27-2-2015 om 09:08: >>>> On 26.02.15 at 19:56, <groen692@grosc.com> wrote: >> Anything planned concerning this? > Thanks for asking, but to be honest I don't understand why you're > (not the first time iirc) pinging me rather than the VMX maintainers > who had promised to do that work. All I can say is that I'm being > told it's still on their todo list, but no-one has found time yet. And > please remember that all of nested virt is still marked preview, i.e. > not fully supported - for reasons like the bug here. > > Jan > >> Jan Beulich schreef op 9-12-2014 om 10:17: >>>>>> On 09.12.14 at 10:09, <groen692@grosc.com> wrote: >>>> Did anyone find the time yet? I'm still more then willing testing any >>>> patches. >>> Just yesterday we were told by Intel that they still can't foresee when >>> they will find time. >>> >>> Jan >>> >>> > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2015-03-04 10:24 ` Jeroen Groenewegen van der Weyden @ 2015-04-07 7:18 ` Li, Liang Z 0 siblings, 0 replies; 18+ messages in thread From: Li, Liang Z @ 2015-04-07 7:18 UTC (permalink / raw) To: Jeroen Groenewegen van der Weyden, Jan Beulich Cc: Tian, Kevin, George Dunlap, Dong, Eddie, xen-devel, Nakajima, Jun, Zhang, Yang Z Hi All, I found a way to preproduce the bug very easy by using the apic->send_IPI_all(NMI_VECTOR) in L2 in a kernel module to trigger MNI. And I have verified that the bug can be fixed as Jan's suggestion, ' the second half of vmx_idtv_reinject() needs to be done without regard to nestedhvm_vcpu_in_guestmode(v)', I will send a patch later. Liang > -----Original Message----- > From: xen-devel-bounces@lists.xen.org [mailto:xen-devel- > bounces@lists.xen.org] On Behalf Of Jeroen Groenewegen van der Weyden > Sent: Wednesday, March 04, 2015 6:24 PM > To: Jan Beulich > Cc: Tian, Kevin; George Dunlap; Dong, Eddie; xen-devel@lists.xen.org; > Nakajima, Jun; Zhang, Yang Z > Subject: Re: [Xen-devel] Crash of guest with nested vmx with Unknown > nested vmexit reason 80000021. > > Hi Jan, > > I am sorry, But had the impression you were somehow the owner of these > feature. My mistake. > I understand the feature is in preview, But are you feeling intense presure > when I ask for some status every 2 months? :-)) > > > BR, > Jeroen > > Jan Beulich schreef op 27-2-2015 om 09:08: > >>>> On 26.02.15 at 19:56, <groen692@grosc.com> wrote: > >> Anything planned concerning this? > > Thanks for asking, but to be honest I don't understand why you're (not > > the first time iirc) pinging me rather than the VMX maintainers who > > had promised to do that work. All I can say is that I'm being told > > it's still on their todo list, but no-one has found time yet. And > > please remember that all of nested virt is still marked preview, i.e. > > not fully supported - for reasons like the bug here. > > > > Jan > > > >> Jan Beulich schreef op 9-12-2014 om 10:17: > >>>>>> On 09.12.14 at 10:09, <groen692@grosc.com> wrote: > >>>> Did anyone find the time yet? I'm still more then willing testing > >>>> any patches. > >>> Just yesterday we were told by Intel that they still can't foresee > >>> when they will find time. > >>> > >>> Jan > >>> > >>> > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Crash of guest with nested vmx with Unknown nested vmexit reason 80000021. 2014-10-10 10:31 ` Jeroen Groenewegen van der Weyden 2014-10-16 6:18 ` Zhang, Yang Z @ 2014-10-16 6:27 ` Zhang, Yang Z 1 sibling, 0 replies; 18+ messages in thread From: Zhang, Yang Z @ 2014-10-16 6:27 UTC (permalink / raw) To: Jeroen Groenewegen van der Weyden, Tian, Kevin, Jan Beulich, Dong, Eddie, Nakajima, Jun Cc: xen-devel [-- Attachment #1: Type: text/plain, Size: 3683 bytes --] > -----Original Message----- > From: Zhang, Yang Z > Sent: Thursday, October 16, 2014 2:18 PM > To: Jeroen Groenewegen van der Weyden; Tian, Kevin; Jan Beulich; Dong, Eddie; > Nakajima, Jun > Cc: xen-devel@lists.xen.org > Subject: RE: [Xen-devel] Crash of guest with nested vmx with Unknown nested > vmexit reason 80000021. > > Jeroen Groenewegen van der Weyden wrote on 2014-10-10: > > Hi all, > > > > Any input from my side necessary to keep this rolling? > > Sorry for the later reply. Yes, this is a known issue to me but I didn't have time > to cook a patch fix it. As Jan pointed out, the NMI handling logic is wrong in > current nested logic. But it is not a trivial task to fix them. I will do it once I have > the time or if you are interesting in it, a patch from you is welcome. You can find the previous discussion here: http://www.gossamer-threads.com/lists/xen/devel/322693?do=post_view_threaded > > > > > BR, > > Jeroen. > > > > Tian, Kevin schreef op 7-10-2014 om 21:56: > >> Add Yang as the nested vmx owner. > >> > >>> From: Jan Beulich [mailto:JBeulich@suse.com] > >>> Sent: Tuesday, October 07, 2014 12:59 AM > >>> > >>>>>> On 30.09.14 at 17:55, <groen692@grosc.com> wrote: > >>>> Recently I updated my openSuse box from 12.3 to 13.1. On this box I > >>>> run xen with several guests. One of these guests is an appliance > >>>> that has 4 kvm guests running. > >>>> When I start this appliance with the nested vmx feature the > >>>> appliance crashes either immediately or after a few minutes. > >>>> > >>>> This same guest was running without a problem on opensuse releases > >>>> 11.4 until 12.3 [...] ==== outup xl demsg > >>>> (XEN) vvmx.c:2459:d5 Unknown nested vmexit reason 80000021. > >>>> (XEN) Failed vm entry (exit reason 0x80000021) caused by invalid > >>>> guest state (0). > >>>> (XEN) ************* VMCS Area ************** [...] > >>> So the problem here is that > >>> > >>>> (XEN) Interruptibility=0008 ActivityState=0000 > >>> VMX_INTR_SHADOW_NMI is set while > >>> > >>>> (XEN) PinBased=0000003f CPUBased=b6b9e5fa SecondaryExec=000004eb > >>> PIN_BASED_VIRTUAL_NMIS is active and > >>> > >>>> (XEN) VMEntry: intr_info=80000202 errcode=5d021101 ilen=00000003 > >>>> (XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003 > >>>> (XEN) reason=80000021 qualification=00000000 > >>>> (XEN) IDTVectoring: info=80000202 errcode=00000000 > >>> an NMI is being injected. This case is explicitly mentioned in Vol > >>> 3 section 31.7.1.2 (Resuming Guest Software after Handling an > >>> Exception). Either there needs to be extra code in vvmx.c to clear > >>> VMX_INTR_SHADOW_NMI (as the second sub-bullet point of the last > >>> bullet point says), or the second half of vmx_idtv_reinject() needs > >>> to be done without regard to nestedhvm_vcpu_in_guestmode(v) (and > >>> maybe also without regard to EXIT_REASON_TASK_SWITCH). > >>> > >>> Speaking of SDM sections: There are quite a few references in the > >>> code that name just section numbers (in the case here, several > >>> references to sections 25.7.1.* exist). These numbers become stale > >>> quite quickly (here they're now 31.7.1.*), so in order to help > >>> digging through issues like the one here, can I please ask one of > >>> you to go through and replace (or at least amend) these numbers with > >>> the sections' titles (which I hope won't get altered that quickly)? > >>> > >>> Jan > >> > >> _______________________________________________ > >> Xen-devel mailing list > >> Xen-devel@lists.xen.org > >> http://lists.xen.org/xen-devel > >> > > > Best regards, > Yang > [-- Attachment #2: Type: message/rfc822, Size: 6548 bytes --] From: Steven Krikstone <skrikstone@gmail.com> To: "Zhang, Yang Z" <yang.z.zhang@intel.com> Cc: "xen-devel@lists.xen.org" <xen-devel@lists.xen.org> Subject: Re: [Xen-devel] [BUG] L2 Guest on hyper-v causes domain_crash Date: Fri, 28 Mar 2014 22:20:47 +0000 Message-ID: <CAD6X+xVsySPm3So2qW0F-ew=dRcBDDTSu6U4p9imNNyFEwpOGQ@mail.gmail.com> On Wed, Mar 26, 2014 at 3:22 AM, Zhang, Yang Z <yang.z.zhang@intel.com<mailto:yang.z.zhang@intel.com>> wrote: Steven Krikstone wrote on 2014-03-26: > Hello, > > Running a FreeBSD L2 guest in Hyper-V on Windows Server 2012 R2 as the > L1 causes the following guest domain crash. > > ### Setup: Intel server running Ubuntu 13.10 with Xen 4.4.1pre > (git:staging-4.4:babcef3 with 1 patch) The Ubuntu server is a stock > install with just the necessary packages installed to run Xen. > Windows Server 2012 R2 is a stock install with the Hyper-V role added. > The following patch from: > http://lists.xen.org/archives/html/xen-devel/2014-02/msg01071.html was > required or else Windows Server VM would hang at the boot splash screen. > > If I set "nestedhvm=0" Server 2012R2 runs without issue (minus a > working Hyper-v role). > > Let me know if you need further logs or traces provided. > > Thanks, > > -steve > > ### xl dmesg [..many L0 PIO lines snipped..] (XEN) L0 PIO 01f7 (XEN) > L0 PIO c220 (XEN) L0 PIO c222 (XEN) L0 PIO c222 (XEN) L0 PIO c220 > (XEN) L0 PIO c222 (XEN) L0 PIO 01f7 (XEN) L0 PIO 01f7 (XEN) L0 PIO > c203 (XEN) L0 PIO c205 (XEN) L0 PIO c203 (XEN) L0 PIO c205 (XEN) L0 > PIO c207 (XEN) L0 PIO c211 (XEN) L0 PIO c213 (XEN) L0 PIO c205 (XEN) > L0 PIO c207 (XEN) > vvmx.c:2477:d3 Unknown nested vmexit reason 80000021. (XEN) Failed vm > entry (exit reason 0x80000021) caused by invalid guest state (0). > (XEN) > ************* VMCS Area ************** (XEN) *** Guest State *** (XEN) > CR0: actual=0x0000000080010031, shadow=0x0000000080050031, > gh_mask=ffffffffffffffff (XEN) CR4: actual=0x00000000001526f8, > shadow=0x00000000001506f8, gh_mask=ffffffffffffffff (XEN) CR3: > actual=0x00000000001a7000, target_count=0 (XEN) > target0=0000000000000000, target1=0000000000000000 (XEN) > target2=0000000000000000, target3=0000000000000000 (XEN) RSP = > 0xffffd00020b3c890 (0xffffd00020b3c890) RIP = 0xfffff80320f0c0ab > (0xfffff80320f0c0ab) (XEN) RFLAGS=0x0000000000000046 > (0x0000000000000046) DR7 = 0x0000000000000400 (XEN) Sysenter > RSP=0000000000000000 CS:RIP=0000:0000000000000000 (XEN) CS: > sel=0x0010, attr=0x0209b, limit=0x00000000, base=0x0000000000000000 (XEN) DS: > sel=0x002b, attr=0x0c0f3, limit=0xffffffff, base=0x0000000000000000 > (XEN) SS: sel=0x0018, attr=0x0c093, limit=0xffffffff, > base=0x0000000000000000 (XEN) ES: sel=0x002b, attr=0x0c0f3, > limit=0xffffffff, base=0x0000000000000000 (XEN) FS: sel=0x0053, > attr=0x040f3, limit=0x0000fc00, base=0x00000000ff9ce000 (XEN) GS: > sel=0x002b, attr=0x0c0f3, limit=0xffffffff, base=0xffffd00020b12000 > (XEN) GDTR: limit=0x0000007f, > base=0xffffd00020b1f040 (XEN) LDTR: sel=0x0000, attr=0x1c000, > limit=0xffffffff, base=0x0000000000000000 (XEN) IDTR: > limit=0x00000fff, base=0xffffd00020b1f0c0 (XEN) TR: sel=0x0040, > attr=0x0008b, limit=0x00000067, base=0xffffd00020b18080 (XEN) Guest > PAT = 0x0000050100070406 (XEN) TSC Offset = fffffefc546de420 (XEN) > DebugCtl=0000000000000000 DebugExceptions=0000000000000000 (XEN) > Interruptibility=0008 ActivityState=0000 Seems the virtual NMI handling for nested is not correct: blocking by NMI is set, but trying to inject a NMI to guest. Best regards, Yang I am more than willing to test any patches you may have. I am also leaving this testbed setup in case you or others need further logs, debugs run, etc. Thanks, -steve [-- Attachment #3: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-04-07 7:18 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-09-30 15:55 Crash of guest with nested vmx with Unknown nested vmexit reason 80000021 Jeroen Groenewegen van der Weyden 2014-10-01 13:20 ` Jan Beulich 2014-10-07 7:58 ` Jan Beulich 2014-10-07 15:16 ` Jeroen Groenewegen van der Weyden 2014-10-07 19:56 ` Tian, Kevin 2014-10-10 10:31 ` Jeroen Groenewegen van der Weyden 2014-10-16 6:18 ` Zhang, Yang Z 2014-10-16 6:41 ` Jan Beulich 2014-10-29 14:17 ` Jeroen Groenewegen van der Weyden 2014-10-29 16:42 ` Jan Beulich 2014-11-03 11:16 ` George Dunlap 2014-12-09 9:09 ` Jeroen Groenewegen van der Weyden 2014-12-09 9:17 ` Jan Beulich 2015-02-26 18:56 ` Jeroen Groenewegen van der Weyden 2015-02-27 8:08 ` Jan Beulich 2015-03-04 10:24 ` Jeroen Groenewegen van der Weyden 2015-04-07 7:18 ` Li, Liang Z 2014-10-16 6:27 ` Zhang, Yang Z
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.