Windows7 crashes inside the VM when starting a certain program

Windows7 crashes inside the VM when starting a certain program

[André Weidemann]

Hi,
I am running Windows7 x64 in a VM which crashes after starting a certain 
game. Actually there are two games both from the same company, that make 
the VM crash after starting them.
Windows crashes right after starting the game. With the 1st game the 
screen goes black as usual and the cursor keeps spinning for 3-5 seconds 
until Windows crashes. With the second game I get to 3D the login 
screen. The game then crashes after logging in.
Windows displays this error message on the first crash:
http://pastebin.com/kMzk9Jif
Windows then finishes writing the crash dump and restarts.
I can reproduce Windows crashing every time I start the game while the 
VM keeps running without any problems.
When Windows reboots after the first crash and the game is started 
again, the message on the following blue screen changes slightly and 
stays the same(except for the addresses) for every following crash:
http://pastebin.com/jVtBc4ZH

I first thought that this might be related to a certain feature in 3D 
acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run 
without any problems. They run a bit choppy on some occasions, but do 
that without crashing Windows7 or the VM.

How can I proceed to investigate what is going wrong?

I am using a qemu-kvm git clone from a few days ago 
(d58931037dbb4fbc2fbb33858629d3fabfd1b0d4).

This is my command line:
/usr/local/bin/qemu-system-x86_64 -boot order=ncd -cpu host -vnc 
192.168.3.1:0 -k de -smp 2,cores=1,threads=2,sockets=1 -drive 
file=/mnt/ntfs/Windows7_120G.img,if=ide,index=0,cache=writeback -m 4096 
-drive file=/dev/cdrom1,if=ide,index=3,media=cdrom,cache=none -device 
pci-assign,host=03:00.0,id=radeon6950,addr=0x11 -net 
nic,model=e1000,macaddr=DE:AD:BE:EF:42:42 -net 
tap,script=/usr/local/bin/qemu-ifup -rtc base=localtime -parallel none 
-soundhw hda -usb -device usb-ehci,id=ehci -usbdevice host:045e:00f9 
-chardev stdio,id=seabios -device 
isa-debugcon,iobase=0x402,chardev=seabios -watchdog ib700 
-watchdog-action debug -monitor telnet:127.0.0.1:11111,server,nowait,nodelay

Regards
  André

Re: Windows7 crashes inside the VM when starting a certain program

[André Weidemann]

On 07.07.2011 07:26, André Weidemann wrote:
> Hi,
> I am running Windows7 x64 in a VM which crashes after starting a certain
> game. Actually there are two games both from the same company, that make
> the VM crash after starting them.
> Windows crashes right after starting the game. With the 1st game the
> screen goes black as usual and the cursor keeps spinning for 3-5 seconds
> until Windows crashes. With the second game I get to 3D the login
> screen. The game then crashes after logging in.
> Windows displays this error message on the first crash:
> http://pastebin.com/kMzk9Jif
> Windows then finishes writing the crash dump and restarts.
> I can reproduce Windows crashing every time I start the game while the
> VM keeps running without any problems.
> When Windows reboots after the first crash and the game is started
> again, the message on the following blue screen changes slightly and
> stays the same(except for the addresses) for every following crash:
> http://pastebin.com/jVtBc4ZH
>
> I first thought that this might be related to a certain feature in 3D
> acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run
> without any problems. They run a bit choppy on some occasions, but do
> that without crashing Windows7 or the VM.
>
> How can I proceed to investigate what is going wrong?

I did some testing and found out that Windows7 does not crash anymore 
when changing "-cpu host" to "-cpu Nehalem". After doing so, the "only" 
thing crashing, is the application itself.
Why is that? What is different between the "real" CPU and the one 
provided by qemu-kvm? How can "-cpu host" cause Windows7 to crash, while 
"-cpu Nehalem" "only" crashes the application.

I then had WinDbg attach to the process in question. When the game 
crashes the debugger reports an Assertion Failure. This seems so happen 
as soon as the game accesses the network through a certain DLL.
To exclude the emulated e1000 hardware as the cause, I  removed it from 
the VM and passed an Intel network card 82574L to it using these lines:

-device pci-assign,host=04:00.0,id=82574L,addr=0x10 \
-net none \

The network card works under Windows7, but the problem of the crashing 
game remains.

Any ideas on how to track the problem are greatly appreciated.


Regards
  André

Re: Windows7 crashes inside the VM when starting a certain program

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2828 bytes --]

On 2011-07-26 07:29, André Weidemann wrote:
> On 07.07.2011 07:26, André Weidemann wrote:
>> Hi,
>> I am running Windows7 x64 in a VM which crashes after starting a certain
>> game. Actually there are two games both from the same company, that make
>> the VM crash after starting them.
>> Windows crashes right after starting the game. With the 1st game the
>> screen goes black as usual and the cursor keeps spinning for 3-5 seconds
>> until Windows crashes. With the second game I get to 3D the login
>> screen. The game then crashes after logging in.
>> Windows displays this error message on the first crash:
>> http://pastebin.com/kMzk9Jif
>> Windows then finishes writing the crash dump and restarts.
>> I can reproduce Windows crashing every time I start the game while the
>> VM keeps running without any problems.
>> When Windows reboots after the first crash and the game is started
>> again, the message on the following blue screen changes slightly and
>> stays the same(except for the addresses) for every following crash:
>> http://pastebin.com/jVtBc4ZH
>>
>> I first thought that this might be related to a certain feature in 3D
>> acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run
>> without any problems. They run a bit choppy on some occasions, but do
>> that without crashing Windows7 or the VM.
>>
>> How can I proceed to investigate what is going wrong?
> 
> I did some testing and found out that Windows7 does not crash anymore
> when changing "-cpu host" to "-cpu Nehalem". After doing so, the "only"
> thing crashing, is the application itself.
> Why is that? What is different between the "real" CPU and the one
> provided by qemu-kvm? How can "-cpu host" cause Windows7 to crash, while
> "-cpu Nehalem" "only" crashes the application.
> 
> I then had WinDbg attach to the process in question. When the game
> crashes the debugger reports an Assertion Failure. This seems so happen
> as soon as the game accesses the network through a certain DLL.
> To exclude the emulated e1000 hardware as the cause, I  removed it from
> the VM and passed an Intel network card 82574L to it using these lines:
> 
> -device pci-assign,host=04:00.0,id=82574L,addr=0x10 \
> -net none \
> 
> The network card works under Windows7, but the problem of the crashing
> game remains.
> 
> Any ideas on how to track the problem are greatly appreciated.

I faced problems more than once when the host used MSIs for an assigned
device while the guest driver decided it's better to stick with legacy
INTx (check the IRQ number under Windows, > 0 means INTx, < 0 MSI). You
can exclude this by adding prefer_msi=off to the pci-assign options. May
require disabling some host devices in case of IRQ conflict (yeah, we
still need to solve that...).

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: Windows7 crashes inside the VM when starting a certain program

[Gleb Natapov]

On Tue, Jul 26, 2011 at 07:29:04AM +0200, André Weidemann wrote:
> On 07.07.2011 07:26, André Weidemann wrote:
> >Hi,
> >I am running Windows7 x64 in a VM which crashes after starting a certain
> >game. Actually there are two games both from the same company, that make
> >the VM crash after starting them.
> >Windows crashes right after starting the game. With the 1st game the
> >screen goes black as usual and the cursor keeps spinning for 3-5 seconds
> >until Windows crashes. With the second game I get to 3D the login
> >screen. The game then crashes after logging in.
> >Windows displays this error message on the first crash:
> >http://pastebin.com/kMzk9Jif
> >Windows then finishes writing the crash dump and restarts.
> >I can reproduce Windows crashing every time I start the game while the
> >VM keeps running without any problems.
> >When Windows reboots after the first crash and the game is started
> >again, the message on the following blue screen changes slightly and
> >stays the same(except for the addresses) for every following crash:
> >http://pastebin.com/jVtBc4ZH
> >
> >I first thought that this might be related to a certain feature in 3D
> >acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run
> >without any problems. They run a bit choppy on some occasions, but do
> >that without crashing Windows7 or the VM.
> >
> >How can I proceed to investigate what is going wrong?
> 
> I did some testing and found out that Windows7 does not crash
> anymore when changing "-cpu host" to "-cpu Nehalem". After doing so,
What is your host cpu (cat /proc/cpuinfo)?

> the "only" thing crashing, is the application itself.
> Why is that? What is different between the "real" CPU and the one
> provided by qemu-kvm? How can "-cpu host" cause Windows7 to crash,
> while "-cpu Nehalem" "only" crashes the application.
> 
> I then had WinDbg attach to the process in question. When the game
> crashes the debugger reports an Assertion Failure. This seems so
> happen as soon as the game accesses the network through a certain
> DLL.
> To exclude the emulated e1000 hardware as the cause, I  removed it
> from the VM and passed an Intel network card 82574L to it using
> these lines:
> 
> -device pci-assign,host=04:00.0,id=82574L,addr=0x10 \
> -net none \
> 
> The network card works under Windows7, but the problem of the
> crashing game remains.
> 
> Any ideas on how to track the problem are greatly appreciated.
> 
> 
> Regards
>  André
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
			Gleb.

Re: Windows7 crashes inside the VM when starting a certain program

[André Weidemann]

Hi,

On 26.07.2011 12:08, Gleb Natapov wrote:
> On Tue, Jul 26, 2011 at 07:29:04AM +0200, André Weidemann wrote:
>> On 07.07.2011 07:26, André Weidemann wrote:
>>> Hi,
>>> I am running Windows7 x64 in a VM which crashes after starting a certain
>>> game. Actually there are two games both from the same company, that make
>>> the VM crash after starting them.
>>> Windows crashes right after starting the game. With the 1st game the
>>> screen goes black as usual and the cursor keeps spinning for 3-5 seconds
>>> until Windows crashes. With the second game I get to 3D the login
>>> screen. The game then crashes after logging in.
>>> Windows displays this error message on the first crash:
>>> http://pastebin.com/kMzk9Jif
>>> Windows then finishes writing the crash dump and restarts.
>>> I can reproduce Windows crashing every time I start the game while the
>>> VM keeps running without any problems.
>>> When Windows reboots after the first crash and the game is started
>>> again, the message on the following blue screen changes slightly and
>>> stays the same(except for the addresses) for every following crash:
>>> http://pastebin.com/jVtBc4ZH
>>>
>>> I first thought that this might be related to a certain feature in 3D
>>> acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run
>>> without any problems. They run a bit choppy on some occasions, but do
>>> that without crashing Windows7 or the VM.
>>>
>>> How can I proceed to investigate what is going wrong?
>>
>> I did some testing and found out that Windows7 does not crash
>> anymore when changing "-cpu host" to "-cpu Nehalem". After doing so,
> What is your host cpu (cat /proc/cpuinfo)?

The server is currently running on 2 out of 8 cores with kernel boot 
parameter "maxcpus=2".

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping        : 5
cpu MHz         : 1596.000
cache size      : 8192 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe 
syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good 
xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 
ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi 
flexpriority ept vpid
bogomips        : 5405.59
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping        : 5
cpu MHz         : 1596.000
cache size      : 8192 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 2
initial apicid  : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe 
syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good 
xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 
ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi 
flexpriority ept vpid
bogomips        : 5404.84
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

>> the "only" thing crashing, is the application itself.
>> Why is that? What is different between the "real" CPU and the one
>> provided by qemu-kvm? How can "-cpu host" cause Windows7 to crash,
>> while "-cpu Nehalem" "only" crashes the application.
>>
>> I then had WinDbg attach to the process in question. When the game
>> crashes the debugger reports an Assertion Failure. This seems so
>> happen as soon as the game accesses the network through a certain
>> DLL.
>> To exclude the emulated e1000 hardware as the cause, I  removed it
>> from the VM and passed an Intel network card 82574L to it using
>> these lines:
>>
>> -device pci-assign,host=04:00.0,id=82574L,addr=0x10 \
>> -net none \
>>
>> The network card works under Windows7, but the problem of the
>> crashing game remains.
>>
>> Any ideas on how to track the problem are greatly appreciated.
>>
>>
>> Regards
>>   André
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> 			Gleb.

  André

Re: Windows7 crashes inside the VM when starting a certain program

[Gleb Natapov]

On Tue, Jul 26, 2011 at 12:57:44PM +0200, André Weidemann wrote:
> Hi,
> 
> On 26.07.2011 12:08, Gleb Natapov wrote:
> >On Tue, Jul 26, 2011 at 07:29:04AM +0200, André Weidemann wrote:
> >>On 07.07.2011 07:26, André Weidemann wrote:
> >>>Hi,
> >>>I am running Windows7 x64 in a VM which crashes after starting a certain
> >>>game. Actually there are two games both from the same company, that make
> >>>the VM crash after starting them.
> >>>Windows crashes right after starting the game. With the 1st game the
> >>>screen goes black as usual and the cursor keeps spinning for 3-5 seconds
> >>>until Windows crashes. With the second game I get to 3D the login
> >>>screen. The game then crashes after logging in.
> >>>Windows displays this error message on the first crash:
> >>>http://pastebin.com/kMzk9Jif
> >>>Windows then finishes writing the crash dump and restarts.
> >>>I can reproduce Windows crashing every time I start the game while the
> >>>VM keeps running without any problems.
> >>>When Windows reboots after the first crash and the game is started
> >>>again, the message on the following blue screen changes slightly and
> >>>stays the same(except for the addresses) for every following crash:
> >>>http://pastebin.com/jVtBc4ZH
> >>>
> >>>I first thought that this might be related to a certain feature in 3D
> >>>acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run
> >>>without any problems. They run a bit choppy on some occasions, but do
> >>>that without crashing Windows7 or the VM.
> >>>
> >>>How can I proceed to investigate what is going wrong?
> >>
> >>I did some testing and found out that Windows7 does not crash
> >>anymore when changing "-cpu host" to "-cpu Nehalem". After doing so,
> >What is your host cpu (cat /proc/cpuinfo)?
> 
> The server is currently running on 2 out of 8 cores with kernel boot
> parameter "maxcpus=2".
> 
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
> pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
> pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
> xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est
> tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow
> vnmi flexpriority ept vpid
Flags that are present on -cpu host but not -cpu Nehalem (excluding vmx
related flags):

vme dts acpi ss ht tm pbe rdtscp constant_tsc arch_perfmon pebs bts rep_good
xtopology nonstop_tsc aperfmperf dtes64 monitor ds_cpl est tm2 xtpr pdcm  ida 

Some of them may be synthetic and some of them may be filtered by KVM.

Can you try to run "-cpu host,-vme,-dts..." (specifying all of those
flags with -). Drop those that qemu does not recognize. See if result
will be the same as with -cpu Nehalem. If yes, then try to find out with
flag make the difference.

--
			Gleb.

Re: Windows7 crashes inside the VM when starting a certain program

[Paolo Bonzini]

On 07/07/2011 07:26 AM, André Weidemann wrote:
> Hi,
> I am running Windows7 x64 in a VM which crashes after starting a certain
> game. Actually there are two games both from the same company, that make
> the VM crash after starting them.
> Windows crashes right after starting the game. With the 1st game the
> screen goes black as usual and the cursor keeps spinning for 3-5 seconds
> until Windows crashes. With the second game I get to 3D the login
> screen. The game then crashes after logging in.
> Windows displays this error message on the first crash:
> http://pastebin.com/kMzk9Jif
> Windows then finishes writing the crash dump and restarts.
> I can reproduce Windows crashing every time I start the game while the
> VM keeps running without any problems.
> When Windows reboots after the first crash and the game is started
> again, the message on the following blue screen changes slightly and
> stays the same(except for the addresses) for every following crash:
> http://pastebin.com/jVtBc4ZH

The blue screens seem to be for the same exception, 
0xC0000096--privileged instruction, only sometimes in user mode (but in 
a system service, which also causes a blue screen) sometimes in kernel mode.

Can you open the produced dump in WinDbg and post a disassemble around 
the failing instruction?

Thanks,

Paolo

Re: Windows7 crashes inside the VM when starting a certain program

[Gleb Natapov]

On Thu, Jul 28, 2011 at 02:01:09PM +0200, Paolo Bonzini wrote:
> On 07/07/2011 07:26 AM, André Weidemann wrote:
> >Hi,
> >I am running Windows7 x64 in a VM which crashes after starting a certain
> >game. Actually there are two games both from the same company, that make
> >the VM crash after starting them.
> >Windows crashes right after starting the game. With the 1st game the
> >screen goes black as usual and the cursor keeps spinning for 3-5 seconds
> >until Windows crashes. With the second game I get to 3D the login
> >screen. The game then crashes after logging in.
> >Windows displays this error message on the first crash:
> >http://pastebin.com/kMzk9Jif
> >Windows then finishes writing the crash dump and restarts.
> >I can reproduce Windows crashing every time I start the game while the
> >VM keeps running without any problems.
> >When Windows reboots after the first crash and the game is started
> >again, the message on the following blue screen changes slightly and
> >stays the same(except for the addresses) for every following crash:
> >http://pastebin.com/jVtBc4ZH
> 
> The blue screens seem to be for the same exception,
> 0xC0000096--privileged instruction, only sometimes in user mode (but
> in a system service, which also causes a blue screen) sometimes in
> kernel mode.
> 
And I assume 0xFFFFF800028D8C0C is the instruction address? If yes then can
you do "x/i 0xFFFFF800028D8C0C" in qemu monitor then (or whatever new
address it will show you).

> Can you open the produced dump in WinDbg and post a disassemble
> around the failing instruction?
> 
> Thanks,
> 
> Paolo
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
			Gleb.

Re: Windows7 crashes inside the VM when starting a certain program

[André Weidemann]

Hi Paolo,

On 28.07.2011 14:01, Paolo Bonzini wrote:
> On 07/07/2011 07:26 AM, André Weidemann wrote:
>> Hi,
>> I am running Windows7 x64 in a VM which crashes after starting a certain
>> game. Actually there are two games both from the same company, that make
>> the VM crash after starting them.
>> Windows crashes right after starting the game. With the 1st game the
>> screen goes black as usual and the cursor keeps spinning for 3-5 seconds
>> until Windows crashes. With the second game I get to 3D the login
>> screen. The game then crashes after logging in.
>> Windows displays this error message on the first crash:
>> http://pastebin.com/kMzk9Jif
>> Windows then finishes writing the crash dump and restarts.
>> I can reproduce Windows crashing every time I start the game while the
>> VM keeps running without any problems.
>> When Windows reboots after the first crash and the game is started
>> again, the message on the following blue screen changes slightly and
>> stays the same(except for the addresses) for every following crash:
>> http://pastebin.com/jVtBc4ZH
>
> The blue screens seem to be for the same exception,
> 0xC0000096--privileged instruction, only sometimes in user mode (but in
> a system service, which also causes a blue screen) sometimes in kernel
> mode.
>
> Can you open the produced dump in WinDbg and post a disassemble around
> the failing instruction?

I haven't used debuggers very much, so I hope I grabbed the correct 
lines from the disassembly:
http://pastebin.com/t3sfvmTg

André

Re: Windows7 crashes inside the VM when starting a certain program

[Avi Kivity]

On 07/28/2011 04:16 PM, André Weidemann wrote:
>> Can you open the produced dump in WinDbg and post a disassemble around
>> the failing instruction?
>
>
> I haven't used debuggers very much, so I hope I grabbed the correct 
> lines from the disassembly:
> http://pastebin.com/t3sfvmTg
>

That's the bug check routine.  Can you go up a frame?

(I'd say how, but I forget).

-- 
error compiling committee.c: too many arguments to function

Re: Windows7 crashes inside the VM when starting a certain program

[Paolo Bonzini]

On 07/28/2011 03:21 PM, Avi Kivity wrote:
>> I haven't used debuggers very much, so I hope I grabbed the correct
>> lines from the disassembly:
>> http://pastebin.com/t3sfvmTg
>
> That's the bug check routine.  Can you go up a frame?

Or just do what Gleb suggested.  Open the dump, type "!analyze -v" and 
cut-paste the address from WinDbg's output into the Disassemble window.

Paolo

Re: Windows7 crashes inside the VM when starting a certain program

[André Weidemann]

Hi,

On 28.07.2011 15:49, Paolo Bonzini wrote:
> On 07/28/2011 03:21 PM, Avi Kivity wrote:
>>> I haven't used debuggers very much, so I hope I grabbed the correct
>>> lines from the disassembly:
>>> http://pastebin.com/t3sfvmTg
>>
>> That's the bug check routine. Can you go up a frame?
>
> Or just do what Gleb suggested. Open the dump, type "!analyze -v" and
> cut-paste the address from WinDbg's output into the Disassemble window.

This is the output of "!analyze -v":
http://pastebin.com/sCZSjr8m

...and this is the output from the disassembly window:
http://pastebin.com/AVZuswkT


André

Re: Windows7 crashes inside the VM when starting a certain program

[Paolo Bonzini]

On 07/28/2011 07:44 PM, André Weidemann wrote:
> Hi,
>
> On 28.07.2011 15:49, Paolo Bonzini wrote:
>> On 07/28/2011 03:21 PM, Avi Kivity wrote:
>>>> I haven't used debuggers very much, so I hope I grabbed the correct
>>>> lines from the disassembly:
>>>> http://pastebin.com/t3sfvmTg
>>>
>>> That's the bug check routine. Can you go up a frame?
>>
>> Or just do what Gleb suggested. Open the dump, type "!analyze -v" and
>> cut-paste the address from WinDbg's output into the Disassemble window.
>
> This is the output of "!analyze -v":
> http://pastebin.com/sCZSjr8m
>
> ...and this is the output from the disassembly window:
> http://pastebin.com/AVZuswkT

Very useful, thanks!

Paolo

Re: Windows7 crashes inside the VM when starting a certain program

[André Weidemann]

On 27.07.2011 10:56, Gleb Natapov wrote:
> On Tue, Jul 26, 2011 at 12:57:44PM +0200, André Weidemann wrote:
>> Hi,
>>
>> On 26.07.2011 12:08, Gleb Natapov wrote:
>>> On Tue, Jul 26, 2011 at 07:29:04AM +0200, André Weidemann wrote:
>>>> On 07.07.2011 07:26, André Weidemann wrote:
>>>>> Hi,
>>>>> I am running Windows7 x64 in a VM which crashes after starting a certain
>>>>> game. Actually there are two games both from the same company, that make
>>>>> the VM crash after starting them.
>>>>> Windows crashes right after starting the game. With the 1st game the
>>>>> screen goes black as usual and the cursor keeps spinning for 3-5 seconds
>>>>> until Windows crashes. With the second game I get to 3D the login
>>>>> screen. The game then crashes after logging in.
>>>>> Windows displays this error message on the first crash:
>>>>> http://pastebin.com/kMzk9Jif
>>>>> Windows then finishes writing the crash dump and restarts.
>>>>> I can reproduce Windows crashing every time I start the game while the
>>>>> VM keeps running without any problems.
>>>>> When Windows reboots after the first crash and the game is started
>>>>> again, the message on the following blue screen changes slightly and
>>>>> stays the same(except for the addresses) for every following crash:
>>>>> http://pastebin.com/jVtBc4ZH
>>>>>
>>>>> I first thought that this might be related to a certain feature in 3D
>>>>> acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run
>>>>> without any problems. They run a bit choppy on some occasions, but do
>>>>> that without crashing Windows7 or the VM.
>>>>>
>>>>> How can I proceed to investigate what is going wrong?
>>>>
>>>> I did some testing and found out that Windows7 does not crash
>>>> anymore when changing "-cpu host" to "-cpu Nehalem". After doing so,
>>> What is your host cpu (cat /proc/cpuinfo)?
>>
>> The server is currently running on 2 out of 8 cores with kernel boot
>> parameter "maxcpus=2".
>>
>> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
>> pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
>> pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
>> xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est
>> tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow
>> vnmi flexpriority ept vpid
> Flags that are present on -cpu host but not -cpu Nehalem (excluding vmx
> related flags):
>
> vme dts acpi ss ht tm pbe rdtscp constant_tsc arch_perfmon pebs bts rep_good
> xtopology nonstop_tsc aperfmperf dtes64 monitor ds_cpl est tm2 xtpr pdcm  ida
>
> Some of them may be synthetic and some of them may be filtered by KVM.
>
> Can you try to run "-cpu host,-vme,-dts..." (specifying all of those
> flags with -). Drop those that qemu does not recognize. See if result
> will be the same as with -cpu Nehalem. If yes, then try to find out with
> flag make the difference.

I started the VM with all flags that differ between the two CPUs. After 
removing the ones qemu-kvm did not recognize, I started the VM again 
with the following line:
-cpu 
host,-vme,-acpi,-ss,-ht,-tm,-pbe,-rdtscp,-dtes64,-monitor,-ds_cpl,-est,-tm2,-xtpr,-pdcm 
\

Running the program under Windows7 inside the VM, caused Windows to 
crash again with a BSoD.
The disassembly of the address fffff8000288320c shows the following:
http://pastebin.com/7yzTYJSG

André

Re: Windows7 crashes inside the VM when starting a certain program

[Gleb Natapov]

On Fri, Jul 29, 2011 at 09:20:35AM +0200, André Weidemann wrote:
> On 27.07.2011 10:56, Gleb Natapov wrote:
> >On Tue, Jul 26, 2011 at 12:57:44PM +0200, André Weidemann wrote:
> >>Hi,
> >>
> >>On 26.07.2011 12:08, Gleb Natapov wrote:
> >>>On Tue, Jul 26, 2011 at 07:29:04AM +0200, André Weidemann wrote:
> >>>>On 07.07.2011 07:26, André Weidemann wrote:
> >>>>>Hi,
> >>>>>I am running Windows7 x64 in a VM which crashes after starting a certain
> >>>>>game. Actually there are two games both from the same company, that make
> >>>>>the VM crash after starting them.
> >>>>>Windows crashes right after starting the game. With the 1st game the
> >>>>>screen goes black as usual and the cursor keeps spinning for 3-5 seconds
> >>>>>until Windows crashes. With the second game I get to 3D the login
> >>>>>screen. The game then crashes after logging in.
> >>>>>Windows displays this error message on the first crash:
> >>>>>http://pastebin.com/kMzk9Jif
> >>>>>Windows then finishes writing the crash dump and restarts.
> >>>>>I can reproduce Windows crashing every time I start the game while the
> >>>>>VM keeps running without any problems.
> >>>>>When Windows reboots after the first crash and the game is started
> >>>>>again, the message on the following blue screen changes slightly and
> >>>>>stays the same(except for the addresses) for every following crash:
> >>>>>http://pastebin.com/jVtBc4ZH
> >>>>>
> >>>>>I first thought that this might be related to a certain feature in 3D
> >>>>>acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run
> >>>>>without any problems. They run a bit choppy on some occasions, but do
> >>>>>that without crashing Windows7 or the VM.
> >>>>>
> >>>>>How can I proceed to investigate what is going wrong?
> >>>>
> >>>>I did some testing and found out that Windows7 does not crash
> >>>>anymore when changing "-cpu host" to "-cpu Nehalem". After doing so,
> >>>What is your host cpu (cat /proc/cpuinfo)?
> >>
> >>The server is currently running on 2 out of 8 cores with kernel boot
> >>parameter "maxcpus=2".
> >>
> >>flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
> >>pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
> >>pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
> >>xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est
> >>tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow
> >>vnmi flexpriority ept vpid
> >Flags that are present on -cpu host but not -cpu Nehalem (excluding vmx
> >related flags):
> >
> >vme dts acpi ss ht tm pbe rdtscp constant_tsc arch_perfmon pebs bts rep_good
> >xtopology nonstop_tsc aperfmperf dtes64 monitor ds_cpl est tm2 xtpr pdcm  ida
> >
> >Some of them may be synthetic and some of them may be filtered by KVM.
> >
> >Can you try to run "-cpu host,-vme,-dts..." (specifying all of those
> >flags with -). Drop those that qemu does not recognize. See if result
> >will be the same as with -cpu Nehalem. If yes, then try to find out with
> >flag make the difference.
> 
> I started the VM with all flags that differ between the two CPUs.
> After removing the ones qemu-kvm did not recognize, I started the VM
> again with the following line:
> -cpu host,-vme,-acpi,-ss,-ht,-tm,-pbe,-rdtscp,-dtes64,-monitor,-ds_cpl,-est,-tm2,-xtpr,-pdcm
> \
> 
> Running the program under Windows7 inside the VM, caused Windows to
> crash again with a BSoD.
> The disassembly of the address fffff8000288320c shows the following:
> http://pastebin.com/7yzTYJSG
> 
Looks like it tries to read MSR_LASTBRANCH_TOS MSR which kvm does not
support. Do you see something interesting in dmesg? I wonder how
availability of the MSR should be checked.

--
			Gleb.

Re: Windows7 crashes inside the VM when starting a certain program

[André Weidemann]

On 29.07.2011 13:37, Gleb Natapov wrote:
> On Fri, Jul 29, 2011 at 09:20:35AM +0200, André Weidemann wrote:
>> On 27.07.2011 10:56, Gleb Natapov wrote:
>>> On Tue, Jul 26, 2011 at 12:57:44PM +0200, André Weidemann wrote:
>>>> Hi,
>>>>
>>>> On 26.07.2011 12:08, Gleb Natapov wrote:
>>>>> On Tue, Jul 26, 2011 at 07:29:04AM +0200, André Weidemann wrote:
>>>>>> On 07.07.2011 07:26, André Weidemann wrote:
>>>>>>> Hi,
>>>>>>> I am running Windows7 x64 in a VM which crashes after starting a certain
>>>>>>> game. Actually there are two games both from the same company, that make
>>>>>>> the VM crash after starting them.
>>>>>>> Windows crashes right after starting the game. With the 1st game the
>>>>>>> screen goes black as usual and the cursor keeps spinning for 3-5 seconds
>>>>>>> until Windows crashes. With the second game I get to 3D the login
>>>>>>> screen. The game then crashes after logging in.
>>>>>>> Windows displays this error message on the first crash:
>>>>>>> http://pastebin.com/kMzk9Jif
>>>>>>> Windows then finishes writing the crash dump and restarts.
>>>>>>> I can reproduce Windows crashing every time I start the game while the
>>>>>>> VM keeps running without any problems.
>>>>>>> When Windows reboots after the first crash and the game is started
>>>>>>> again, the message on the following blue screen changes slightly and
>>>>>>> stays the same(except for the addresses) for every following crash:
>>>>>>> http://pastebin.com/jVtBc4ZH
>>>>>>>
>>>>>>> I first thought that this might be related to a certain feature in 3D
>>>>>>> acceleration being used, but Futuremark 3DMark Vantage or 3DMark 11 run
>>>>>>> without any problems. They run a bit choppy on some occasions, but do
>>>>>>> that without crashing Windows7 or the VM.
>>>>>>>
>>>>>>> How can I proceed to investigate what is going wrong?
>>>>>>
>>>>>> I did some testing and found out that Windows7 does not crash
>>>>>> anymore when changing "-cpu host" to "-cpu Nehalem". After doing so,
>>>>> What is your host cpu (cat /proc/cpuinfo)?
>>>>
>>>> The server is currently running on 2 out of 8 cores with kernel boot
>>>> parameter "maxcpus=2".
>>>>
>>>> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
>>>> pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
>>>> pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
>>>> xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est
>>>> tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow
>>>> vnmi flexpriority ept vpid
>>> Flags that are present on -cpu host but not -cpu Nehalem (excluding vmx
>>> related flags):
>>>
>>> vme dts acpi ss ht tm pbe rdtscp constant_tsc arch_perfmon pebs bts rep_good
>>> xtopology nonstop_tsc aperfmperf dtes64 monitor ds_cpl est tm2 xtpr pdcm  ida
>>>
>>> Some of them may be synthetic and some of them may be filtered by KVM.
>>>
>>> Can you try to run "-cpu host,-vme,-dts..." (specifying all of those
>>> flags with -). Drop those that qemu does not recognize. See if result
>>> will be the same as with -cpu Nehalem. If yes, then try to find out with
>>> flag make the difference.
>>
>> I started the VM with all flags that differ between the two CPUs.
>> After removing the ones qemu-kvm did not recognize, I started the VM
>> again with the following line:
>> -cpu host,-vme,-acpi,-ss,-ht,-tm,-pbe,-rdtscp,-dtes64,-monitor,-ds_cpl,-est,-tm2,-xtpr,-pdcm
>> \
>>
>> Running the program under Windows7 inside the VM, caused Windows to
>> crash again with a BSoD.
>> The disassembly of the address fffff8000288320c shows the following:
>> http://pastebin.com/7yzTYJSG
>>
> Looks like it tries to read MSR_LASTBRANCH_TOS MSR which kvm does not
> support. Do you see something interesting in dmesg? I wonder how
> availability of the MSR should be checked.

You are right about the MSR. The log file from the time of occurrence 
shows the following:

Jul  4 18:55:12 server kernel: [ 1528.618647] kvm: 3123: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul  4 18:55:12 server kernel: [ 1528.619444] kvm: 3123: cpu1 unhandled 
rdmsr: 0x1c9


André

Re: Windows7 crashes inside the VM when starting a certain program

[Avi Kivity]

On 07/30/2011 03:29 PM, André Weidemann wrote:
>> Looks like it tries to read MSR_LASTBRANCH_TOS MSR which kvm does not
>> support. Do you see something interesting in dmesg? I wonder how
>> availability of the MSR should be checked.
>
>
> You are right about the MSR. The log file from the time of occurrence 
> shows the following:
>
> Jul  4 18:55:12 server kernel: [ 1528.618647] kvm: 3123: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul  4 18:55:12 server kernel: [ 1528.619444] kvm: 3123: cpu1 
> unhandled rdmsr: 0x1c9
>

You can try

$ rmmod kvm-intel kvm
$ modprobe kvm ignore_msrs=1
$ modprobe kvm-intel

(or kvm-amd, as the case may be)

This will work around the issue, but the application may not function 
correctly.

-- 
error compiling committee.c: too many arguments to function

Re: Windows7 crashes inside the VM when starting a certain program

[Gleb Natapov]

On Sun, Jul 31, 2011 at 11:16:00AM +0300, Avi Kivity wrote:
> On 07/30/2011 03:29 PM, André Weidemann wrote:
> >>Looks like it tries to read MSR_LASTBRANCH_TOS MSR which kvm does not
> >>support. Do you see something interesting in dmesg? I wonder how
> >>availability of the MSR should be checked.
> >
> >
> >You are right about the MSR. The log file from the time of
> >occurrence shows the following:
> >
> >Jul  4 18:55:12 server kernel: [ 1528.618647] kvm: 3123: cpu1
> >kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >Jul  4 18:55:12 server kernel: [ 1528.619444] kvm: 3123: cpu1
> >unhandled rdmsr: 0x1c9
> >
> 
> You can try
> 
> $ rmmod kvm-intel kvm
> $ modprobe kvm ignore_msrs=1
> $ modprobe kvm-intel
> 
> (or kvm-amd, as the case may be)
> 
> This will work around the issue, but the application may not
> function correctly.
> 
I wonder why Windows tries to use this MSR with -cpu host, but not with -cpu
Nehalem. Both cpus have this MSR (it is supported back to P4 it seams). 

--
			Gleb.