stub domain crash under heavy disk i/o

stub domain crash under heavy disk i/o

[Bart Van Kerckhove]

Dear list,

I have observed apparently random crashes on a windows 2008 HVM domain (no 
special drivers installed) with disk/net running in a stub domain.
The easiest way for me to reproduce this is to create lots of disk i/o, in 
my case using sisoft sandra.

I have tested this on xen 3.4.0 as well as 3.4.1rc10, also reproduced it in 
windows 2003 x64
These crashes do not occur when running without stubdom.

I find this in the qemu-dm-win2k8.log :

Cannot handle page request order 0!
Page fault at linear address 0x0, rip 0xe8ef0, regs 0x5bf088, sp 0x5bf130, 
our_sp 0x5bf050, code 2
Thread: main
RIP: e030:[<00000000000e8ef0>]
RSP: e02b:00000000005bf130  EFLAGS: 00010202
RAX: 0000000000000000 RBX: 00000000005bf188 RCX: 0000000000000020
RDX: 0000000000001000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 00000000005bf178 R08: 0101010101010101 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000001e96aa8 R12: 000000200354a000
R13: 0000000000000003 R14: 0000000000000000 R15: ffff800000000000
base is 0x5bf178 caller is 0x3067
base is 0x5bf2a8 caller is 0x28447
base is 0x5bf2c8 caller is 0x6a80c
base is 0x5bf318 caller is 0x6af5e
base is 0x5bf368 caller is 0x6b04f
base is 0x5bf3b8 caller is 0x68edc
base is 0x5bf3f8 caller is 0x68f7e
base is 0x5bf448 caller is 0x68fb8
base is 0x5bf458 caller is 0x5ba29
base is 0x5bf488 caller is 0x5a563
base is 0x5bf4a8 caller is 0x3480
base is 0x5bf4b8 caller is 0x35a9
base is 0x5bf4c8 caller is 0x21647
base is 0x5bf4d8 caller is 0x217aa
base is 0x5bf578 caller is 0x21ce8
base is 0x5bf5c8 caller is 0x4a9d
base is 0x5bf638 caller is 0x21ee1
base is 0x5bf698 caller is 0x9376
base is 0x5bfe18 caller is 0xcae73
base is 0x5bffe8 caller is 0x33da

5bf120: 30 f1 5b 00 00 00 00 00 2b e0 00 00 00 00 00 00
5bf130: bd 56 0e 00 00 00 00 00 25 d0 5d 26 02 00 00 00
5bf140: 80 8a 4b 6b 01 00 00 00 03 00 00 00 02 00 00 00
5bf150: 00 a0 54 03 20 00 00 00 00 a0 54 03 20 00 00 00

5bf160: 48 c4 6b 00 00 00 00 00 00 f0 ff ff ff ff 0f 00
5bf170: 03 00 00 00 00 00 00 00 a8 f2 5b 00 00 00 00 00
5bf180: 67 30 00 00 00 00 00 00 00 00 00 00 00 80 ff ff
5bf190: 03 00 00 00 00 00 00 00 00 f0 ff ff ff ff 0f 00

e8ee0: 48 81 fa 00 01 00 00 72 78 48 c1 e9 07 0f 1f 00
e8ef0: 48 0f c3 07 48 0f c3 47 08 48 0f c3 47 10 48 0f
e8f00: c3 47 18 48 0f c3 47 20 48 0f c3 47 28 48 0f c3
e8f10: 47 30 48 0f c3 47 38 48 0f c3 47 40 48 0f c3 47
Pagetable walk from virt 0, base 555000:
 L4 = 00000001ead58067 (0x556000)  [offset = 0]
  L3 = 00000001ead57067 (0x557000)  [offset = 0]
   L2 = 00000001ead56067 (0x558000)  [offset = 0]
    L1 = 0000000000000000 [offset = 0]


Thanks for any insights you have to share!

-Bart Van Kerckhove 

Re: stub domain crash under heavy disk i/o

[Keith Coleman]

Unfortunately this is a known issue that I have discussed with Keir
and Stefano. It will almost certainly not be fixed in time for the
3.4.1 release that should happen this week.

Keith Coleman


On Sun, Aug 2, 2009 at 5:58 AM, Bart Van Kerckhove<bart@it-ss.be> wrote:
> Dear list,
>
> I have observed apparently random crashes on a windows 2008 HVM domain (no
> special drivers installed) with disk/net running in a stub domain.
> The easiest way for me to reproduce this is to create lots of disk i/o, in
> my case using sisoft sandra.
>
> I have tested this on xen 3.4.0 as well as 3.4.1rc10, also reproduced it in
> windows 2003 x64
> These crashes do not occur when running without stubdom.
>
> I find this in the qemu-dm-win2k8.log :
>
> Cannot handle page request order 0!
> Page fault at linear address 0x0, rip 0xe8ef0, regs 0x5bf088, sp 0x5bf130,
> our_sp 0x5bf050, code 2
> Thread: main
> RIP: e030:[<00000000000e8ef0>]
> RSP: e02b:00000000005bf130  EFLAGS: 00010202
> RAX: 0000000000000000 RBX: 00000000005bf188 RCX: 0000000000000020
> RDX: 0000000000001000 RSI: 0000000000000000 RDI: 0000000000000000
> RBP: 00000000005bf178 R08: 0101010101010101 R09: 0000000000000000
> R10: 0000000000000000 R11: 0000000001e96aa8 R12: 000000200354a000
> R13: 0000000000000003 R14: 0000000000000000 R15: ffff800000000000
> base is 0x5bf178 caller is 0x3067
> base is 0x5bf2a8 caller is 0x28447
> base is 0x5bf2c8 caller is 0x6a80c
> base is 0x5bf318 caller is 0x6af5e
> base is 0x5bf368 caller is 0x6b04f
> base is 0x5bf3b8 caller is 0x68edc
> base is 0x5bf3f8 caller is 0x68f7e
> base is 0x5bf448 caller is 0x68fb8
> base is 0x5bf458 caller is 0x5ba29
> base is 0x5bf488 caller is 0x5a563
> base is 0x5bf4a8 caller is 0x3480
> base is 0x5bf4b8 caller is 0x35a9
> base is 0x5bf4c8 caller is 0x21647
> base is 0x5bf4d8 caller is 0x217aa
> base is 0x5bf578 caller is 0x21ce8
> base is 0x5bf5c8 caller is 0x4a9d
> base is 0x5bf638 caller is 0x21ee1
> base is 0x5bf698 caller is 0x9376
> base is 0x5bfe18 caller is 0xcae73
> base is 0x5bffe8 caller is 0x33da
>
> 5bf120: 30 f1 5b 00 00 00 00 00 2b e0 00 00 00 00 00 00
> 5bf130: bd 56 0e 00 00 00 00 00 25 d0 5d 26 02 00 00 00
> 5bf140: 80 8a 4b 6b 01 00 00 00 03 00 00 00 02 00 00 00
> 5bf150: 00 a0 54 03 20 00 00 00 00 a0 54 03 20 00 00 00
>
> 5bf160: 48 c4 6b 00 00 00 00 00 00 f0 ff ff ff ff 0f 00
> 5bf170: 03 00 00 00 00 00 00 00 a8 f2 5b 00 00 00 00 00
> 5bf180: 67 30 00 00 00 00 00 00 00 00 00 00 00 80 ff ff
> 5bf190: 03 00 00 00 00 00 00 00 00 f0 ff ff ff ff 0f 00
>
> e8ee0: 48 81 fa 00 01 00 00 72 78 48 c1 e9 07 0f 1f 00
> e8ef0: 48 0f c3 07 48 0f c3 47 08 48 0f c3 47 10 48 0f
> e8f00: c3 47 18 48 0f c3 47 20 48 0f c3 47 28 48 0f c3
> e8f10: 47 30 48 0f c3 47 38 48 0f c3 47 40 48 0f c3 47
> Pagetable walk from virt 0, base 555000:
> L4 = 00000001ead58067 (0x556000)  [offset = 0]
>  L3 = 00000001ead57067 (0x557000)  [offset = 0]
>  L2 = 00000001ead56067 (0x558000)  [offset = 0]
>   L1 = 0000000000000000 [offset = 0]
>
>
> Thanks for any insights you have to share!
>
> -Bart Van Kerckhove
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>