bug in _shadow_prealloc during migration of PV domU

bug in _shadow_prealloc during migration of PV domU

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 5153 bytes --]

I was testing 'virsh migrate domU host' and did some libvirtd debugging
on 'host'. This means the migration was attempted a few times, but did
not actually start because libvirtd was in gdb. Not sure if libvirt on
the sender does anything with the domU before a connection to the remote
host is fully established.

Finally I installed the fixed libvirtd on 'host' and started the
migration again. This time the sender died like this:

-- 22:anonymi -- time-stamp -- 2018-04-11 22:18:11 --
(XEN) sh error: _shadow_prealloc(): Can't pre-allocate 1 shadow pages!
(XEN)   shadow pages total = 5, free = 0, p2m=0
(XEN) Xen BUG at common.c:1315
(XEN) ----[ Xen-4.11.20180410T125709.50f8ba84a5-4.xen_unstable  x86_64  debug=n   Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    e008:[<ffff82d08032bdd8>] common.c#_shadow_prealloc+0x478/0x4f0
(XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor (d0v0)
(XEN) rax: ffff83043dd8e02c   rbx: ffff8303393e9000   rcx: 0000000000000000
(XEN) rdx: ffff83043dd87fff   rsi: 000000000000000a   rdi: ffff82d08043c6b8
(XEN) rbp: 0000000000000001   rsp: ffff83043dd87b78   r8:  ffff83043dd90000
(XEN) r9:  0000000000008000   r10: 0000000000000000   r11: 0000000000000001
(XEN) r12: 0000000000000020   r13: 0000000000000000   r14: ffff82d08057ffd8
(XEN) r15: ffff83043dd87fff   cr0: 0000000080050033   cr4: 00000000000026e0
(XEN) cr3: 000000039253f000   cr2: ffff8800a2b6c1b0
(XEN) fsb: 00007f80c8424700   gsb: ffff880140400000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen code around <ffff82d08032bdd8> (common.c#_shadow_prealloc+0x478/0x4f0):
(XEN)  ea 31 c0 e8 d8 08 f2 ff <0f> 0b 31 c9 e9 c3 fe ff ff 31 c9 e9 ad fe ff ff
(XEN) Xen stack trace from rsp=ffff83043dd87b78:
(XEN)    0000000000000000 ffff83043dd87b90 0000000000000000 0000000000000000
(XEN)    000000003dd87fff ffff8300bf1f4000 ffff8303393e9000 0000000000000008
(XEN)    ffff82d0803abf40 ffff83043dd87fff ffffffffffffffff ffff82d08032ea78
(XEN)    0000000000101000 ffff8300bf1f4000 0000000000101000 ffff8303393e9650
(XEN)    ffff82d08057ffc0 ffff82d08032ef18 ffff83043dd16000 ffff8303393e9000
(XEN)    ffff82d08057ffc0 ffff82d08032f059 ffff8303393e9000 0000000000000001
(XEN)    0000000000000024 ffff83043dd16000 00007f80c8439004 ffff82d0803125b5
(XEN)    ffff83043dd16000 ffff8303393e9000 ffff83043dd87d98 ffff82d08026db1e
(XEN)    0000000000000000 0000000000000000 000000000000000c 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff8300bf1fb000
(XEN)    ffff83043dd28188 ffff82d08056a500 0000000000000246 ffff82d08057f180
(XEN)    0000000000000206 ffff82d0802344ea 0000000000000000 ffff83043dd16000
(XEN)    000000000038b83c 00007f80c8439004 ffff83043dd87d98 0000000000000024
(XEN)    0000000000000000 ffff8303393e9000 ffffffffffffffff ffff82d080205f5e
(XEN)    ffff8300bf583000 ffff83043ddb00d0 0000000000000000 ffff82d08020c535
(XEN)    ffff83043ddb00d0 ffff83043ddb00c0 ffff83043ddb0010 07ff82d000000003
(XEN)    ffff82d08035781b ffff82d08020bc8a 0000000000000000 ffff82d080553c80
(XEN)    000000100000000a 0000000000000002 0000000000000002 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff82d08032bdd8>] common.c#_shadow_prealloc+0x478/0x4f0
(XEN)    [<ffff82d08032ea78>] common.c#sh_update_paging_modes+0x1f8/0x390
(XEN)    [<ffff82d08032ef18>] common.c#shadow_one_bit_enable+0x88/0x110
(XEN)    [<ffff82d08032f059>] common.c#sh_enable_log_dirty+0xb9/0x120
(XEN)    [<ffff82d0803125b5>] paging_log_dirty_enable+0x45/0x60
(XEN)    [<ffff82d08026db1e>] arch_do_domctl+0xcee/0x2450
(XEN)    [<ffff82d0802344ea>] vcpu_wake+0x12a/0x390
(XEN)    [<ffff82d080205f5e>] do_domctl+0xcce/0x17e0
(XEN)    [<ffff82d08020c535>] event_fifo.c#evtchn_fifo_set_pending+0x235/0x350
(XEN)    [<ffff82d08035781b>] common_interrupt+0x9b/0x110
(XEN)    [<ffff82d08020bc8a>] evtchn_check_pollers+0x1a/0xa0
(XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
(XEN)    [<ffff82d080205290>] do_domctl+0/0x17e0
(XEN)    [<ffff82d080351318>] pv_hypercall+0x138/0x200
(XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
(XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
(XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
(XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
(XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
(XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
(XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
(XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
(XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
(XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
(XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
(XEN)    [<ffff82d08035748f>] lstar_enter+0x10f/0x120
(XEN) 
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 1:
(XEN) Xen BUG at common.c:1315
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...

I will see if I can reproduce it.

Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: bug in _shadow_prealloc during migration of PV domU

[Jan Beulich]

>>> On 11.04.18 at 22:32, <olaf@aepfle.de> wrote:
> I was testing 'virsh migrate domU host' and did some libvirtd debugging
> on 'host'. This means the migration was attempted a few times, but did
> not actually start because libvirtd was in gdb. Not sure if libvirt on
> the sender does anything with the domU before a connection to the remote
> host is fully established.
> 
> Finally I installed the fixed libvirtd on 'host' and started the
> migration again. This time the sender died like this:
> 
> -- 22:anonymi -- time-stamp -- 2018-04-11 22:18:11 --
> (XEN) sh error: _shadow_prealloc(): Can't pre-allocate 1 shadow pages!
> (XEN)   shadow pages total = 5, free = 0, p2m=0
> (XEN) Xen BUG at common.c:1315
> (XEN) ----[ Xen-4.11.20180410T125709.50f8ba84a5-4.xen_unstable  x86_64  
> debug=n   Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    e008:[<ffff82d08032bdd8>] common.c#_shadow_prealloc+0x478/0x4f0
> (XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor (d0v0)
> (XEN) rax: ffff83043dd8e02c   rbx: ffff8303393e9000   rcx: 0000000000000000
> (XEN) rdx: ffff83043dd87fff   rsi: 000000000000000a   rdi: ffff82d08043c6b8
> (XEN) rbp: 0000000000000001   rsp: ffff83043dd87b78   r8:  ffff83043dd90000
> (XEN) r9:  0000000000008000   r10: 0000000000000000   r11: 0000000000000001
> (XEN) r12: 0000000000000020   r13: 0000000000000000   r14: ffff82d08057ffd8
> (XEN) r15: ffff83043dd87fff   cr0: 0000000080050033   cr4: 00000000000026e0
> (XEN) cr3: 000000039253f000   cr2: ffff8800a2b6c1b0
> (XEN) fsb: 00007f80c8424700   gsb: ffff880140400000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen code around <ffff82d08032bdd8> 
> (common.c#_shadow_prealloc+0x478/0x4f0):
> (XEN)  ea 31 c0 e8 d8 08 f2 ff <0f> 0b 31 c9 e9 c3 fe ff ff 31 c9 e9 ad fe ff 
> ff
> (XEN) Xen stack trace from rsp=ffff83043dd87b78:
> (XEN)    0000000000000000 ffff83043dd87b90 0000000000000000 0000000000000000
> (XEN)    000000003dd87fff ffff8300bf1f4000 ffff8303393e9000 0000000000000008
> (XEN)    ffff82d0803abf40 ffff83043dd87fff ffffffffffffffff ffff82d08032ea78
> (XEN)    0000000000101000 ffff8300bf1f4000 0000000000101000 ffff8303393e9650
> (XEN)    ffff82d08057ffc0 ffff82d08032ef18 ffff83043dd16000 ffff8303393e9000
> (XEN)    ffff82d08057ffc0 ffff82d08032f059 ffff8303393e9000 0000000000000001
> (XEN)    0000000000000024 ffff83043dd16000 00007f80c8439004 ffff82d0803125b5
> (XEN)    ffff83043dd16000 ffff8303393e9000 ffff83043dd87d98 ffff82d08026db1e
> (XEN)    0000000000000000 0000000000000000 000000000000000c 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 ffff8300bf1fb000
> (XEN)    ffff83043dd28188 ffff82d08056a500 0000000000000246 ffff82d08057f180
> (XEN)    0000000000000206 ffff82d0802344ea 0000000000000000 ffff83043dd16000
> (XEN)    000000000038b83c 00007f80c8439004 ffff83043dd87d98 0000000000000024
> (XEN)    0000000000000000 ffff8303393e9000 ffffffffffffffff ffff82d080205f5e
> (XEN)    ffff8300bf583000 ffff83043ddb00d0 0000000000000000 ffff82d08020c535
> (XEN)    ffff83043ddb00d0 ffff83043ddb00c0 ffff83043ddb0010 07ff82d000000003
> (XEN)    ffff82d08035781b ffff82d08020bc8a 0000000000000000 ffff82d080553c80
> (XEN)    000000100000000a 0000000000000002 0000000000000002 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff82d08032bdd8>] common.c#_shadow_prealloc+0x478/0x4f0
> (XEN)    [<ffff82d08032ea78>] common.c#sh_update_paging_modes+0x1f8/0x390
> (XEN)    [<ffff82d08032ef18>] common.c#shadow_one_bit_enable+0x88/0x110
> (XEN)    [<ffff82d08032f059>] common.c#sh_enable_log_dirty+0xb9/0x120
> (XEN)    [<ffff82d0803125b5>] paging_log_dirty_enable+0x45/0x60
> (XEN)    [<ffff82d08026db1e>] arch_do_domctl+0xcee/0x2450
> (XEN)    [<ffff82d0802344ea>] vcpu_wake+0x12a/0x390
> (XEN)    [<ffff82d080205f5e>] do_domctl+0xcce/0x17e0
> (XEN)    [<ffff82d08020c535>] event_fifo.c#evtchn_fifo_set_pending+0x235/0x350
> (XEN)    [<ffff82d08035781b>] common_interrupt+0x9b/0x110
> (XEN)    [<ffff82d08020bc8a>] evtchn_check_pollers+0x1a/0xa0
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080205290>] do_domctl+0/0x17e0
> (XEN)    [<ffff82d080351318>] pv_hypercall+0x138/0x200
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d08035748f>] lstar_enter+0x10f/0x120
> (XEN) 
> (XEN) 
> (XEN) ****************************************
> (XEN) Panic on CPU 1:
> (XEN) Xen BUG at common.c:1315
> (XEN) ****************************************
> (XEN) 
> (XEN) Reboot in five seconds...
> 
> I will see if I can reproduce it.

That would be helpful (ideally with debug=y); iirc Andrew has seen
this once but then wasn't able to repro. Also Cc-ing Tim. Pretty
clearly the question is how we've ended up with just 5 pages in the
pool. But independent of that I wonder whether
shadow_one_bit_enable() wouldn't better call
shadow_set_allocation() when total_pages is below
shadow_min_acceptable_pages() (or alternatively fail in that
case); perhaps the conditional around the call should simply be
removed.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: bug in _shadow_prealloc during migration of PV domU

[Juergen Gross]

On 11/04/18 22:32, Olaf Hering wrote:
> I was testing 'virsh migrate domU host' and did some libvirtd debugging
> on 'host'. This means the migration was attempted a few times, but did
> not actually start because libvirtd was in gdb. Not sure if libvirt on
> the sender does anything with the domU before a connection to the remote
> host is fully established.
> 
> Finally I installed the fixed libvirtd on 'host' and started the
> migration again. This time the sender died like this:
> 
> -- 22:anonymi -- time-stamp -- 2018-04-11 22:18:11 --
> (XEN) sh error: _shadow_prealloc(): Can't pre-allocate 1 shadow pages!
> (XEN)   shadow pages total = 5, free = 0, p2m=0
> (XEN) Xen BUG at common.c:1315
> (XEN) ----[ Xen-4.11.20180410T125709.50f8ba84a5-4.xen_unstable  x86_64  debug=n   Not tainted ]----
> (XEN) CPU:    1
> (XEN) RIP:    e008:[<ffff82d08032bdd8>] common.c#_shadow_prealloc+0x478/0x4f0
> (XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor (d0v0)
> (XEN) rax: ffff83043dd8e02c   rbx: ffff8303393e9000   rcx: 0000000000000000
> (XEN) rdx: ffff83043dd87fff   rsi: 000000000000000a   rdi: ffff82d08043c6b8
> (XEN) rbp: 0000000000000001   rsp: ffff83043dd87b78   r8:  ffff83043dd90000
> (XEN) r9:  0000000000008000   r10: 0000000000000000   r11: 0000000000000001
> (XEN) r12: 0000000000000020   r13: 0000000000000000   r14: ffff82d08057ffd8
> (XEN) r15: ffff83043dd87fff   cr0: 0000000080050033   cr4: 00000000000026e0
> (XEN) cr3: 000000039253f000   cr2: ffff8800a2b6c1b0
> (XEN) fsb: 00007f80c8424700   gsb: ffff880140400000   gss: 0000000000000000
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen code around <ffff82d08032bdd8> (common.c#_shadow_prealloc+0x478/0x4f0):
> (XEN)  ea 31 c0 e8 d8 08 f2 ff <0f> 0b 31 c9 e9 c3 fe ff ff 31 c9 e9 ad fe ff ff
> (XEN) Xen stack trace from rsp=ffff83043dd87b78:
> (XEN)    0000000000000000 ffff83043dd87b90 0000000000000000 0000000000000000
> (XEN)    000000003dd87fff ffff8300bf1f4000 ffff8303393e9000 0000000000000008
> (XEN)    ffff82d0803abf40 ffff83043dd87fff ffffffffffffffff ffff82d08032ea78
> (XEN)    0000000000101000 ffff8300bf1f4000 0000000000101000 ffff8303393e9650
> (XEN)    ffff82d08057ffc0 ffff82d08032ef18 ffff83043dd16000 ffff8303393e9000
> (XEN)    ffff82d08057ffc0 ffff82d08032f059 ffff8303393e9000 0000000000000001
> (XEN)    0000000000000024 ffff83043dd16000 00007f80c8439004 ffff82d0803125b5
> (XEN)    ffff83043dd16000 ffff8303393e9000 ffff83043dd87d98 ffff82d08026db1e
> (XEN)    0000000000000000 0000000000000000 000000000000000c 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 ffff8300bf1fb000
> (XEN)    ffff83043dd28188 ffff82d08056a500 0000000000000246 ffff82d08057f180
> (XEN)    0000000000000206 ffff82d0802344ea 0000000000000000 ffff83043dd16000
> (XEN)    000000000038b83c 00007f80c8439004 ffff83043dd87d98 0000000000000024
> (XEN)    0000000000000000 ffff8303393e9000 ffffffffffffffff ffff82d080205f5e
> (XEN)    ffff8300bf583000 ffff83043ddb00d0 0000000000000000 ffff82d08020c535
> (XEN)    ffff83043ddb00d0 ffff83043ddb00c0 ffff83043ddb0010 07ff82d000000003
> (XEN)    ffff82d08035781b ffff82d08020bc8a 0000000000000000 ffff82d080553c80
> (XEN)    000000100000000a 0000000000000002 0000000000000002 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff82d08032bdd8>] common.c#_shadow_prealloc+0x478/0x4f0
> (XEN)    [<ffff82d08032ea78>] common.c#sh_update_paging_modes+0x1f8/0x390
> (XEN)    [<ffff82d08032ef18>] common.c#shadow_one_bit_enable+0x88/0x110
> (XEN)    [<ffff82d08032f059>] common.c#sh_enable_log_dirty+0xb9/0x120
> (XEN)    [<ffff82d0803125b5>] paging_log_dirty_enable+0x45/0x60
> (XEN)    [<ffff82d08026db1e>] arch_do_domctl+0xcee/0x2450
> (XEN)    [<ffff82d0802344ea>] vcpu_wake+0x12a/0x390
> (XEN)    [<ffff82d080205f5e>] do_domctl+0xcce/0x17e0
> (XEN)    [<ffff82d08020c535>] event_fifo.c#evtchn_fifo_set_pending+0x235/0x350
> (XEN)    [<ffff82d08035781b>] common_interrupt+0x9b/0x110
> (XEN)    [<ffff82d08020bc8a>] evtchn_check_pollers+0x1a/0xa0
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080205290>] do_domctl+0/0x17e0
> (XEN)    [<ffff82d080351318>] pv_hypercall+0x138/0x200
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d080357422>] lstar_enter+0xa2/0x120
> (XEN)    [<ffff82d08035742e>] lstar_enter+0xae/0x120
> (XEN)    [<ffff82d08035748f>] lstar_enter+0x10f/0x120
> (XEN) 
> (XEN) 
> (XEN) ****************************************
> (XEN) Panic on CPU 1:
> (XEN) Xen BUG at common.c:1315
> (XEN) ****************************************
> (XEN) 
> (XEN) Reboot in five seconds...
> 
> I will see if I can reproduce it.

Unfortunately I can reproduce that easily.

Unfortunately because this happens with my XPTI series after the first
failed migration when I try to migrate again.

I guess this is related to some missing cleanup when suspending the
guest failed (e.g. due to a timeout).


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: bug in _shadow_prealloc during migration of PV domU

[Juergen Gross]

On 12/04/18 10:01, Juergen Gross wrote:
> On 11/04/18 22:32, Olaf Hering wrote:
>> I was testing 'virsh migrate domU host' and did some libvirtd debugging
>> on 'host'. This means the migration was attempted a few times, but did
>> not actually start because libvirtd was in gdb. Not sure if libvirt on
>> the sender does anything with the domU before a connection to the remote
>> host is fully established.
>>
>> Finally I installed the fixed libvirtd on 'host' and started the
>> migration again. This time the sender died like this:
> Unfortunately I can reproduce that easily.
> 
> Unfortunately because this happens with my XPTI series after the first
> failed migration when I try to migrate again.
> 
> I guess this is related to some missing cleanup when suspending the
> guest failed (e.g. due to a timeout).

Here is more data:

The first migrate attempt failed with:



# xl migrate 1 localhost
Password:
migration target: Ready to receive domain.
Saving to migration stream new xl format (info 0x3/0x0/1218)
Loading new save file <incoming migration stream> (new xl fmt info
0x3/0x0/1218)
 Savefile contains xl domain config in JSON format
Parsing config from <saved>
xc: info: Saving domain 1, type x86 PV
xc: info: Found x86 PV domain from Xen 4.11
xc: info: Restoring domain
libxl: error:
libxl_dom_suspend.c:262:domain_suspend_common_pvcontrol_suspending:
Domain 1:guest didn't acknowledge suspend, cancelling request
xc: error: Domain has not been suspended: shutdown 0, reason 255:
Internal error
xc: error: Save failed (0 = Success): Internal error
libxl: error: libxl_stream_write.c:350:libxl__xc_domain_save_done:
Domain 1:saving domain: domain did not respond to suspend request: Success
migration sender: libxl_domain_suspend failed (rc=-8)
xc: error: Failed to read Record Header from stream (0 = Success):
Internal error
xc: error: Restore failed (0 = Success): Internal error
libxl: error: libxl_stream_read.c:850:libxl__xc_domain_restore_done:
restoring domain: Success
libxl: error: libxl_create.c:1265:domcreate_rebuild_done: Domain
2:cannot (re-)build domain: -3
libxl: error: libxl_domain.c:1034:libxl__destroy_domid: Domain
2:Non-existant domain
libxl: error: libxl_domain.c:993:domain_destroy_callback: Domain
2:Unable to destroy guest
libxl: error: libxl_domain.c:920:domain_destroy_cb: Domain 2:Destruction
of domain failed
migration target: Domain creation failed (code -3).
libxl: info: libxl_exec.c:118:libxl_report_child_exitstatus: migration
transport process [2393] exited with error status 1
Migration failed, failed to suspend at sender.


The second attempt immediately produced:

(XEN) sh error: _shadow_prealloc(): Can't pre-allocate 1 shadow pages!
(XEN)   shadow pages total = 6, free = 0, p2m=0
(XEN) Xen BUG at common.c:1315
(XEN) ----[ Xen-4.11-unstable  x86_64  debug=y   Tainted:  C   ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d08033f8ff>]
common.c#_shadow_prealloc+0x5b1/0x638
(XEN) RFLAGS: 0000000000010292   CONTEXT: hypervisor (d0v2)
(XEN) rax: 0000000000000200   rbx: ffff83021e125000   rcx: 0000000000000000
(XEN) rdx: ffff8300dba87fff   rsi: 000000000000000a   rdi: ffff82d0804766b8
(XEN) rbp: ffff8300dba87af8   rsp: ffff8300dba87aa8   r8:  ffff830217f78000
(XEN) r9:  0000000000000001   r10: 0000000000000000   r11: 0000000000000001
(XEN) r12: 0000000000000020   r13: 0000000000000000   r14: ffff82d0805bfff8
(XEN) r15: ffff8300dba87fff   cr0: 0000000080050033   cr4: 00000000001526e0
(XEN) cr3: 00000000d0747000   cr2: 00007f231d31f272
(XEN) fsb: 00007f231d93f700   gsb: ffff88020f700000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen code around <ffff82d08033f8ff>
(common.c#_shadow_prealloc+0x5b1/0x638):
(XEN)  00 00 00 e8 f4 29 f1 ff <0f> 0b f6 40 10 40 0f 85 2d fb ff ff e9
21 fb ff
(XEN) Xen stack trace from rsp=ffff8300dba87aa8:
(XEN)    ffff8300dba87ac0 000000011e125000 0000000000000000 0000000000000000
(XEN)    0000000000000000 ffff8300db7fe000 ffff83021e125000 0000000000000002
(XEN)    ffff82d0803e77c0 0000000000000195 ffff8300dba87b08 ffff82d08033f9a6
(XEN)    ffff8300dba87b48 ffff82d08034325b 00007f231d960004 ffff8300db7fe000
(XEN)    0000000000001000 0000000000000000 00007f231d960004 ffff830217f3b000
(XEN)    ffff8300dba87b68 ffff82d0803435a2 ffff8300dba87b78 ffff83021e125000
(XEN)    ffff8300dba87b88 ffff82d08034391a ffff83021e125000 ffff83021e125650
(XEN)    ffff8300dba87bb8 ffff82d080343a33 0000000000000001 ffff83021e125000
(XEN)    0000000000000001 0000000000000000 ffff8300dba87bd8 ffff82d080321f47
(XEN)    ffff83021e125000 ffff8300dba87d98 ffff8300dba87c68 ffff82d080322943
(XEN)    ffff8300dba87c98 ffff82d08022df69 80100000c06e5067 ffff8300dba87fff
(XEN)    ffff8300d8dfe000 0000000000000001 ffff830217f3b000 0000000400000025
(XEN)    0000000000000008 0000000000000000 0000000000000000 ffff8300dba87d88
(XEN)    ffff83021e125000 00007f231d960004 00007f231d960004 ffff830217f3b000
(XEN)    ffff8300dba87d28 ffff82d080272a4c ffff8300dba87d08 ffff82d080290269
(XEN)    0000000000000206 00000000000c0733 0000000000000000 ffff8300dba87cd0
(XEN)    0000000000000000 ffff830217f3b000 ffff8200400088f8 0000000000000000
(XEN)    ffff8300dba87d08 0000000000000000 0000000000000009 0000000000000206
(XEN)    ffff820040008000 0000000000000000 0000000000000292 ffff83021e125000
(XEN)    0000000000000000 00007f231d960004 0000000000000000 deadbeefdeadf00d
(XEN) Xen call trace:
(XEN)    [<ffff82d08033f8ff>] common.c#_shadow_prealloc+0x5b1/0x638
(XEN)    [<ffff82d08033f9a6>] shadow_prealloc+0x20/0x22
(XEN)    [<ffff82d08034325b>] common.c#sh_update_paging_modes+0xf5/0x3de
(XEN)    [<ffff82d0803435a2>] common.c#sh_new_mode+0x5e/0x6e
(XEN)    [<ffff82d08034391a>] common.c#shadow_one_bit_enable+0xd3/0xf2
(XEN)    [<ffff82d080343a33>] common.c#sh_enable_log_dirty+0xfa/0x14d
(XEN)    [<ffff82d080321f47>] paging_log_dirty_enable+0x47/0x61
(XEN)    [<ffff82d080322943>] paging_domctl+0x1cc/0xaca
(XEN)    [<ffff82d080272a4c>] arch_do_domctl+0x219/0x2648
(XEN)    [<ffff82d080206cb4>] do_domctl+0x1872/0x1bce
(XEN)    [<ffff82d08036c2aa>] pv_hypercall+0x1f4/0x43e
(XEN)    [<ffff82d0803734a5>] lstar_enter+0x115/0x120
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Xen BUG at common.c:1315
(XEN) ****************************************


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel