* Deadlocks by p2m_lock and event_lock
@ 2012-03-09 10:58 Hao, Xudong
2012-03-09 11:20 ` Tim Deegan
0 siblings, 1 reply; 18+ messages in thread
From: Hao, Xudong @ 2012-03-09 10:58 UTC (permalink / raw)
To: JBeulich, Andres Lagar-Cavilla; +Cc: xen-devel, Keir Fraser, Zhang, Xiantao
[-- Attachment #1: Type: text/plain, Size: 1479 bytes --]
Hi, Jan and Andres
Here we met a deadlocks issue by p2m_lock and event_lock on Xen. The issue appears from the series of patches "Synchronized p2m lookups, Xen 24770~", deadlocks may happen on a specifical case: Assign a PCIe device with MSI-x capability to HVM guest(in case of EPT).
I've dump the all processor registers when dom0 hang( attach the log). The deadlocks happen as follows:
====CPU0===
map_domain_pirq() Grab event_lock
/
Pci_enable_msi()
/
msix_capability_init()
/
p2m_change_entry_type_global() Trying to acquire p2m_lock
====CPU9===
hvm_hap_nested_page_fault() -> get_gfn_type_access() Grab p2m_lock
/
handle_mmio()
/
...
/
notify_via_xen_event_channel() Trying to acquire event_lock
The event_lock is used anywhere in Xen, I only have a patch of workaround this issue for proposal, but not for the final fix. Any good suggestion?
diff -r f61120046915 xen/arch/x86/irq.c
--- a/xen/arch/x86/irq.c Wed Mar 07 11:50:31 2012 +0100
+++ b/xen/arch/x86/irq.c Sat Mar 10 02:06:18 2012 +0800
@@ -1875,10 +1875,12 @@ int map_domain_pirq(
if ( !cpu_has_apic )
goto done;
+ spin_unlock(&d->event_lock);
pdev = pci_get_pdev(msi->seg, msi->bus, msi->devfn);
ret = pci_enable_msi(msi, &msi_desc);
if ( ret )
goto done;
+ spin_lock(&d->event_lock);
spin_lock_irqsave(&desc->lock, flags);
Best Regards,
Xudong Hao
[-- Attachment #2: registers_dump.log --]
[-- Type: application/octet-stream, Size: 55096 bytes --]
(XEN) 'd' pressed -> dumping registers
(XEN)
(XEN) *** Dumping CPU0 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff82c480125ef5>] _spin_lock+0x1d/0xa8
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000000 rbx: ffff83028eea11a8 rcx: ffff83028eea11a8
(XEN) rdx: 0000000000000000 rsi: 0000000000000002 rdi: ffff83028eea11ac
(XEN) rbp: ffff82c4802af228 rsp: ffff82c4802af220 r8: 00000000ffffffff
(XEN) r9: 0000000000020000 r10: 00000000000a1936 r11: 0000000000000000
(XEN) r12: ffff83028eea1000 r13: ffff83028eea11a8 r14: 0000000000000000
(XEN) r15: ffff83028eea12e0 cr0: 0000000080050033 cr4: 00000000000026f0
(XEN) cr3: 000000028ee96000 cr2: 0000000000e06000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff82c4802af220:
(XEN) 0000000000000002 ffff82c4802af258 ffff82c4801070d3 0000000000000282
(XEN) ffff83011e725020 ffff830053c3f000 ffff83011e726000 ffff82c4802af2a8
(XEN) ffff82c4801b65a3 00000000a1935001 0000000000000000 ffff82c4802af2a8
(XEN) ffff83011e725020 0000000000000001 ffff83028eea19c0 00000000000a1936
(XEN) 0000000000000001 ffff82c4802af378 ffff82c4801bb47a ffff82c4802af308
(XEN) ffff82c4801d81c5 00000007802af2d8 ffff82c4802af380 ffff82c4802af388
(XEN) ffff83028e874ef0 ffff83028e874ef0 0000000000000001 0000000000000002
(XEN) 0000000000000000 ffff82c4802af388 ffff82c4801df86c 0000000000000000
(XEN) ffff83028eea1000 ffff82c4802af448 ffff82c4802af44c 00000009000a1935
(XEN) ffff82c4802af368 ffff82c4801bf398 0000000000000002 ffff83011e725020
(XEN) 0000000000000000 ffff8302bf9163f0 ffff83011e725020 ffff82c4802af3d8
(XEN) ffff82c4801b5a78 ffff82c4802af3d8 ffff82c4801d81c5 ffff8800000a1935
(XEN) ffff82c4802af448 ffff82c48025f360 ffff830053c3f000 0000000000000001
(XEN) 0000000000000000 0000000000000001 ffff83011e725020 ffff82c4802af488
(XEN) ffff82c4801aa542 ffff82c400000001 ffff8800000a1935 0100000000000001
(XEN) 0000000000000000 0100000000000000 ffff82c4802af4e8 00000000000a1935
(XEN) 0000000000000001 ffff83011e725000 0100000000000935 ffff82c4802af4e0
(XEN) ffff82c4802af910 0000000000000007 0000000000000000 00000000000000a1
(XEN) 0000000000000000 0000000000000001 ffff82c4802afb68 0000000000000003
(XEN) ffff830053c3f000 ffff82c4802af4a8 ffff82c4801aa72b 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c480125ef5>] _spin_lock+0x1d/0xa8
(XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
(XEN) [<ffff82c4801b65a3>] hvm_buffered_io_send+0x1f1/0x21b
(XEN) [<ffff82c4801bb47a>] stdvga_intercept_mmio+0x491/0x4c7
(XEN) [<ffff82c4801b5a78>] hvm_io_intercept+0x218/0x244
(XEN) [<ffff82c4801aa542>] hvmemul_do_io+0x55a/0x716
(XEN) [<ffff82c4801aa72b>] hvmemul_do_mmio+0x2d/0x2f
(XEN) [<ffff82c4801aaf53>] hvmemul_write+0x181/0x1a2
(XEN) [<ffff82c4801962a0>] x86_emulate+0xcad3/0xfbdf
(XEN) [<ffff82c4801a997e>] hvm_emulate_one+0x120/0x1af
(XEN) [<ffff82c4801b60eb>] handle_mmio+0x4e/0x1d1
(XEN) [<ffff82c4801af519>] hvm_hap_nested_page_fault+0x1e7/0x331
(XEN) [<ffff82c4801d1b29>] vmx_vmexit_handler+0x152f/0x17dc
(XEN)
(XEN) *** Dumping CPU0 guest state (d4:v1): ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 0
(XEN) RIP: 0010:[<ffffffff812a8c07>]
(XEN) RFLAGS: 0000000000010286 CONTEXT: hvm guest
(XEN) rax: ffff8800000a1935 rbx: ffff88001ce50000 rcx: 0000000000000000
(XEN) rdx: ffff88001ce51936 rsi: ffff8800000a2000 rdi: ffffffff81f05a68
(XEN) rbp: ffff88001cd63c38 rsp: ffff88001cd63bf8 r8: 0000000000000246
(XEN) r9: 0000000080042000 r10: 0000000000000000 r11: 0000000000000000
(XEN) r12: 0000000000000001 r13: 0000000000000004 r14: 0000000000000001
(XEN) r15: 000000000000000e cr0: 0000000080050033 cr4: 00000000000006e0
(XEN) cr3: 000000001a3a3000 cr2: 0000000000e06000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0018 cs: 0010
(XEN)
(XEN) *** Dumping CPU1 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 1
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013ffe4750 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ef3ac00 rdi: ffff83013ef37ea0
(XEN) rbp: ffff83013ef37ee0 rsp: ffff83013ef37e78 r8: 00000181855eee00
(XEN) r9: ffff8300bf300060 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ffe4810 r13: 000001824869555a r14: 00000181854f6e4c
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000002b1c05000 cr2: ffff880245009198
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ef37e78:
(XEN) ffff82c48019daa1 ffff82c480121d0e ffff83013ef37ef8 ffff82c4802d0880
(XEN) 00000000ffffffff 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 0000042b0001c893 ffff83013ef37f18 ffff82c48025bc80 ffff83013ef37f18
(XEN) 00000000ffffffff ffff83013ef37f10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300bf30a000 ffff8300bf300000 0000000000000001 ffff83013ef37d88
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000010
(XEN) ffff88025c8c5f00 ffff88025c8c4010 0000000000000246 0000015d662fe080
(XEN) ffff88026e752cc0 0000000000000000 0000000000000000 ffffffff810013aa
(XEN) 0000000000000000 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN) ffffffff810013aa 000000000000e033 0000000000000246 ffff88025c8c5ee8
(XEN) 000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN) 000000000000beef 0000000000000001 ffff8300bf30a000 0000003cbec3e680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU2 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 2
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013ffe4c50 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ffdbc00 rdi: ffff83013ffd7ea0
(XEN) rbp: ffff83013ffd7ee0 rsp: ffff83013ffd7e78 r8: 000000000000265a
(XEN) r9: ffff8300beef5060 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ffe4d10 r13: 0000018252a989dd r14: 000001818bde31bd
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000002b1c05000 cr2: ffff88025c2ef9f0
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ffd7e78:
(XEN) ffff82c48019daa1 ffff83013ffd7f18 ffffffffffffffff ffff83013ffd7ed0
(XEN) 0000000080125395 0000000000000000 0000000000000000 ffff82c48025bc80
(XEN) 0000041800095d6c ffff83013ffd7f18 ffff82c48025bc80 ffff83013ffd7f18
(XEN) 00000000ffffffff ffff83013ffd7f10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300bf0fd000 ffff8300beef5000 0000000000000002 ffff83013ffd7d88
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000013
(XEN) ffff88025c8dbf00 ffff88025c8da010 0000000000000246 0000015d65a68c40
(XEN) ffff88026e7a3cc0 0000000000000000 0000000000000000 ffffffff810013aa
(XEN) 0000000000000000 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN) ffffffff810013aa 000000000000e033 0000000000000246 ffff88025c8dbee8
(XEN) 000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN) 000000000000beef 0000000000000002 ffff8300bf0fd000 0000003cbfcdf680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU3 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 3
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013ffcc170 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ffcdc00 rdi: ffff83013ffc7ea0
(XEN) rbp: ffff83013ffc7ee0 rsp: ffff83013ffc7e78 r8: 0000000000000001
(XEN) r9: ffff8300bf303060 r10: ffff8300bf306060 r11: 000001828bde544c
(XEN) r12: ffff83013ffcc230 r13: 000001825ce81753 r14: 0000018252a9a7f5
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000002b1c05000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ffc7e78:
(XEN) ffff82c48019daa1 ffff83013ffc7f18 ffffffffffffffff ffff83013ffc7ed0
(XEN) 0000000080125395 0000000000000000 0000000000000000 ffff82c48025bc80
(XEN) 000016c20070d5b7 ffff83013ffc7f18 ffff82c48025bc80 ffff83013ffc7f18
(XEN) 00000000ffffffff ffff83013ffc7f10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300beefe000 ffff8300bf306000 0000000000000003 ffff83013ffc7d88
(XEN) 0000000000000000 0000000000000000 0000000000000000 000000000000000a
(XEN) ffff88025c8b9f00 ffff88025c8b8010 0000000000000246 0000015d67428900
(XEN) ffff88026e6b0cc0 0000000000000000 0000000000000000 ffffffff810013aa
(XEN) 0000000000000000 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN) ffffffff810013aa 000000000000e033 0000000000000246 ffff88025c8b9ee8
(XEN) 000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN) 000000000000beef 0000000000000003 ffff8300beefe000 0000003cbfcd1680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU4 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 4
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013ffcc670 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ef7fc00 rdi: ffff83013ef77ea0
(XEN) rbp: ffff83013ef77ee0 rsp: ffff83013ef77e78 r8: 00000181855eee00
(XEN) r9: ffff8300beef6060 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ffcc730 r13: 0000018267279b5b r14: 0000018184f86f9d
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000002b1c05000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ef77e78:
(XEN) ffff82c48019daa1 ffff82c480121d0e ffff83013ef77ef8 ffff82c4802d0a00
(XEN) 00000000ffffffff 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 00001a7000092a4c ffff83013ef77f18 ffff82c48025bc80 ffff83013ef77f18
(XEN) 00000000ffffffff ffff83013ef77f10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300bf309000 ffff8300beef6000 0000000000000004 ffff83013ef77d88
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000012
(XEN) ffff88025c8d9f00 ffff88025c8d8010 0000000000000246 0000015d65d45300
(XEN) ffff88026e788cc0 0000000000000000 0000000000000000 ffffffff810013aa
(XEN) 0000000000000000 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN) ffffffff810013aa 000000000000e033 0000000000000246 ffff88025c8d9ee8
(XEN) 000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN) 000000000000beef 0000000000000004 ffff8300bf309000 0000003cbec83680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU5 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 5
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ffccb70 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ef7ac00 rdi: ffff83013ffccbd0
(XEN) rbp: ffff83013ef6fe70 rsp: ffff83013ef6fe50 r8: 0000000000000001
(XEN) r9: ffff8300bf7f2060 r10: ffff83013ffcc9c8 r11: 0000018271711002
(XEN) r12: ffff83013ffccbd0 r13: 0000000000000001 r14: 0000018271633517
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000054796000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ef6fe50:
(XEN) 00000000ffffffff ffff83013ffccb70 ffff83013ffccbd0 0000000000000001
(XEN) ffff83013ef6fee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ef6fef8
(XEN) ffff82c4802d0a80 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000001620000016c ffff83013ef6ff18 ffff82c48025bc80
(XEN) ffff83013ef6ff18 00000000ffffffff ffff83013ef6ff10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300bf308000 ffff8300bf301000 0000000000000005
(XEN) ffff83013ef6fd88 0000000000000000 0000000000000000 0000000000000000
(XEN) 000000000000000f ffff88025c8c3f00 ffff88025c8c2010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8c3ee8 000000000000e02b 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000beef 0000000000000005 ffff8300bf308000
(XEN) 0000003cbec7e680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU6 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 6
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ef64090 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ef65c00 rdi: ffff83013ef640f0
(XEN) rbp: ffff83013ef5fe70 rsp: ffff83013ef5fe50 r8: 0000000000000000
(XEN) r9: 0000000000000002 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ef640f0 r13: 0000000000000001 r14: 000001827c885ba5
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000055e70000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ef5fe50:
(XEN) 00000000ffffffff ffff83013ef64090 ffff83013ef640f0 0000000000000001
(XEN) ffff83013ef5fee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ef5fef8
(XEN) ffff82c4802d0b00 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003d30000041b ffff83013ef5ff18 ffff82c48025bc80
(XEN) ffff83013ef5ff18 00000000ffffffff ffff83013ef5ff10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300bf0ff000 ffff8300bf7f5000 0000000000000006
(XEN) ffff83013ef5fd88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000003 ffff88025c8a9f00 ffff88025c8a8010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8a9ee8 000000000000e02b 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000006 ffff8300bf0ff000
(XEN) 0000003cbec69680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU7 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 7
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ef64590 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ef57c00 rdi: ffff83013ef645f0
(XEN) rbp: ffff83013ef4fe70 rsp: ffff83013ef4fe50 r8: 0000000000000000
(XEN) r9: 0000000000000003 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ef645f0 r13: 0000000000000001 r14: 0000018287adcf36
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000002b1c05000 cr2: ffff88025b0372f8
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ef4fe50:
(XEN) 00000000ffffffff ffff83013ef64590 ffff83013ef645f0 0000000000000001
(XEN) ffff83013ef4fee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ef4fef8
(XEN) ffff82c4802d0b80 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003ce00000480 ffff83013ef4ff18 ffff82c48025bc80
(XEN) ffff83013ef4ff18 00000000ffffffff ffff83013ef4ff10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300bf0fe000 ffff8300beef4000 0000000000000007
(XEN) ffff83013ef4fd88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000014 ffff88025c8ddf00 ffff88025c8dc010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8ddee8 000000000000e02b 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000007 ffff8300bf0fe000
(XEN) 0000003cbec5b680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU8 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 8
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ef64a90 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ef52c00 rdi: ffff83013ef64af0
(XEN) rbp: ffff83013ef47e70 rsp: ffff83013ef47e50 r8: 0000018292c8a800
(XEN) r9: ffff8300bf7f7060 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ef64af0 r13: 0000000000000001 r14: 0000018292c3cd98
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000000530e6000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ef47e50:
(XEN) 00000000ffffffff ffff83013ef64a90 ffff83013ef64af0 0000000000000001
(XEN) ffff83013ef47ee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ef47ef8
(XEN) ffff82c4802d0c00 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000001730000017e ffff83013ef47f18 ffff82c48025bc80
(XEN) ffff83013ef47f18 00000000ffffffff ffff83013ef47f10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300beefd000 ffff8300bf7f7000 0000000000000008
(XEN) ffff83013ef47d88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000001 ffff88025c88df00 ffff88025c88c010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c88dee8 000000000000e02b 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000008 ffff8300beefd000
(XEN) 0000003cbec56680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU9 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 9
(XEN) RIP: e008:[<ffff82c480125ca3>] _spin_lock_recursive+0xf/0x1b
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000000 rbx: ffff83028e874ef0 rcx: 00000000f175c430
(XEN) rdx: 0000000000000000 rsi: 0000000000000005 rdi: ffff83028e874ef0
(XEN) rbp: ffff83013ff77c88 rsp: ffff83013ff77c80 r8: ffff83013fff8004
(XEN) r9: 000000000000001f r10: 0000000000000001 r11: 0000000000000282
(XEN) r12: 0000000000000005 r13: 0000000000000005 r14: ffff83028e874ef0
(XEN) r15: 0000000000000003 cr0: 0000000080050033 cr4: 00000000000026f0
(XEN) cr3: 000000006c6bc000 cr2: ffff8802421c0088
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff77c80:
(XEN) ffff83028e874ef0 ffff83013ff77cb8 ffff82c4801d9e8d 0000000000002000
(XEN) 0000000000000188 ffff83011e862b20 ffff83013ff77db8 ffff83013ff77d88
(XEN) ffff82c48016418c ffff83013ff77cd8 ffff82c480136166 c00283013ff77ce8
(XEN) ffff83011e862bc0 ffff830100000003 ffff82c480135dc9 ffff83013ff77d38
(XEN) 00000000000fba10 0000007200004d70 00000000fba10000 0000000100000000
(XEN) 0000000000000000 0000000000000000 00000000000fba10 0000000000000070
(XEN) 0000000100000000 ffff83010e8a3b50 000000113ef25ca0 000000000000007f
(XEN) ffff83028eea1000 ffff83013ee07280 0000000000000072 00000000ffffffed
(XEN) ffff83011e862b20 ffff83013ff77df8 ffff82c480166482 ffff83013ff77dc8
(XEN) ffff83013ff77eb8 ffff830200000000 0000000000000072 ffff83013ff77e90
(XEN) ffff83010e8a3180 ffff83013ff77df8 ffff83013ff77eb8 0000000000000072
(XEN) 0000000000000000 ffff83013ff77e90 ffff83013ff77e94 ffff83013ff77e68
(XEN) ffff82c480179ca3 000000000000beef 000000000000beef 000000000000beef
(XEN) ffff83010000007f ffff830100000001 ffff83028eea1000 ffff83013ff77e68
(XEN) 000000000000000d 000000000166f004 ffff8300bf658000 0000000000000007
(XEN) ffff88025183df80 ffff83013ff77ef8 ffff82c48017a416 00007fffd4a82bf0
(XEN) 0000000000000001 0000000000000004 ffffffffffffffff 0000008800000001
(XEN) 0000000000000000 00000000fba10000 0000000000000000 0000007288010000
(XEN) 0000000000000000 00000000fba10000 ffff82c48016a024 ffff8300bf658000
(XEN) 0000000000000007 00007fffd4a82bc0 0000000000000007 00007cfec00880c7
(XEN) Xen call trace:
(XEN) [<ffff82c480125ca3>] _spin_lock_recursive+0xf/0x1b
(XEN) [<ffff82c4801d9e8d>] p2m_change_entry_type_global+0xb9/0x17e
(XEN) [<ffff82c48016418c>] pci_enable_msi+0x636/0xa28
(XEN) [<ffff82c480166482>] map_domain_pirq+0x2d7/0x41d
(XEN) [<ffff82c480179ca3>] physdev_map_pirq+0x483/0x546
(XEN) [<ffff82c48017a416>] do_physdev_op+0x6b0/0x1152
(XEN) [<ffff82c48021efa8>] syscall_enter+0xc8/0x122
(XEN)
(XEN) *** Dumping CPU9 guest state (d0:v0): ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 9
(XEN) RIP: e033:[<ffffffff8100142a>]
(XEN) RFLAGS: 0000000000000282 EM: 0 CONTEXT: pv guest
(XEN) rax: 0000000000000021 rbx: 00007fffd4a82bc0 rcx: ffffffff8100142a
(XEN) rdx: 0000000000000000 rsi: 000000000166f004 rdi: 000000000000000d
(XEN) rbp: ffff88025b033e98 rsp: ffff88025b033e40 r8: 0000000000000000
(XEN) r9: 0000000000000009 r10: 0000000000000000 r11: 0000000000000282
(XEN) r12: 0000000000000007 r13: 00007fffd4a82bc0 r14: 0000000000000007
(XEN) r15: ffff88025183df80 cr0: 0000000080050033 cr4: 00000000000026f0
(XEN) cr3: 000000006c6bc000 cr2: 000000323d411fa6
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033
(XEN) Guest stack trace from rsp=ffff88025b033e40:
(XEN) 0000000000000246 0000000000000000 ffffffff813137db 0000000000000021
(XEN) 000000000000000d 000000000166f004 0000000000000000 0000000000000000
(XEN) 0000000000000000 ffffffff81052662 ffff880245d46a80 ffff88025b033f28
(XEN) ffffffff8118a5b8 aaaaaaaaaaaaaaaa ffff88025b0300d8 aaaaaaaaaaaaaaaa
(XEN) aaaaaaaaaaaaaaaa 0000000000000001 0000000000000000 ffffffff8105264f
(XEN) ffffffff8104c129 ffff88026e5a0270 ffff88026e5a0a70 ffff88026e5a0270
(XEN) ffff880245d46a80 0000000000000007 0000000000305000 00007fffd4a82bc0
(XEN) 0000000000000000 ffff88025b033f78 ffffffff8118ab19 ffff88025b033f48
(XEN) 00000001816d5b99 00007fffd4a82b70 00000000ffffffea 00007fffd4a82cbc
(XEN) 000000000164e160 0000000000000051 00000000016fa4c0 00007fffd4a82ba0
(XEN) ffffffff816dd092 0000000000000246 0000000000000000 0000000000100000
(XEN) 0000000000000000 0000000000000010 0000000000000000 00007fffd4a82bc0
(XEN) 0000000000305000 0000000000000007 0000000000000010 00000033cd2dd847
(XEN) 000000000000e033 0000000000000246 00007fffd4a82b98 000000000000e02b
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)
(XEN) *** Dumping CPU10 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 10
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ff784b0 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ff6fc00 rdi: ffff83013ff78510
(XEN) rbp: ffff83013ff67e70 rsp: ffff83013ff67e50 r8: 0000000000000000
(XEN) r9: 0000000000000003 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ff78510 r13: 0000000000000001 r14: 00000182b9916689
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000055edf000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff67e50:
(XEN) 00000000ffffffff ffff83013ff784b0 ffff83013ff78510 0000000000000001
(XEN) ffff83013ff67ee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ff67ef8
(XEN) ffff82c4802d0d00 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003cd0000057b ffff83013ff67f18 ffff82c48025bc80
(XEN) ffff83013ff67f18 00000000ffffffff ffff83013ff67f10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300bf7fb000 ffff8300bf7f1000 000000000000000a
(XEN) ffff83013ff67d88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000007 ffff88025c8b3f00 ffff88025c8b2010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8b3ee8 000000000000e02b 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000beef 000000000000000a ffff8300bf7fb000
(XEN) 0000003cbfc73680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU11 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 11
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ff789b0 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ff6ac00 rdi: ffff83013ff78a10
(XEN) rbp: ffff83013ff5fe70 rsp: ffff83013ff5fe50 r8: 0000000000000000
(XEN) r9: 0000000000000003 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ff78a10 r13: 0000000000000001 r14: 00000182c4b6beba
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000055edf000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff5fe50:
(XEN) 00000000ffffffff ffff83013ff789b0 ffff83013ff78a10 0000000000000001
(XEN) ffff83013ff5fee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ff5fef8
(XEN) ffff82c4802d0d80 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003d200000406 ffff83013ff5ff18 ffff82c48025bc80
(XEN) ffff83013ff5ff18 00000000ffffffff ffff83013ff5ff10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300bf7fa000 ffff8300bf7f0000 000000000000000b
(XEN) ffff83013ff5fd88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000008 ffff88025c8b5f00 ffff88025c8b4010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8b5ee8 000000000000e02b 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000beef 000000000000000b ffff8300bf7fa000
(XEN) 0000003cbfc6e680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU12 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 12
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013ff53010 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ff55c00 rdi: ffff83013ff4fea0
(XEN) rbp: ffff83013ff4fee0 rsp: ffff83013ff4fe78 r8: 00000182b2f86700
(XEN) r9: ffff8300beef7060 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ff530d0 r13: 00000182cfdd8f3d r14: 00000182b28e0598
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000000533b2000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff4fe78:
(XEN) ffff82c48019daa1 ffff82c480121d0e ffff83013ff4fef8 ffff82c4802d0e00
(XEN) 00000000ffffffff 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 00001b71002017b7 ffff83013ff4ff18 ffff82c48025bc80 ffff83013ff4ff18
(XEN) 00000000ffffffff ffff83013ff4ff10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300bf7f9000 ffff8300beef7000 000000000000000c ffff83013ff4fd88
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000011
(XEN) ffff88025c8c7f00 ffff88025c8c6010 0000000000000246 0000017f9416da40
(XEN) ffff88026e76dcc0 0000000000000000 0000000000000000 ffffffff810013aa
(XEN) 0000000000000000 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN) ffffffff810013aa 000000000000e033 0000000000000246 ffff88025c8c7ee8
(XEN) 000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000000c ffff8300bf7f9000 0000003cbfc59680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU13 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 13
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ff533d0 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ff52c00 rdi: ffff83013ff53430
(XEN) rbp: ffff83013ff47e70 rsp: ffff83013ff47e50 r8: 0000000000000000
(XEN) r9: 0000000000000004 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ff53430 r13: 0000000000000001 r14: 00000182da1cda25
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000057221000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff47e50:
(XEN) 00000000ffffffff ffff83013ff533d0 ffff83013ff53430 0000000000000001
(XEN) ffff83013ff47ee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ff47ef8
(XEN) ffff82c4802d0e80 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003c600000411 ffff83013ff47f18 ffff82c48025bc80
(XEN) ffff83013ff47f18 00000000ffffffff ffff83013ff47f10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300bf7f8000 ffff8300bf7f6000 000000000000000d
(XEN) ffff83013ff47d88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000002 ffff88025c88ff00 ffff88025c88e010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c88fee8 000000000000e02b 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000beef 000000000000000d ffff8300bf7f8000
(XEN) 0000003cbfc56680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU14 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 14
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ff538d0 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ff50c00 rdi: ffff83013ff53930
(XEN) rbp: ffff83013ff3fe70 rsp: ffff83013ff3fe50 r8: 0000000000000000
(XEN) r9: 0000000000000005 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ff53930 r13: 0000000000000001 r14: 00000182e54219bb
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000057360000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff3fe50:
(XEN) 00000000ffffffff ffff83013ff538d0 ffff83013ff53930 0000000000000001
(XEN) ffff83013ff3fee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ff3fef8
(XEN) ffff82c4802d0f00 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003cd00000482 ffff83013ff3ff18 ffff82c48025bc80
(XEN) ffff83013ff3ff18 00000000ffffffff ffff83013ff3ff10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300beefb000 ffff8300beef2000 000000000000000e
(XEN) ffff83013ff3fd88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000016 ffff88025c8e1f00 ffff88025c8e0010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8e1ee8 000000000000e02b 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000beef 000000000000000e ffff8300beefb000
(XEN) 0000003cbfc54680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU15 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 15
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013ff53dd0 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ff36c00 rdi: ffff83013ff2fea0
(XEN) rbp: ffff83013ff2fee0 rsp: ffff83013ff2fe78 r8: 0000000000000001
(XEN) r9: ffff8300bf307060 r10: ffff83013ff53c28 r11: 00000182c0155600
(XEN) r12: ffff83013ff53e90 r13: 00000182f06a6301 r14: 00000182bf3a9e8e
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000002b1c05000 cr2: 00007fe5b267751e
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff2fe78:
(XEN) ffff82c48019daa1 ffff83013ff2ff18 ffffffffffffffff ffff83013ff2fed0
(XEN) 0000000080125395 0000000000000000 0000000000000000 ffff82c48025bc80
(XEN) 00001124000f684a ffff83013ff2ff18 ffff82c48025bc80 ffff83013ff2ff18
(XEN) 00000000ffffffff ffff83013ff2ff10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300beefa000 ffff8300bf307000 000000000000000f ffff83013ff2fd88
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000009
(XEN) ffff88025c8b7f00 ffff88025c8b6010 0000000000000246 0000015d677f9200
(XEN) ffff88026e695cc0 0000000000000000 0000000000000000 ffffffff810013aa
(XEN) 0000000000000000 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN) ffffffff810013aa 000000000000e033 0000000000000246 ffff88025c8b7ee8
(XEN) 000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000000f ffff8300beefa000 0000003cbfc3a680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU16 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 16
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff8302bd2082f0 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ff34c00 rdi: ffff8302bd208350
(XEN) rbp: ffff83013ff27e70 rsp: ffff83013ff27e50 r8: 0000000000000000
(XEN) r9: 0000000000000002 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff8302bd208350 r13: 0000000000000001 r14: 00000182faa8238a
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000057204000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff27e50:
(XEN) 00000000ffffffff ffff8302bd2082f0 ffff8302bd208350 0000000000000001
(XEN) ffff83013ff27ee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ff27ef8
(XEN) ffff82c4802d1000 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003c3000004b0 ffff83013ff27f18 ffff82c48025bc80
(XEN) ffff83013ff27f18 00000000ffffffff ffff83013ff27f10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300beef9000 ffff8300bf7f3000 0000000000000010
(XEN) ffff83013ff27d88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000005 ffff88025c8adf00 ffff88025c8ac010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8adee8 000000000000e02b 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000010 ffff8300beef9000
(XEN) 0000003cbfc38680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU17 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 17
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff8302bd2087f0 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ff32c00 rdi: ffff8302bd208850
(XEN) rbp: ffff83013ff1fe70 rsp: ffff83013ff1fe50 r8: 0000000000000000
(XEN) r9: ffff8300beef3060 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff8302bd208850 r13: 0000000000000001 r14: 0000018305cd88f5
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000051d78000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff1fe50:
(XEN) 00000000ffffffff ffff8302bd2087f0 ffff8302bd208850 0000000000000001
(XEN) ffff83013ff1fee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ff1fef8
(XEN) ffff82c4802d1080 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000001630000016e ffff83013ff1ff18 ffff82c48025bc80
(XEN) ffff83013ff1ff18 00000000ffffffff ffff83013ff1ff10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300beef8000 ffff8300beef3000 0000000000000011
(XEN) ffff83013ff1fd88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000015 ffff88025c8dff00 ffff88025c8de010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8dfee8 000000000000e02b 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000011 ffff8300beef8000
(XEN) 0000003cbfc36680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU18 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 18
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff8302bd208cf0 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ff30c00 rdi: ffff83013ff17ea0
(XEN) rbp: ffff83013ff17ee0 rsp: ffff83013ff17e78 r8: 0000018286444f80
(XEN) r9: ffff8300bf304060 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff8302bd208db0 r13: 0000018310f73831 r14: 00000182863e2b2f
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000002b1c05000 cr2: ffff88024516c408
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff17e78:
(XEN) ffff82c48019daa1 ffff82c480121d0e ffff83013ff17ef8 ffff82c4802d1100
(XEN) 00000000ffffffff 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 000001c700039035 ffff83013ff17f18 ffff82c48025bc80 ffff83013ff17f18
(XEN) 00000000ffffffff ffff83013ff17f10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300bf65f000 ffff8300bf304000 0000000000000012 ffff83013ff17d88
(XEN) 0000000000000000 0000000000000000 0000000000000000 000000000000000c
(XEN) ffff88025c8bdf00 ffff88025c8bc010 0000000000000246 0000015d66f63dc0
(XEN) ffff88026e6e6cc0 0000000000000000 0000000000000000 ffffffff810013aa
(XEN) 0000000000000000 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN) ffffffff810013aa 000000000000e033 0000000000000246 ffff88025c8bdee8
(XEN) 000000000000e02b 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000012 ffff8300bf65f000 0000003cbfc34680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU19 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 19
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013ff0d210 rcx: 0000000000000001
(XEN) rdx: 00000000000000e8 rsi: ffff83013ff0ec00 rdi: ffff83013ff0d270
(XEN) rbp: ffff83013ff07e70 rsp: ffff83013ff07e50 r8: 0000000000000000
(XEN) r9: 0000000000000004 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ff0d270 r13: 0000000000000001 r14: 000001831b338300
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 0000000057360000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013ff07e50:
(XEN) 00000000ffffffff ffff83013ff0d210 ffff83013ff0d270 0000000000000001
(XEN) ffff83013ff07ee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013ff07ef8
(XEN) ffff82c4802d1180 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003c0000003e9 ffff83013ff07f18 ffff82c48025bc80
(XEN) ffff83013ff07f18 00000000ffffffff ffff83013ff07f10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300bf65e000 ffff8300beef1000 0000000000000013
(XEN) ffff83013ff07d88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000017 ffff88025c8e3f00 ffff88025c8e2010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8e3ee8 000000000000e02b 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000beef 0000000000000013 ffff8300bf65e000
(XEN) 0000003cbfc12680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU20 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 20
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013ff0d710 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ff0bc00 rdi: ffff83013efffea0
(XEN) rbp: ffff83013efffee0 rsp: ffff83013efffe78 r8: 0000018286dce600
(XEN) r9: ffff8300bf305060 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013ff0d7d0 r13: 00000183265f900a r14: 00000182866fb9ac
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000002b1c05000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013efffe78:
(XEN) ffff82c48019daa1 ffff82c480121d0e ffff83013efffef8 ffff82c4802d1200
(XEN) 00000000ffffffff 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 00001c280007f583 ffff83013effff18 ffff82c48025bc80 ffff83013effff18
(XEN) 00000000ffffffff ffff83013effff10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300bf65d000 ffff8300bf305000 0000000000000014 ffff83013efffd88
(XEN) 0000000000000000 0000000000000000 0000000000000000 000000000000000b
(XEN) ffff88025c8bbf00 ffff88025c8ba010 0000000000000246 0000015d67240480
(XEN) ffff88026e6cbcc0 0000000000000000 0000000000000000 ffffffff810013aa
(XEN) 0000000000000000 00000000deadbeef 00000000deadbeef 0000010000000000
(XEN) ffffffff810013aa 000000000000e033 0000000000000246 ffff88025c8bbee8
(XEN) 000000000000e02b 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000014 ffff8300bf65d000 0000003cbfc0f680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU21 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 21
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013ff0dc10 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013ff09c00 rdi: ffff83013eff7ea0
(XEN) rbp: ffff83013eff7ee0 rsp: ffff83013eff7e78 r8: 0000000000001c44
(XEN) r9: 000000000000002a r10: 0000000000000000 r11: 0000000000000001
(XEN) r12: ffff83013ff0dcd0 r13: 0000018330a0df77 r14: 00000183265fb20d
(XEN) r15: 0000000000000003 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000000bf0b4000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff83013eff7e78:
(XEN) ffff82c48019daa1 ffff83013eff7f18 ffffffffffffffff ffff83013eff7ed0
(XEN) 0000000080125395 0000000000000000 0000000000000000 ffff82c48025bc80
(XEN) 00000f3f0008b47e ffff83013eff7f18 ffff82c48025bc80 ffff83013eff7f18
(XEN) 00000000ffffffff ffff83013eff7f10 ffff82c48015a20b ffffffff810da100
(XEN) ffff8300bf65c000 ffff8300bf7f7000 0000000000000015 ffff83013eff7d88
(XEN) ffffffff810da100 ffff88026e5bdc80 00000000000ceb6d ffff88025c88de38
(XEN) ffff88026e5b3d28 ffff88026e5bddc0 0000000000000001 0000000000000000
(XEN) 0000000000000001 0000000000000000 0000000000000000 ffff8802580d1a48
(XEN) ffff88026e5bdc80 0000000000000000 0000000000000000 000000f900000000
(XEN) ffffffff810d330d 000000000000e033 0000000000000202 ffff88026e5b3d08
(XEN) 000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN) 000000000000beef 0000000000000015 ffff8300bf65c000 0000003cbfc0d680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU22 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 22
(XEN) RIP: e008:[<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: ffff82c48025b314 rbx: ffff83013efee130 rcx: 0000000000000001
(XEN) rdx: 0000000000002f40 rsi: ffff83013efefc00 rdi: ffff83013efe7ea0
(XEN) rbp: ffff83013efe7ee0 rsp: ffff83013efe7e78 r8: 000000000000183c
(XEN) r9: 0000000000000029 r10: 00000000fffffffe r11: 00000000001c6970
(XEN) r12: ffff83013efee1f0 r13: 000001833ae346b1 r14: 0000017e5e22d8a9
(XEN) r15: 0000000000000003 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 00000000bf0b4000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff83013efe7e78:
(XEN) ffff82c48019daa1 ffff83013efe7f18 ffffffffffffffff ffff83013efe7ed0
(XEN) 0000000080125395 0000000000000000 0000000000000000 ffff82c48025bc80
(XEN) 00000c830009351a ffff83013efe7f18 ffff82c48025bc80 ffff83013efe7f18
(XEN) 00000000ffffffff ffff83013efe7f10 ffff82c48015a20b 0000000000000000
(XEN) ffff8300bf65b000 ffff8300bf7f0000 0000000000000016 ffff83013efe7d88
(XEN) 0000000000000000 0000000000000000 00000000000ceb6d 0000000000000008
(XEN) ffff88026e670f38 ffff88026e67adc0 0000000000000246 0000000000000000
(XEN) 0000000000000001 0000000000000000 0000000000000008 0000000000000001
(XEN) ffff88026e66d000 0000000000000002 0000000000000000 000000f900000000
(XEN) ffffffff810d330d 000000000000e033 0000000000000202 ffff88026e670f18
(XEN) 000000000000e02b 000000000000beef 000000000000beef 000000000000beef
(XEN) 000000000000beef 0000000000000016 ffff8300bf65b000 0000003cbecf3680
(XEN) 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019c1c0>] lapic_timer_nop+0x0/0x6
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
(XEN) *** Dumping CPU23 host state: ***
(XEN) ----[ Xen-4.2-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 23
(XEN) RIP: e008:[<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) RFLAGS: 0000000000000246 CONTEXT: hypervisor
(XEN) rax: 0000000000000003 rbx: ffff83013efee630 rcx: 0000000000000001
(XEN) rdx: 00000000000000e9 rsi: ffff83013efecc00 rdi: ffff83013efee690
(XEN) rbp: ffff83013efdfe70 rsp: ffff83013efdfe50 r8: 00000183452848bb
(XEN) r9: ffff83028eea16a0 r10: 00000000deadbeef r11: 0000000000000246
(XEN) r12: ffff83013efee690 r13: 0000000000000001 r14: 00000183451a5a2b
(XEN) r15: 0000000000000002 cr0: 000000008005003b cr4: 00000000000026f0
(XEN) cr3: 000000005601d000 cr2: ffffffffff600400
(XEN) ds: 002b es: 002b fs: 0000 gs: 0000 ss: e010 cs: e008
(XEN) Xen stack trace from rsp=ffff83013efdfe50:
(XEN) 00000000ffffffff ffff83013efee630 ffff83013efee690 0000000000000001
(XEN) ffff83013efdfee0 ffff82c48019d8e0 ffff82c480121d0e ffff83013efdfef8
(XEN) ffff82c4802d1380 00000000ffffffff 0000000000000000 0000000000000000
(XEN) ffffffffffffffff 000003c700000511 ffff83013efdff18 ffff82c48025bc80
(XEN) ffff83013efdff18 00000000ffffffff ffff83013efdff10 ffff82c48015a20b
(XEN) 0000000000000000 ffff8300bf65a000 ffff8300bf7f4000 0000000000000017
(XEN) ffff83013efdfd88 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000004 ffff88025c8abf00 ffff88025c8aa010 0000000000000246
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) ffffffff810013aa 0000000000000000 00000000deadbeef 00000000deadbeef
(XEN) 0000010000000000 ffffffff810013aa 000000000000e033 0000000000000246
(XEN) ffff88025c8abee8 000000000000e02b 000000000000beef 000000000000beef
(XEN) 000000000000beef 000000000000beef 0000000000000017 ffff8300bf65a000
(XEN) 0000003cbecf0680 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82c48019d58f>] acpi_idle_do_entry+0x109/0x118
(XEN) [<ffff82c48019d8e0>] acpi_processor_idle+0x2b7/0x501
(XEN) [<ffff82c48015a20b>] idle_loop+0x6c/0x7d
(XEN)
[ 1830.719675] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 7, t=1140194 jiffies)
[ 2010.751675] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 7, t=1320226 jiffies)
[ 2190.783675] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 7, t=1500258 jiffies)
[ 2370.815677] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 7, t=1680290 jiffies)
[ 2550.847670] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 7, t=1860322 jiffies)
[ 2730.879674] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 7, t=2040354 jiffies)
[ 2910.911677] INFO: rcu_sched_state detected stalls on CPUs/tasks: { 0} (detected by 7, t=2220386 jiffies)
[-- Attachment #3: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-09 10:58 Deadlocks by p2m_lock and event_lock Hao, Xudong
@ 2012-03-09 11:20 ` Tim Deegan
2012-03-09 11:44 ` Hao, Xudong
0 siblings, 1 reply; 18+ messages in thread
From: Tim Deegan @ 2012-03-09 11:20 UTC (permalink / raw)
To: Hao, Xudong
Cc: xen-devel, Keir Fraser, Andres Lagar-Cavilla, JBeulich, Zhang, Xiantao
Hi,
At 10:58 +0000 on 09 Mar (1331290728), Hao, Xudong wrote:
> ====CPU0===
> map_domain_pirq() Grab event_lock
> /
> Pci_enable_msi()
> /
> msix_capability_init()
> /
> p2m_change_entry_type_global() Trying to acquire p2m_lock
>
> ====CPU9===
> hvm_hap_nested_page_fault() -> get_gfn_type_access() Grab p2m_lock
> /
> handle_mmio()
> /
> ...
> /
> notify_via_xen_event_channel() Trying to acquire event_lock
>
>
> The event_lock is used anywhere in Xen, I only have a patch of workaround this issue for proposal, but not for the final fix. Any good suggestion?
>
> diff -r f61120046915 xen/arch/x86/irq.c
> --- a/xen/arch/x86/irq.c Wed Mar 07 11:50:31 2012 +0100
> +++ b/xen/arch/x86/irq.c Sat Mar 10 02:06:18 2012 +0800
> @@ -1875,10 +1875,12 @@ int map_domain_pirq(
> if ( !cpu_has_apic )
> goto done;
>
> + spin_unlock(&d->event_lock);
> pdev = pci_get_pdev(msi->seg, msi->bus, msi->devfn);
> ret = pci_enable_msi(msi, &msi_desc);
> if ( ret )
> goto done;
> + spin_lock(&d->event_lock);
>
> spin_lock_irqsave(&desc->lock, flags);
>
> Best Regards,
> Xudong Hao
I don't know about the event lock, but it seems unwise to call in to
handle_mmio with a gfn lock held. How about fixing the other path?
diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
--- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
+++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
@@ -1324,10 +1324,11 @@ int hvm_hap_nested_page_fault(unsigned l
if ( (p2mt == p2m_mmio_dm) ||
(access_w && (p2mt == p2m_ram_ro)) )
{
+ put_gfn(p2m->domain, gfn);
if ( !handle_mmio() )
hvm_inject_exception(TRAP_gp_fault, 0, 0);
rc = 1;
- goto out_put_gfn;
+ goto out;
}
#ifdef __x86_64__
@@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
out_put_gfn:
put_gfn(p2m->domain, gfn);
+out:
if ( paged )
p2m_mem_paging_populate(v->domain, gfn);
if ( req_ptr )
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-09 11:20 ` Tim Deegan
@ 2012-03-09 11:44 ` Hao, Xudong
2012-03-09 16:29 ` Andres Lagar-Cavilla
0 siblings, 1 reply; 18+ messages in thread
From: Hao, Xudong @ 2012-03-09 11:44 UTC (permalink / raw)
To: Tim Deegan
Cc: xen-devel, Keir Fraser, Andres Lagar-Cavilla, JBeulich, Zhang, Xiantao
> -----Original Message-----
> From: Tim Deegan [mailto:tim@xen.org]
> Sent: Friday, March 09, 2012 7:20 PM
> To: Hao, Xudong
> Cc: JBeulich@suse.com; Andres Lagar-Cavilla; xen-devel@lists.xensource.com;
> Keir Fraser; Zhang, Xiantao
> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>
> Hi,
>
> At 10:58 +0000 on 09 Mar (1331290728), Hao, Xudong wrote:
> > ====CPU0===
> > map_domain_pirq() Grab event_lock
> > /
> > Pci_enable_msi()
> > /
> > msix_capability_init()
> > /
> > p2m_change_entry_type_global() Trying to acquire p2m_lock
> >
> > ====CPU9===
> > hvm_hap_nested_page_fault() -> get_gfn_type_access() Grab p2m_lock
> > /
> > handle_mmio()
> > /
> > ...
> > /
> > notify_via_xen_event_channel() Trying to acquire event_lock
> >
> >
> > The event_lock is used anywhere in Xen, I only have a patch of workaround
> this issue for proposal, but not for the final fix. Any good suggestion?
> >
> > diff -r f61120046915 xen/arch/x86/irq.c
> > --- a/xen/arch/x86/irq.c Wed Mar 07 11:50:31 2012 +0100
> > +++ b/xen/arch/x86/irq.c Sat Mar 10 02:06:18 2012 +0800
> > @@ -1875,10 +1875,12 @@ int map_domain_pirq(
> > if ( !cpu_has_apic )
> > goto done;
> >
> > + spin_unlock(&d->event_lock);
> > pdev = pci_get_pdev(msi->seg, msi->bus, msi->devfn);
> > ret = pci_enable_msi(msi, &msi_desc);
> > if ( ret )
> > goto done;
> > + spin_lock(&d->event_lock);
> >
> > spin_lock_irqsave(&desc->lock, flags);
> >
> > Best Regards,
> > Xudong Hao
>
> I don't know about the event lock, but it seems unwise to call in to
> handle_mmio with a gfn lock held. How about fixing the other path?
>
> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
> @@ -1324,10 +1324,11 @@ int hvm_hap_nested_page_fault(unsigned l
> if ( (p2mt == p2m_mmio_dm) ||
> (access_w && (p2mt == p2m_ram_ro)) )
> {
> + put_gfn(p2m->domain, gfn);
> if ( !handle_mmio() )
> hvm_inject_exception(TRAP_gp_fault, 0, 0);
> rc = 1;
> - goto out_put_gfn;
> + goto out;
> }
>
> #ifdef __x86_64__
> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
>
> out_put_gfn:
> put_gfn(p2m->domain, gfn);
> +out:
> if ( paged )
> p2m_mem_paging_populate(v->domain, gfn);
> if ( req_ptr )
Yes, that's fine to release the p2m lock earlier than handle_mmio.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-09 11:44 ` Hao, Xudong
@ 2012-03-09 16:29 ` Andres Lagar-Cavilla
2012-03-09 16:55 ` Tim Deegan
0 siblings, 1 reply; 18+ messages in thread
From: Andres Lagar-Cavilla @ 2012-03-09 16:29 UTC (permalink / raw)
To: Hao, Xudong
Cc: Keir Fraser, xen-devel, Tim Deegan, Zhang, Xiantao, JBeulich@suse.com
>
>> -----Original Message-----
>> From: Tim Deegan [mailto:tim@xen.org]
>> Sent: Friday, March 09, 2012 7:20 PM
>> To: Hao, Xudong
>> Cc: JBeulich@suse.com; Andres Lagar-Cavilla;
>> xen-devel@lists.xensource.com;
>> Keir Fraser; Zhang, Xiantao
>> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>>
>> Hi,
>>
>> At 10:58 +0000 on 09 Mar (1331290728), Hao, Xudong wrote:
>> > ====CPU0===
>> > map_domain_pirq() Grab event_lock
>> > /
>> > Pci_enable_msi()
>> > /
>> > msix_capability_init()
>> > /
>> > p2m_change_entry_type_global() Trying to acquire p2m_lock
>> >
>> > ====CPU9===
>> > hvm_hap_nested_page_fault() -> get_gfn_type_access() Grab p2m_lock
>> > /
>> > handle_mmio()
>> > /
>> > ...
>> > /
>> > notify_via_xen_event_channel() Trying to acquire event_lock
>> >
>> >
>> > The event_lock is used anywhere in Xen, I only have a patch of
>> workaround
>> this issue for proposal, but not for the final fix. Any good suggestion?
>> >
>> > diff -r f61120046915 xen/arch/x86/irq.c
>> > --- a/xen/arch/x86/irq.c Wed Mar 07 11:50:31 2012 +0100
>> > +++ b/xen/arch/x86/irq.c Sat Mar 10 02:06:18 2012 +0800
>> > @@ -1875,10 +1875,12 @@ int map_domain_pirq(
>> > if ( !cpu_has_apic )
>> > goto done;
>> >
>> > + spin_unlock(&d->event_lock);
>> > pdev = pci_get_pdev(msi->seg, msi->bus, msi->devfn);
>> > ret = pci_enable_msi(msi, &msi_desc);
>> > if ( ret )
>> > goto done;
>> > + spin_lock(&d->event_lock);
>> >
>> > spin_lock_irqsave(&desc->lock, flags);
>> >
>> > Best Regards,
>> > Xudong Hao
>>
>> I don't know about the event lock, but it seems unwise to call in to
>> handle_mmio with a gfn lock held. How about fixing the other path?
>>
>> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
>> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
>> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
>> @@ -1324,10 +1324,11 @@ int hvm_hap_nested_page_fault(unsigned l
>> if ( (p2mt == p2m_mmio_dm) ||
>> (access_w && (p2mt == p2m_ram_ro)) )
>> {
>> + put_gfn(p2m->domain, gfn);
>> if ( !handle_mmio() )
>> hvm_inject_exception(TRAP_gp_fault, 0, 0);
>> rc = 1;
>> - goto out_put_gfn;
>> + goto out;
>> }
>>
>> #ifdef __x86_64__
>> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
>>
>> out_put_gfn:
>> put_gfn(p2m->domain, gfn);
>> +out:
>> if ( paged )
>> p2m_mem_paging_populate(v->domain, gfn);
>> if ( req_ptr )
>
> Yes, that's fine to release the p2m lock earlier than handle_mmio.
Ack
Thanks,
Andres
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-09 16:29 ` Andres Lagar-Cavilla
@ 2012-03-09 16:55 ` Tim Deegan
2012-03-13 7:51 ` Hao, Xudong
0 siblings, 1 reply; 18+ messages in thread
From: Tim Deegan @ 2012-03-09 16:55 UTC (permalink / raw)
To: Andres Lagar-Cavilla
Cc: Keir Fraser, xen-devel, Hao, Xudong, JBeulich@suse.com, Zhang, Xiantao
At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
> >> I don't know about the event lock, but it seems unwise to call in to
> >> handle_mmio with a gfn lock held. How about fixing the other path?
> >>
> >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
> >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
> >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
> >> @@ -1324,10 +1324,11 @@ int hvm_hap_nested_page_fault(unsigned l
> >> if ( (p2mt == p2m_mmio_dm) ||
> >> (access_w && (p2mt == p2m_ram_ro)) )
> >> {
> >> + put_gfn(p2m->domain, gfn);
> >> if ( !handle_mmio() )
> >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
> >> rc = 1;
> >> - goto out_put_gfn;
> >> + goto out;
> >> }
> >>
> >> #ifdef __x86_64__
> >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
> >>
> >> out_put_gfn:
> >> put_gfn(p2m->domain, gfn);
> >> +out:
> >> if ( paged )
> >> p2m_mem_paging_populate(v->domain, gfn);
> >> if ( req_ptr )
> >
> > Yes, that's fine to release the p2m lock earlier than handle_mmio.
>
> Ack
OK, applied.
Tim.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-09 16:55 ` Tim Deegan
@ 2012-03-13 7:51 ` Hao, Xudong
2012-03-13 15:27 ` Andres Lagar-Cavilla
` (2 more replies)
0 siblings, 3 replies; 18+ messages in thread
From: Hao, Xudong @ 2012-03-13 7:51 UTC (permalink / raw)
To: Tim Deegan, Andres Lagar-Cavilla
Cc: xen-devel, Keir Fraser, JBeulich@suse.com, Zhang, Xiantao
Hi, Tim and Andres
The patch fix part of this issue. In handle_mmio, function hvmemul_do_io() is called and p2m lock was held again by calling get_gfn_unshare(), still trigger a deadlocks.
(XEN) Xen call trace:
(XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
(XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
(XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
(XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
(XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
(XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
(XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
(XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
(XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
(XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
(XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
(XEN) [<ffff82c4801afd72>] hvm_hap_nested_page_fault+0x210/0x37f
(XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
Thanks,
-Xudong
> -----Original Message-----
> From: Tim Deegan [mailto:tim@xen.org]
> Sent: Saturday, March 10, 2012 12:56 AM
> To: Andres Lagar-Cavilla
> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com; Zhang, Xiantao;
> JBeulich@suse.com
> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>
> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
> > >> I don't know about the event lock, but it seems unwise to call in
> > >> to handle_mmio with a gfn lock held. How about fixing the other path?
> > >>
> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
> > >> @@ -1324,10 +1324,11 @@ int hvm_hap_nested_page_fault(unsigned l
> > >> if ( (p2mt == p2m_mmio_dm) ||
> > >> (access_w && (p2mt == p2m_ram_ro)) )
> > >> {
> > >> + put_gfn(p2m->domain, gfn);
> > >> if ( !handle_mmio() )
> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
> > >> rc = 1;
> > >> - goto out_put_gfn;
> > >> + goto out;
> > >> }
> > >>
> > >> #ifdef __x86_64__
> > >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
> > >>
> > >> out_put_gfn:
> > >> put_gfn(p2m->domain, gfn);
> > >> +out:
> > >> if ( paged )
> > >> p2m_mem_paging_populate(v->domain, gfn);
> > >> if ( req_ptr )
> > >
> > > Yes, that's fine to release the p2m lock earlier than handle_mmio.
> >
> > Ack
>
> OK, applied.
>
> Tim.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-13 7:51 ` Hao, Xudong
@ 2012-03-13 15:27 ` Andres Lagar-Cavilla
2012-03-13 18:26 ` Andres Lagar-Cavilla
2012-03-13 18:45 ` Andres Lagar-Cavilla
2 siblings, 0 replies; 18+ messages in thread
From: Andres Lagar-Cavilla @ 2012-03-13 15:27 UTC (permalink / raw)
To: Hao, Xudong
Cc: Keir Fraser, xen-devel, Tim Deegan, JBeulich@suse.com, Zhang, Xiantao
> Hi, Tim and Andres
> The patch fix part of this issue. In handle_mmio, function hvmemul_do_io()
> is called and p2m lock was held again by calling get_gfn_unshare(), still
> trigger a deadlocks.
>
> (XEN) Xen call trace:
> (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
> (XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
> (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
> (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
> (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
> (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
> (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
> (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
> (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
> (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
> (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
> (XEN) [<ffff82c4801afd72>] hvm_hap_nested_page_fault+0x210/0x37f
> (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
>
> Thanks,
Thanks for the report, very useful. I'll look into it
Andres
> -Xudong
>
>> -----Original Message-----
>> From: Tim Deegan [mailto:tim@xen.org]
>> Sent: Saturday, March 10, 2012 12:56 AM
>> To: Andres Lagar-Cavilla
>> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> Xiantao;
>> JBeulich@suse.com
>> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>>
>> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
>> > >> I don't know about the event lock, but it seems unwise to call in
>> > >> to handle_mmio with a gfn lock held. How about fixing the other
>> path?
>> > >>
>> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
>> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
>> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
>> > >> @@ -1324,10 +1324,11 @@ int hvm_hap_nested_page_fault(unsigned l
>> > >> if ( (p2mt == p2m_mmio_dm) ||
>> > >> (access_w && (p2mt == p2m_ram_ro)) )
>> > >> {
>> > >> + put_gfn(p2m->domain, gfn);
>> > >> if ( !handle_mmio() )
>> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
>> > >> rc = 1;
>> > >> - goto out_put_gfn;
>> > >> + goto out;
>> > >> }
>> > >>
>> > >> #ifdef __x86_64__
>> > >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
>> > >>
>> > >> out_put_gfn:
>> > >> put_gfn(p2m->domain, gfn);
>> > >> +out:
>> > >> if ( paged )
>> > >> p2m_mem_paging_populate(v->domain, gfn);
>> > >> if ( req_ptr )
>> > >
>> > > Yes, that's fine to release the p2m lock earlier than handle_mmio.
>> >
>> > Ack
>>
>> OK, applied.
>>
>> Tim.
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-13 7:51 ` Hao, Xudong
2012-03-13 15:27 ` Andres Lagar-Cavilla
@ 2012-03-13 18:26 ` Andres Lagar-Cavilla
2012-03-14 9:20 ` Jan Beulich
2012-03-13 18:45 ` Andres Lagar-Cavilla
2 siblings, 1 reply; 18+ messages in thread
From: Andres Lagar-Cavilla @ 2012-03-13 18:26 UTC (permalink / raw)
To: Hao, Xudong
Cc: Keir Fraser, xen-devel, Tim Deegan, JBeulich@suse.com, Zhang, Xiantao
> Hi, Tim and Andres
> The patch fix part of this issue. In handle_mmio, function hvmemul_do_io()
> is called and p2m lock was held again by calling get_gfn_unshare(), still
> trigger a deadlocks.
I have a question before I dive into lock untangling
msix_capability_init ->
p2m_change_entry_type_global(dev->domain, p2m_mmio_direct, p2m_mmio_direct);
Huh? This achieves ... nothing. Almost. It flushes a bunch of TLBs, but
that can be done with significantly less effort. Am I missing something?
Andres
>
> (XEN) Xen call trace:
> (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
> (XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
> (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
> (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
> (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
> (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
> (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
> (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
> (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
> (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
> (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
> (XEN) [<ffff82c4801afd72>] hvm_hap_nested_page_fault+0x210/0x37f
> (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
>
> Thanks,
> -Xudong
>
>> -----Original Message-----
>> From: Tim Deegan [mailto:tim@xen.org]
>> Sent: Saturday, March 10, 2012 12:56 AM
>> To: Andres Lagar-Cavilla
>> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> Xiantao;
>> JBeulich@suse.com
>> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>>
>> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
>> > >> I don't know about the event lock, but it seems unwise to call in
>> > >> to handle_mmio with a gfn lock held. How about fixing the other
>> path?
>> > >>
>> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
>> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
>> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
>> > >> @@ -1324,10 +1324,11 @@ int hvm_hap_nested_page_fault(unsigned l
>> > >> if ( (p2mt == p2m_mmio_dm) ||
>> > >> (access_w && (p2mt == p2m_ram_ro)) )
>> > >> {
>> > >> + put_gfn(p2m->domain, gfn);
>> > >> if ( !handle_mmio() )
>> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
>> > >> rc = 1;
>> > >> - goto out_put_gfn;
>> > >> + goto out;
>> > >> }
>> > >>
>> > >> #ifdef __x86_64__
>> > >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
>> > >>
>> > >> out_put_gfn:
>> > >> put_gfn(p2m->domain, gfn);
>> > >> +out:
>> > >> if ( paged )
>> > >> p2m_mem_paging_populate(v->domain, gfn);
>> > >> if ( req_ptr )
>> > >
>> > > Yes, that's fine to release the p2m lock earlier than handle_mmio.
>> >
>> > Ack
>>
>> OK, applied.
>>
>> Tim.
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-13 7:51 ` Hao, Xudong
2012-03-13 15:27 ` Andres Lagar-Cavilla
2012-03-13 18:26 ` Andres Lagar-Cavilla
@ 2012-03-13 18:45 ` Andres Lagar-Cavilla
2012-03-14 7:12 ` Hao, Xudong
2 siblings, 1 reply; 18+ messages in thread
From: Andres Lagar-Cavilla @ 2012-03-13 18:45 UTC (permalink / raw)
To: Hao, Xudong
Cc: Keir Fraser, xen-devel, Tim Deegan, JBeulich@suse.com, Zhang, Xiantao
> Hi, Tim and Andres
> The patch fix part of this issue. In handle_mmio, function hvmemul_do_io()
> is called and p2m lock was held again by calling get_gfn_unshare(), still
> trigger a deadlocks.
Typically hvmemul_do_io gets the zero gfn, because in many cases that's
the 'rma_gpa' it is passed. However, in the case of mmio, and particularly
stdvga, ram_gpa is the data to be copied to the framebuffer. So it is in
principle ok to get_gfn in hvmemul_do_io.
There are two solutions
1. msix_capability_init does not call p2m_change_entry_type_global. See my
previous email. If we want to resync the
EPT/NPT/traditional/VTD/IOMMU/superduper TLBs, we can just do that
explicitly. I hope.
2. hvmemul_do_io does gets a ref on the mfn underlying ram_gpa, and holds
that for the critical section, instead of the p2m lock. One way to achieve
this is
/* Check for paged out page */
ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
if ( this or that )
{ ... handle ... }
if ( mfn_valid(ram_mfn) )
get_page(mfn_to_page(ram_mfn, curr->domain));
put_gfn(curr->domain, ram_gfn)
/* replace all put_gfn in all exit paths by put_page */
This will ensure the target page is live and sane while not holding the
p2m lock. Xudong, did that make sense? Do you think you could try that and
report back?
Thanks!
Andres
>
> (XEN) Xen call trace:
> (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
> (XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
> (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
> (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
> (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
> (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
> (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
> (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
> (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
> (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
> (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
> (XEN) [<ffff82c4801afd72>] hvm_hap_nested_page_fault+0x210/0x37f
> (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
>
> Thanks,
> -Xudong
>
>> -----Original Message-----
>> From: Tim Deegan [mailto:tim@xen.org]
>> Sent: Saturday, March 10, 2012 12:56 AM
>> To: Andres Lagar-Cavilla
>> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> Xiantao;
>> JBeulich@suse.com
>> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>>
>> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
>> > >> I don't know about the event lock, but it seems unwise to call in
>> > >> to handle_mmio with a gfn lock held. How about fixing the other
>> path?
>> > >>
>> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
>> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
>> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
>> > >> @@ -1324,10 +1324,11 @@ int hvm_hap_nested_page_fault(unsigned l
>> > >> if ( (p2mt == p2m_mmio_dm) ||
>> > >> (access_w && (p2mt == p2m_ram_ro)) )
>> > >> {
>> > >> + put_gfn(p2m->domain, gfn);
>> > >> if ( !handle_mmio() )
>> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
>> > >> rc = 1;
>> > >> - goto out_put_gfn;
>> > >> + goto out;
>> > >> }
>> > >>
>> > >> #ifdef __x86_64__
>> > >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
>> > >>
>> > >> out_put_gfn:
>> > >> put_gfn(p2m->domain, gfn);
>> > >> +out:
>> > >> if ( paged )
>> > >> p2m_mem_paging_populate(v->domain, gfn);
>> > >> if ( req_ptr )
>> > >
>> > > Yes, that's fine to release the p2m lock earlier than handle_mmio.
>> >
>> > Ack
>>
>> OK, applied.
>>
>> Tim.
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-13 18:45 ` Andres Lagar-Cavilla
@ 2012-03-14 7:12 ` Hao, Xudong
2012-03-14 8:28 ` Zhang, Yang Z
` (2 more replies)
0 siblings, 3 replies; 18+ messages in thread
From: Hao, Xudong @ 2012-03-14 7:12 UTC (permalink / raw)
To: andres
Cc: Keir Fraser, xen-devel, Tim Deegan, JBeulich@suse.com, Zhang, Xiantao
I prefer to the 2nd, I made a patch and testing show it works.
diff -r 5d20d2f6ffed xen/arch/x86/hvm/emulate.c
--- a/xen/arch/x86/hvm/emulate.c Fri Mar 09 16:54:24 2012 +0000
+++ b/xen/arch/x86/hvm/emulate.c Wed Mar 14 15:11:52 2012 -0400
@@ -60,20 +60,23 @@
ioreq_t *p = get_ioreq(curr);
unsigned long ram_gfn = paddr_to_pfn(ram_gpa);
p2m_type_t p2mt;
- mfn_t ram_mfn;
+ unsigned long ram_mfn;
int rc;
/* Check for paged out page */
- ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
+ ram_mfn = mfn_x(get_gfn_unshare(curr->domain, ram_gfn, &p2mt));
+ if ( mfn_valid(ram_mfn) )
+ get_page(mfn_to_page(ram_mfn), curr->domain);
+ put_gfn(curr->domain, ram_gfn);
if ( p2m_is_paging(p2mt) )
{
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
p2m_mem_paging_populate(curr->domain, ram_gfn);
return X86EMUL_RETRY;
}
if ( p2m_is_shared(p2mt) )
{
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_RETRY;
}
@@ -87,7 +90,7 @@
ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
if ( dir == IOREQ_READ )
memset(p_data, ~0, size);
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_UNHANDLEABLE;
}
@@ -108,7 +111,7 @@
unsigned int bytes = vio->mmio_large_write_bytes;
if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
{
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_OKAY;
}
}
@@ -120,7 +123,7 @@
{
memcpy(p_data, &vio->mmio_large_read[addr - pa],
size);
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_OKAY;
}
}
@@ -134,7 +137,7 @@
vio->io_state = HVMIO_none;
if ( p_data == NULL )
{
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_UNHANDLEABLE;
}
goto finish_access;
@@ -144,11 +147,11 @@
(addr == (vio->mmio_large_write_pa +
vio->mmio_large_write_bytes)) )
{
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_RETRY;
}
default:
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_UNHANDLEABLE;
}
@@ -156,7 +159,7 @@
{
gdprintk(XENLOG_WARNING, "WARNING: io already pending (%d)?\n",
p->state);
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_UNHANDLEABLE;
}
@@ -208,7 +211,7 @@
if ( rc != X86EMUL_OKAY )
{
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return rc;
}
@@ -244,7 +247,7 @@
}
}
- put_gfn(curr->domain, ram_gfn);
+ put_page(mfn_to_page(ram_mfn));
return X86EMUL_OKAY;
}
Thanks,
-Xudong
> -----Original Message-----
> From: Andres Lagar-Cavilla [mailto:andres@lagarcavilla.org]
> Sent: Wednesday, March 14, 2012 2:46 AM
> To: Hao, Xudong
> Cc: Tim Deegan; Keir Fraser; xen-devel@lists.xensource.com; Zhang, Xiantao;
> JBeulich@suse.com
> Subject: RE: [Xen-devel] Deadlocks by p2m_lock and event_lock
>
> > Hi, Tim and Andres
> > The patch fix part of this issue. In handle_mmio, function
> > hvmemul_do_io() is called and p2m lock was held again by calling
> > get_gfn_unshare(), still trigger a deadlocks.
>
> Typically hvmemul_do_io gets the zero gfn, because in many cases that's the
> 'rma_gpa' it is passed. However, in the case of mmio, and particularly stdvga,
> ram_gpa is the data to be copied to the framebuffer. So it is in principle ok to
> get_gfn in hvmemul_do_io.
>
> There are two solutions
> 1. msix_capability_init does not call p2m_change_entry_type_global. See my
> previous email. If we want to resync the
> EPT/NPT/traditional/VTD/IOMMU/superduper TLBs, we can just do that
> explicitly. I hope.
>
> 2. hvmemul_do_io does gets a ref on the mfn underlying ram_gpa, and holds
> that for the critical section, instead of the p2m lock. One way to achieve this is
>
> /* Check for paged out page */
> ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
> if ( this or that )
> { ... handle ... }
> if ( mfn_valid(ram_mfn) )
> get_page(mfn_to_page(ram_mfn, curr->domain));
> put_gfn(curr->domain, ram_gfn)
>
> /* replace all put_gfn in all exit paths by put_page */
>
> This will ensure the target page is live and sane while not holding the p2m lock.
> Xudong, did that make sense? Do you think you could try that and report back?
>
> Thanks!
> Andres
>
> >
> > (XEN) Xen call trace:
> > (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
> > (XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
> > (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
> > (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
> > (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
> > (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
> > (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
> > (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
> > (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
> > (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
> > (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
> > (XEN) [<ffff82c4801afd72>] hvm_hap_nested_page_fault+0x210/0x37f
> > (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
> >
> > Thanks,
> > -Xudong
> >
> >> -----Original Message-----
> >> From: Tim Deegan [mailto:tim@xen.org]
> >> Sent: Saturday, March 10, 2012 12:56 AM
> >> To: Andres Lagar-Cavilla
> >> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
> >> Xiantao; JBeulich@suse.com
> >> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
> >>
> >> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
> >> > >> I don't know about the event lock, but it seems unwise to call
> >> > >> in to handle_mmio with a gfn lock held. How about fixing the
> >> > >> other
> >> path?
> >> > >>
> >> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
> >> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
> >> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
> >> > >> @@ -1324,10 +1324,11 @@ int
> hvm_hap_nested_page_fault(unsigned l
> >> > >> if ( (p2mt == p2m_mmio_dm) ||
> >> > >> (access_w && (p2mt == p2m_ram_ro)) )
> >> > >> {
> >> > >> + put_gfn(p2m->domain, gfn);
> >> > >> if ( !handle_mmio() )
> >> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
> >> > >> rc = 1;
> >> > >> - goto out_put_gfn;
> >> > >> + goto out;
> >> > >> }
> >> > >>
> >> > >> #ifdef __x86_64__
> >> > >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
> >> > >>
> >> > >> out_put_gfn:
> >> > >> put_gfn(p2m->domain, gfn);
> >> > >> +out:
> >> > >> if ( paged )
> >> > >> p2m_mem_paging_populate(v->domain, gfn);
> >> > >> if ( req_ptr )
> >> > >
> >> > > Yes, that's fine to release the p2m lock earlier than handle_mmio.
> >> >
> >> > Ack
> >>
> >> OK, applied.
> >>
> >> Tim.
> >
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-14 7:12 ` Hao, Xudong
@ 2012-03-14 8:28 ` Zhang, Yang Z
2012-03-14 14:20 ` Andres Lagar-Cavilla
2012-03-14 15:10 ` Andres Lagar-Cavilla
2 siblings, 0 replies; 18+ messages in thread
From: Zhang, Yang Z @ 2012-03-14 8:28 UTC (permalink / raw)
To: Hao, Xudong, andres
Cc: Tim Deegan, xen-devel, Keir Fraser, Zhang, Xiantao, JBeulich@suse.com
The get_page() and put_page() should be used in pairs. You cannot call put_page() separately when get_page() is not called before.
best regards
yang
> -----Original Message-----
> From: xen-devel-bounces@lists.xen.org
> [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Hao, Xudong
> Sent: Wednesday, March 14, 2012 3:13 PM
> To: andres@lagarcavilla.org
> Cc: Keir Fraser; xen-devel@lists.xensource.com; Tim Deegan; JBeulich@suse.com;
> Zhang, Xiantao
> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>
> I prefer to the 2nd, I made a patch and testing show it works.
>
> diff -r 5d20d2f6ffed xen/arch/x86/hvm/emulate.c
> --- a/xen/arch/x86/hvm/emulate.c Fri Mar 09 16:54:24 2012 +0000
> +++ b/xen/arch/x86/hvm/emulate.c Wed Mar 14 15:11:52 2012 -0400
> @@ -60,20 +60,23 @@
> ioreq_t *p = get_ioreq(curr);
> unsigned long ram_gfn = paddr_to_pfn(ram_gpa);
> p2m_type_t p2mt;
> - mfn_t ram_mfn;
> + unsigned long ram_mfn;
> int rc;
>
> /* Check for paged out page */
> - ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
> + ram_mfn = mfn_x(get_gfn_unshare(curr->domain, ram_gfn, &p2mt));
> + if ( mfn_valid(ram_mfn) )
> + get_page(mfn_to_page(ram_mfn), curr->domain);
> + put_gfn(curr->domain, ram_gfn);
> if ( p2m_is_paging(p2mt) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> p2m_mem_paging_populate(curr->domain, ram_gfn);
> return X86EMUL_RETRY;
> }
> if ( p2m_is_shared(p2mt) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_RETRY;
> }
>
> @@ -87,7 +90,7 @@
> ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
> if ( dir == IOREQ_READ )
> memset(p_data, ~0, size);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -108,7 +111,7 @@
> unsigned int bytes = vio->mmio_large_write_bytes;
> if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
> }
> @@ -120,7 +123,7 @@
> {
> memcpy(p_data, &vio->mmio_large_read[addr - pa],
> size);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
> }
> @@ -134,7 +137,7 @@
> vio->io_state = HVMIO_none;
> if ( p_data == NULL )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
> goto finish_access;
> @@ -144,11 +147,11 @@
> (addr == (vio->mmio_large_write_pa +
> vio->mmio_large_write_bytes)) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_RETRY;
> }
> default:
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -156,7 +159,7 @@
> {
> gdprintk(XENLOG_WARNING, "WARNING: io already pending
> (%d)?\n",
> p->state);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -208,7 +211,7 @@
>
> if ( rc != X86EMUL_OKAY )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return rc;
> }
>
> @@ -244,7 +247,7 @@
> }
> }
>
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
>
>
> Thanks,
> -Xudong
>
>
> > -----Original Message-----
> > From: Andres Lagar-Cavilla [mailto:andres@lagarcavilla.org]
> > Sent: Wednesday, March 14, 2012 2:46 AM
> > To: Hao, Xudong
> > Cc: Tim Deegan; Keir Fraser; xen-devel@lists.xensource.com; Zhang, Xiantao;
> > JBeulich@suse.com
> > Subject: RE: [Xen-devel] Deadlocks by p2m_lock and event_lock
> >
> > > Hi, Tim and Andres
> > > The patch fix part of this issue. In handle_mmio, function
> > > hvmemul_do_io() is called and p2m lock was held again by calling
> > > get_gfn_unshare(), still trigger a deadlocks.
> >
> > Typically hvmemul_do_io gets the zero gfn, because in many cases that's the
> > 'rma_gpa' it is passed. However, in the case of mmio, and particularly stdvga,
> > ram_gpa is the data to be copied to the framebuffer. So it is in principle ok to
> > get_gfn in hvmemul_do_io.
> >
> > There are two solutions
> > 1. msix_capability_init does not call p2m_change_entry_type_global. See my
> > previous email. If we want to resync the
> > EPT/NPT/traditional/VTD/IOMMU/superduper TLBs, we can just do that
> > explicitly. I hope.
> >
> > 2. hvmemul_do_io does gets a ref on the mfn underlying ram_gpa, and holds
> > that for the critical section, instead of the p2m lock. One way to achieve this is
> >
> > /* Check for paged out page */
> > ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
> > if ( this or that )
> > { ... handle ... }
> > if ( mfn_valid(ram_mfn) )
> > get_page(mfn_to_page(ram_mfn, curr->domain));
> > put_gfn(curr->domain, ram_gfn)
> >
> > /* replace all put_gfn in all exit paths by put_page */
> >
> > This will ensure the target page is live and sane while not holding the p2m lock.
> > Xudong, did that make sense? Do you think you could try that and report back?
> >
> > Thanks!
> > Andres
> >
> > >
> > > (XEN) Xen call trace:
> > > (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
> > > (XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
> > > (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
> > > (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
> > > (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
> > > (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
> > > (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
> > > (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
> > > (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
> > > (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
> > > (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
> > > (XEN) [<ffff82c4801afd72>] hvm_hap_nested_page_fault+0x210/0x37f
> > > (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
> > >
> > > Thanks,
> > > -Xudong
> > >
> > >> -----Original Message-----
> > >> From: Tim Deegan [mailto:tim@xen.org]
> > >> Sent: Saturday, March 10, 2012 12:56 AM
> > >> To: Andres Lagar-Cavilla
> > >> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
> > >> Xiantao; JBeulich@suse.com
> > >> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
> > >>
> > >> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
> > >> > >> I don't know about the event lock, but it seems unwise to call
> > >> > >> in to handle_mmio with a gfn lock held. How about fixing the
> > >> > >> other
> > >> path?
> > >> > >>
> > >> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
> > >> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
> > >> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
> > >> > >> @@ -1324,10 +1324,11 @@ int
> > hvm_hap_nested_page_fault(unsigned l
> > >> > >> if ( (p2mt == p2m_mmio_dm) ||
> > >> > >> (access_w && (p2mt == p2m_ram_ro)) )
> > >> > >> {
> > >> > >> + put_gfn(p2m->domain, gfn);
> > >> > >> if ( !handle_mmio() )
> > >> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
> > >> > >> rc = 1;
> > >> > >> - goto out_put_gfn;
> > >> > >> + goto out;
> > >> > >> }
> > >> > >>
> > >> > >> #ifdef __x86_64__
> > >> > >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
> > >> > >>
> > >> > >> out_put_gfn:
> > >> > >> put_gfn(p2m->domain, gfn);
> > >> > >> +out:
> > >> > >> if ( paged )
> > >> > >> p2m_mem_paging_populate(v->domain, gfn);
> > >> > >> if ( req_ptr )
> > >> > >
> > >> > > Yes, that's fine to release the p2m lock earlier than handle_mmio.
> > >> >
> > >> > Ack
> > >>
> > >> OK, applied.
> > >>
> > >> Tim.
> > >
> >
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-13 18:26 ` Andres Lagar-Cavilla
@ 2012-03-14 9:20 ` Jan Beulich
2012-03-14 14:18 ` Andres Lagar-Cavilla
0 siblings, 1 reply; 18+ messages in thread
From: Jan Beulich @ 2012-03-14 9:20 UTC (permalink / raw)
To: andres; +Cc: Keir Fraser, xen-devel, Xudong Hao, Xiantao Zhang, Tim Deegan
>>> On 13.03.12 at 19:26, "Andres Lagar-Cavilla" <andres@lagarcavilla.org> wrote:
>> Hi, Tim and Andres
>> The patch fix part of this issue. In handle_mmio, function hvmemul_do_io()
>> is called and p2m lock was held again by calling get_gfn_unshare(), still
>> trigger a deadlocks.
>
> I have a question before I dive into lock untangling
>
> msix_capability_init ->
> p2m_change_entry_type_global(dev->domain, p2m_mmio_direct, p2m_mmio_direct);
>
> Huh? This achieves ... nothing. Almost. It flushes a bunch of TLBs, but
> that can be done with significantly less effort. Am I missing something?
Yes - the purpose of this isn't to flush any TLBs, but to enforce the
immediately preceding addition to the mmio_ro_ranges range set.
Jan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-14 9:20 ` Jan Beulich
@ 2012-03-14 14:18 ` Andres Lagar-Cavilla
0 siblings, 0 replies; 18+ messages in thread
From: Andres Lagar-Cavilla @ 2012-03-14 14:18 UTC (permalink / raw)
To: Jan Beulich; +Cc: Keir Fraser, xen-devel, Xudong Hao, Xiantao Zhang, Tim Deegan
>>>> On 13.03.12 at 19:26, "Andres Lagar-Cavilla" <andres@lagarcavilla.org>
>>>> wrote:
>>> Hi, Tim and Andres
>>> The patch fix part of this issue. In handle_mmio, function
>>> hvmemul_do_io()
>>> is called and p2m lock was held again by calling get_gfn_unshare(),
>>> still
>>> trigger a deadlocks.
>>
>> I have a question before I dive into lock untangling
>>
>> msix_capability_init ->
>> p2m_change_entry_type_global(dev->domain, p2m_mmio_direct,
>> p2m_mmio_direct);
>>
>> Huh? This achieves ... nothing. Almost. It flushes a bunch of TLBs, but
>> that can be done with significantly less effort. Am I missing something?
>
> Yes - the purpose of this isn't to flush any TLBs, but to enforce the
> immediately preceding addition to the mmio_ro_ranges range set.
Because p2m entries of type mmio_direct have their permissions
(re)computed as a function of rangesets. Got it. Thanks.
Andres
>
> Jan
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-14 7:12 ` Hao, Xudong
2012-03-14 8:28 ` Zhang, Yang Z
@ 2012-03-14 14:20 ` Andres Lagar-Cavilla
2012-03-14 15:10 ` Andres Lagar-Cavilla
2 siblings, 0 replies; 18+ messages in thread
From: Andres Lagar-Cavilla @ 2012-03-14 14:20 UTC (permalink / raw)
To: Hao, Xudong
Cc: Keir Fraser, xen-devel, Tim Deegan, JBeulich@suse.com, Zhang, Xiantao
> I prefer to the 2nd, I made a patch and testing show it works.
Not only is it preferable, as per Jan's email, it is the only way to go.
Thanks for putting together the patch and testing it. However, let me
rework it a bit (I'll add your Signed-off). If the new version works well,
then let's request to get it applied.
Cheers
Andres
>
> diff -r 5d20d2f6ffed xen/arch/x86/hvm/emulate.c
> --- a/xen/arch/x86/hvm/emulate.c Fri Mar 09 16:54:24 2012 +0000
> +++ b/xen/arch/x86/hvm/emulate.c Wed Mar 14 15:11:52 2012 -0400
> @@ -60,20 +60,23 @@
> ioreq_t *p = get_ioreq(curr);
> unsigned long ram_gfn = paddr_to_pfn(ram_gpa);
> p2m_type_t p2mt;
> - mfn_t ram_mfn;
> + unsigned long ram_mfn;
> int rc;
>
> /* Check for paged out page */
> - ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
> + ram_mfn = mfn_x(get_gfn_unshare(curr->domain, ram_gfn, &p2mt));
> + if ( mfn_valid(ram_mfn) )
> + get_page(mfn_to_page(ram_mfn), curr->domain);
> + put_gfn(curr->domain, ram_gfn);
> if ( p2m_is_paging(p2mt) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> p2m_mem_paging_populate(curr->domain, ram_gfn);
> return X86EMUL_RETRY;
> }
> if ( p2m_is_shared(p2mt) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_RETRY;
> }
>
> @@ -87,7 +90,7 @@
> ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
> if ( dir == IOREQ_READ )
> memset(p_data, ~0, size);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -108,7 +111,7 @@
> unsigned int bytes = vio->mmio_large_write_bytes;
> if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
> }
> @@ -120,7 +123,7 @@
> {
> memcpy(p_data, &vio->mmio_large_read[addr - pa],
> size);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
> }
> @@ -134,7 +137,7 @@
> vio->io_state = HVMIO_none;
> if ( p_data == NULL )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
> goto finish_access;
> @@ -144,11 +147,11 @@
> (addr == (vio->mmio_large_write_pa +
> vio->mmio_large_write_bytes)) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_RETRY;
> }
> default:
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -156,7 +159,7 @@
> {
> gdprintk(XENLOG_WARNING, "WARNING: io already pending (%d)?\n",
> p->state);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -208,7 +211,7 @@
>
> if ( rc != X86EMUL_OKAY )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return rc;
> }
>
> @@ -244,7 +247,7 @@
> }
> }
>
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
>
>
> Thanks,
> -Xudong
>
>
>> -----Original Message-----
>> From: Andres Lagar-Cavilla [mailto:andres@lagarcavilla.org]
>> Sent: Wednesday, March 14, 2012 2:46 AM
>> To: Hao, Xudong
>> Cc: Tim Deegan; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> Xiantao;
>> JBeulich@suse.com
>> Subject: RE: [Xen-devel] Deadlocks by p2m_lock and event_lock
>>
>> > Hi, Tim and Andres
>> > The patch fix part of this issue. In handle_mmio, function
>> > hvmemul_do_io() is called and p2m lock was held again by calling
>> > get_gfn_unshare(), still trigger a deadlocks.
>>
>> Typically hvmemul_do_io gets the zero gfn, because in many cases that's
>> the
>> 'rma_gpa' it is passed. However, in the case of mmio, and particularly
>> stdvga,
>> ram_gpa is the data to be copied to the framebuffer. So it is in
>> principle ok to
>> get_gfn in hvmemul_do_io.
>>
>> There are two solutions
>> 1. msix_capability_init does not call p2m_change_entry_type_global. See
>> my
>> previous email. If we want to resync the
>> EPT/NPT/traditional/VTD/IOMMU/superduper TLBs, we can just do that
>> explicitly. I hope.
>>
>> 2. hvmemul_do_io does gets a ref on the mfn underlying ram_gpa, and
>> holds
>> that for the critical section, instead of the p2m lock. One way to
>> achieve this is
>>
>> /* Check for paged out page */
>> ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
>> if ( this or that )
>> { ... handle ... }
>> if ( mfn_valid(ram_mfn) )
>> get_page(mfn_to_page(ram_mfn, curr->domain));
>> put_gfn(curr->domain, ram_gfn)
>>
>> /* replace all put_gfn in all exit paths by put_page */
>>
>> This will ensure the target page is live and sane while not holding the
>> p2m lock.
>> Xudong, did that make sense? Do you think you could try that and report
>> back?
>>
>> Thanks!
>> Andres
>>
>> >
>> > (XEN) Xen call trace:
>> > (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
>> > (XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
>> > (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
>> > (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
>> > (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
>> > (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
>> > (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
>> > (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
>> > (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
>> > (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
>> > (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
>> > (XEN) [<ffff82c4801afd72>] hvm_hap_nested_page_fault+0x210/0x37f
>> > (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
>> >
>> > Thanks,
>> > -Xudong
>> >
>> >> -----Original Message-----
>> >> From: Tim Deegan [mailto:tim@xen.org]
>> >> Sent: Saturday, March 10, 2012 12:56 AM
>> >> To: Andres Lagar-Cavilla
>> >> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> >> Xiantao; JBeulich@suse.com
>> >> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>> >>
>> >> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
>> >> > >> I don't know about the event lock, but it seems unwise to call
>> >> > >> in to handle_mmio with a gfn lock held. How about fixing the
>> >> > >> other
>> >> path?
>> >> > >>
>> >> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
>> >> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
>> >> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
>> >> > >> @@ -1324,10 +1324,11 @@ int
>> hvm_hap_nested_page_fault(unsigned l
>> >> > >> if ( (p2mt == p2m_mmio_dm) ||
>> >> > >> (access_w && (p2mt == p2m_ram_ro)) )
>> >> > >> {
>> >> > >> + put_gfn(p2m->domain, gfn);
>> >> > >> if ( !handle_mmio() )
>> >> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
>> >> > >> rc = 1;
>> >> > >> - goto out_put_gfn;
>> >> > >> + goto out;
>> >> > >> }
>> >> > >>
>> >> > >> #ifdef __x86_64__
>> >> > >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
>> >> > >>
>> >> > >> out_put_gfn:
>> >> > >> put_gfn(p2m->domain, gfn);
>> >> > >> +out:
>> >> > >> if ( paged )
>> >> > >> p2m_mem_paging_populate(v->domain, gfn);
>> >> > >> if ( req_ptr )
>> >> > >
>> >> > > Yes, that's fine to release the p2m lock earlier than
>> handle_mmio.
>> >> >
>> >> > Ack
>> >>
>> >> OK, applied.
>> >>
>> >> Tim.
>> >
>>
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-14 7:12 ` Hao, Xudong
2012-03-14 8:28 ` Zhang, Yang Z
2012-03-14 14:20 ` Andres Lagar-Cavilla
@ 2012-03-14 15:10 ` Andres Lagar-Cavilla
2012-03-15 2:19 ` Hao, Xudong
2 siblings, 1 reply; 18+ messages in thread
From: Andres Lagar-Cavilla @ 2012-03-14 15:10 UTC (permalink / raw)
To: Hao, Xudong
Cc: Keir Fraser, xen-devel, Tim Deegan, JBeulich@suse.com, Zhang, Xiantao
Can you give this a try? (Tim, Keir, Jan, if Xudong reports success, can
you please apply?)
Thanks,
Andres
# HG changeset patch
# User Andres Lagar-Cavilla <andres@lagarcavilla.org>
# Date 1331737660 14400
# Node ID fe10f0433f6279091c193127d95d4d39b44a72ed
# Parent 5d20d2f6ffed0a49f030f04a8870f1926babbcbf
x86/mm: Fix deadlock between p2m and event channel locks.
The hvm io emulation code holds the p2m lock for the duration of the
emulation, which may include sending an event to qemu. On a separate path,
map_domain_pirq grabs the event channel and p2m locks in opposite order.
Fix this by ensuring liveness of the ram_gfn used by io emulation, with a
page ref.
Reported-by: "Hao, Xudong" <xudong.hao@intel.com>
Signed-off-by: "Hao, Xudong" <xudong.hao@intel.com>
Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
diff -r 5d20d2f6ffed -r fe10f0433f62 xen/arch/x86/hvm/emulate.c
--- a/xen/arch/x86/hvm/emulate.c
+++ b/xen/arch/x86/hvm/emulate.c
@@ -77,6 +77,17 @@ static int hvmemul_do_io(
return X86EMUL_RETRY;
}
+ /* Maintain a ref on the mfn to ensure liveness. Put the gfn
+ * to avoid potential deadlock wrt event channel lock, later. */
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ if ( !get_page(mfn_to_page(mfn_x(ram_mfn)),
+ curr->domain) )
+ {
+ put_gfn(curr->domain, ram_gfn);
+ return X86EMUL_RETRY;
+ }
+ put_gfn(curr->domain, ram_gfn);
+
/*
* Weird-sized accesses have undefined behaviour: we discard writes
* and read all-ones.
@@ -87,7 +98,8 @@ static int hvmemul_do_io(
ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
if ( dir == IOREQ_READ )
memset(p_data, ~0, size);
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return X86EMUL_UNHANDLEABLE;
}
@@ -108,7 +120,8 @@ static int hvmemul_do_io(
unsigned int bytes = vio->mmio_large_write_bytes;
if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
{
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return X86EMUL_OKAY;
}
}
@@ -120,7 +133,8 @@ static int hvmemul_do_io(
{
memcpy(p_data, &vio->mmio_large_read[addr - pa],
size);
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return X86EMUL_OKAY;
}
}
@@ -134,7 +148,8 @@ static int hvmemul_do_io(
vio->io_state = HVMIO_none;
if ( p_data == NULL )
{
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return X86EMUL_UNHANDLEABLE;
}
goto finish_access;
@@ -144,11 +159,13 @@ static int hvmemul_do_io(
(addr == (vio->mmio_large_write_pa +
vio->mmio_large_write_bytes)) )
{
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return X86EMUL_RETRY;
}
default:
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return X86EMUL_UNHANDLEABLE;
}
@@ -156,7 +173,8 @@ static int hvmemul_do_io(
{
gdprintk(XENLOG_WARNING, "WARNING: io already pending (%d)?\n",
p->state);
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return X86EMUL_UNHANDLEABLE;
}
@@ -208,7 +226,8 @@ static int hvmemul_do_io(
if ( rc != X86EMUL_OKAY )
{
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return rc;
}
@@ -244,7 +263,8 @@ static int hvmemul_do_io(
}
}
- put_gfn(curr->domain, ram_gfn);
+ if ( mfn_valid(mfn_x(ram_mfn)) )
+ put_page(mfn_to_page(mfn_x(ram_mfn)));
return X86EMUL_OKAY;
}
> I prefer to the 2nd, I made a patch and testing show it works.
>
> diff -r 5d20d2f6ffed xen/arch/x86/hvm/emulate.c
> --- a/xen/arch/x86/hvm/emulate.c Fri Mar 09 16:54:24 2012 +0000
> +++ b/xen/arch/x86/hvm/emulate.c Wed Mar 14 15:11:52 2012 -0400
> @@ -60,20 +60,23 @@
> ioreq_t *p = get_ioreq(curr);
> unsigned long ram_gfn = paddr_to_pfn(ram_gpa);
> p2m_type_t p2mt;
> - mfn_t ram_mfn;
> + unsigned long ram_mfn;
> int rc;
>
> /* Check for paged out page */
> - ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
> + ram_mfn = mfn_x(get_gfn_unshare(curr->domain, ram_gfn, &p2mt));
> + if ( mfn_valid(ram_mfn) )
> + get_page(mfn_to_page(ram_mfn), curr->domain);
> + put_gfn(curr->domain, ram_gfn);
> if ( p2m_is_paging(p2mt) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> p2m_mem_paging_populate(curr->domain, ram_gfn);
> return X86EMUL_RETRY;
> }
> if ( p2m_is_shared(p2mt) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_RETRY;
> }
>
> @@ -87,7 +90,7 @@
> ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
> if ( dir == IOREQ_READ )
> memset(p_data, ~0, size);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -108,7 +111,7 @@
> unsigned int bytes = vio->mmio_large_write_bytes;
> if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
> }
> @@ -120,7 +123,7 @@
> {
> memcpy(p_data, &vio->mmio_large_read[addr - pa],
> size);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
> }
> @@ -134,7 +137,7 @@
> vio->io_state = HVMIO_none;
> if ( p_data == NULL )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
> goto finish_access;
> @@ -144,11 +147,11 @@
> (addr == (vio->mmio_large_write_pa +
> vio->mmio_large_write_bytes)) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_RETRY;
> }
> default:
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -156,7 +159,7 @@
> {
> gdprintk(XENLOG_WARNING, "WARNING: io already pending (%d)?\n",
> p->state);
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -208,7 +211,7 @@
>
> if ( rc != X86EMUL_OKAY )
> {
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return rc;
> }
>
> @@ -244,7 +247,7 @@
> }
> }
>
> - put_gfn(curr->domain, ram_gfn);
> + put_page(mfn_to_page(ram_mfn));
> return X86EMUL_OKAY;
> }
>
>
> Thanks,
> -Xudong
>
>
>> -----Original Message-----
>> From: Andres Lagar-Cavilla [mailto:andres@lagarcavilla.org]
>> Sent: Wednesday, March 14, 2012 2:46 AM
>> To: Hao, Xudong
>> Cc: Tim Deegan; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> Xiantao;
>> JBeulich@suse.com
>> Subject: RE: [Xen-devel] Deadlocks by p2m_lock and event_lock
>>
>> > Hi, Tim and Andres
>> > The patch fix part of this issue. In handle_mmio, function
>> > hvmemul_do_io() is called and p2m lock was held again by calling
>> > get_gfn_unshare(), still trigger a deadlocks.
>>
>> Typically hvmemul_do_io gets the zero gfn, because in many cases that's
>> the
>> 'rma_gpa' it is passed. However, in the case of mmio, and particularly
>> stdvga,
>> ram_gpa is the data to be copied to the framebuffer. So it is in
>> principle ok to
>> get_gfn in hvmemul_do_io.
>>
>> There are two solutions
>> 1. msix_capability_init does not call p2m_change_entry_type_global. See
>> my
>> previous email. If we want to resync the
>> EPT/NPT/traditional/VTD/IOMMU/superduper TLBs, we can just do that
>> explicitly. I hope.
>>
>> 2. hvmemul_do_io does gets a ref on the mfn underlying ram_gpa, and
>> holds
>> that for the critical section, instead of the p2m lock. One way to
>> achieve this is
>>
>> /* Check for paged out page */
>> ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
>> if ( this or that )
>> { ... handle ... }
>> if ( mfn_valid(ram_mfn) )
>> get_page(mfn_to_page(ram_mfn, curr->domain));
>> put_gfn(curr->domain, ram_gfn)
>>
>> /* replace all put_gfn in all exit paths by put_page */
>>
>> This will ensure the target page is live and sane while not holding the
>> p2m lock.
>> Xudong, did that make sense? Do you think you could try that and report
>> back?
>>
>> Thanks!
>> Andres
>>
>> >
>> > (XEN) Xen call trace:
>> > (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
>> > (XEN) [<ffff82c4801070d3>] notify_via_xen_event_channel+0x21/0x106
>> > (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
>> > (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
>> > (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
>> > (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
>> > (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
>> > (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
>> > (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
>> > (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
>> > (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
>> > (XEN) [<ffff82c4801afd72>] hvm_hap_nested_page_fault+0x210/0x37f
>> > (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
>> >
>> > Thanks,
>> > -Xudong
>> >
>> >> -----Original Message-----
>> >> From: Tim Deegan [mailto:tim@xen.org]
>> >> Sent: Saturday, March 10, 2012 12:56 AM
>> >> To: Andres Lagar-Cavilla
>> >> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> >> Xiantao; JBeulich@suse.com
>> >> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>> >>
>> >> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
>> >> > >> I don't know about the event lock, but it seems unwise to call
>> >> > >> in to handle_mmio with a gfn lock held. How about fixing the
>> >> > >> other
>> >> path?
>> >> > >>
>> >> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
>> >> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
>> >> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
>> >> > >> @@ -1324,10 +1324,11 @@ int
>> hvm_hap_nested_page_fault(unsigned l
>> >> > >> if ( (p2mt == p2m_mmio_dm) ||
>> >> > >> (access_w && (p2mt == p2m_ram_ro)) )
>> >> > >> {
>> >> > >> + put_gfn(p2m->domain, gfn);
>> >> > >> if ( !handle_mmio() )
>> >> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
>> >> > >> rc = 1;
>> >> > >> - goto out_put_gfn;
>> >> > >> + goto out;
>> >> > >> }
>> >> > >>
>> >> > >> #ifdef __x86_64__
>> >> > >> @@ -1379,6 +1380,7 @@ int hvm_hap_nested_page_fault(unsigned l
>> >> > >>
>> >> > >> out_put_gfn:
>> >> > >> put_gfn(p2m->domain, gfn);
>> >> > >> +out:
>> >> > >> if ( paged )
>> >> > >> p2m_mem_paging_populate(v->domain, gfn);
>> >> > >> if ( req_ptr )
>> >> > >
>> >> > > Yes, that's fine to release the p2m lock earlier than
>> handle_mmio.
>> >> >
>> >> > Ack
>> >>
>> >> OK, applied.
>> >>
>> >> Tim.
>> >
>>
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-14 15:10 ` Andres Lagar-Cavilla
@ 2012-03-15 2:19 ` Hao, Xudong
2012-03-15 3:37 ` Andres Lagar-Cavilla
2012-03-15 10:44 ` Tim Deegan
0 siblings, 2 replies; 18+ messages in thread
From: Hao, Xudong @ 2012-03-15 2:19 UTC (permalink / raw)
To: andres
Cc: Keir Fraser, xen-devel, Tim Deegan, JBeulich@suse.com, Zhang, Xiantao
Works by tested.
Thanks,
-Xudong
> -----Original Message-----
> From: Andres Lagar-Cavilla [mailto:andres@lagarcavilla.org]
> Sent: Wednesday, March 14, 2012 11:11 PM
> To: Hao, Xudong
> Cc: Tim Deegan; Keir Fraser; xen-devel@lists.xensource.com; Zhang, Xiantao;
> JBeulich@suse.com
> Subject: RE: [Xen-devel] Deadlocks by p2m_lock and event_lock
>
> Can you give this a try? (Tim, Keir, Jan, if Xudong reports success, can you
> please apply?)
>
> Thanks,
> Andres
>
> # HG changeset patch
> # User Andres Lagar-Cavilla <andres@lagarcavilla.org> # Date 1331737660
> 14400 # Node ID fe10f0433f6279091c193127d95d4d39b44a72ed
> # Parent 5d20d2f6ffed0a49f030f04a8870f1926babbcbf
> x86/mm: Fix deadlock between p2m and event channel locks.
>
> The hvm io emulation code holds the p2m lock for the duration of the emulation,
> which may include sending an event to qemu. On a separate path,
> map_domain_pirq grabs the event channel and p2m locks in opposite order.
>
> Fix this by ensuring liveness of the ram_gfn used by io emulation, with a page
> ref.
>
> Reported-by: "Hao, Xudong" <xudong.hao@intel.com>
> Signed-off-by: "Hao, Xudong" <xudong.hao@intel.com>
> Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
>
> diff -r 5d20d2f6ffed -r fe10f0433f62 xen/arch/x86/hvm/emulate.c
> --- a/xen/arch/x86/hvm/emulate.c
> +++ b/xen/arch/x86/hvm/emulate.c
> @@ -77,6 +77,17 @@ static int hvmemul_do_io(
> return X86EMUL_RETRY;
> }
>
> + /* Maintain a ref on the mfn to ensure liveness. Put the gfn
> + * to avoid potential deadlock wrt event channel lock, later. */
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + if ( !get_page(mfn_to_page(mfn_x(ram_mfn)),
> + curr->domain) )
> + {
> + put_gfn(curr->domain, ram_gfn);
> + return X86EMUL_RETRY;
> + }
> + put_gfn(curr->domain, ram_gfn);
> +
> /*
> * Weird-sized accesses have undefined behaviour: we discard writes
> * and read all-ones.
> @@ -87,7 +98,8 @@ static int hvmemul_do_io(
> ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
> if ( dir == IOREQ_READ )
> memset(p_data, ~0, size);
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -108,7 +120,8 @@ static int hvmemul_do_io(
> unsigned int bytes = vio->mmio_large_write_bytes;
> if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return X86EMUL_OKAY;
> }
> }
> @@ -120,7 +133,8 @@ static int hvmemul_do_io(
> {
> memcpy(p_data, &vio->mmio_large_read[addr - pa],
> size);
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return X86EMUL_OKAY;
> }
> }
> @@ -134,7 +148,8 @@ static int hvmemul_do_io(
> vio->io_state = HVMIO_none;
> if ( p_data == NULL )
> {
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return X86EMUL_UNHANDLEABLE;
> }
> goto finish_access;
> @@ -144,11 +159,13 @@ static int hvmemul_do_io(
> (addr == (vio->mmio_large_write_pa +
> vio->mmio_large_write_bytes)) )
> {
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return X86EMUL_RETRY;
> }
> default:
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -156,7 +173,8 @@ static int hvmemul_do_io(
> {
> gdprintk(XENLOG_WARNING, "WARNING: io already pending
> (%d)?\n",
> p->state);
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return X86EMUL_UNHANDLEABLE;
> }
>
> @@ -208,7 +226,8 @@ static int hvmemul_do_io(
>
> if ( rc != X86EMUL_OKAY )
> {
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return rc;
> }
>
> @@ -244,7 +263,8 @@ static int hvmemul_do_io(
> }
> }
>
> - put_gfn(curr->domain, ram_gfn);
> + if ( mfn_valid(mfn_x(ram_mfn)) )
> + put_page(mfn_to_page(mfn_x(ram_mfn)));
> return X86EMUL_OKAY;
> }
>
>
>
>
> > I prefer to the 2nd, I made a patch and testing show it works.
> >
> > diff -r 5d20d2f6ffed xen/arch/x86/hvm/emulate.c
> > --- a/xen/arch/x86/hvm/emulate.c Fri Mar 09 16:54:24 2012 +0000
> > +++ b/xen/arch/x86/hvm/emulate.c Wed Mar 14 15:11:52 2012 -0400
> > @@ -60,20 +60,23 @@
> > ioreq_t *p = get_ioreq(curr);
> > unsigned long ram_gfn = paddr_to_pfn(ram_gpa);
> > p2m_type_t p2mt;
> > - mfn_t ram_mfn;
> > + unsigned long ram_mfn;
> > int rc;
> >
> > /* Check for paged out page */
> > - ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
> > + ram_mfn = mfn_x(get_gfn_unshare(curr->domain, ram_gfn, &p2mt));
> > + if ( mfn_valid(ram_mfn) )
> > + get_page(mfn_to_page(ram_mfn), curr->domain);
> > + put_gfn(curr->domain, ram_gfn);
> > if ( p2m_is_paging(p2mt) )
> > {
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > p2m_mem_paging_populate(curr->domain, ram_gfn);
> > return X86EMUL_RETRY;
> > }
> > if ( p2m_is_shared(p2mt) )
> > {
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_RETRY;
> > }
> >
> > @@ -87,7 +90,7 @@
> > ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
> > if ( dir == IOREQ_READ )
> > memset(p_data, ~0, size);
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_UNHANDLEABLE;
> > }
> >
> > @@ -108,7 +111,7 @@
> > unsigned int bytes = vio->mmio_large_write_bytes;
> > if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
> > {
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_OKAY;
> > }
> > }
> > @@ -120,7 +123,7 @@
> > {
> > memcpy(p_data, &vio->mmio_large_read[addr - pa],
> > size);
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_OKAY;
> > }
> > }
> > @@ -134,7 +137,7 @@
> > vio->io_state = HVMIO_none;
> > if ( p_data == NULL )
> > {
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_UNHANDLEABLE;
> > }
> > goto finish_access;
> > @@ -144,11 +147,11 @@
> > (addr == (vio->mmio_large_write_pa +
> > vio->mmio_large_write_bytes)) )
> > {
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_RETRY;
> > }
> > default:
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_UNHANDLEABLE;
> > }
> >
> > @@ -156,7 +159,7 @@
> > {
> > gdprintk(XENLOG_WARNING, "WARNING: io already pending
> (%d)?\n",
> > p->state);
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_UNHANDLEABLE;
> > }
> >
> > @@ -208,7 +211,7 @@
> >
> > if ( rc != X86EMUL_OKAY )
> > {
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return rc;
> > }
> >
> > @@ -244,7 +247,7 @@
> > }
> > }
> >
> > - put_gfn(curr->domain, ram_gfn);
> > + put_page(mfn_to_page(ram_mfn));
> > return X86EMUL_OKAY;
> > }
> >
> >
> > Thanks,
> > -Xudong
> >
> >
> >> -----Original Message-----
> >> From: Andres Lagar-Cavilla [mailto:andres@lagarcavilla.org]
> >> Sent: Wednesday, March 14, 2012 2:46 AM
> >> To: Hao, Xudong
> >> Cc: Tim Deegan; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
> >> Xiantao; JBeulich@suse.com
> >> Subject: RE: [Xen-devel] Deadlocks by p2m_lock and event_lock
> >>
> >> > Hi, Tim and Andres
> >> > The patch fix part of this issue. In handle_mmio, function
> >> > hvmemul_do_io() is called and p2m lock was held again by calling
> >> > get_gfn_unshare(), still trigger a deadlocks.
> >>
> >> Typically hvmemul_do_io gets the zero gfn, because in many cases
> >> that's the 'rma_gpa' it is passed. However, in the case of mmio, and
> >> particularly stdvga, ram_gpa is the data to be copied to the
> >> framebuffer. So it is in principle ok to get_gfn in hvmemul_do_io.
> >>
> >> There are two solutions
> >> 1. msix_capability_init does not call p2m_change_entry_type_global.
> >> See my previous email. If we want to resync the
> >> EPT/NPT/traditional/VTD/IOMMU/superduper TLBs, we can just do that
> >> explicitly. I hope.
> >>
> >> 2. hvmemul_do_io does gets a ref on the mfn underlying ram_gpa, and
> >> holds that for the critical section, instead of the p2m lock. One way
> >> to achieve this is
> >>
> >> /* Check for paged out page */
> >> ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
> >> if ( this or that )
> >> { ... handle ... }
> >> if ( mfn_valid(ram_mfn) )
> >> get_page(mfn_to_page(ram_mfn, curr->domain));
> >> put_gfn(curr->domain, ram_gfn)
> >>
> >> /* replace all put_gfn in all exit paths by put_page */
> >>
> >> This will ensure the target page is live and sane while not holding
> >> the p2m lock.
> >> Xudong, did that make sense? Do you think you could try that and
> >> report back?
> >>
> >> Thanks!
> >> Andres
> >>
> >> >
> >> > (XEN) Xen call trace:
> >> > (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
> >> > (XEN) [<ffff82c4801070d3>]
> notify_via_xen_event_channel+0x21/0x106
> >> > (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
> >> > (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
> >> > (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
> >> > (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
> >> > (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
> >> > (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
> >> > (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
> >> > (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
> >> > (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
> >> > (XEN) [<ffff82c4801afd72>]
> hvm_hap_nested_page_fault+0x210/0x37f
> >> > (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
> >> >
> >> > Thanks,
> >> > -Xudong
> >> >
> >> >> -----Original Message-----
> >> >> From: Tim Deegan [mailto:tim@xen.org]
> >> >> Sent: Saturday, March 10, 2012 12:56 AM
> >> >> To: Andres Lagar-Cavilla
> >> >> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com;
> >> >> Zhang, Xiantao; JBeulich@suse.com
> >> >> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
> >> >>
> >> >> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
> >> >> > >> I don't know about the event lock, but it seems unwise to
> >> >> > >> call in to handle_mmio with a gfn lock held. How about
> >> >> > >> fixing the other
> >> >> path?
> >> >> > >>
> >> >> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
> >> >> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
> >> >> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
> >> >> > >> @@ -1324,10 +1324,11 @@ int
> >> hvm_hap_nested_page_fault(unsigned l
> >> >> > >> if ( (p2mt == p2m_mmio_dm) ||
> >> >> > >> (access_w && (p2mt == p2m_ram_ro)) )
> >> >> > >> {
> >> >> > >> + put_gfn(p2m->domain, gfn);
> >> >> > >> if ( !handle_mmio() )
> >> >> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
> >> >> > >> rc = 1;
> >> >> > >> - goto out_put_gfn;
> >> >> > >> + goto out;
> >> >> > >> }
> >> >> > >>
> >> >> > >> #ifdef __x86_64__
> >> >> > >> @@ -1379,6 +1380,7 @@ int
> hvm_hap_nested_page_fault(unsigned
> >> >> > >> l
> >> >> > >>
> >> >> > >> out_put_gfn:
> >> >> > >> put_gfn(p2m->domain, gfn);
> >> >> > >> +out:
> >> >> > >> if ( paged )
> >> >> > >> p2m_mem_paging_populate(v->domain, gfn);
> >> >> > >> if ( req_ptr )
> >> >> > >
> >> >> > > Yes, that's fine to release the p2m lock earlier than
> >> handle_mmio.
> >> >> >
> >> >> > Ack
> >> >>
> >> >> OK, applied.
> >> >>
> >> >> Tim.
> >> >
> >>
> >
> >
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-15 2:19 ` Hao, Xudong
@ 2012-03-15 3:37 ` Andres Lagar-Cavilla
2012-03-15 10:44 ` Tim Deegan
1 sibling, 0 replies; 18+ messages in thread
From: Andres Lagar-Cavilla @ 2012-03-15 3:37 UTC (permalink / raw)
To: Hao, Xudong
Cc: Keir Fraser, xen-devel, Tim Deegan, JBeulich@suse.com, Zhang, Xiantao
> Works by tested.
>
> Thanks,
Thanks to you!
Andres
> -Xudong
>
>
>> -----Original Message-----
>> From: Andres Lagar-Cavilla [mailto:andres@lagarcavilla.org]
>> Sent: Wednesday, March 14, 2012 11:11 PM
>> To: Hao, Xudong
>> Cc: Tim Deegan; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> Xiantao;
>> JBeulich@suse.com
>> Subject: RE: [Xen-devel] Deadlocks by p2m_lock and event_lock
>>
>> Can you give this a try? (Tim, Keir, Jan, if Xudong reports success, can
>> you
>> please apply?)
>>
>> Thanks,
>> Andres
>>
>> # HG changeset patch
>> # User Andres Lagar-Cavilla <andres@lagarcavilla.org> # Date 1331737660
>> 14400 # Node ID fe10f0433f6279091c193127d95d4d39b44a72ed
>> # Parent 5d20d2f6ffed0a49f030f04a8870f1926babbcbf
>> x86/mm: Fix deadlock between p2m and event channel locks.
>>
>> The hvm io emulation code holds the p2m lock for the duration of the
>> emulation,
>> which may include sending an event to qemu. On a separate path,
>> map_domain_pirq grabs the event channel and p2m locks in opposite order.
>>
>> Fix this by ensuring liveness of the ram_gfn used by io emulation, with
>> a page
>> ref.
>>
>> Reported-by: "Hao, Xudong" <xudong.hao@intel.com>
>> Signed-off-by: "Hao, Xudong" <xudong.hao@intel.com>
>> Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
>>
>> diff -r 5d20d2f6ffed -r fe10f0433f62 xen/arch/x86/hvm/emulate.c
>> --- a/xen/arch/x86/hvm/emulate.c
>> +++ b/xen/arch/x86/hvm/emulate.c
>> @@ -77,6 +77,17 @@ static int hvmemul_do_io(
>> return X86EMUL_RETRY;
>> }
>>
>> + /* Maintain a ref on the mfn to ensure liveness. Put the gfn
>> + * to avoid potential deadlock wrt event channel lock, later. */
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + if ( !get_page(mfn_to_page(mfn_x(ram_mfn)),
>> + curr->domain) )
>> + {
>> + put_gfn(curr->domain, ram_gfn);
>> + return X86EMUL_RETRY;
>> + }
>> + put_gfn(curr->domain, ram_gfn);
>> +
>> /*
>> * Weird-sized accesses have undefined behaviour: we discard writes
>> * and read all-ones.
>> @@ -87,7 +98,8 @@ static int hvmemul_do_io(
>> ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
>> if ( dir == IOREQ_READ )
>> memset(p_data, ~0, size);
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return X86EMUL_UNHANDLEABLE;
>> }
>>
>> @@ -108,7 +120,8 @@ static int hvmemul_do_io(
>> unsigned int bytes = vio->mmio_large_write_bytes;
>> if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
>> {
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return X86EMUL_OKAY;
>> }
>> }
>> @@ -120,7 +133,8 @@ static int hvmemul_do_io(
>> {
>> memcpy(p_data, &vio->mmio_large_read[addr - pa],
>> size);
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return X86EMUL_OKAY;
>> }
>> }
>> @@ -134,7 +148,8 @@ static int hvmemul_do_io(
>> vio->io_state = HVMIO_none;
>> if ( p_data == NULL )
>> {
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return X86EMUL_UNHANDLEABLE;
>> }
>> goto finish_access;
>> @@ -144,11 +159,13 @@ static int hvmemul_do_io(
>> (addr == (vio->mmio_large_write_pa +
>> vio->mmio_large_write_bytes)) )
>> {
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return X86EMUL_RETRY;
>> }
>> default:
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return X86EMUL_UNHANDLEABLE;
>> }
>>
>> @@ -156,7 +173,8 @@ static int hvmemul_do_io(
>> {
>> gdprintk(XENLOG_WARNING, "WARNING: io already pending
>> (%d)?\n",
>> p->state);
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return X86EMUL_UNHANDLEABLE;
>> }
>>
>> @@ -208,7 +226,8 @@ static int hvmemul_do_io(
>>
>> if ( rc != X86EMUL_OKAY )
>> {
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return rc;
>> }
>>
>> @@ -244,7 +263,8 @@ static int hvmemul_do_io(
>> }
>> }
>>
>> - put_gfn(curr->domain, ram_gfn);
>> + if ( mfn_valid(mfn_x(ram_mfn)) )
>> + put_page(mfn_to_page(mfn_x(ram_mfn)));
>> return X86EMUL_OKAY;
>> }
>>
>>
>>
>>
>> > I prefer to the 2nd, I made a patch and testing show it works.
>> >
>> > diff -r 5d20d2f6ffed xen/arch/x86/hvm/emulate.c
>> > --- a/xen/arch/x86/hvm/emulate.c Fri Mar 09 16:54:24 2012 +0000
>> > +++ b/xen/arch/x86/hvm/emulate.c Wed Mar 14 15:11:52 2012 -0400
>> > @@ -60,20 +60,23 @@
>> > ioreq_t *p = get_ioreq(curr);
>> > unsigned long ram_gfn = paddr_to_pfn(ram_gpa);
>> > p2m_type_t p2mt;
>> > - mfn_t ram_mfn;
>> > + unsigned long ram_mfn;
>> > int rc;
>> >
>> > /* Check for paged out page */
>> > - ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
>> > + ram_mfn = mfn_x(get_gfn_unshare(curr->domain, ram_gfn, &p2mt));
>> > + if ( mfn_valid(ram_mfn) )
>> > + get_page(mfn_to_page(ram_mfn), curr->domain);
>> > + put_gfn(curr->domain, ram_gfn);
>> > if ( p2m_is_paging(p2mt) )
>> > {
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > p2m_mem_paging_populate(curr->domain, ram_gfn);
>> > return X86EMUL_RETRY;
>> > }
>> > if ( p2m_is_shared(p2mt) )
>> > {
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_RETRY;
>> > }
>> >
>> > @@ -87,7 +90,7 @@
>> > ASSERT(p_data != NULL); /* cannot happen with a REP prefix */
>> > if ( dir == IOREQ_READ )
>> > memset(p_data, ~0, size);
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_UNHANDLEABLE;
>> > }
>> >
>> > @@ -108,7 +111,7 @@
>> > unsigned int bytes = vio->mmio_large_write_bytes;
>> > if ( (addr >= pa) && ((addr + size) <= (pa + bytes)) )
>> > {
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_OKAY;
>> > }
>> > }
>> > @@ -120,7 +123,7 @@
>> > {
>> > memcpy(p_data, &vio->mmio_large_read[addr - pa],
>> > size);
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_OKAY;
>> > }
>> > }
>> > @@ -134,7 +137,7 @@
>> > vio->io_state = HVMIO_none;
>> > if ( p_data == NULL )
>> > {
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_UNHANDLEABLE;
>> > }
>> > goto finish_access;
>> > @@ -144,11 +147,11 @@
>> > (addr == (vio->mmio_large_write_pa +
>> > vio->mmio_large_write_bytes)) )
>> > {
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_RETRY;
>> > }
>> > default:
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_UNHANDLEABLE;
>> > }
>> >
>> > @@ -156,7 +159,7 @@
>> > {
>> > gdprintk(XENLOG_WARNING, "WARNING: io already pending
>> (%d)?\n",
>> > p->state);
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_UNHANDLEABLE;
>> > }
>> >
>> > @@ -208,7 +211,7 @@
>> >
>> > if ( rc != X86EMUL_OKAY )
>> > {
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return rc;
>> > }
>> >
>> > @@ -244,7 +247,7 @@
>> > }
>> > }
>> >
>> > - put_gfn(curr->domain, ram_gfn);
>> > + put_page(mfn_to_page(ram_mfn));
>> > return X86EMUL_OKAY;
>> > }
>> >
>> >
>> > Thanks,
>> > -Xudong
>> >
>> >
>> >> -----Original Message-----
>> >> From: Andres Lagar-Cavilla [mailto:andres@lagarcavilla.org]
>> >> Sent: Wednesday, March 14, 2012 2:46 AM
>> >> To: Hao, Xudong
>> >> Cc: Tim Deegan; Keir Fraser; xen-devel@lists.xensource.com; Zhang,
>> >> Xiantao; JBeulich@suse.com
>> >> Subject: RE: [Xen-devel] Deadlocks by p2m_lock and event_lock
>> >>
>> >> > Hi, Tim and Andres
>> >> > The patch fix part of this issue. In handle_mmio, function
>> >> > hvmemul_do_io() is called and p2m lock was held again by calling
>> >> > get_gfn_unshare(), still trigger a deadlocks.
>> >>
>> >> Typically hvmemul_do_io gets the zero gfn, because in many cases
>> >> that's the 'rma_gpa' it is passed. However, in the case of mmio, and
>> >> particularly stdvga, ram_gpa is the data to be copied to the
>> >> framebuffer. So it is in principle ok to get_gfn in hvmemul_do_io.
>> >>
>> >> There are two solutions
>> >> 1. msix_capability_init does not call p2m_change_entry_type_global.
>> >> See my previous email. If we want to resync the
>> >> EPT/NPT/traditional/VTD/IOMMU/superduper TLBs, we can just do that
>> >> explicitly. I hope.
>> >>
>> >> 2. hvmemul_do_io does gets a ref on the mfn underlying ram_gpa, and
>> >> holds that for the critical section, instead of the p2m lock. One way
>> >> to achieve this is
>> >>
>> >> /* Check for paged out page */
>> >> ram_mfn = get_gfn_unshare(curr->domain, ram_gfn, &p2mt);
>> >> if ( this or that )
>> >> { ... handle ... }
>> >> if ( mfn_valid(ram_mfn) )
>> >> get_page(mfn_to_page(ram_mfn, curr->domain));
>> >> put_gfn(curr->domain, ram_gfn)
>> >>
>> >> /* replace all put_gfn in all exit paths by put_page */
>> >>
>> >> This will ensure the target page is live and sane while not holding
>> >> the p2m lock.
>> >> Xudong, did that make sense? Do you think you could try that and
>> >> report back?
>> >>
>> >> Thanks!
>> >> Andres
>> >>
>> >> >
>> >> > (XEN) Xen call trace:
>> >> > (XEN) [<ffff82c4801261a3>] _spin_lock+0x1b/0xa8
>> >> > (XEN) [<ffff82c4801070d3>]
>> notify_via_xen_event_channel+0x21/0x106
>> >> > (XEN) [<ffff82c4801b6883>] hvm_buffered_io_send+0x1f1/0x21b
>> >> > (XEN) [<ffff82c4801bbd3a>] stdvga_intercept_mmio+0x491/0x4c7
>> >> > (XEN) [<ffff82c4801b5d58>] hvm_io_intercept+0x218/0x244
>> >> > (XEN) [<ffff82c4801aa931>] hvmemul_do_io+0x55a/0x716
>> >> > (XEN) [<ffff82c4801aab1a>] hvmemul_do_mmio+0x2d/0x2f
>> >> > (XEN) [<ffff82c4801ab239>] hvmemul_write+0x181/0x1a2
>> >> > (XEN) [<ffff82c4801963f0>] x86_emulate+0xcad3/0xfbdf
>> >> > (XEN) [<ffff82c4801a9d2e>] hvm_emulate_one+0x120/0x1af
>> >> > (XEN) [<ffff82c4801b63cb>] handle_mmio+0x4e/0x1d1
>> >> > (XEN) [<ffff82c4801afd72>]
>> hvm_hap_nested_page_fault+0x210/0x37f
>> >> > (XEN) [<ffff82c4801d2419>] vmx_vmexit_handler+0x1523/0x17d0
>> >> >
>> >> > Thanks,
>> >> > -Xudong
>> >> >
>> >> >> -----Original Message-----
>> >> >> From: Tim Deegan [mailto:tim@xen.org]
>> >> >> Sent: Saturday, March 10, 2012 12:56 AM
>> >> >> To: Andres Lagar-Cavilla
>> >> >> Cc: Hao, Xudong; Keir Fraser; xen-devel@lists.xensource.com;
>> >> >> Zhang, Xiantao; JBeulich@suse.com
>> >> >> Subject: Re: [Xen-devel] Deadlocks by p2m_lock and event_lock
>> >> >>
>> >> >> At 08:29 -0800 on 09 Mar (1331281767), Andres Lagar-Cavilla wrote:
>> >> >> > >> I don't know about the event lock, but it seems unwise to
>> >> >> > >> call in to handle_mmio with a gfn lock held. How about
>> >> >> > >> fixing the other
>> >> >> path?
>> >> >> > >>
>> >> >> > >> diff -r 04673ecb9d78 xen/arch/x86/hvm/hvm.c
>> >> >> > >> --- a/xen/arch/x86/hvm/hvm.c Thu Mar 08 16:40:05 2012 +0000
>> >> >> > >> +++ b/xen/arch/x86/hvm/hvm.c Fri Mar 09 11:15:25 2012 +0000
>> >> >> > >> @@ -1324,10 +1324,11 @@ int
>> >> hvm_hap_nested_page_fault(unsigned l
>> >> >> > >> if ( (p2mt == p2m_mmio_dm) ||
>> >> >> > >> (access_w && (p2mt == p2m_ram_ro)) )
>> >> >> > >> {
>> >> >> > >> + put_gfn(p2m->domain, gfn);
>> >> >> > >> if ( !handle_mmio() )
>> >> >> > >> hvm_inject_exception(TRAP_gp_fault, 0, 0);
>> >> >> > >> rc = 1;
>> >> >> > >> - goto out_put_gfn;
>> >> >> > >> + goto out;
>> >> >> > >> }
>> >> >> > >>
>> >> >> > >> #ifdef __x86_64__
>> >> >> > >> @@ -1379,6 +1380,7 @@ int
>> hvm_hap_nested_page_fault(unsigned
>> >> >> > >> l
>> >> >> > >>
>> >> >> > >> out_put_gfn:
>> >> >> > >> put_gfn(p2m->domain, gfn);
>> >> >> > >> +out:
>> >> >> > >> if ( paged )
>> >> >> > >> p2m_mem_paging_populate(v->domain, gfn);
>> >> >> > >> if ( req_ptr )
>> >> >> > >
>> >> >> > > Yes, that's fine to release the p2m lock earlier than
>> >> handle_mmio.
>> >> >> >
>> >> >> > Ack
>> >> >>
>> >> >> OK, applied.
>> >> >>
>> >> >> Tim.
>> >> >
>> >>
>> >
>> >
>>
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: Deadlocks by p2m_lock and event_lock
2012-03-15 2:19 ` Hao, Xudong
2012-03-15 3:37 ` Andres Lagar-Cavilla
@ 2012-03-15 10:44 ` Tim Deegan
1 sibling, 0 replies; 18+ messages in thread
From: Tim Deegan @ 2012-03-15 10:44 UTC (permalink / raw)
To: Hao, Xudong
Cc: xen-devel, Keir Fraser, andres, JBeulich@suse.com, Zhang, Xiantao
At 02:19 +0000 on 15 Mar (1331777960), Hao, Xudong wrote:
> Works by tested.
Righto. Applied, thanks.
Tim.
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2012-03-15 10:44 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-03-09 10:58 Deadlocks by p2m_lock and event_lock Hao, Xudong
2012-03-09 11:20 ` Tim Deegan
2012-03-09 11:44 ` Hao, Xudong
2012-03-09 16:29 ` Andres Lagar-Cavilla
2012-03-09 16:55 ` Tim Deegan
2012-03-13 7:51 ` Hao, Xudong
2012-03-13 15:27 ` Andres Lagar-Cavilla
2012-03-13 18:26 ` Andres Lagar-Cavilla
2012-03-14 9:20 ` Jan Beulich
2012-03-14 14:18 ` Andres Lagar-Cavilla
2012-03-13 18:45 ` Andres Lagar-Cavilla
2012-03-14 7:12 ` Hao, Xudong
2012-03-14 8:28 ` Zhang, Yang Z
2012-03-14 14:20 ` Andres Lagar-Cavilla
2012-03-14 15:10 ` Andres Lagar-Cavilla
2012-03-15 2:19 ` Hao, Xudong
2012-03-15 3:37 ` Andres Lagar-Cavilla
2012-03-15 10:44 ` Tim Deegan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.