Re: VMX test report for Xen 4.5-unstable-C/S 28634

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

create !
title -1 "linux, S3 resume of PVHVM fails - missing call to xen_arch_post_suspend?"

Looking at the stack I think we are missing these calls:

86         xen_arch_post_suspend(si->cancelled);                                   
 87         gnttab_resume();                                                        
 88                                                                                 
 89         if (!si->cancelled) {                                                   
 90                 xen_irq_resume();                                               
 91                 xen_timer_resume();                                             
 92         }                                                                       G

which are called via xen_suspend but I don't see them being
hooked up in the generic suspend/resume calls - that is
hooked up via register_syscore_ops. Perhaps adding them
in would do it.

On Wed, Aug 06, 2014 at 07:57:31AM +0000, Hu, Robert wrote:
> Sorry Konrad for late response. It was nearly swallowed up until Chao reminds me.
> > -----Original Message-----
> > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > Sent: Monday, August 4, 2014 10:00 PM
> > To: Hu, Robert
> > Cc: xen-devel@lists.xen.org; Zhou, Chao
> > Subject: Re: [Xen-devel] VMX test report for Xen 4.5-unstable-C/S 28634
> > 
> > On Mon, Mar 31, 2014 at 08:52:54AM +0000, Hu, Robert wrote:
> > > Hi All,
> > > This is a report based on our testing for Xen 4.5-unstable on Intel platforms.
> > > Test environment:
> > > Xen: Xen 4.5-unstable with qemu-upstream-unstable.git
> > > Changeset: 28634:b75cbf39ebe4
> > > Dom0: Linux kernel 3.13.7
> > > Hardware: Intel Sandy Bridge, Ivy Bridge, Haswell
> > >
> > > Newly-and-Open bugs (0):
> > >
> > > Reopened bug(1)
> > > 1. Guest hang after resume from S3
> > >   http://bugzilla-archived.xenproject.org//bugzilla/show_bug.cgi?id=1828
> > 
> > Hey Hu,
> > 
> > Considering that Liu is no long working for Intel and in the past he
> > had been working on this bug:
> >  - Is there somebody else from Intel who is looking at this?
> Not until now we'll get some one look into.
> >  - Is this still an issue? (It looks like yes - as you have to compile the guest
> >    without CONFIG_XEN_PVHVM? - but that implies a Linux issue not
> >    hypervisor - btw in 3.17 there will be an 'xen_nopv' bootline parameter
> >    that will disable the PV drivers/codepaths in HVM guests).
> > 
> > If this is an issue, could you post the details fully on xen-devel so
> > that we can open a tracker bug instead of using the bugzilla system?
> > 
> Yes, this is still an issue, even with CONFIG_XEN_PVHVM=y. I just retest this on latest Xen commit 8b24b07e, with Dom0 stable kernel 3.15.8, as well as guest kernel. Attached is guest kernel config file.
> > Thank you!
> 
> Environment:
> ------------
> Service Arch (ia32/ia32e/IA64): ia32e
> Guest Arch (ia32/ia32e/IA64): ia32e
> Guest OS Type (Linux/Windows): Linux
> Commit No.: 8b24b07e  
> Hardware: Ivytown-EP
> Other:
> 
> Bug detailed description:
> --------------------------
> Guest hang after resume from S3.  We see call trace from guest console after we 'xl trigger $domid s3resume'.
> 
> 
> Reproduce steps:
> ----------------
> 1. Start a rhel6u5 guest, with kernel 3.15.8 in our test. 
> 2. Suspend guest via 'echo mem > /sys/power/state' command.
> 3. Resume guest from dom0 via 'xl trigger $domid s3resume' command.
> 4. Verify guest works fine via ping and ssh from dom0.
> 5. Check guest's base info(such as ifconfig).
> 
> Current result:
> ----------------
> Step 4:
> Guest isn't successfully awaken. We see call trace from guest console.
> 
> 
> 
> Expected result:
> ----------------
> Guest can be waked up and works fine.
> 
> 
> Call trace we captured
> -----------------
> Last login: Wed Aug  6 14:59:59 on ttyS0
> [root@rhel6 ~]# [   73.356160] PM: Syncing filesystems ... done.
> [   73.525388] Freezing user space processes ... (elapsed 0.001 seconds) done.
> [   73.534449] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
> [   73.545581] Suspending console(s) (use no_console_suspend to debug)
> [   73.554958] PM: suspend of devices complete after 3.652 msecs
> [   73.555104] PM: late suspend of devices complete after 0.144 msecs
> [   73.559043] PM: noirq suspend of devices complete after 3.937 msecs
> [   73.559046] ACPI: Preparing to enter system sleep state S3
> [   73.559183] PM: Saving platform NVS memory
> [   73.559183] Disabling non-boot CPUs ...
> [   73.560863] smpboot: CPU 1 is now offline
> [   73.562594] smpboot: CPU 2 is now offline
> [   73.564262] smpboot: CPU 3 is now offline
> [   73.565220] ACPI: Low-level resume complete
> [   73.565220] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S0_] (20140214/hwxface-580)
> [   73.565220] PM: Restoring platform NVS memory
> [   73.565220] Enabling non-boot CPUs ...
> [   73.565220] x86: Booting SMP configuration:
> [   73.565220] smpboot: Booting Node 0 Processor 1 APIC 0x2
> [   73.578993] cpu 1 spinlock event irq 74
> [   73.579214] CPU1 is up
> [   73.579316] smpboot: Booting Node 0 Processor 2 APIC 0x4
> [   73.580206] cpu 2 spinlock event irq 79
> [   73.593680] CPU2 is up
> [   73.593781] smpboot: Booting Node 0 Processor 3 APIC 0x6
> [   73.594672] cpu 3 spinlock event irq 84
> [   73.608201] CPU3 is up
> [   73.630088] ACPI: Waking up from system sleep state S3
> [   73.641880] PM: noirq resume of devices complete after 11.456 msecs
> [   73.642040] PM: early resume of devices complete after 0.082 msecs
> [   73.642124] pci 0000:00:01.0: PIIX3: Enabling Passive Release
> [   73.644845] xen:grant_table: WARNING: g.e. 0x324 still in use!
> [   73.644849] xen:grant_table: WARNING: g.e. 0x341 still in use!
> [   73.644850] xen:grant_table: WARNING: g.e. 0x308 still in use!
> [   73.644852] xen:grant_table: WARNING: g.e. 0x336 still in use!
> [   73.644853] xen:grant_table: WARNING: g.e. 0x30d still in use!
> [   73.644854] xen:grant_table: WARNING: g.e. 0x349 still in use!
> [   73.644856] xen:grant_table: WARNING: g.e. 0x334 still in use!
> [   73.644857] xen:grant_table: WARNING: g.e. 0x306 still in use!
> [   73.644858] xen:grant_table: WARNING: g.e. 0x320 still in use!
> [   73.644859] xen:grant_table: WARNING: g.e. 0x9 still in use!
> [   73.644861] xen:grant_table: WARNING: g.e. 0x314 still in use!
> [   73.644862] xen:grant_table: WARNING: g.e. 0x340 still in use!
> [   73.644863] xen:grant_table: WARNING: g.e. 0x355 still in use!
> [   73.644864] xen:grant_table: WARNING: g.e. 0x305 still in use!
> [   73.644866] xen:grant_table: WARNING: g.e. 0x35a still in use!
> [   73.644867] xen:grant_table: WARNING: g.e. 0x31a still in use!
> [   73.644868] xen:grant_table: WARNING: g.e. 0x32c still in use!
> [   73.644869] xen:grant_table: WARNING: g.e. 0x317 still in use!
> [   73.644870] xen:grant_table: WARNING: g.e. 0x304 still in use!
> [   73.644871] xen:grant_table: WARNING: g.e. 0x303 still in use!
> [   73.644872] xen:grant_table: WARNING: g.e. 0x32f still in use!
> [   73.644874] xen:grant_table: WARNING: g.e. 0x30e still in use!
> [   73.644875] xen:grant_table: WARNING: g.e. 0x328 still in use!
> [   73.644876] xen:grant_table: WARNING: g.e. 0x33f still in use!
> [   73.644878] xen:grant_table: WARNING: g.e. 0x31b still in use!
> [   73.644879] xen:grant_table: WARNING: g.e. 0x333 still in use!
> [   73.644880] xen:grant_table: WARNING: g.e. 0x30b still in use!
> [   73.644881] xen:grant_table: WARNING: g.e. 0x337 still in use!
> [   73.644882] xen:grant_table: WARNING: g.e. 0x348 still in use!
> [   73.644883] xen:grant_table: WARNING: g.e. 0x32a still in use!
> [   73.644884] xen:grant_table: WARNING: g.e. 0x342 still in use!
> [   73.644885] xen:grant_table: WARNING: g.e. 0x31e still in use!
> [   73.644887] xen:grant_table: WARNING: g.e. 0x31d still in use!
> [   73.644891] xen:grant_table: WARNING: g.e. 0x30f still in use!
> [   73.644892] xen:grant_table: WARNING: g.e. 0x35c still in use!
> [   73.644893] xen:grant_table: WARNING: g.e. 0x311 still in use!
> [   73.644894] xen:grant_table: WARNING: g.e. 0x32d still in use!
> [   73.644896] xen:grant_table: WARNING: g.e. 0x31c still in use!
> [   73.644897] xen:grant_table: WARNING: g.e. 0x335 still in use!
> [   73.644898] xen:grant_table: WARNING: g.e. 0x339 still in use!
> [   73.644899] xen:grant_table: WARNING: g.e. 0x307 still in use!
> [   73.644900] xen:grant_table: WARNING: g.e. 0x34a still in use!
> [   73.644901] xen:grant_table: WARNING: g.e. 0x330 still in use!
> [   73.644902] xen:grant_table: WARNING: g.e. 0x316 still in use!
> [   73.644903] xen:grant_table: WARNING: g.e. 0x358 still in use!
> [   73.644904] xen:grant_table: WARNING: g.e. 0x33a still in use!
> [   73.644905] xen:grant_table: WARNING: g.e. 0x329 still in use!
> [   73.644907] xen:grant_table: WARNING: g.e. 0x30c still in use!
> [   73.644908] xen:grant_table: WARNING: g.e. 0x352 still in use!
> [   73.644909] xen:grant_table: WARNING: g.e. 0x325 still in use!
> [   73.644910] xen:grant_table: WARNING: g.e. 0x344 still in use!
> [   73.644915] xen:grant_table: WARNING: g.e. 0x338 still in use!
> [   73.644916] xen:grant_table: WARNING: g.e. 0x331 still in use!
> [   73.644917] xen:grant_table: WARNING: g.e. 0x359 still in use!
> [   73.644918] xen:grant_table: WARNING: g.e. 0x332 still in use!
> [   73.644919] xen:grant_table: WARNING: g.e. 0x33b still in use!
> [   73.644920] xen:grant_table: WARNING: g.e. 0x322 still in use!
> [   73.644921] xen:grant_table: WARNING: g.e. 0x35b still in use!
> [   73.644923] xen:grant_table: WARNING: g.e. 0x30a still in use!
> [   73.644924] xen:grant_table: WARNING: g.e. 0x312 still in use!
> [   73.644925] xen:grant_table: WARNING: g.e. 0x326 still in use!
> [   73.644926] xen:grant_table: WARNING: g.e. 0x33d still in use!
> [   73.644927] xen:grant_table: WARNING: g.e. 0x310 still in use!
> [   73.644928] xen:grant_table: WARNING: g.e. 0x347 still in use!
> [   73.644929] xen:grant_table: WARNING: g.e. 0x309 still in use!
> [   73.644931] xen:grant_table: WARNING: g.e. 0x323 still in use!
> [   73.644932] xen:grant_table: WARNING: g.e. 0x351 still in use!
> [   73.644933] xen:grant_table: WARNING: g.e. 0x346 still in use!
> [   73.644934] xen:grant_table: WARNING: g.e. 0x33e still in use!
> [   73.644937] xen:grant_table: WARNING: g.e. 0x34c still in use!
> [   73.644938] xen:grant_table: WARNING: g.e. 0x34b still in use!
> [   73.644939] xen:grant_table: WARNING: g.e. 0x32e still in use!
> [   73.644940] xen:grant_table: WARNING: g.e. 0x319 still in use!
> [   73.644941] xen:grant_table: WARNING: g.e. 0x34d still in use!
> [   73.644942] xen:grant_table: WARNING: g.e. 0x34e still in use!
> [   73.644943] xen:grant_table: WARNING: g.e. 0x353 still in use!
> [   73.644945] xen:grant_table: WARNING: g.e. 0x343 still in use!
> [   73.644945] xen:grant_table: WARNING: g.e. 0x32b still in use!
> [   73.644947] xen:grant_table: WARNING: g.e. 0x318 still in use!
> [   73.644948] xen:grant_table: WARNING: g.e. 0x313 still in use!
> [   73.644949] xen:grant_table: WARNING: g.e. 0x354 still in use!
> [   73.644950] xen:grant_table: WARNING: g.e. 0x315 still in use!
> [   73.644951] xen:grant_table: WARNING: g.e. 0x31f still in use!
> [   73.644952] xen:grant_table: WARNING: g.e. 0x33c still in use!
> [   73.644953] xen:grant_table: WARNING: g.e. 0x357 still in use!
> [   73.644954] xen:grant_table: WARNING: g.e. 0x356 still in use!
> [   73.644957] xen:grant_table: WARNING: g.e. 0x321 still in use!
> [   73.644958] xen:grant_table: WARNING: g.e. 0x345 still in use!
> [   73.644959] xen:grant_table: WARNING: g.e. 0x350 still in use!
> [   73.644960] xen:grant_table: WARNING: g.e. 0x34f still in use!
> [   73.644961] xen:grant_table: WARNING: g.e. 0x327 still in use!
> [   73.645030] xen:grant_table: WARNING: g.e. 0x8 still in use!
> [   73.650803] xen:grant_table: WARNING: g.e. 0x300 still in use!
> [   73.650804] xen:grant_table: WARNING: g.e. 0x301 still in use!
> [   73.663230] PM: resume of devices complete after 21.188 msecs
> [   74.350929] Restarting tasks ... done.
> [   94.578054] INFO: rcu_sched detected stalls on CPUs/tasks: { 1} (detected by 0, t=21002 jiffies, g=716, c=715, q=174)
> [   94.579048] sending NMI to all CPUs:
> [   94.579048] NMI backtrace for cpu 0
> [   94.579048] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.15.8 #1
> [   94.579048] Hardware name: Xen HVM domU, BIOS 4.5-unstable 08/04/2014
> [   94.579048] task: ffffffff81c1b460 ti: ffffffff81c00000 task.ti: ffffffff81c00000
> [   94.579048] RIP: 0010:[<ffffffff81099640>]  [<ffffffff81099640>] default_send_IPI_mask_sequence_phys+0xc0/0x130
> [   94.579048] RSP: 0018:ffff88000d603ba8  EFLAGS: 00000046
> [   94.579048] RAX: 0000000000000400 RBX: 0000000000000002 RCX: 0000000000000000
> [   94.579048] RDX: 0000000000000000 RSI: 0000000000000040 RDI: 0000000000000300
> [   94.579048] RBP: ffff88000d603be8 R08: ffffffff81cad508 R09: 0000000000000000
> [   94.579048] R10: 00000000000002f1 R11: 00000000000002f1 R12: 000000000000b024
> [   94.579048] R13: ffffffff81cad508 R14: 0000000000000400 R15: 0000000000000082
> [   94.579048] FS:  0000000000000000(0000) GS:ffff88000d600000(0000) knlGS:0000000000000000
> [   94.579048] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   94.579048] CR2: 0000000000ac54c0 CR3: 000000000b276000 CR4: 00000000001406f0
> [   94.579048] Stack:
> [   94.579048]  0000000000000000 0000000000000000 ffff88000d603bc8 0000000000000000
> [   94.579048]  ffffffff81c38fc0 ffffffff81c39080 0000000000000000 ffffffff81cad500
> [   94.579048]  ffff88000d603bf8 ffffffff8109d522 ffff88000d603c18 ffffffff810996f7
> [   94.579048] Call Trace:
> [   94.579048]  <IRQ> 
> [   94.579048]  [<ffffffff8109d522>] physflat_send_IPI_all+0x12/0x20
> [   94.579048]  [<ffffffff810996f7>] arch_trigger_all_cpu_backtrace+0x47/0x80
> [   94.579048]  [<ffffffff81104bd2>] rcu_check_callbacks+0x692/0x6b0
> [   94.579048]  [<ffffffff810be763>] update_process_times+0x43/0x80
> [   94.579048]  [<ffffffff8111137c>] tick_sched_handle+0x3c/0x50
> [   94.579048]  [<ffffffff8111158f>] tick_sched_timer+0x4f/0x90
> [   94.579048]  [<ffffffff810d8dff>] __run_hrtimer+0x8f/0x1e0
> [   94.579048]  [<ffffffff81111540>] ? tick_nohz_handler+0xc0/0xc0
> [   94.579048]  [<ffffffff810d91f6>] hrtimer_interrupt+0x106/0x280
> [   94.579048]  [<ffffffff81058baf>] xen_timer_interrupt+0x2f/0x130
> [   94.579048]  [<ffffffff810fc9ed>] handle_irq_event_percpu+0x5d/0x1f0
> [   94.579048]  [<ffffffff810ffab3>] handle_percpu_irq+0x43/0x60
> [   94.579048]  [<ffffffff810fc932>] generic_handle_irq+0x22/0x30
> [   94.579048]  [<ffffffff81357aec>] evtchn_fifo_handle_events+0x14c/0x170
> [   94.579048]  [<ffffffff81354c78>] __xen_evtchn_do_upcall+0x48/0x90
> [   94.579048]  [<ffffffff81355422>] xen_evtchn_do_upcall+0x32/0x50
> [   94.579048]  [<ffffffff8170e66d>] xen_hvm_callback_vector+0x6d/0x80
> [   94.579048]  <EOI> 
> [   94.579048]  [<ffffffff810e6b48>] ? sched_clock_cpu+0x98/0xc0
> [   94.579048]  [<ffffffff810a1bb6>] ? native_safe_halt+0x6/0x10
> [   94.579048]  [<ffffffff8106cacd>] default_idle+0x2d/0xd0
> [   94.579048]  [<ffffffff8106c14a>] arch_cpu_idle+0xa/0x10
> [   94.579048]  [<ffffffff810f3831>] cpu_startup_entry+0x1a1/0x3e0
> [   94.579048]  [<ffffffff816f5bdd>] rest_init+0x6d/0x70
> [   94.579048]  [<ffffffff81cd22b6>] start_kernel+0x49a/0x4a7
> [   94.579048]  [<ffffffff81cd1c3a>] ? repair_env_string+0x5b/0x5b
> [   94.579048]  [<ffffffff81cd1117>] ? early_idt_handlers+0x117/0x120
> [   94.579048]  [<ffffffff81cd13e9>] x86_64_start_reservations+0x2a/0x2c
> [   94.579048]  [<ffffffff81cd183a>] x86_64_start_kernel+0x1ea/0x1f9
> [   94.579048] Code: 0f 1f 00 f3 90 8b 04 25 00 93 5f ff f6 c4 10 75 f2 c1 e1 18 89 0c 25 10 93 5f ff 83 fb 02 44 89 f0 0f 45 c3 89 04 25 00 93 5f ff <eb> 9e 41 f7 c7 00 02 00 00 75 23 48 83 3d b5 6f b9 00 00 74 2d 
> [   74.365746] NMI backtrace for cpu 1
> [   74.365746] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.15.8 #1
> [   74.365746] Hardware name: Xen HVM domU, BIOS 4.5-unstable 08/04/2014
> [   74.365746] task: ffff88000e49cf50 ti: ffff88000e4ec000 task.ti: ffff88000e4ec000
> [   74.365746] RIP: 0010:[<ffffffff810a1bb6>]  [<ffffffff810a1bb6>] native_safe_halt+0x6/0x10
> [   74.365746] RSP: 0018:ffff88000e4efe88  EFLAGS: 00000282
> [   74.365746] RAX: 0000000000000000 RBX: ffff88000e4ec010 RCX: ffff88000e4ec010
> [   74.365746] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> [   74.365746] RBP: ffff88000e4efe88 R08: 0000000000000000 R09: 0000000000000000
> [   74.365746] R10: 00000000000ed800 R11: 0000000000000000 R12: 0000000000000001
> [   74.365746] R13: ffffffff81cad508 R14: ffff88000e4ec010 R15: 0000000000000000
> [   74.365746] FS:  0000000000000000(0000) GS:ffff88000d620000(0000) knlGS:0000000000000000
> [   74.365746] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   74.365746] CR2: 00007f465aa759a0 CR3: 000000000c169000 CR4: 00000000001406e0
> [   74.365746] Stack:
> [   74.365746]  ffff88000e4efea8 ffffffff8106cacd ffff88000e4ec010 ffff88000e4ec010
> [   74.365746]  ffff88000e4efeb8 ffffffff8106c14a ffff88000e4eff28 ffffffff810f3831
> [   74.365746]  ffff88000d62bdc0 0000000000000282 ffff88000e4ec010 ffff88000e4ec010
> [   74.365746] Call Trace:
> [   74.365746]  [<ffffffff8106cacd>] default_idle+0x2d/0xd0
> [   74.365746]  [<ffffffff8106c14a>] arch_cpu_idle+0xa/0x10
> [   74.365746]  [<ffffffff810f3831>] cpu_startup_entry+0x1a1/0x3e0
> [   74.365746]  [<ffffffff810961ab>] start_secondary+0x19b/0x200
> [   74.365746] Code: 00 00 00 00 00 55 48 89 e5 fa c9 c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb c9 c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <c9> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 c9 c3 66 0f 1f 84 
> [   74.365709] NMI backtrace for cpu 3
> [   74.365709] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 3.15.8 #1
> [   74.365709] Hardware name: Xen HVM domU, BIOS 4.5-unstable 08/04/2014
> [   74.365709] task: ffff88000e49e0f0 ti: ffff88000e4f4000 task.ti: ffff88000e4f4000
> [   74.365709] RIP: 0010:[<ffffffff810a1bb6>]  [<ffffffff810a1bb6>] native_safe_halt+0x6/0x10
> [   74.365709] RSP: 0018:ffff88000e4f7e88  EFLAGS: 00000282
> [   74.365709] RAX: 0000000000000000 RBX: ffff88000e4f4010 RCX: ffff88000e4f4010
> [   74.365709] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> [   74.365709] RBP: ffff88000e4f7e88 R08: 0000000000000000 R09: 0000000000000000
> [   74.365709] R10: 00000000000ed800 R11: 0000000000000000 R12: 0000000000000003
> [   74.365709] R13: ffffffff81cad508 R14: ffff88000e4f4010 R15: 0000000000000000
> [   74.365709] FS:  0000000000000000(0000) GS:ffff88000d660000(0000) knlGS:0000000000000000
> [   74.365709] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   74.365709] CR2: 00007faf6c003ac8 CR3: 000000000c169000 CR4: 00000000001406e0
> [   74.365709] Stack:
> [   74.365709]  ffff88000e4f7ea8 ffffffff8106cacd ffff88000e4f4010 ffff88000e4f4010
> [   74.365709]  ffff88000e4f7eb8 ffffffff8106c14a ffff88000e4f7f28 ffffffff810f3831
> [   74.365709]  0000000000000282 ffff88000e4f7ee0 ffff88000e4f4010 ffff88000e4f4010
> [   74.365709] Call Trace:
> [   74.365709]  [<ffffffff8106cacd>] default_idle+0x2d/0xd0
> [   74.365709]  [<ffffffff8106c14a>] arch_cpu_idle+0xa/0x10
> [   74.365709]  [<ffffffff810f3831>] cpu_startup_entry+0x1a1/0x3e0
> [   74.365709]  [<ffffffff810961ab>] start_secondary+0x19b/0x200
> [   74.365709] Code: 00 00 00 00 00 55 48 89 e5 fa c9 c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb c9 c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <c9> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 c9 c3 66 0f 1f 84 
> [   74.365731] NMI backtrace for cpu 2
> [   74.365731] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 3.15.8 #1
> [   74.365731] Hardware name: Xen HVM domU, BIOS 4.5-unstable 08/04/2014
> [   74.365731] task: ffff88000e49d820 ti: ffff88000e4f0000 task.ti: ffff88000e4f0000
> [   74.365731] RIP: 0010:[<ffffffff810a1bb6>]  [<ffffffff810a1bb6>] native_safe_halt+0x6/0x10
> [   74.365731] RSP: 0018:ffff88000e4f3e88  EFLAGS: 00000282
> [   74.365731] RAX: 0000000000000000 RBX: ffff88000e4f0010 RCX: ffff88000e4f0010
> [   74.365731] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> [   74.365731] RBP: ffff88000e4f3e88 R08: 0000000000000000 R09: 0000000000000000
> [   74.365731] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000002
> [   74.365731] R13: ffffffff81cad508 R14: ffff88000e4f0010 R15: 0000000000000000
> [   74.365731] FS:  0000000000000000(0000) GS:ffff88000d640000(0000) knlGS:0000000000000000
> [   74.365731] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   74.365731] CR2: ffffffffff600400 CR3: 000000000c169000 CR4: 00000000001406e0
> [   74.365731] Stack:
> [   74.365731]  ffff88000e4f3ea8 ffffffff8106cacd ffff88000e4f0010 ffff88000e4f0010
> [   74.365731]  ffff88000e4f3eb8 ffffffff8106c14a ffff88000e4f3f28 ffffffff810f3831
> [   74.365731]  0000000000000282 ffff88000e4f3ee0 ffff88000e4f0010 ffff88000e4f0010
> [   74.365731] Call Trace:
> [   74.365731]  [<ffffffff8106cacd>] default_idle+0x2d/0xd0
> [   74.365731]  [<ffffffff8106c14a>] arch_cpu_idle+0xa/0x10
> [   74.365731]  [<ffffffff810f3831>] cpu_startup_entry+0x1a1/0x3e0
> [   74.365731]  [<ffffffff810961ab>] start_secondary+0x19b/0x200
> [   74.365731] Code: 00 00 00 00 00 55 48 89 e5 fa c9 c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb c9 c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <c9> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 c9 c3 66 0f 1f 84 
> [   94.579048] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 627.500 msecs
> 
> 

thanks