Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

I'm running Xen-4.7.0_08-452 + linux kernel 4.7.0-4.g89a2ada-default on X86_64 UEFI hardware.

If I boot without Xen hypervisor enabled it boots fine.

If I boot with Xen enabled it PANICs:
	
	(XEN) [2016-07-26 22:05:33] Hardware Dom0 crashed: rebooting machine in 5 seconds.
	(XEN) [2016-07-26 22:05:33] APIC error on CPU0: 40(00)
	(XEN) [2016-07-26 22:05:38] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
	(XEN) [2016-07-26 22:05:38] CPU:    0
	(XEN) [2016-07-26 22:05:38] RIP:    e008:[<000000009e7463c6>] 000000009e7463c6
	(XEN) [2016-07-26 22:05:38] RFLAGS: 0000000000010202   CONTEXT: hypervisor (d0v0)
	(XEN) [2016-07-26 22:05:38] rax: 0000000000000003   rbx: 0000000000000000   rcx: 0000000000000000
	(XEN) [2016-07-26 22:05:38] rdx: 000000009e7467a0   rsi: 0000000000000000   rdi: 0000000000000000
	(XEN) [2016-07-26 22:05:38] rbp: 0000000000000000   rsp: ffff83008ce27d78   r8:  ffff83008ce27db8
	(XEN) [2016-07-26 22:05:38] r9:  ffff83008ce27da8   r10: 0000000000000000   r11: 0000000000000000
	(XEN) [2016-07-26 22:05:38] r12: 0000000000000000   r13: 0000000000000cf9   r14: 0000000000000065
	(XEN) [2016-07-26 22:05:38] r15: ffff830000000000   cr0: 0000000080050033   cr4: 00000000001526e0
	(XEN) [2016-07-26 22:05:38] cr3: 000000084b4f6000   cr2: 0000000000000018
	(XEN) [2016-07-26 22:05:38] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
	(XEN) [2016-07-26 22:05:38] Xen code around <000000009e7463c6> (000000009e7463c6):
	(XEN) [2016-07-26 22:05:38]  0f 48 8b 44 24 40 8b ce <ff> 50 08 3b d8 0f 4c d8 48 ff c7 48 3b 7c 24 30
	(XEN) [2016-07-26 22:05:38] Xen stack trace from rsp=ffff83008ce27d78:
	(XEN) [2016-07-26 22:05:38]    ffff83084b4b51c0 0000000000000000 0000000000000000 ffff82d0801670f6
	(XEN) [2016-07-26 22:05:38]    ffff83008ce27db0 0000000000000000 00000000001526e0 0000000000000206
	(XEN) [2016-07-26 22:05:38]    0000000000000003 0000000841e06000 0000000000000000 000000009efe42f6
	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 efff000000000000 ffff82d0807fe000
	(XEN) [2016-07-26 22:05:38]    000000084b4f6000 ffff82d08022f94a 0000000841e06000 0000000000000000
	(XEN) [2016-07-26 22:05:38]    0000000000000007 000000000000e008 0000000000000296 0000000000000000
	(XEN) [2016-07-26 22:05:38]    00000000fffffffe ffff82d08018bcc8 0000000000000000 0000138800000008
	(XEN) [2016-07-26 22:05:38]    000083008ce27ea8 0000000000000000 ffff83008ce27eb8 0000000000000003
	(XEN) [2016-07-26 22:05:38]    0000000000000003 ffff83084b4b5000 ffff83084b4b51c0 0000000000000000
	(XEN) [2016-07-26 22:05:38]    0000000000000000 ffff82d08012bf0d 0000000000000003 ffff82d08012bfaf
	(XEN) [2016-07-26 22:05:38]    ffffffff81e03f28 ffff82d080105871 0000000000000000 ffff83008ce27fff
	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 ffffffff81e03f28 ffff82d080105978
	(XEN) [2016-07-26 22:05:38]    ffff830092826000 ffff82d080197f8e 0000000000000001 ffff82d08022cc55
	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 ffffffff81e03f28 0000000000000000
	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 0000000000000067 0000000000007ff0
	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000200 0000000000000000 0000000100000000
	(XEN) [2016-07-26 22:05:38]    000000016f141000 ffffffff81efb2c0 000000016f141000 0000010e00000004
	(XEN) [2016-07-26 22:05:38]    ffffffff81f6374c 000000000000e033 0000000000000246 ffffffff81e03e50
	(XEN) [2016-07-26 22:05:38]    000000000000e02b 0000000000000000 0000000000000000 0000000000000000
	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 ffff830092826000 0000000000000000
	(XEN) [2016-07-26 22:05:38] Xen call trace:
	(XEN) [2016-07-26 22:05:38]    [<000000009e7463c6>] 000000009e7463c6
	(XEN) [2016-07-26 22:05:38]    [<ffff82d0801670f6>] i387.c#_vcpu_save_fpu+0x86/0x190
	(XEN) [2016-07-26 22:05:38]    [<ffff82d08022f94a>] efi_reset_system+0x3a/0x60
	(XEN) [2016-07-26 22:05:38]    [<ffff82d08018bcc8>] machine_restart+0x208/0x2d0
	(XEN) [2016-07-26 22:05:38]    [<ffff82d08012bf0d>] shutdown.c#maybe_reboot+0x3d/0x40
	(XEN) [2016-07-26 22:05:38]    [<ffff82d08012bfaf>] hwdom_shutdown+0x9f/0xf0
	(XEN) [2016-07-26 22:05:38]    [<ffff82d080105871>] domain_shutdown+0xf1/0x100
	(XEN) [2016-07-26 22:05:38]    [<ffff82d080105978>] __domain_crash_synchronous+0x18/0x30
	(XEN) [2016-07-26 22:05:38]    [<ffff82d080197f8e>] asm_domain_crash_synchronous+0x3e/0x40
	(XEN) [2016-07-26 22:05:38]    [<ffff82d08022cc55>] entry.o#handle_exception_saved+0x9b/0xa4
	(XEN) [2016-07-26 22:05:38] 
	(XEN) [2016-07-26 22:05:38] 
	(XEN) [2016-07-26 22:05:38] ****************************************
	(XEN) [2016-07-26 22:05:38] Panic on CPU 0:
	(XEN) [2016-07-26 22:05:38] GENERAL PROTECTION FAULT
	(XEN) [2016-07-26 22:05:38] [error_code=0000]
	(XEN) [2016-07-26 22:05:38] ****************************************
	(XEN) [2016-07-26 22:05:38] 
	(XEN) [2016-07-26 22:05:38] Reboot in five seconds...

This server used to be running OK as recently as last week with this config.  Some recent upgrade - kernel, qemu, xen, etc - may have broken something.  Most notable was the the upgrade to kernel 4.6.3 -> 4.7.0.

What other debug info can help figure out this specific problem?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

> What other debug info can help figure out this specific problem?

I found this post with some suggestions and additional references

  Troubleshooting UEFI related problems
   https://www.qubes-os.org/doc/uefi-troubleshooting/

I tried different combinations of

   /mapbs, /noexitboot

on the EFI cmd line, and 

    efi=attr=uc

on the Xen command line.

None of them fix or look like they effect the crash.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Andrew Cooper]

On 27/07/16 00:32, lists@ssl-mail.com wrote:
> I'm running Xen-4.7.0_08-452 + linux kernel 4.7.0-4.g89a2ada-default on X86_64 UEFI hardware.
>
> If I boot without Xen hypervisor enabled it boots fine.
>
> If I boot with Xen enabled it PANICs:
> 	
> 	(XEN) [2016-07-26 22:05:33] Hardware Dom0 crashed: rebooting machine in 5 seconds.

The Dom0 kernel crashed.  As a result, Xen tried to reboot.

> 	(XEN) [2016-07-26 22:05:33] APIC error on CPU0: 40(00)
> 	(XEN) [2016-07-26 22:05:38] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
> 	(XEN) [2016-07-26 22:05:38] CPU:    0
> 	(XEN) [2016-07-26 22:05:38] RIP:    e008:[<000000009e7463c6>] 000000009e7463c6
> 	(XEN) [2016-07-26 22:05:38] RFLAGS: 0000000000010202   CONTEXT: hypervisor (d0v0)
> 	(XEN) [2016-07-26 22:05:38] rax: 0000000000000003   rbx: 0000000000000000   rcx: 0000000000000000
> 	(XEN) [2016-07-26 22:05:38] rdx: 000000009e7467a0   rsi: 0000000000000000   rdi: 0000000000000000
> 	(XEN) [2016-07-26 22:05:38] rbp: 0000000000000000   rsp: ffff83008ce27d78   r8:  ffff83008ce27db8
> 	(XEN) [2016-07-26 22:05:38] r9:  ffff83008ce27da8   r10: 0000000000000000   r11: 0000000000000000
> 	(XEN) [2016-07-26 22:05:38] r12: 0000000000000000   r13: 0000000000000cf9   r14: 0000000000000065
> 	(XEN) [2016-07-26 22:05:38] r15: ffff830000000000   cr0: 0000000080050033   cr4: 00000000001526e0
> 	(XEN) [2016-07-26 22:05:38] cr3: 000000084b4f6000   cr2: 0000000000000018
> 	(XEN) [2016-07-26 22:05:38] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
> 	(XEN) [2016-07-26 22:05:38] Xen code around <000000009e7463c6> (000000009e7463c6):
> 	(XEN) [2016-07-26 22:05:38]  0f 48 8b 44 24 40 8b ce <ff> 50 08 3b d8 0f 4c d8 48 ff c7 48 3b 7c 24 30

This disassembles to

callq  *0x8(%rax)

and %rax looks like an implausible value for a function pointer.  This
particular issue is definitely an EFI firmware issue.

> 	(XEN) [2016-07-26 22:05:38] Xen stack trace from rsp=ffff83008ce27d78:
> 	(XEN) [2016-07-26 22:05:38]    ffff83084b4b51c0 0000000000000000 0000000000000000 ffff82d0801670f6
> 	(XEN) [2016-07-26 22:05:38]    ffff83008ce27db0 0000000000000000 00000000001526e0 0000000000000206
> 	(XEN) [2016-07-26 22:05:38]    0000000000000003 0000000841e06000 0000000000000000 000000009efe42f6
> 	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 efff000000000000 ffff82d0807fe000
> 	(XEN) [2016-07-26 22:05:38]    000000084b4f6000 ffff82d08022f94a 0000000841e06000 0000000000000000
> 	(XEN) [2016-07-26 22:05:38]    0000000000000007 000000000000e008 0000000000000296 0000000000000000
> 	(XEN) [2016-07-26 22:05:38]    00000000fffffffe ffff82d08018bcc8 0000000000000000 0000138800000008
> 	(XEN) [2016-07-26 22:05:38]    000083008ce27ea8 0000000000000000 ffff83008ce27eb8 0000000000000003
> 	(XEN) [2016-07-26 22:05:38]    0000000000000003 ffff83084b4b5000 ffff83084b4b51c0 0000000000000000
> 	(XEN) [2016-07-26 22:05:38]    0000000000000000 ffff82d08012bf0d 0000000000000003 ffff82d08012bfaf
> 	(XEN) [2016-07-26 22:05:38]    ffffffff81e03f28 ffff82d080105871 0000000000000000 ffff83008ce27fff
> 	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 ffffffff81e03f28 ffff82d080105978
> 	(XEN) [2016-07-26 22:05:38]    ffff830092826000 ffff82d080197f8e 0000000000000001 ffff82d08022cc55
> 	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 ffffffff81e03f28 0000000000000000
> 	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 0000000000000067 0000000000007ff0
> 	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000200 0000000000000000 0000000100000000
> 	(XEN) [2016-07-26 22:05:38]    000000016f141000 ffffffff81efb2c0 000000016f141000 0000010e00000004
> 	(XEN) [2016-07-26 22:05:38]    ffffffff81f6374c 000000000000e033 0000000000000246 ffffffff81e03e50
> 	(XEN) [2016-07-26 22:05:38]    000000000000e02b 0000000000000000 0000000000000000 0000000000000000
> 	(XEN) [2016-07-26 22:05:38]    0000000000000000 0000000000000000 ffff830092826000 0000000000000000
> 	(XEN) [2016-07-26 22:05:38] Xen call trace:
> 	(XEN) [2016-07-26 22:05:38]    [<000000009e7463c6>] 000000009e7463c6
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d0801670f6>] i387.c#_vcpu_save_fpu+0x86/0x190
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d08022f94a>] efi_reset_system+0x3a/0x60
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d08018bcc8>] machine_restart+0x208/0x2d0
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d08012bf0d>] shutdown.c#maybe_reboot+0x3d/0x40
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d08012bfaf>] hwdom_shutdown+0x9f/0xf0
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d080105871>] domain_shutdown+0xf1/0x100
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d080105978>] __domain_crash_synchronous+0x18/0x30
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d080197f8e>] asm_domain_crash_synchronous+0x3e/0x40
> 	(XEN) [2016-07-26 22:05:38]    [<ffff82d08022cc55>] entry.o#handle_exception_saved+0x9b/0xa4
> 	(XEN) [2016-07-26 22:05:38] 
> 	(XEN) [2016-07-26 22:05:38] 
> 	(XEN) [2016-07-26 22:05:38] ****************************************
> 	(XEN) [2016-07-26 22:05:38] Panic on CPU 0:
> 	(XEN) [2016-07-26 22:05:38] GENERAL PROTECTION FAULT
> 	(XEN) [2016-07-26 22:05:38] [error_code=0000]
> 	(XEN) [2016-07-26 22:05:38] ****************************************
> 	(XEN) [2016-07-26 22:05:38] 
> 	(XEN) [2016-07-26 22:05:38] Reboot in five seconds...
>
> This server used to be running OK as recently as last week with this config.  Some recent upgrade - kernel, qemu, xen, etc - may have broken something.  Most notable was the the upgrade to kernel 4.6.3 -> 4.7.0.

I presume you mean an upgrade of the dom0 Linux kernel from 4.6.3 to 4.7.0?

> What other debug info can help figure out this specific problem?

This is first a Linux crash, followed by bad knock-on behaviour.

For the knockon behaviour, does Linux 4.6.3 encounter the same reboot
crash with Xen 4.7.0?

For the Linux crash, can you boot Linux with "earlyprintk=xen" and see
if that provides more help as to what went wrong?

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Jul 27, 2016, at 08:50 AM, Andrew Cooper wrote:
> This disassembles to
> 
> callq  *0x8(%rax)
> 
> and %rax looks like an implausible value for a function pointer.  This
> particular issue is definitely an EFI firmware issue.

With all the reference to & around EFI I kinda figured ...

> I presume you mean an upgrade of the dom0 Linux kernel from 4.6.3 to 4.7.0?

Yep.

> > What other debug info can help figure out this specific problem?
> 
> This is first a Linux crash, followed by bad knock-on behaviour.
> 
> For the knockon behaviour, does Linux 4.6.3 encounter the same reboot crash with Xen 4.7.0?

It didn't ... a week or so ago.  I.e. it was working fine.  Or at least I can say it wasn't crashing and I could boot Dom0 & my DomUs.

At the moment, I'm stuck in "use what the distro packaging provides"-land.

Which mean that I don't have a drop-back to 4.6.3 available.

Couple of things I'm gonna do to make sure that doesn't happen again in the future.  But for today - I'm stuck with 4.7 kernel.

> For the Linux crash, can you boot Linux with "earlyprintk=xen" and see
> if that provides more help as to what went wrong?

Here's serial console output with grub2 log parameters included as

	kernel:     earlyprintk=xen,keep debug loglevel=8
	hypervisor: loglvl=all guest_loglvl=all sync_console console_to_ring


Loading Xen 4.7.0_08-452 with Linux 4.7.0-4.g89a2ada-default ...Loading Xen 4.7.0_08-452 with Linux 4.7.0-4.g89a2ada-default ...

/EndEntire
/EndEntire
file path: file path: /ACPI(a0341d0,0)/ACPI(a0341d0,0)/PCI(1,1c)/PCI(1,1c)/PCI(0,0)/PCI(0,0)/PCI(0,1)/PCI(0,1)/PCI(0,0)/PCI(0,0)/HardwareVendor
(cf31fac5-c24e-11d2-85f3-00a0c93ec93b)[1: 
/HardwareVendor(cf31fac5-c24e-11d2-85f3-00a0c93ec93b)[1: 88 88 ]]/HD(2,1000,96000,c5cc9661271ee648
,2,2)/HD(2,1000,96000,c5cc9661271ee648,2,2)/File(\EFI\OPENSUSE)
/File(\EFI\OPENSUSE)/File(xen-4.7.0_08-452.efi)/File(xen-4.7.0_08-452.efi)/EndEntire
/EndEntire
Xen 4.7.0_08-452 (c/s ) EFI loader
Using configuration file 'xen-4.7.0_08-452.cfg'
vmlinuz-4.7.0-4.g89a2ada-default: 0x000000008bef9000-0x000000008c5064c0
initrd-4.7.0-4.g89a2ada-default: 0x000000008af57000-0x000000008bef8b2c
0x0000:0x00:0x19.0x0: ROM: 0x10000 bytes at 0x92a25018
0x0000:0x03:0x00.0x0: ROM: 0x8000 bytes at 0x92a1c018
0x0000:0x0f:0x00.0x0: ROM: 0x10800 bytes at 0x929fb018
 __  __            _  _   _____ ___     ___   ___     _  _  ____ ____  
 \ \/ /___ _ __   | || | |___  / _ \   / _ \ ( _ )   | || || ___|___ \ 
  \  // _ \ '_ \  | || |_   / / | | | | | | |/ _ \ __| || ||___ \ __) |
  /  \  __/ | | | |__   _| / /| |_| | | |_| | (_) |__|__   _|__) / __/ 
 /_/\_\___|_| |_|    |_|(_)_/(_)___/___\___/ \___/      |_||____/_____|
                                  |_____|                              
(XEN) Xen version 4.7.0_08-452 (abuild@suse.de) (gcc (SUSE Linux) 4.8.5) debug=n Thu Jun 23 15:45:38 UTC 2016
(XEN) Latest ChangeSet: 
(XEN) Console output is synchronous.
(XEN) Bootloader: EFI
(XEN) Command line: dom0_mem=4096M,max:4096M dom0_max_vcpus=1 dom0_vcpus_pin=true cpuidle=1 cpufreq=xen com1=115200,8n1,pci console=com1,vga console_timestamps consol
e_to_ring conring_size=64  log_buf_len=16M sched_debug apic_verbosity=verbose loglvl=all guest_loglvl=all noreboot=true sync_console=true 
(XEN) Video information:
(XEN)  VGA is graphics mode 800x600, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 6 EDD information structures
(XEN) EFI RAM map:
(XEN)  0000000000000000 - 0000000000008000 (reserved)
(XEN)  0000000000008000 - 0000000000048000 (usable)
(XEN)  0000000000048000 - 0000000000059000 (reserved)
(XEN)  0000000000059000 - 000000000005f000 (usable)
(XEN)  000000000005f000 - 00000000000a0000 (reserved)
(XEN)  0000000000100000 - 000000008d93e000 (usable)
(XEN)  000000008d93e000 - 000000008d945000 (ACPI NVS)
(XEN)  000000008d945000 - 000000008e25a000 (reserved)
(XEN)  000000008e25a000 - 000000008e260000 (usable)
(XEN)  000000008e260000 - 000000008e6a1000 (reserved)
(XEN)  000000008e6a1000 - 0000000091782000 (usable)
(XEN)  0000000091782000 - 00000000919ea000 (reserved)
(XEN)  00000000919ea000 - 0000000091a1d000 (usable)
(XEN)  0000000091a1d000 - 0000000091a7a000 (reserved)
(XEN)  0000000091a7a000 - 0000000091ae0000 (usable)
(XEN)  0000000091ae0000 - 0000000091b86000 (reserved)
(XEN)  0000000091b86000 - 0000000091bba000 (usable)
(XEN)  0000000091bba000 - 0000000091bbb000 (reserved)
(XEN)  0000000091bbb000 - 0000000091bbd000 (usable)
(XEN)  0000000091bbd000 - 0000000091bbe000 (reserved)
(XEN)  0000000091bbe000 - 0000000091bc6000 (usable)
(XEN)  0000000091bc6000 - 0000000091bc9000 (reserved)
(XEN)  0000000091bc9000 - 0000000091bdc000 (usable)
(XEN)  0000000091bdc000 - 0000000091be7000 (reserved)
(XEN)  0000000091be7000 - 0000000091c68000 (usable)
(XEN)  0000000091c68000 - 0000000091ce5000 (reserved)
(XEN)  0000000091ce5000 - 0000000091d2f000 (usable)
(XEN)  0000000091d2f000 - 000000009208d000 (reserved)
(XEN)  000000009208d000 - 00000000920d4000 (usable)
(XEN)  00000000920d4000 - 000000009214a000 (reserved)
(XEN)  000000009214a000 - 000000009217d000 (usable)
(XEN)  000000009217d000 - 0000000092295000 (reserved)
(XEN)  0000000092295000 - 00000000922b4000 (usable)
(XEN)  00000000922b4000 - 0000000092358000 (reserved)
(XEN)  0000000092358000 - 0000000092362000 (usable)
(XEN)  0000000092362000 - 0000000092461000 (reserved)
(XEN)  0000000092461000 - 0000000092466000 (usable)
(XEN)  0000000092466000 - 00000000925b5000 (reserved)
(XEN)  00000000925b5000 - 00000000925b7000 (usable)
(XEN)  00000000925b7000 - 00000000925bd000 (reserved)
(XEN)  00000000925bd000 - 00000000925bf000 (usable)
(XEN)  00000000925bf000 - 00000000925c5000 (reserved)
(XEN)  00000000925c5000 - 00000000925c7000 (usable)
(XEN)  00000000925c7000 - 00000000925ce000 (reserved)
(XEN)  00000000925ce000 - 00000000925d1000 (usable)
(XEN)  00000000925d1000 - 00000000925e4000 (reserved)
(XEN)  00000000925e4000 - 00000000925e7000 (usable)
(XEN)  00000000925e7000 - 00000000925ea000 (reserved)
(XEN)  00000000925ea000 - 00000000925ed000 (usable)
(XEN)  00000000925ed000 - 0000000092824000 (reserved)
(XEN)  0000000092824000 - 0000000092826000 (usable)
(XEN)  0000000092826000 - 0000000092860000 (reserved)
(XEN)  0000000092860000 - 0000000092862000 (usable)
(XEN)  0000000092862000 - 000000009286a000 (reserved)
(XEN)  000000009286a000 - 000000009286c000 (usable)
(XEN)  000000009286c000 - 0000000092872000 (reserved)
(XEN)  0000000092872000 - 0000000092874000 (usable)
(XEN)  0000000092874000 - 000000009287c000 (reserved)
(XEN)  000000009287c000 - 000000009287d000 (usable)
(XEN)  000000009287d000 - 000000009e1a0000 (reserved)
(XEN)  000000009e1a0000 - 000000009e430000 (usable)
(XEN)  000000009e430000 - 000000009e845000 (reserved)
(XEN)  000000009e845000 - 000000009e891000 (usable)
(XEN)  000000009e891000 - 000000009e9c5000 (ACPI NVS)
(XEN)  000000009e9c5000 - 000000009f000000 (reserved)
(XEN)  00000000f0000000 - 00000000f8000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000085e000000 (usable)
(XEN) ACPI: RSDP 9E995000, 0024 (r2 SUPERM)
(XEN) ACPI: XSDT 9E995090, 009C (r1 SUPERM SMCI--MB  1072009 AMI     10013)
(XEN) ACPI: FACP 9E9A32E8, 010C (r5 SUPERM SMCI--MB  1072009 AMI     10013)
(XEN) ACPI: DSDT 9E9951C0, E121 (r2 SUPERM SMCI--MB        0 INTL 20120711)
(XEN) ACPI: FACS 9E9C4F80, 0040
(XEN) ACPI: APIC 9E9A33F8, 0072 (r3 SUPERM SMCI--MB  1072009 AMI     10013)
(XEN) ACPI: FPDT 9E9A3470, 0044 (r1 SUPERM SMCI--MB  1072009 AMI     10013)
(XEN) ACPI: FIDT 9E9A34B8, 009C (r1 SUPERM SMCI--MB  1072009 AMI     10013)
(XEN) ACPI: SSDT 9E9A3558, 0C7D (r2 Ther_R Ther_Rvp     1000 INTL 20120711)
(XEN) ACPI: SSDT 9E9A41D8, 0539 (r2  PmRef  Cpu0Ist     3000 INTL 20051117)
(XEN) ACPI: SSDT 9E9A4718, 0B74 (r2 CpuRef  CpuSsdt     3000 INTL 20051117)
(XEN) ACPI: SSDT 9E9A5290, 02DE (r2  PmRef  Cpu0Tst     3000 INTL 20051117)
(XEN) ACPI: SSDT 9E9A5570, 0348 (r2  PmRef    ApTst     3000 INTL 20051117)
(XEN) ACPI: MCFG 9E9A58B8, 003C (r1 SUPERM SMCI--MB  1072009 MSFT       97)
(XEN) ACPI: HPET 9E9A58F8, 0038 (r1 SUPERM SMCI--MB  1072009 AMI.        5)
(XEN) ACPI: SSDT 9E9A5930, 0397 (r1 SataRe SataTabl     1000 INTL 20120711)
(XEN) ACPI: SSDT 9E9A5CC8, 57F6 (r2 SaSsdt  SaSsdt      3000 INTL 20120711)
(XEN) ACPI: ASF! 9E9AB4C0, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) ACPI: DMAR 9E9AB568, 0080 (r1 INTEL      BDW         1 INTL        1)
(XEN) System RAM: 32494MB (33274608kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000085e000000
(XEN) Domain heap initialised
(XEN) Allocated console ring of 64 KiB.
(XEN) vesafb: framebuffer at 0xd1000000, mapped to 0xffff82c000201000, using 1920k, total 1920k
(XEN) vesafb: mode is 800x600x32, linelength=3200, font 8x8
(XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
(XEN) SMBIOS 2.7 present.
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1808
(XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1804,1:0], pm1x_evt[1:1800,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 9e9c4f80/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[9e9c4f8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) [VT-D]Host address width 39
(XEN) [VT-D]found ACPI_DMAR_DRHD:
(XEN) [VT-D]  dmaru->address = fed90000
(XEN) [VT-D]drhd->address = fed90000 iommu->reg = ffff82c0003e2000
(XEN) [VT-D]cap = d2008c20660462 ecap = f010da
(XEN) [VT-D] IOAPIC: 0000:f0:1f.0
(XEN) [VT-D] MSI HPET: 0000:f0:0f.0
(XEN) [VT-D]  flags: INCLUDE_ALL
(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D] endpoint: 0000:00:1d.0
(XEN) [VT-D] endpoint: 0000:00:1a.0
(XEN) [VT-D] endpoint: 0000:00:14.0
(XEN) [VT-D]  RMRR region: base_addr 9eea5000 end_address 9eeb3fff
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) mapped APIC to ffff82cfffffb000 (fee00000)
(XEN) mapped IOAPIC to ffff82cfffffa000 (fec00000)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Not enabling x2APIC (upon firmware request)
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  WARNING: This is experimental software in development.
(XEN)  Use at your own risk.
(XEN)  load_window_shift: 18
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: core
(XEN) csched2_dom_init: Initializing domain 32767
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Detected 3092.938 MHz processor.
(XEN) EFI memory map: (mapping BootServices)
(XEN)  0000000000000-0000000007fff type=3 attr=000000000000000f
(XEN)  0000000008000-0000000047fff type=7 attr=000000000000000f
(XEN)  0000000048000-0000000057fff type=4 attr=000000000000000f
(XEN)  0000000058000-0000000058fff type=0 attr=000000000000000f
(XEN)  0000000059000-000000005bfff type=7 attr=000000000000000f
(XEN)  000000005c000-000000005efff type=2 attr=000000000000000f
(XEN)  000000005f000-000000005ffff type=4 attr=000000000000000f
(XEN)  0000000060000-000000009efff type=3 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
(XEN)  0000000100000-000006635cfff type=7 attr=000000000000000f
(XEN)  000006635d000-000008a7b0fff type=2 attr=000000000000000f
(XEN)  000008a7b1000-000008af56fff type=7 attr=000000000000000f
(XEN)  000008af57000-000008c506fff type=2 attr=000000000000000f
(XEN)  000008c507000-000008d93dfff type=1 attr=000000000000000f
(XEN)  000008d93e000-000008d944fff type=10 attr=000000000000000f
(XEN)  000008d945000-000008da98fff type=4 attr=000000000000000f
(XEN)  000008da99000-000008e255fff type=3 attr=000000000000000f
(XEN)  000008e256000-000008e259fff type=4 attr=000000000000000f
(XEN)  000008e25a000-000008e25ffff type=7 attr=000000000000000f
(XEN)  000008e260000-000008e264fff type=4 attr=000000000000000f
(XEN)  000008e265000-000008e26efff type=3 attr=000000000000000f
(XEN)  000008e26f000-000008e275fff type=4 attr=000000000000000f
(XEN)  000008e276000-000008e27ffff type=3 attr=000000000000000f
(XEN)  000008e280000-000008e28afff type=4 attr=000000000000000f
(XEN)  000008e28b000-000008e691fff type=6 attr=800000000000000f
(XEN)  000008e692000-000008e6a0fff type=4 attr=000000000000000f
(XEN)  000008e6a1000-000008e6aefff type=7 attr=000000000000000f
(XEN)  000008e6af000-000008e6affff type=2 attr=000000000000000f
(XEN)  000008e6b0000-0000091781fff type=7 attr=000000000000000f
(XEN)  0000091782000-00000919e9fff type=4 attr=000000000000000f
(XEN)  00000919ea000-0000091a1cfff type=7 attr=000000000000000f
(XEN)  0000091a1d000-0000091a79fff type=4 attr=000000000000000f
(XEN)  0000091a7a000-0000091adffff type=7 attr=000000000000000f
(XEN)  0000091ae0000-0000091b85fff type=4 attr=000000000000000f
(XEN)  0000091b86000-0000091bb9fff type=7 attr=000000000000000f
(XEN)  0000091bba000-0000091bbafff type=4 attr=000000000000000f
(XEN)  0000091bbb000-0000091bbcfff type=7 attr=000000000000000f
(XEN)  0000091bbd000-0000091bbdfff type=4 attr=000000000000000f
(XEN)  0000091bbe000-0000091bc5fff type=7 attr=000000000000000f
(XEN)  0000091bc6000-0000091bc8fff type=4 attr=000000000000000f
(XEN)  0000091bc9000-0000091bdbfff type=7 attr=000000000000000f
(XEN)  0000091bdc000-0000091be6fff type=4 attr=000000000000000f
(XEN)  0000091be7000-0000091c67fff type=7 attr=000000000000000f
(XEN)  0000091c68000-0000091ce4fff type=4 attr=000000000000000f
(XEN)  0000091ce5000-0000091d2efff type=7 attr=000000000000000f
(XEN)  0000091d2f000-000009208cfff type=4 attr=000000000000000f
(XEN)  000009208d000-00000920d3fff type=7 attr=000000000000000f
(XEN)  00000920d4000-0000092149fff type=4 attr=000000000000000f
(XEN)  000009214a000-000009217cfff type=7 attr=000000000000000f
(XEN)  000009217d000-0000092294fff type=4 attr=000000000000000f
(XEN)  0000092295000-00000922b3fff type=7 attr=000000000000000f
(XEN)  00000922b4000-0000092357fff type=4 attr=000000000000000f
(XEN)  0000092358000-0000092361fff type=7 attr=000000000000000f
(XEN)  0000092362000-0000092460fff type=4 attr=000000000000000f
(XEN)  0000092461000-0000092465fff type=7 attr=000000000000000f
(XEN)  0000092466000-00000925b4fff type=4 attr=000000000000000f
(XEN)  00000925b5000-00000925b6fff type=7 attr=000000000000000f
(XEN)  00000925b7000-00000925bcfff type=4 attr=000000000000000f
(XEN)  00000925bd000-00000925befff type=7 attr=000000000000000f
(XEN)  00000925bf000-00000925c4fff type=4 attr=000000000000000f
(XEN)  00000925c5000-00000925c6fff type=7 attr=000000000000000f
(XEN)  00000925c7000-00000925cdfff type=4 attr=000000000000000f
(XEN)  00000925ce000-00000925d0fff type=7 attr=000000000000000f
(XEN)  00000925d1000-00000925e3fff type=4 attr=000000000000000f
(XEN)  00000925e4000-00000925e6fff type=7 attr=000000000000000f
(XEN)  00000925e7000-00000925e9fff type=4 attr=000000000000000f
(XEN)  00000925ea000-00000925ecfff type=7 attr=000000000000000f
(XEN)  00000925ed000-0000092823fff type=4 attr=000000000000000f
(XEN)  0000092824000-0000092825fff type=7 attr=000000000000000f
(XEN)  0000092826000-000009285ffff type=4 attr=000000000000000f
(XEN)  0000092860000-0000092861fff type=7 attr=000000000000000f
(XEN)  0000092862000-0000092869fff type=4 attr=000000000000000f
(XEN)  000009286a000-000009286bfff type=7 attr=000000000000000f
(XEN)  000009286c000-0000092871fff type=4 attr=000000000000000f
(XEN)  0000092872000-0000092873fff type=7 attr=000000000000000f
(XEN)  0000092874000-000009287bfff type=4 attr=000000000000000f
(XEN)  000009287c000-000009287cfff type=7 attr=000000000000000f
(XEN)  000009287d000-000009e19ffff type=4 attr=000000000000000f
(XEN)  000009e1a0000-000009e42ffff type=7 attr=000000000000000f
(XEN)  000009e430000-000009e79ffff type=3 attr=000000000000000f
(XEN)  000009e7a0000-000009e7e9fff type=0 attr=000000000000000f
(XEN)  000009e7ea000-000009e844fff type=0 attr=000000000000000f
(XEN)  000009e845000-000009e890fff type=7 attr=000000000000000f
(XEN)  000009e891000-000009e9affff type=10 attr=000000000000000f
(XEN)  000009e9b0000-000009e9c0fff type=10 attr=000000000000000f
(XEN)  000009e9c1000-000009e9c4fff type=10 attr=000000000000000f
(XEN)  000009e9c5000-000009eabffff type=6 attr=800000000000000f
(XEN)  000009eac0000-000009ec3bfff type=6 attr=800000000000000f
(XEN)  000009ec3c000-000009ec3dfff type=6 attr=800000000000000f
(XEN)  000009ec3e000-000009eef0fff type=6 attr=800000000000000f
(XEN)  000009eef1000-000009eef2fff type=6 attr=800000000000000f
(XEN)  000009eef3000-000009ef8efff type=6 attr=800000000000000f
(XEN)  000009ef8f000-000009efa6fff type=5 attr=800000000000000f
(XEN)  000009efa7000-000009effefff type=5 attr=800000000000000f
(XEN)  000009efff000-000009effffff type=4 attr=000000000000000f
(XEN)  0000100000000-000085dffffff type=7 attr=000000000000000f
(XEN)  00000f0000000-00000f7ffffff type=11 attr=8000000000000001
(XEN)  00000fec00000-00000fec00fff type=11 attr=8000000000000001
(XEN)  00000fed00000-00000fed03fff type=11 attr=8000000000000001
(XEN)  00000fed1c000-00000fed1ffff type=11 attr=8000000000000001
(XEN)  00000fee00000-00000fee00fff type=11 attr=8000000000000001
(XEN)  00000ff000000-00000ffffffff type=11 attr=8000000000000001
(XEN) Initing memory sharing.
(XEN) alt table ffff82d08064f710 -> ffff82d080650d18
(XEN) spurious 8259A interrupt: IRQ7.
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 2
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) Suppress EOI broadcast on CPU#0
(XEN) enabled ExtINT on CPU#0
(XEN) ENABLING IO-APIC IRQs
(XEN) init IO_APIC IRQs
(XEN)  IO-APIC (apicid-pin) 8-0, 8-16, 8-17, 8-18, 8-19, 8-20, 8-21, 8-22, 8-23 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=0 pin2=0
(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #8 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #8......
(XEN) .... register #00: 08000000
(XEN) .......    : physical APIC id: 08
(XEN) .......    : Delivery Type: 0
(XEN) .......    : LTS          : 0
(XEN) .... register #01: 00170020
(XEN) .......     : max redirection entries: 0017
(XEN) .......     : PRQ implemented: 0
(XEN) .......     : IO APIC version: 0020
(XEN) .... IRQ redirection table:
(XEN)  NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:   
(XEN)  00 000 00  1    0    0   0   0    0    7    00
(XEN)  01 001 01  0    0    0   0   0    1    1    30
(XEN)  02 001 01  0    0    0   0   0    1    1    F0
(XEN)  03 001 01  0    0    0   0   0    1    1    38
(XEN)  04 001 01  0    0    0   0   0    1    1    F1
(XEN)  05 001 01  0    0    0   0   0    1    1    40
(XEN)  06 001 01  0    0    0   0   0    1    1    48
(XEN)  07 001 01  0    0    0   0   0    1    1    50
(XEN)  08 001 01  0    0    0   0   0    1    1    58
(XEN)  09 001 01  1    1    0   0   0    1    1    60
(XEN)  0a 001 01  0    0    0   0   0    1    1    68
(XEN)  0b 001 01  0    0    0   0   0    1    1    70
(XEN)  0c 001 01  0    0    0   0   0    1    1    78
(XEN)  0d 001 01  0    0    0   0   0    1    1    88
(XEN)  0e 001 01  0    0    0   0   0    1    1    90
(XEN)  0f 001 01  0    0    0   0   0    1    1    98
(XEN)  10 000 00  1    0    0   0   0    0    0    00
(XEN)  11 000 00  1    0    0   0   0    0    0    00
(XEN)  12 000 00  1    0    0   0   0    0    0    00
(XEN)  13 017 07  1    0    0   0   0    1    2    80
(XEN)  14 000 00  1    0    0   0   0    0    0    00
(XEN)  15 000 00  1    0    0   0   0    0    0    00
(XEN)  16 000 00  1    0    0   0   0    0    0    00
(XEN)  17 000 00  1    0    0   0   0    0    0    00
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ48 -> 0:1
(XEN) IRQ56 -> 0:3
(XEN) IRQ241 -> 0:4
(XEN) IRQ64 -> 0:5
(XEN) IRQ72 -> 0:6
(XEN) IRQ80 -> 0:7
(XEN) IRQ88 -> 0:8
(XEN) IRQ96 -> 0:9
(XEN) IRQ104 -> 0:10
(XEN) IRQ112 -> 0:11
(XEN) IRQ120 -> 0:12
(XEN) IRQ136 -> 0:13
(XEN) IRQ144 -> 0:14
(XEN) IRQ152 -> 0:15
(XEN) .................................... done.
(XEN) Using local APIC timer interrupts.
(XEN) calibrating APIC timer ...
(XEN) ..... CPU clock speed is 3092.9190 MHz.
(XEN) ..... host bus clock speed is 99.7715 MHz.
(XEN) ..... bus_scale = 0x662c
(XEN) TSC deadline timer enabled
(XEN) [2016-07-27 16:08:29] Platform timer is 14.318MHz HPET
(XEN) [2016-07-27 16:08:29] mwait-idle: MWAIT substates: 0x42120
(XEN) [2016-07-27 16:08:29] mwait-idle: v0.4.1 model 0x3c
(XEN) [2016-07-27 16:08:29] mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) [2016-07-27 16:08:29] VMX: Supported advanced features:
(XEN) [2016-07-27 16:08:29]  - APIC MMIO access virtualisation
(XEN) [2016-07-27 16:08:29]  - APIC TPR shadow
(XEN) [2016-07-27 16:08:29]  - Extended Page Tables (EPT)
(XEN) [2016-07-27 16:08:29]  - Virtual-Processor Identifiers (VPID)
(XEN) [2016-07-27 16:08:29]  - Virtual NMI
(XEN) [2016-07-27 16:08:29]  - MSR direct-access bitmap
(XEN) [2016-07-27 16:08:29]  - Unrestricted Guest
(XEN) [2016-07-27 16:08:29]  - VMCS shadowing
(XEN) [2016-07-27 16:08:29]  - VM Functions
(XEN) [2016-07-27 16:08:29] HVM: ASIDs enabled.
(XEN) [2016-07-27 16:08:29] HVM: VMX enabled
(XEN) [2016-07-27 16:08:29] HVM: Hardware Assisted Paging (HAP) detected
(XEN) [2016-07-27 16:08:29] HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) [2016-07-27 16:08:29] Suppress EOI broadcast on CPU#1
(XEN) [2016-07-27 16:08:29] masked ExtINT on CPU#1
(XEN) [2016-07-27 16:08:29] Adding cpu 1 to runqueue 1
(XEN) [2016-07-27 16:08:29] [VT-D]INTR-REMAP: Request device [0000:f0:1f.0] fault index 0, iommu reg = ffff82c0003e2000
(XEN) [2016-07-27 16:08:29]  First cpu on runqueue, activating
(XEN) [2016-07-27 16:08:29] [VT-D]INTR-REMAP: reason 25 - Blocked a compatibility format interrupt request
(XEN) [2016-07-27 16:08:29] Suppress EOI broadcast on CPU#2
(XEN) [2016-07-27 16:08:29] masked ExtINT on CPU#2
(XEN) [2016-07-27 16:08:29] Adding cpu 2 to runqueue 2
(XEN) [2016-07-27 16:08:29]  First cpu on runqueue, activating
(XEN) [2016-07-27 16:08:29] Suppress EOI broadcast on CPU#3
(XEN) [2016-07-27 16:08:29] masked ExtINT on CPU#3
(XEN) [2016-07-27 16:08:29] Adding cpu 3 to runqueue 3
(XEN) [2016-07-27 16:08:29]  First cpu on runqueue, activating
(XEN) [2016-07-27 16:08:29] Brought up 4 CPUs
(XEN) [2016-07-27 16:08:29] ACPI sleep modes: S3
(XEN) [2016-07-27 16:08:29] VPMU: disabled
(XEN) [2016-07-27 16:08:29] mcheck_poll: Machine check polling timer started.
(XEN) [2016-07-27 16:08:29] Dom0 has maximum 216 PIRQs
(XEN) [2016-07-27 16:08:29] csched2_dom_init: Initializing domain 0
(XEN) [2016-07-27 16:08:29] csched2_vcpu_insert: Inserting d0v0
(XEN) [2016-07-27 16:08:29] NX (Execute Disable) protection active
(XEN) [2016-07-27 16:08:29] *** LOADING DOMAIN 0 ***
(XEN) [2016-07-27 16:08:29]  Xen  kernel: 64-bit, lsb, compat32
(XEN) [2016-07-27 16:08:29]  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2290000
(XEN) [2016-07-27 16:08:29] PHYSICAL MEMORY ARRANGEMENT:
(XEN) [2016-07-27 16:08:29]  Dom0 alloc.:   0000000840000000->0000000844000000 (1028190 pages to be allocated)
(XEN) [2016-07-27 16:08:30]  Init. ramdisk: 000000085d05e000->000000085dfffb2c
(XEN) [2016-07-27 16:08:30] VIRTUAL MEMORY ARRANGEMENT:
(XEN) [2016-07-27 16:08:30]  Loaded kernel: ffffffff81000000->ffffffff82290000
(XEN) [2016-07-27 16:08:30]  Init. ramdisk: 0000000000000000->0000000000000000
(XEN) [2016-07-27 16:08:30]  Phys-Mach map: 0000008000000000->0000008000800000
(XEN) [2016-07-27 16:08:30]  Start info:    ffffffff82290000->ffffffff822904b4
(XEN) [2016-07-27 16:08:30]  Page tables:   ffffffff82291000->ffffffff822a6000
(XEN) [2016-07-27 16:08:30]  Boot stack:    ffffffff822a6000->ffffffff822a7000
(XEN) [2016-07-27 16:08:30]  TOTAL:         ffffffff80000000->ffffffff82400000
(XEN) [2016-07-27 16:08:30]  ENTRY ADDRESS: ffffffff81f42180
(XEN) [2016-07-27 16:08:30] Dom0 has maximum 1 VCPUs
(XEN) [2016-07-27 16:08:32] Bogus DMIBAR 0xfed18001 on 0000:00:00.0
(XEN) [2016-07-27 16:08:32] [VT-D]iommu_enable_translation: iommu->reg = ffff82c0003e2000
(XEN) [2016-07-27 16:08:32] Scrubbing Free RAM on 1 nodes using 4 CPUs
(XEN) [2016-07-27 16:08:32] ...................................................................done.
(XEN) [2016-07-27 16:08:34] Initial low memory virq threshold set at 0x4000 pages.
(XEN) [2016-07-27 16:08:34] Std. Loglevel: All
(XEN) [2016-07-27 16:08:34] Guest Loglevel: All
(XEN) [2016-07-27 16:08:34] **********************************************
(XEN) [2016-07-27 16:08:34] ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) [2016-07-27 16:08:34] ******* This option is intended to aid debugging of Xen by ensuring
(XEN) [2016-07-27 16:08:34] ******* that all output is synchronously delivered on the serial line.
(XEN) [2016-07-27 16:08:34] ******* However it can introduce SIGNIFICANT latencies and affect
(XEN) [2016-07-27 16:08:34] ******* timekeeping. It is NOT recommended for production use!
(XEN) [2016-07-27 16:08:34] **********************************************
(XEN) [2016-07-27 16:08:34] 3... 2... 1... 
(XEN) [2016-07-27 16:08:37] Xen is relinquishing VGA console.
(XEN) [2016-07-27 16:08:37] *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) [2016-07-27 16:08:37] Freed 328kB init memory.
mapping kernel into physical memory
about to get started...
[    0.000000] Linux version 4.7.0-4.g89a2ada-default (geeko@buildhost) (gcc version 6.1.1 20160707 [gcc-6-branch revision 238088] (SUSE Linux) ) #1 SMP PREEMPT Mon Jul 25 08:42:47 UTC 2016
 (89a2ada)
[    0.000000] Command line: root=/dev/mapper/VG0_ROOT  resume=/dev/mapper/VG0_SWAP  video
=vesa:off video=efifb:1024x768  xencons=xvc console=tty0 console=hvc0  clocksource=xen net.ifnames=1 biosdevname=1 pcie
_aspm=off mce=off  nomodeset drm.modeset=1 nouveau.modeset=1 showopts noquiet log_buf_len=10M print_fatal_signals=1 systemd.log_target=kmsg systemd.log_lev
el=info earlyprintk=xen debug loglevel=8 
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[    0.000000] x86/fpu: Using 'eager' FPU context switches.
[    0.000000] Released 0 page(s)
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] Xen: [mem 0x0000000000000000-0x0000000000007fff] reserved
[    0.000000] Xen: [mem 0x0000000000008000-0x0000000000047fff] usable
[    0.000000] Xen: [mem 0x0000000000048000-0x0000000000058fff] reserved
[    0.000000] Xen: [mem 0x0000000000059000-0x000000000005efff] usable
[    0.000000] Xen: [mem 0x000000000005f000-0x00000000000fffff] reserved
[    0.000000] Xen: [mem 0x0000000000100000-0x000000008d93dfff] usable
[    0.000000] Xen: [mem 0x000000008d93e000-0x000000008d944fff] ACPI NVS
[    0.000000] Xen: [mem 0x000000008d945000-0x000000008e259fff] reserved
[    0.000000] Xen: [mem 0x000000008e25a000-0x000000008e25ffff] usable
[    0.000000] Xen: [mem 0x000000008e260000-0x000000008e6a0fff] reserved
[    0.000000] Xen: [mem 0x000000008e6a1000-0x0000000091781fff] usable
[    0.000000] Xen: [mem 0x0000000091782000-0x00000000919e9fff] reserved
[    0.000000] Xen: [mem 0x00000000919ea000-0x0000000091a1cfff] usable
[    0.000000] Xen: [mem 0x0000000091a1d000-0x0000000091a79fff] reserved
[    0.000000] Xen: [mem 0x0000000091a7a000-0x0000000091adffff] usable
[    0.000000] Xen: [mem 0x0000000091ae0000-0x0000000091b85fff] reserved
[    0.000000] Xen: [mem 0x0000000091b86000-0x0000000091bb9fff] usable
[    0.000000] Xen: [mem 0x0000000091bba000-0x0000000091bbafff] reserved
[    0.000000] Xen: [mem 0x0000000091bbb000-0x0000000091bbcfff] usable
[    0.000000] Xen: [mem 0x0000000091bbd000-0x0000000091bbdfff] reserved
[    0.000000] Xen: [mem 0x0000000091bbe000-0x0000000091bc5fff] usable
[    0.000000] Xen: [mem 0x0000000091bc6000-0x0000000091bc8fff] reserved
[    0.000000] Xen: [mem 0x0000000091bc9000-0x0000000091bdbfff] usable
[    0.000000] Xen: [mem 0x0000000091bdc000-0x0000000091be6fff] reserved
[    0.000000] Xen: [mem 0x0000000091be7000-0x0000000091c67fff] usable
[    0.000000] Xen: [mem 0x0000000091c68000-0x0000000091ce4fff] reserved
[    0.000000] Xen: [mem 0x0000000091ce5000-0x0000000091d2efff] usable
[    0.000000] Xen: [mem 0x0000000091d2f000-0x000000009208cfff] reserved
[    0.000000] Xen: [mem 0x000000009208d000-0x00000000920d3fff] usable
[    0.000000] Xen: [mem 0x00000000920d4000-0x0000000092149fff] reserved
[    0.000000] Xen: [mem 0x000000009214a000-0x000000009217cfff] usable
[    0.000000] Xen: [mem 0x000000009217d000-0x0000000092294fff] reserved
[    0.000000] Xen: [mem 0x0000000092295000-0x00000000922b3fff] usable
[    0.000000] Xen: [mem 0x00000000922b4000-0x0000000092357fff] reserved
[    0.000000] Xen: [mem 0x0000000092358000-0x0000000092361fff] usable
[    0.000000] Xen: [mem 0x0000000092362000-0x0000000092460fff] reserved
[    0.000000] Xen: [mem 0x0000000092461000-0x0000000092465fff] usable
[    0.000000] Xen: [mem 0x0000000092466000-0x00000000925b4fff] reserved
[    0.000000] Xen: [mem 0x00000000925b5000-0x00000000925b6fff] usable
[    0.000000] Xen: [mem 0x00000000925b7000-0x00000000925bcfff] reserved
[    0.000000] Xen: [mem 0x00000000925bd000-0x00000000925befff] usable
[    0.000000] Xen: [mem 0x00000000925bf000-0x00000000925c4fff] reserved
[    0.000000] Xen: [mem 0x00000000925c5000-0x00000000925c6fff] usable
[    0.000000] Xen: [mem 0x00000000925c7000-0x00000000925cdfff] reserved
[    0.000000] Xen: [mem 0x00000000925ce000-0x00000000925d0fff] usable
[    0.000000] Xen: [mem 0x00000000925d1000-0x00000000925e3fff] reserved
[    0.000000] Xen: [mem 0x00000000925e4000-0x00000000925e6fff] usable
[    0.000000] Xen: [mem 0x00000000925e7000-0x00000000925e9fff] reserved
[    0.000000] Xen: [mem 0x00000000925ea000-0x00000000925ecfff] usable
[    0.000000] Xen: [mem 0x00000000925ed000-0x0000000092823fff] reserved
[    0.000000] Xen: [mem 0x0000000092824000-0x0000000092825fff] usable
[    0.000000] Xen: [mem 0x0000000092826000-0x000000009285ffff] reserved
[    0.000000] Xen: [mem 0x0000000092860000-0x0000000092861fff] usable
[    0.000000] Xen: [mem 0x0000000092862000-0x0000000092869fff] reserved
[    0.000000] Xen: [mem 0x000000009286a000-0x000000009286bfff] usable
[    0.000000] Xen: [mem 0x000000009286c000-0x0000000092871fff] reserved
[    0.000000] Xen: [mem 0x0000000092872000-0x0000000092873fff] usable
[    0.000000] Xen: [mem 0x0000000092874000-0x000000009287bfff] reserved
[    0.000000] Xen: [mem 0x000000009287c000-0x000000009287cfff] usable
[    0.000000] Xen: [mem 0x000000009287d000-0x000000009e19ffff] reserved
[    0.000000] Xen: [mem 0x000000009e1a0000-0x000000009e42ffff] usable
[    0.000000] Xen: [mem 0x000000009e430000-0x000000009e844fff] reserved
[    0.000000] Xen: [mem 0x000000009e845000-0x000000009e890fff] usable
[    0.000000] Xen: [mem 0x000000009e891000-0x000000009e9c4fff] ACPI NVS
[    0.000000] Xen: [mem 0x000000009e9c5000-0x000000009effffff] reserved
[    0.000000] Xen: [mem 0x00000000f0000000-0x00000000f7ffffff] reserved
[    0.000000] Xen: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] Xen: [mem 0x00000000fed00000-0x00000000fed03fff] reserved
[    0.000000] Xen: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[    0.000000] Xen: [mem 0x00000000fed90000-0x00000000fed90fff] reserved
[    0.000000] Xen: [mem 0x00000000fee00000-0x00000000feefffff] reserved
[    0.000000] Xen: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] Xen: [mem 0x0000000100000000-0x000000016f143fff] usable
[    0.000000] bootconsole [xenboot0] enabled
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] efi: EFI v2.31 by American Megatrends
[    0.000000] efi:  ESRT=0x9ef8d998  ACPI 2.0=0x9e995000  ACPI=0x9e995000  SMBIOS=0xf04c0  MPS=0xfd480 
[    0.000000] efi: EFI_MEMMAP is not enabled.
[    0.000000] esrt: ESRT header is not in the memory map.
[    0.000000] SMBIOS 2.7 present.
[    0.000000] DMI: Supermicro X10SAT/X10SAT, BIOS 3.0 05/26/2015
[    0.000000] Hypervisor detected: Xen
[    0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000000] e820: last_pfn = 0x16f144 max_arch_pfn = 0x400000000
[    0.000000] MTRR: Disabled
[    0.000000] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[    0.000000] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WC  WP  UC  UC  
[    0.000000] e820: last_pfn = 0x9e891 max_arch_pfn = 0x400000000
(XEN) [2016-07-27 16:08:38] d0v0: unhandled page fault (ec=0000)
(XEN) [2016-07-27 16:08:38] Pagetable walk from 0000000000000018:
(XEN) [2016-07-27 16:08:38]  L4[0x000] = 0000000000000000 ffffffffffffffff
(XEN) [2016-07-27 16:08:38] domain_crash_sync called from entry.S: fault at ffff82d08022ca4a entry.o#create_bounce_frame+0x137/0x146
(XEN) [2016-07-27 16:08:38] Domain 0 (vcpu#0) crashed on cpu#0:
(XEN) [2016-07-27 16:08:38] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
(XEN) [2016-07-27 16:08:38] CPU:    0
(XEN) [2016-07-27 16:08:38] RIP:    e033:[<ffffffff81f6374c>]
(XEN) [2016-07-27 16:08:38] RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest (d0v0)
(XEN) [2016-07-27 16:08:38] rax: 0000000000000000   rbx: 0000000000000000   rcx: 0000000100000000
(XEN) [2016-07-27 16:08:38] rdx: 000000016f144000   rsi: ffffffff81efb2c0   rdi: 000000016f144000
(XEN) [2016-07-27 16:08:38] rbp: 0000000000000000   rsp: ffffffff81e03e50   r8:  0000000000000200
(XEN) [2016-07-27 16:08:38] r9:  0000000000000000   r10: 0000000000007ff0   r11: 0000000000000067
(XEN) [2016-07-27 16:08:38] r12: 0000000000000000   r13: ffffffff81e03f28   r14: 0000000000000000
(XEN) [2016-07-27 16:08:38] r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526e0
(XEN) [2016-07-27 16:08:38] cr3: 0000000841e06000   cr2: 0000000000000018
(XEN) [2016-07-27 16:08:38] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
(XEN) [2016-07-27 16:08:38] Guest stack trace from rsp=ffffffff81e03e50:
(XEN) [2016-07-27 16:08:38]    0000000100000000 0000000000000067 0000000000000000 ffffffff81f6374c
(XEN) [2016-07-27 16:08:38]    000000010000e030 0000000000010046 ffffffff81e03e98 000000000000e02b
(XEN) [2016-07-27 16:08:38]    ffffffff820e2ce0 0000000001000000 0000000001290000 0000000000000000
(XEN) [2016-07-27 16:08:38]    ffffffff81e03f28 ffffffff81f4d940 ffffffff8118b419 ffffffff00000010
(XEN) [2016-07-27 16:08:38]    ffffffff81e03f28 ffffffff81e03ee0 000000000000001a 0000000000000000
(XEN) [2016-07-27 16:08:38]    ffffffffffffffff 0000000000000000 ffffffff81fe9920 0000000000000000
(XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 ffffffff81f42b92 ffffffff81ff62e0
(XEN) [2016-07-27 16:08:38]    ffffffff81e03f68 ffffffff81e03f64 0000000000000000 0000000000000000
(XEN) [2016-07-27 16:08:38]    ffffffff81f486b1 000306c300000000 0000000100100800 0300000100000032
(XEN) [2016-07-27 16:08:38]    0000000000000005 0000000000000000 0000000000000000 0000000000000000
(XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 0f00000060c0c748 ccccccccccccc305
(XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
(XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
(XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
(XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
(XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
(XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
(XEN) [2016-07-27 16:08:38] Hardware Dom0 crashed: 'noreboot' set - not rebooting.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Andrew Cooper]

On 27/07/16 17:21, lists@ssl-mail.com wrote:
> [    0.000000] DMI: Supermicro X10SAT/X10SAT, BIOS 3.0 05/26/2015
> [    0.000000] Hypervisor detected: Xen
> [    0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
> [    0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
> [    0.000000] e820: last_pfn = 0x16f144 max_arch_pfn = 0x400000000
> [    0.000000] MTRR: Disabled
> [    0.000000] x86/PAT: MTRRs disabled, skipping PAT initialization too.
> [    0.000000] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WC  WP  UC  UC  
> [    0.000000] e820: last_pfn = 0x9e891 max_arch_pfn = 0x400000000
> (XEN) [2016-07-27 16:08:38] d0v0: unhandled page fault (ec=0000)
> (XEN) [2016-07-27 16:08:38] Pagetable walk from 0000000000000018:
> (XEN) [2016-07-27 16:08:38]  L4[0x000] = 0000000000000000 ffffffffffffffff
> (XEN) [2016-07-27 16:08:38] domain_crash_sync called from entry.S: fault at ffff82d08022ca4a entry.o#create_bounce_frame+0x137/0x146
> (XEN) [2016-07-27 16:08:38] Domain 0 (vcpu#0) crashed on cpu#0:
> (XEN) [2016-07-27 16:08:38] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
> (XEN) [2016-07-27 16:08:38] CPU:    0
> (XEN) [2016-07-27 16:08:38] RIP:    e033:[<ffffffff81f6374c>]
> (XEN) [2016-07-27 16:08:38] RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest (d0v0)
> (XEN) [2016-07-27 16:08:38] rax: 0000000000000000   rbx: 0000000000000000   rcx: 0000000100000000
> (XEN) [2016-07-27 16:08:38] rdx: 000000016f144000   rsi: ffffffff81efb2c0   rdi: 000000016f144000
> (XEN) [2016-07-27 16:08:38] rbp: 0000000000000000   rsp: ffffffff81e03e50   r8:  0000000000000200
> (XEN) [2016-07-27 16:08:38] r9:  0000000000000000   r10: 0000000000007ff0   r11: 0000000000000067
> (XEN) [2016-07-27 16:08:38] r12: 0000000000000000   r13: ffffffff81e03f28   r14: 0000000000000000
> (XEN) [2016-07-27 16:08:38] r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526e0
> (XEN) [2016-07-27 16:08:38] cr3: 0000000841e06000   cr2: 0000000000000018
> (XEN) [2016-07-27 16:08:38] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> (XEN) [2016-07-27 16:08:38] Guest stack trace from rsp=ffffffff81e03e50:
> (XEN) [2016-07-27 16:08:38]    0000000100000000 0000000000000067 0000000000000000 ffffffff81f6374c
> (XEN) [2016-07-27 16:08:38]    000000010000e030 0000000000010046 ffffffff81e03e98 000000000000e02b
> (XEN) [2016-07-27 16:08:38]    ffffffff820e2ce0 0000000001000000 0000000001290000 0000000000000000
> (XEN) [2016-07-27 16:08:38]    ffffffff81e03f28 ffffffff81f4d940 ffffffff8118b419 ffffffff00000010
> (XEN) [2016-07-27 16:08:38]    ffffffff81e03f28 ffffffff81e03ee0 000000000000001a 0000000000000000
> (XEN) [2016-07-27 16:08:38]    ffffffffffffffff 0000000000000000 ffffffff81fe9920 0000000000000000
> (XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 ffffffff81f42b92 ffffffff81ff62e0
> (XEN) [2016-07-27 16:08:38]    ffffffff81e03f68 ffffffff81e03f64 0000000000000000 0000000000000000
> (XEN) [2016-07-27 16:08:38]    ffffffff81f486b1 000306c300000000 0000000100100800 0300000100000032
> (XEN) [2016-07-27 16:08:38]    0000000000000005 0000000000000000 0000000000000000 0000000000000000
> (XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) [2016-07-27 16:08:38]    0000000000000000 0000000000000000 0f00000060c0c748 ccccccccccccc305
> (XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
> (XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
> (XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
> (XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
> (XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
> (XEN) [2016-07-27 16:08:38]    cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc
> (XEN) [2016-07-27 16:08:38] Hardware Dom0 crashed: 'noreboot' set - not rebooting.

This looks suspiciously like the issue which was fixed by c/s
d6b186c1e2d852a92c43f090d0d8fad4704d51ef "x86/xen: avoid m2p lookup when
setting early page table entries", but that fix is present in Linux 4.7.0

Can you check to see whether the commit is included in your kernel?

Failing that, can you find out exactly where the kernel crashed?  You
need to manually decode ffffffff81f6374c with the debug symbols.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Jul 27, 2016, at 09:34 AM, Andrew Cooper wrote:
> This looks suspiciously like the issue which was fixed by c/s
> d6b186c1e2d852a92c43f090d0d8fad4704d51ef "x86/xen: avoid m2p lookup when
> setting early page table entries", but that fix is present in Linux 4.7.0
> 
> Can you check to see whether the commit is included in your kernel?

This

 https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable.git/+/d6b186c1e2d852a92c43f090d0d8fad4704d51ef

looks like the Suse team was involved,

  Reviewed-by: Juergen Gross <jgross@suse.com>

But checking this kernel

 rpm -q kernel-source
  kernel-source-4.7.0-4.1.g89a2ada.noarch
 rpm -q --changelog kernel-source | egrep -i "m2p|d6b186c1"
  - guarantee M2P to be invisible to user mode.

it doesn't look like it's in here.

> Failing that, can you find out exactly where the kernel crashed?  You
> need to manually decode ffffffff81f6374c with the debug symbols.

Sure can try.  I'm gonna have to read-up on how .  Atm no clue.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Andrew Cooper]

On 27/07/16 17:54, lists@ssl-mail.com wrote:
>
> On Wed, Jul 27, 2016, at 09:34 AM, Andrew Cooper wrote:
>> This looks suspiciously like the issue which was fixed by c/s
>> d6b186c1e2d852a92c43f090d0d8fad4704d51ef "x86/xen: avoid m2p lookup when
>> setting early page table entries", but that fix is present in Linux 4.7.0
>>
>> Can you check to see whether the commit is included in your kernel?
> This
>
>  https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable.git/+/d6b186c1e2d852a92c43f090d0d8fad4704d51ef
>
> looks like the Suse team was involved,
>
>   Reviewed-by: Juergen Gross <jgross@suse.com>
>
> But checking this kernel
>
>  rpm -q kernel-source
>   kernel-source-4.7.0-4.1.g89a2ada.noarch
>  rpm -q --changelog kernel-source | egrep -i "m2p|d6b186c1"
>   - guarantee M2P to be invisible to user mode.
>
> it doesn't look like it's in here.

Juergen: Any ideas?

>
>> Failing that, can you find out exactly where the kernel crashed?  You
>> need to manually decode ffffffff81f6374c with the debug symbols.
> Sure can try.  I'm gonna have to read-up on how .  Atm no clue.

addr2line -e /path/to/linux/debug/symbols ffffffff81f6374c

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Jul 27, 2016, at 09:56 AM, Andrew Cooper wrote:
> >> Failing that, can you find out exactly where the kernel crashed?  You
> >> need to manually decode ffffffff81f6374c with the debug symbols.
> > Sure can try.  I'm gonna have to read-up on how .  Atm no clue.
> 
> addr2line -e /path/to/linux/debug/symbols ffffffff81f6374c

Thanks.

For whatever reason it the dev debug symbols aren't available for the official Kernel:Stable branch,

	http://software.opensuse.org/package/kernel-default-debuginfo?search_term=kernel-default-debuginfo

unlike for the release 4.1.x kernel.

But there is this from an unofficial repo

	https://build.opensuse.org/package/binaries/hardware:nvdimm/kernel-default?repository=openSUSE_Leap_42.1

I'm not sure if that's good enough.  Anyway if I directly install that one

	http://download.opensuse.org/repositories/hardware:/nvdimm/openSUSE_Leap_42.1/x86_64/kernel-default-debuginfo-4.7.0-5.1.x86_64.rpm

Then

	rpm -ql kernel-default-debuginfo | grep linux
		/usr/lib/debug/boot/vmlinux-4.7.0-5-default.debug

	addr2line -e /usr/lib/debug/boot/vmlinux-4.7.0-5-default.debug ffffffff81f6374c
		/usr/src/debug/kernel-default-4.7.0/linux-4.7/linux-obj/../arch/x86/mm/numa_emulation.c:444

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Andrew Cooper]

On 27/07/16 19:22, lists@ssl-mail.com wrote:
>
> On Wed, Jul 27, 2016, at 09:56 AM, Andrew Cooper wrote:
>>>> Failing that, can you find out exactly where the kernel crashed?  You
>>>> need to manually decode ffffffff81f6374c with the debug symbols.
>>> Sure can try.  I'm gonna have to read-up on how .  Atm no clue.
>> addr2line -e /path/to/linux/debug/symbols ffffffff81f6374c
> Thanks.
>
> For whatever reason it the dev debug symbols aren't available for the official Kernel:Stable branch,
>
> 	http://software.opensuse.org/package/kernel-default-debuginfo?search_term=kernel-default-debuginfo
>
> unlike for the release 4.1.x kernel.
>
> But there is this from an unofficial repo
>
> 	https://build.opensuse.org/package/binaries/hardware:nvdimm/kernel-default?repository=openSUSE_Leap_42.1
>
> I'm not sure if that's good enough.

Sadly not.  The debug symbols need to be specific to the exact binary
you booted.

Any change in the compilation will result in the translation being
useless.  What addr2line is doing is saying "which specific bit of
source code did the compiler/linker end up putting at $X".

I wouldn't worry too much - I expect (the lack of) the aforementioned
changeset is the cause of your issues.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Jul 27, 2016, at 11:28 AM, Andrew Cooper wrote:
> > I'm not sure if that's good enough.
> 
> Sadly not.  The debug symbols need to be specific to the exact binary
> you booted.
> 
> Any change in the compilation will result in the translation being
> useless.  What addr2line is doing is saying "which specific bit of
> source code did the compiler/linker end up putting at $X".

Got it.  Weird that they don't put the .debuginfo rpms in there.  While I was searching around kernel bug reports over at the distro there's lots of posts telling people to debug.  Not sure then how you do it without the debug symbols.

Guess you have to build your own kernel.

That's one of the things I'm thinking of doing , along with Xen, eventually so I don't have to waste days hoping packages get fixed when someone helps out with a suggestion to try, like you did.  But I never got the "build it all yourself" working when I tried awhile ago.  

> I wouldn't worry too much - I expect (the lack of) the aforementioned
> changeset is the cause of your issues.

Ok we'll wait and see if there's an update to try, I guess.

Thanks.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Jul 27, 2016, at 11:36 AM, lists@ssl-mail.com wrote:
> On Wed, Jul 27, 2016, at 11:28 AM, Andrew Cooper wrote:
> > > I'm not sure if that's good enough.
> > 
> > Sadly not.  The debug symbols need to be specific to the exact binary
> > you booted.
> > 
> > Any change in the compilation will result in the translation being
> > useless.  What addr2line is doing is saying "which specific bit of
> > source code did the compiler/linker end up putting at $X".
> 
> Got it.  Weird that they don't put the .debuginfo rpms in there.  While I was searching around kernel bug reports over at the distro there's lots of posts telling people to debug.  Not sure then how you do it without the debug symbols.
> 
> Guess you have to build your own kernel.

I got my hands on a 'matched set'

	rpm -qa kernel-default\*
		kernel-default-4.7.0-5.1.x86_64
		kernel-default-devel-4.7.0-5.1.x86_64
		kernel-default-debuginfo-4.7.0-5.1.x86_64

reboot to Xen, still crashes

		(XEN) [2016-07-28 00:13:18] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
		(XEN) [2016-07-28 00:13:18] CPU:    0
>>>		(XEN) [2016-07-28 00:13:18] RIP:    e033:[<ffffffff81f63eb0>]
		(XEN) [2016-07-28 00:13:18] RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest (d0v0)
		(XEN) [2016-07-28 00:13:18] rax: 0000000000000000   rbx: 0000000000000000   rcx: 000000016f144000
		(XEN) [2016-07-28 00:13:18] rdx: 0000000100000000   rsi: 000000016f144000   rdi: fffffffffffff000
		(XEN) [2016-07-28 00:13:18] rbp: 0000000001000000   rsp: ffffffff81e03e50   r8:  ffffffff81efb0c0
		(XEN) [2016-07-28 00:13:18] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000100000000
		(XEN) [2016-07-28 00:13:18] r12: 0000000000000000   r13: 0000000000000000   r14: ffffffff81e03f28
		(XEN) [2016-07-28 00:13:18] r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526e0
		(XEN) [2016-07-28 00:13:18] cr3: 0000000841e06000   cr2: 0000000000000018
		(XEN) [2016-07-28 00:13:18] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
		(XEN) [2016-07-28 00:13:18] Guest stack trace from rsp=ffffffff81e03e50:

check ar the RIP addr

	addr2line -e /usr/lib/debug/boot/vmlinux-4.7.0-5-default.debug ffffffff81f63eb0
		/usr/src/debug/kernel-default-4.7.0/linux-4.7/linux-obj/../arch/x86/platform/efi/efi.c:123

in source

	@ https://github.com/torvalds/linux/blob/v4.7/arch/x86/platform/efi/efi.c

		...
		void __init efi_find_mirror(void)
		{
			efi_memory_desc_t *md;
			u64 mirror_size = 0, total_size = 0;

			for_each_efi_memory_desc(md) {
				unsigned long long start = md->phys_addr;
123				unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;

				total_size += size;
				if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
					memblock_mark_mirror(start, size);
					mirror_size += size;
				}
			}
			if (mirror_size)
				pr_info("Memory: %lldM/%lldM mirrored memory\n",
					mirror_size>>20, total_size>>20);
		}
		...

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Jul 27, 2016, at 05:28 PM, lists@ssl-mail.com wrote:
> 123				unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;

If I'm reading it right, that originated in this pull

Date	Mon, 16 May 2016 16:43:11 +0200
From	Ingo Molnar <>
Subject	[GIT PULL] EFI changes for v4.7
https://lkml.org/lkml/2016/5/16/244

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Konrad Rzeszutek Wilk]

> > > Sadly not.  The debug symbols need to be specific to the exact binary
> > > you booted.
> > > 
> > > Any change in the compilation will result in the translation being
> > > useless.  What addr2line is doing is saying "which specific bit of
> > > source code did the compiler/linker end up putting at $X".
> > 
> > Got it.  Weird that they don't put the .debuginfo rpms in there.  While I was searching around kernel bug reports over at the distro there's lots of posts telling people to debug.  Not sure then how you do it without the debug symbols.
> > 
> > Guess you have to build your own kernel.
> 
> I got my hands on a 'matched set'
> 
> 	rpm -qa kernel-default\*
> 		kernel-default-4.7.0-5.1.x86_64
> 		kernel-default-devel-4.7.0-5.1.x86_64
> 		kernel-default-debuginfo-4.7.0-5.1.x86_64
> 
> reboot to Xen, still crashes
> 
> 		(XEN) [2016-07-28 00:13:18] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
> 		(XEN) [2016-07-28 00:13:18] CPU:    0
> >>>		(XEN) [2016-07-28 00:13:18] RIP:    e033:[<ffffffff81f63eb0>]
> 		(XEN) [2016-07-28 00:13:18] RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest (d0v0)
> 		(XEN) [2016-07-28 00:13:18] rax: 0000000000000000   rbx: 0000000000000000   rcx: 000000016f144000
> 		(XEN) [2016-07-28 00:13:18] rdx: 0000000100000000   rsi: 000000016f144000   rdi: fffffffffffff000
> 		(XEN) [2016-07-28 00:13:18] rbp: 0000000001000000   rsp: ffffffff81e03e50   r8:  ffffffff81efb0c0
> 		(XEN) [2016-07-28 00:13:18] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000100000000
> 		(XEN) [2016-07-28 00:13:18] r12: 0000000000000000   r13: 0000000000000000   r14: ffffffff81e03f28
> 		(XEN) [2016-07-28 00:13:18] r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526e0
> 		(XEN) [2016-07-28 00:13:18] cr3: 0000000841e06000   cr2: 0000000000000018
> 		(XEN) [2016-07-28 00:13:18] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> 		(XEN) [2016-07-28 00:13:18] Guest stack trace from rsp=ffffffff81e03e50:
> 
> check ar the RIP addr
> 
> 	addr2line -e /usr/lib/debug/boot/vmlinux-4.7.0-5-default.debug ffffffff81f63eb0
> 		/usr/src/debug/kernel-default-4.7.0/linux-4.7/linux-obj/../arch/x86/platform/efi/efi.c:123
> 
> in source
> 
> 	@ https://github.com/torvalds/linux/blob/v4.7/arch/x86/platform/efi/efi.c
> 
> 		...
> 		void __init efi_find_mirror(void)
> 		{
> 			efi_memory_desc_t *md;
> 			u64 mirror_size = 0, total_size = 0;
> 
> 			for_each_efi_memory_desc(md) {
> 				unsigned long long start = md->phys_addr;
> 123				unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> 
> 				total_size += size;
> 				if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
> 					memblock_mark_mirror(start, size);
> 					mirror_size += size;
> 				}
> 			}
> 			if (mirror_size)
> 				pr_info("Memory: %lldM/%lldM mirrored memory\n",
> 					mirror_size>>20, total_size>>20);
> 		}
> 		...
> 

+CC-ing Daniel.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

anyone need any addl info from my end to help ?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Daniel Kiper]

On Wed, Jul 27, 2016 at 09:09:52PM -0400, Konrad Rzeszutek Wilk wrote:
> > > > Sadly not.  The debug symbols need to be specific to the exact binary
> > > > you booted.
> > > >
> > > > Any change in the compilation will result in the translation being
> > > > useless.  What addr2line is doing is saying "which specific bit of
> > > > source code did the compiler/linker end up putting at $X".
> > >
> > > Got it.  Weird that they don't put the .debuginfo rpms in there.  While I was searching around kernel bug reports over at the distro there's lots of posts telling people to debug.  Not sure then how you do it without the debug symbols.
> > >
> > > Guess you have to build your own kernel.
> >
> > I got my hands on a 'matched set'
> >
> > 	rpm -qa kernel-default\*
> > 		kernel-default-4.7.0-5.1.x86_64
> > 		kernel-default-devel-4.7.0-5.1.x86_64
> > 		kernel-default-debuginfo-4.7.0-5.1.x86_64
> >
> > reboot to Xen, still crashes
> >
> > 		(XEN) [2016-07-28 00:13:18] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
> > 		(XEN) [2016-07-28 00:13:18] CPU:    0
> > >>>		(XEN) [2016-07-28 00:13:18] RIP:    e033:[<ffffffff81f63eb0>]
> > 		(XEN) [2016-07-28 00:13:18] RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest (d0v0)
> > 		(XEN) [2016-07-28 00:13:18] rax: 0000000000000000   rbx: 0000000000000000   rcx: 000000016f144000
> > 		(XEN) [2016-07-28 00:13:18] rdx: 0000000100000000   rsi: 000000016f144000   rdi: fffffffffffff000
> > 		(XEN) [2016-07-28 00:13:18] rbp: 0000000001000000   rsp: ffffffff81e03e50   r8:  ffffffff81efb0c0
> > 		(XEN) [2016-07-28 00:13:18] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000100000000
> > 		(XEN) [2016-07-28 00:13:18] r12: 0000000000000000   r13: 0000000000000000   r14: ffffffff81e03f28
> > 		(XEN) [2016-07-28 00:13:18] r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526e0
> > 		(XEN) [2016-07-28 00:13:18] cr3: 0000000841e06000   cr2: 0000000000000018
> > 		(XEN) [2016-07-28 00:13:18] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> > 		(XEN) [2016-07-28 00:13:18] Guest stack trace from rsp=ffffffff81e03e50:
> >
> > check ar the RIP addr
> >
> > 	addr2line -e /usr/lib/debug/boot/vmlinux-4.7.0-5-default.debug ffffffff81f63eb0
> > 		/usr/src/debug/kernel-default-4.7.0/linux-4.7/linux-obj/../arch/x86/platform/efi/efi.c:123
> >
> > in source
> >
> > 	@ https://github.com/torvalds/linux/blob/v4.7/arch/x86/platform/efi/efi.c
> >
> > 		...
> > 		void __init efi_find_mirror(void)
> > 		{
> > 			efi_memory_desc_t *md;
> > 			u64 mirror_size = 0, total_size = 0;
> >
> > 			for_each_efi_memory_desc(md) {
> > 				unsigned long long start = md->phys_addr;
> > 123				unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> >
> > 				total_size += size;
> > 				if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
> > 					memblock_mark_mirror(start, size);
> > 					mirror_size += size;
> > 				}
> > 			}
> > 			if (mirror_size)
> > 				pr_info("Memory: %lldM/%lldM mirrored memory\n",
> > 					mirror_size>>20, total_size>>20);
> > 		}
> > 		...
> >
>
> +CC-ing Daniel.

Hmmm.... Could you provide full console dump from Xen and Linux kernel?

Daniel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

> Hmmm.... Could you provide full console dump from Xen and Linux kernel?

Will serial console output with these options

	kernel:     earlyprintk=xen,keep debug loglevel=8
	hypervisor: loglvl=all guest_loglvl=all sync_console console_to_ring

do?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Daniel Kiper]

On Thu, Jul 28, 2016 at 11:25:42AM -0700, lists@ssl-mail.com wrote:
> > Hmmm.... Could you provide full console dump from Xen and Linux kernel?
>
> Will serial console output with these options
>
> 	kernel:     earlyprintk=xen,keep debug loglevel=8
> 	hypervisor: loglvl=all guest_loglvl=all sync_console console_to_ring
>
> do?

I think that you should add to above mentioned hypervisor command line
at least "com1=115200,8n1 console=com1". Of course this is an example.
You should find your serial port details and configure it properly.

Daniel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On 07/28/2016 11:25 AM, lists@ssl-mail.com wrote:>
> Hmmm.... Could you provide full console dump from Xen and Linux kernel?
> 
> Will serial console output with these options
> 
> 	kernel:     earlyprintk=xen,keep debug loglevel=8
> 	hypervisor: loglvl=all guest_loglvl=all sync_console console_to_ring
> 
> do?

I'll just assume it does.

So full console output from boot -> crash now doesn't look any different than 

	https://lists.xen.org/archives/html/xen-devel/2016-07/msg02814.html

		On 07/27/2016 08:50 AM, Andrew Cooper wrote
		>> For the Linux crash, can you boot Linux with "earlyprintk=xen" and see
		>> if that provides more help as to what went wrong?
		>
		>Here's serial console output with grub2 log parameters included as
		>
		>        kernel:     earlyprintk=xen,keep debug loglevel=8
		>        hypervisor: loglvl=all guest_loglvl=all sync_console console_to_ring


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Jul 27, 2016, at 09:34 AM, Andrew Cooper wrote:
> This looks suspiciously like the issue which was fixed by c/s
> d6b186c1e2d852a92c43f090d0d8fad4704d51ef "x86/xen: avoid m2p lookup when
> setting early page table entries", but that fix is present in Linux 4.7.0
> 
> Can you check to see whether the commit is included in your kernel?
> 
> Failing that, can you find out exactly where the kernel crashed?  You
> need to manually decode ffffffff81f6374c with the debug symbols.
> 
> ~Andrew

Is an eventual fix to whatever is causing this crash likely in the kernel or Xen code?

I've managed to try a couple of different kernels, older version, with no luck.  Crash still happens.

Getting my hands on an older Xen package is turning out to be a bit tougher.  Still working on it.

Not sure what specifically changed to break this , since this server clearly was running not that long ago.

I'd like to at least figure out what I can drop back to so I can get my server and guests back up and running.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Konrad Rzeszutek Wilk]

On Fri, Jul 29, 2016 at 07:36:57AM -0700, lists@ssl-mail.com wrote:
> On Wed, Jul 27, 2016, at 09:34 AM, Andrew Cooper wrote:
> > This looks suspiciously like the issue which was fixed by c/s
> > d6b186c1e2d852a92c43f090d0d8fad4704d51ef "x86/xen: avoid m2p lookup when
> > setting early page table entries", but that fix is present in Linux 4.7.0
> > 
> > Can you check to see whether the commit is included in your kernel?
> > 
> > Failing that, can you find out exactly where the kernel crashed?  You
> > need to manually decode ffffffff81f6374c with the debug symbols.
> > 
> > ~Andrew
> 
> Is an eventual fix to whatever is causing this crash likely in the kernel or Xen code?
> 
> I've managed to try a couple of different kernels, older version, with no luck.  Crash still happens.
> 
> Getting my hands on an older Xen package is turning out to be a bit tougher.  Still working on it.
> 
> Not sure what specifically changed to break this , since this server clearly was running not that long ago.

did you apply the patch that Vitaly pointed out?
> 
> I'd like to at least figure out what I can drop back to so I can get my server and guests back up and running.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Fri, Jul 29, 2016, at 08:42 AM, Konrad Rzeszutek Wilk wrote:
> did you apply the patch that Vitaly pointed out?

No.  It wasn't clear that it was anything more than a question to "double-check".   There wasn't any further comment on my reply.

I'm depending on working packages for now.  Like I said earlier building my own packages is something I haven't gotten to work to the point that I can boot up without causing other problems yet.

I'm trying to get to that point, but not there today.

If that's a 'must fix this' patch, then I don't quite get why it doesn't just get added to the distro packages.  The devs from the distro are obviously here.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Konrad Rzeszutek Wilk]

On Fri, Jul 29, 2016 at 08:57:59AM -0700, lists@ssl-mail.com wrote:
> 
> 
> On Fri, Jul 29, 2016, at 08:42 AM, Konrad Rzeszutek Wilk wrote:
> > did you apply the patch that Vitaly pointed out?
> 
> No.  It wasn't clear that it was anything more than a question to "double-check".   There wasn't any further comment on my reply.
> 
> I'm depending on working packages for now.  Like I said earlier building my own packages is something I haven't gotten to work to the point that I can boot up without causing other problems yet.
> 
> I'm trying to get to that point, but not there today.
> 
> If that's a 'must fix this' patch, then I don't quite get why it doesn't just get added to the distro packages.  The devs from the distro are obviously here.

It may very well be added.

But having extra test-confirmation is always good.




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Fri, Jul 29, 2016, at 09:03 AM, Konrad Rzeszutek Wilk wrote:
> It may very well be added.

Just fyi, it's not in here.  Yet.

> But having extra test-confirmation is always good.

Right, and I'm glad to do that.  I'd like to.  Goal is to keep moving the ball forward.

And I've been testing what I've been asked to when I can do it with packages that I have and that work.

I just haven't figured out yet how to use their 'build system' to build a kernel and get it working to boot.

I built a kernel the 'upstream' way but that won't boot Xen at all.  Sure it's something I'm not doing right, I know.

So

(1) I'll keep trying to DIY and get something running
(2) As soon as the distro patches their packages with the patches you all say it should have, then I'll test them and report back

IF there's a different combination of different OS + packages for a Kernel version + Xen version that is known to work with UEFI Dom0 & UEFI Guests then I'm all ears.

A friend's starting to test KVM with QEMU + OVMF/UEFI on another box, and so far so good I guess.  I don't want to use it for a bunch of reasons, so I'm sticking it out in here.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Fri, Jul 29, 2016, at 09:03 AM, Konrad Rzeszutek Wilk wrote:
> It may very well be added.
> 
> But having extra test-confirmation is always good.

looking at the patch

	diff --git a/include/linux/efi.h b/include/linux/efi.h
	index c2db3ca..f196dd0 100644
	--- a/include/linux/efi.h
	+++ b/include/linux/efi.h
	@@ -1005,7 +1005,7 @@  extern int efi_memattr_apply_permissions(struct mm_struct *mm,
	 /* Iterate through an efi_memory_map */
	 #define for_each_efi_memory_desc_in_map(m, md)				   \
		for ((md) = (m)->map;						   \
	-	     (md) <= (efi_memory_desc_t *)((m)->map_end - (m)->desc_size); \
	+	     ((void *)(md) + (m)->desc_size) <= (m)->map_end;		   \
		     (md) = (void *)(md) + (m)->desc_size)

	 /**

and in the source used in the distro build

	wget https://build.opensuse.org/source/Kernel:stable/kernel-default/linux-4.7.tar.xz?rev=88182fafc706b2366e0251692bd4b7e7
	tar zxf linux-4.7.tar.xz
	cd linux-4.7/include/linux/
	edit efi.h

it looks like the patch is in there already

	1005	/* Iterate through an efi_memory_map */
			#define for_each_efi_memory_desc_in_map(m, md)				   \
				for ((md) = (m)->map;						   \
>>>				     ((void *)(md) + (m)->desc_size) <= (m)->map_end;		   \
				     (md) = (void *)(md) + (m)->desc_size)

even if it's not mentioned in the changelog.

The binary built with this source crashes as reported.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[BUG] Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

Horse is already WAY out of the barn on this, but just realizing the instructions say to

> Please tag your subject line with a '[BUG]' prefix.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Jan Beulich]

>>> On 01.08.16 at 22:11, <lists@ssl-mail.com> wrote:
> On Fri, Jul 29, 2016, at 09:03 AM, Konrad Rzeszutek Wilk wrote:
>> It may very well be added.
>> 
>> But having extra test-confirmation is always good.
> 
> looking at the patch
> 
> 	diff --git a/include/linux/efi.h b/include/linux/efi.h
> 	index c2db3ca..f196dd0 100644
> 	--- a/include/linux/efi.h
> 	+++ b/include/linux/efi.h
> 	@@ -1005,7 +1005,7 @@  extern int efi_memattr_apply_permissions(struct mm_struct *mm,
> 	 /* Iterate through an efi_memory_map */
> 	 #define for_each_efi_memory_desc_in_map(m, md)				   \
> 		for ((md) = (m)->map;						   \
> 	-	     (md) <= (efi_memory_desc_t *)((m)->map_end - (m)->desc_size); \
> 	+	     ((void *)(md) + (m)->desc_size) <= (m)->map_end;		   \
> 		     (md) = (void *)(md) + (m)->desc_size)
> 
> 	 /**
> 
> and in the source used in the distro build
> 
> 	wget 
> https://build.opensuse.org/source/Kernel:stable/kernel-default/linux-4.7.tar.
> xz?rev=88182fafc706b2366e0251692bd4b7e7
> 	tar zxf linux-4.7.tar.xz
> 	cd linux-4.7/include/linux/
> 	edit efi.h
> 
> it looks like the patch is in there already
> 
> 	1005	/* Iterate through an efi_memory_map */
> 			#define for_each_efi_memory_desc_in_map(m, md)				   \
> 				for ((md) = (m)->map;						   \
>>>>				     ((void *)(md) + (m)->desc_size) <= (m)->map_end;		   \
> 				     (md) = (void *)(md) + (m)->desc_size)
> 
> even if it's not mentioned in the changelog.

Obviously it's not mentioned, as it's in the base tarball.

But said patch is likely insufficient for the Xen case: I suspect
efi.memmap.desc_size never gets set in this case, and hence
the <= above will always be true, despite md being NULL. Can
you try

			     ((void *)(md) + (m)->desc_size - 1) < (m)->map_end;		   \

instead? But in the end no memory map related function should
ever be called in the kernel in the first place when running under
Xen - one of the many weaknesses of the upstream pv-ops
approach: Problems like this need to be found actively instead of
getting detected at build time.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Mon, Aug 1, 2016, at 11:57 PM, Jan Beulich wrote:
> Obviously it's not mentioned, as it's in the base tarball.

Not obvious at all.  What seemed obvious is that the changelog would show all the changes.

It doesn't and it wasn't mentioned.  Now I know.

> Can you try
> 
> 			     ((void *)(md) + (m)->desc_size - 1) < (m)->map_end;		   \
> 
> instead?

Sure, as soon as I can figure out how to successfully patch & build either the distro kernel, or a vanilla one that'll boot the system.

Can't do either one just yet.  Working on it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

> > Can you try
> > 
> > 			     ((void *)(md) + (m)->desc_size - 1) < (m)->map_end;		   \
> > 
> > instead?

with the 'baseline' as referenced + a patched kernel

	> Can you try
	>	     ((void *)(md) + (m)->desc_size - 1) < (m)->map_end;		   \

with efi cmd line opts: +"/mapbs"

The system now boots correctly

	uname -rm
		4.7.0-6-default x86_64

	xl list
		Name                                        ID   Mem VCPUs      State   Time(s)
		Domain-0                                     0  4096     1     r-----      52.4
		g1                                           1  2049     1     -b----      15.0
		g2                                           2  1025     1     -b----      15.5
		g3                                           3  1025     1     -b----      16.2
		g4                                           4  1025     1     -b----      19.6

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Jan Beulich]

>>> On 02.08.16 at 21:02, <lists@ssl-mail.com> wrote:
>> > Can you try
>> > 
>> > 			     ((void *)(md) + (m)->desc_size - 1) < (m)->map_end;		   \
>> > 
>> > instead?
> 
> with the 'baseline' as referenced + a patched kernel
> 
> 	> Can you try
> 	>	     ((void *)(md) + (m)->desc_size - 1) < (m)->map_end;		   \
> 
> with efi cmd line opts: +"/mapbs"
> 
> The system now boots correctly

Thanks. Does the use of /mapbs really matter for booting? I was
assuming it would be relevant only for shutdown/reboot?

I'd like to credit you with Reported-by and Tested-by tags, but those
will look odd without an real name. Please let me know if I should
rather not add such tags to the patch or what your real name is.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Aug 3, 2016, at 02:01 AM, Jan Beulich wrote:
> Thanks. Does the use of /mapbs really matter for booting? I was
> assuming it would be relevant only for shutdown/reboot?

It has no effect on boot. With or without the "/mapbs" it boots Xen OK.

Without the "/mapbs" the system used to crash on reboot, but still continued with a reboot.  Reboot with the "/mapbs" was OK.

Now, even with the "/mapbs" the system crashes on reboot.


	[  196.052042] reboot: Restarting system
	(XEN) [2016-08-03 13:06:54] Hardware Dom0 shutdown: rebooting machine
	(XEN) [2016-08-03 13:06:54] APIC error on CPU0: 40(00)
	(XEN) [2016-08-03 13:06:54] ----[ Xen-4.7.0_09-454  x86_64  debug=n  Not tainted ]----
	(XEN) [2016-08-03 13:06:54] CPU:    0
	(XEN) [2016-08-03 13:06:54] RIP:    e008:[<000000009e746340>] 000000009e746340
	(XEN) [2016-08-03 13:06:54] RFLAGS: 0000000000010202   CONTEXT: hypervisor (d0v0)
	(XEN) [2016-08-03 13:06:54] rax: 000000009e746340   rbx: 0000000000000000   rcx: 0000000000000000
	(XEN) [2016-08-03 13:06:54] rdx: 0000000000000000   rsi: 0000000000000000   rdi: 0000000000000000
	(XEN) [2016-08-03 13:06:54] rbp: 0000000000000000   rsp: ffff83008ce27dc0   r8:  0000000000000000
	(XEN) [2016-08-03 13:06:54] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
	(XEN) [2016-08-03 13:06:54] r12: 0000000000000000   r13: 0000000000000cf9   r14: 0000000000000065
	(XEN) [2016-08-03 13:06:54] r15: ffff830000000000   cr0: 0000000080050033   cr4: 00000000001526e0
	(XEN) [2016-08-03 13:06:54] cr3: 000000084b4e5000   cr2: 000000009e746340
	(XEN) [2016-08-03 13:06:54] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
	(XEN) [2016-08-03 13:06:54] Xen code around <000000009e746340> (000000009e746340):
	(XEN) [2016-08-03 13:06:54]  00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
	(XEN) [2016-08-03 13:06:54] Xen stack trace from rsp=ffff83008ce27dc0:
	(XEN) [2016-08-03 13:06:55]    000000009efe42f6 0000000000000cf9 ffff82d08022f741 efff000000000000
	(XEN) [2016-08-03 13:06:55]    ffff82d0807fe000 000000084b4e5000 ffff82d08022f94a 000000085da98000
	(XEN) [2016-08-03 13:06:55]    0000000000000000 0000000000000007 000000000000e008 0000000000000292
	Initializing Adapter 0

	...
	XEN) [2016-08-03 13:30:44] Xen call trace:
	(XEN) [2016-08-03 13:30:44]    [<000000009e7463c6>] 000000009e7463c6
	(XEN) [2016-08-03 13:30:44]    [<ffff82d0801670f6>] i387.c#_vcpu_save_fpu+0x86/0x190
	(XEN) [2016-08-03 13:30:44]    [<ffff82d08022f94a>] efi_reset_system+0x3a/0x60
	(XEN) [2016-08-03 13:30:44]    [<ffff82d08018bcc8>] machine_restart+0x208/0x2d0
	(XEN) [2016-08-03 13:30:44]    [<ffff82d08012bf8d>] hwdom_shutdown+0x7d/0xf0
	(XEN) [2016-08-03 13:30:44]    [<ffff82d080105871>] domain_shutdown+0xf1/0x100
	(XEN) [2016-08-03 13:30:44]    [<ffff82d08012b042>] do_sched_op+0x1b2/0x460
	(XEN) [2016-08-03 13:30:44]    [<ffff82d08022c65b>] lstar_enter+0x9b/0xa0
	(XEN) [2016-08-03 13:30:44]
	(XEN) [2016-08-03 13:30:44]
	(XEN) [2016-08-03 13:30:44] ****************************************
	(XEN) [2016-08-03 13:30:44] Panic on CPU 0:
	(XEN) [2016-08-03 13:30:44] GENERAL PROTECTION FAULT
	(XEN) [2016-08-03 13:30:44] [error_code=0000]
	(XEN) [2016-08-03 13:30:44] ****************************************

And still reboots.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Jan Beulich]

>>> On 03.08.16 at 15:33, <lists@ssl-mail.com> wrote:
> On Wed, Aug 3, 2016, at 02:01 AM, Jan Beulich wrote:
>> Thanks. Does the use of /mapbs really matter for booting? I was
>> assuming it would be relevant only for shutdown/reboot?
> 
> It has no effect on boot. With or without the "/mapbs" it boots Xen OK.

As expected.

> Without the "/mapbs" the system used to crash on reboot, but still continued 
> with a reboot.  Reboot with the "/mapbs" was OK.
> 
> Now, even with the "/mapbs" the system crashes on reboot.

But afaict in a different way:

> 	[  196.052042] reboot: Restarting system
> 	(XEN) [2016-08-03 13:06:54] Hardware Dom0 shutdown: rebooting machine
> 	(XEN) [2016-08-03 13:06:54] APIC error on CPU0: 40(00)
> 	(XEN) [2016-08-03 13:06:54] ----[ Xen-4.7.0_09-454  x86_64  debug=n  Not tainted ]----
> 	(XEN) [2016-08-03 13:06:54] CPU:    0
> 	(XEN) [2016-08-03 13:06:54] RIP:    e008:[<000000009e746340>] 000000009e746340
>[...]
> 	(XEN) [2016-08-03 13:30:44] ****************************************
> 	(XEN) [2016-08-03 13:30:44] Panic on CPU 0:
> 	(XEN) [2016-08-03 13:30:44] GENERAL PROTECTION FAULT
> 	(XEN) [2016-08-03 13:30:44] [error_code=0000]
> 	(XEN) [2016-08-03 13:30:44] ****************************************

A #GP fault in firmware code. Not much we can do about, I'm afraid,
except for having you go with one of the mentioned workarounds
(and perhaps getting in touch with the BIOS vendor).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Andrew Cooper]

On 03/08/16 14:57, Jan Beulich wrote:
>>>> On 03.08.16 at 15:33, <lists@ssl-mail.com> wrote:
>> On Wed, Aug 3, 2016, at 02:01 AM, Jan Beulich wrote:
>>> Thanks. Does the use of /mapbs really matter for booting? I was
>>> assuming it would be relevant only for shutdown/reboot?
>> It has no effect on boot. With or without the "/mapbs" it boots Xen OK.
> As expected.
>
>> Without the "/mapbs" the system used to crash on reboot, but still continued 
>> with a reboot.  Reboot with the "/mapbs" was OK.
>>
>> Now, even with the "/mapbs" the system crashes on reboot.
> But afaict in a different way:
>
>> 	[  196.052042] reboot: Restarting system
>> 	(XEN) [2016-08-03 13:06:54] Hardware Dom0 shutdown: rebooting machine
>> 	(XEN) [2016-08-03 13:06:54] APIC error on CPU0: 40(00)
>> 	(XEN) [2016-08-03 13:06:54] ----[ Xen-4.7.0_09-454  x86_64  debug=n  Not tainted ]----
>> 	(XEN) [2016-08-03 13:06:54] CPU:    0
>> 	(XEN) [2016-08-03 13:06:54] RIP:    e008:[<000000009e746340>] 000000009e746340
>> [...]
>> 	(XEN) [2016-08-03 13:30:44] ****************************************
>> 	(XEN) [2016-08-03 13:30:44] Panic on CPU 0:
>> 	(XEN) [2016-08-03 13:30:44] GENERAL PROTECTION FAULT
>> 	(XEN) [2016-08-03 13:30:44] [error_code=0000]
>> 	(XEN) [2016-08-03 13:30:44] ****************************************
> A #GP fault in firmware code. Not much we can do about, I'm afraid,
> except for having you go with one of the mentioned workarounds
> (and perhaps getting in touch with the BIOS vendor).

One bit you snipped was

(XEN) [2016-08-03 13:06:54] Xen code around <000000009e746340>
(000000009e746340):
(XEN) [2016-08-03 13:06:54]  00 00 00 00 00 00 00 00 <00> 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00
(XEN) [2016-08-03 13:06:54] Xen stack trace from rsp=ffff83008ce27dc0:

which shows that the EFI firmware has ended up in a block of zeroes. 
This clearly isn't conducive to its overall health.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

> > A #GP fault in firmware code. Not much we can do about, I'm afraid,
> > except for having you go with one of the mentioned workarounds

I tried

	+ efi=no-rs
	+ efi=attr=uc

on the Xen cmd line.

With efi=attr=uc, crashes on reboot with or without /mapbs

With efi=no-rs, reboots OK with or without /mapbs

So *today's* simplest working combination seems to be 

- /mapbs, + efi=no-rs

Two weeks  or so ago, it was just

+ /mapbs

> > (and perhaps getting in touch with the BIOS vendor).

The BIOS vendor refuses to talk to end users.  They deflect to the Motherboard vendor.

The motherboard vendor is Supermicro.  We have four different servers running on their motherboards.

Supermicro are not interested in a small user's problems.  Their responses have included

	- we don't support linux
	- we don't support Xen
	- use Microsoft WIndows
	- we don't write the BIOS. there's nothing we can do.

and finally just ignoring our emails.

At least Supermicro, being a 'server grade' mobo provider, has tech support you can get to that seem aware of linux & Xen.

The 'consumer grade' providers were just completely hopeless the moment you mention linux, xen, and or server.

And to be honest, what exactly would *I* tell them?  "It doesn't work"? It's not like I have any real idea what's broken in detail or how to fix it.

Unless devs that know what they're talking about, and are from a big vendor or project with visible presence or clout, get in touch with them nothing will change. 

> One bit you snipped was
> 
> (XEN) [2016-08-03 13:06:54] Xen code around <000000009e746340>
> (000000009e746340):
> (XEN) [2016-08-03 13:06:54]  00 00 00 00 00 00 00 00 <00> 00 00 00 00 00
> 00 00 00 00 00 00 00 00 00 00
> (XEN) [2016-08-03 13:06:54] Xen stack trace from rsp=ffff83008ce27dc0:
> 
> which shows that the EFI firmware has ended up in a block of zeroes. 
> This clearly isn't conducive to its overall health.

What options to the Xen or EFI command line can fix that?
Or is that a Xen or kernel code fix?
Or again a "BIOS issue"?

Fwiw 3 of those 4 servers are now being migrated to other Hypervisor/Container platforms.

So far KVM, FreeBSD and Microsoft are all booting & rebooting on UEFI hardware OK. Or at least not seeing crashes.  Haven't gotten much further in testing than that yet.

I'm not clear why 'BIOS' problems are only showing up when we use Xen, and only since 'recent' upgrades (it was working a few weeks ago), and now need (at least) efi=no-rs when they didn't before.

If I tell Supermicro that their motherboard works on these other platforms, particularly Microsoft, but not with Xen, I'm pretty sure I know exactly what they'll say.  And as an enduser I don't have the detailed knowledge to argue with them.  Let alone get them to do anything about it.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Aug 3, 2016, at 07:50 AM, lists@ssl-mail.com wrote:
> So *today's* simplest working combination seems to be 

After the sytem is booted with

	+ patched kernel
	- /mapbs
	+ efi=no-rs

I now get tons of these at serial console

	(XEN) [2016-08-03 15:23:25] d1v0 Over-allocation for domain 1: 524545 > 524544
	(XEN) [2016-08-03 15:23:39] d2v2 Over-allocation for domain 2: 524545 > 524544
	(XEN) [2016-08-03 15:23:57] d1v0 Over-allocation for domain 1: 524545 > 524544
	(XEN) [2016-08-03 15:24:11] d2v2 Over-allocation for domain 2: 524545 > 524544
	(XEN) [2016-08-03 15:24:30] d1v0 Over-allocation for domain 1: 524545 > 524544
	(XEN) [2016-08-03 15:24:43] d2v2 Over-allocation for domain 2: 524545 > 524544
	(XEN) [2016-08-03 15:25:02] d1v0 Over-allocation for domain 1: 524545 > 524544
	(XEN) [2016-08-03 15:25:15] d2v2 Over-allocation for domain 2: 524545 > 524544
	(XEN) [2016-08-03 15:25:34] d1v0 Over-allocation for domain 1: 524545 > 524544

That's with logging set to

	loglvl=warning guest_loglvl=none/warning

Not sure it matters other than being useless noise.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Wed, Aug 3, 2016, at 02:01 AM, Jan Beulich wrote:
> > with the 'baseline' as referenced + a patched kernel
> > 
> > 	> Can you try
> > 	>	     ((void *)(md) + (m)->desc_size - 1) < (m)->map_end;		   \
> > 
> > with efi cmd line opts: +"/mapbs"
> > 
> > The system now boots correctly

fyi, there's at least one disagreement re: the suggested - and ack'd as working - patch,

  https://lists.opensuse.org/opensuse-kernel/2016-08/msg00008.html

"That's incorrect IMO. Either -1 or <. Not both -1 and <."

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Vitaly Kuznetsov]

lists@ssl-mail.com writes:

> On Wed, Jul 27, 2016, at 11:36 AM, lists@ssl-mail.com wrote:
>> On Wed, Jul 27, 2016, at 11:28 AM, Andrew Cooper wrote:
>> > > I'm not sure if that's good enough.
>> > 
>> > Sadly not.  The debug symbols need to be specific to the exact binary
>> > you booted.
>> > 
>> > Any change in the compilation will result in the translation being
>> > useless.  What addr2line is doing is saying "which specific bit of
>> > source code did the compiler/linker end up putting at $X".
>> 
>> Got it.  Weird that they don't put the .debuginfo rpms in there.
>> While I was searching around kernel bug reports over at the distro
>> there's lots of posts telling people to debug.  Not sure then how
>> you do it without the debug symbols.
>> 
>> Guess you have to build your own kernel.
>
> I got my hands on a 'matched set'
>
> 	rpm -qa kernel-default\*
> 		kernel-default-4.7.0-5.1.x86_64
> 		kernel-default-devel-4.7.0-5.1.x86_64
> 		kernel-default-debuginfo-4.7.0-5.1.x86_64
>
> reboot to Xen, still crashes
>
> 		(XEN) [2016-07-28 00:13:18] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
> 		(XEN) [2016-07-28 00:13:18] CPU:    0
>>>>		(XEN) [2016-07-28 00:13:18] RIP:    e033:[<ffffffff81f63eb0>]
> 		(XEN) [2016-07-28 00:13:18] RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest (d0v0)
> 		(XEN) [2016-07-28 00:13:18] rax: 0000000000000000   rbx: 0000000000000000   rcx: 000000016f144000
> 		(XEN) [2016-07-28 00:13:18] rdx: 0000000100000000   rsi: 000000016f144000   rdi: fffffffffffff000
> 		(XEN) [2016-07-28 00:13:18] rbp: 0000000001000000   rsp: ffffffff81e03e50   r8:  ffffffff81efb0c0
> 		(XEN) [2016-07-28 00:13:18] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000100000000
> 		(XEN) [2016-07-28 00:13:18] r12: 0000000000000000   r13: 0000000000000000   r14: ffffffff81e03f28
> 		(XEN) [2016-07-28 00:13:18] r15: 0000000000000000   cr0: 0000000080050033   cr4: 00000000001526e0
> 		(XEN) [2016-07-28 00:13:18] cr3: 0000000841e06000   cr2: 0000000000000018
> 		(XEN) [2016-07-28 00:13:18] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
> 		(XEN) [2016-07-28 00:13:18] Guest stack trace from rsp=ffffffff81e03e50:
>
> check ar the RIP addr
>
> 	addr2line -e /usr/lib/debug/boot/vmlinux-4.7.0-5-default.debug ffffffff81f63eb0
> 		/usr/src/debug/kernel-default-4.7.0/linux-4.7/linux-obj/../arch/x86/platform/efi/efi.c:123
>
> in source
>
> 	@ https://github.com/torvalds/linux/blob/v4.7/arch/x86/platform/efi/efi.c
>
> 		...
> 		void __init efi_find_mirror(void)
> 		{
> 			efi_memory_desc_t *md;
> 			u64 mirror_size = 0, total_size = 0;
>
> 			for_each_efi_memory_desc(md) {
> 				unsigned long long start = md->phys_addr;
> 123				unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
>

While I see that you're running linux-4.7 could you please double-check
that it has the following:

commit 55f1ea15216a5a14c96738bd5284100a00ffa9dc
Author: Vitaly Kuznetsov <vkuznets@redhat.com>
Date:   Tue May 31 11:23:43 2016 +0100

    efi: Fix for_each_efi_memory_desc_in_map() for empty memmaps

?

-- 
  Vitaly

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Thu, Jul 28, 2016, at 07:09 AM, Vitaly Kuznetsov wrote:
> While I see that you're running linux-4.7 could you please double-check
> that it has the following:
> 
> commit 55f1ea15216a5a14c96738bd5284100a00ffa9dc
> Author: Vitaly Kuznetsov <vkuznets@redhat.com>
> Date:   Tue May 31 11:23:43 2016 +0100
> 
>     efi: Fix for_each_efi_memory_desc_in_map() for empty memmaps

Checking here

  rpm -q --changelog kernel-default | egrep -i "55f1ea15|for_each_efi_memory_desc_in_map|kuznets|memmaps"

returns nothing.  Doesn't look like it's in there.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Juergen Gross]

On 27/07/16 18:56, Andrew Cooper wrote:
> On 27/07/16 17:54, lists@ssl-mail.com wrote:
>>
>> On Wed, Jul 27, 2016, at 09:34 AM, Andrew Cooper wrote:
>>> This looks suspiciously like the issue which was fixed by c/s
>>> d6b186c1e2d852a92c43f090d0d8fad4704d51ef "x86/xen: avoid m2p lookup when
>>> setting early page table entries", but that fix is present in Linux 4.7.0
>>>
>>> Can you check to see whether the commit is included in your kernel?
>> This
>>
>>  https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable.git/+/d6b186c1e2d852a92c43f090d0d8fad4704d51ef
>>
>> looks like the Suse team was involved,
>>
>>   Reviewed-by: Juergen Gross <jgross@suse.com>
>>
>> But checking this kernel
>>
>>  rpm -q kernel-source
>>   kernel-source-4.7.0-4.1.g89a2ada.noarch
>>  rpm -q --changelog kernel-source | egrep -i "m2p|d6b186c1"
>>   - guarantee M2P to be invisible to user mode.
>>
>> it doesn't look like it's in here.
> 
> Juergen: Any ideas?

I'm rather sure it is a different problem for the following reasons:

- The failed virtual address is wrong. It is for sure no address
  related to the m2p table.

- The kernel booted a little bit further up here. So we have already
  passed the critical section where the faulty m2p lookup occurred.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Jan Beulich]

>>> On 27.07.16 at 01:32, <lists@ssl-mail.com> wrote:
> I'm running Xen-4.7.0_08-452 + linux kernel 4.7.0-4.g89a2ada-default on 
> X86_64 UEFI hardware.
> 
> If I boot without Xen hypervisor enabled it boots fine.
> 
> If I boot with Xen enabled it PANICs:
> 	
> 	(XEN) [2016-07-26 22:05:33] Hardware Dom0 crashed: rebooting machine in 5 seconds.
> 	(XEN) [2016-07-26 22:05:33] APIC error on CPU0: 40(00)
> 	(XEN) [2016-07-26 22:05:38] ----[ Xen-4.7.0_08-452  x86_64  debug=n  Tainted:    C ]----
> 	(XEN) [2016-07-26 22:05:38] CPU:    0
> 	(XEN) [2016-07-26 22:05:38] RIP:    e008:[<000000009e7463c6>] 000000009e7463c6

Taking this together with

(XEN)  000009e430000-000009e79ffff type=3 attr=000000000000000f

from your later post with a full log, I can't see why /mapbs wouldn't
at least change the picture (I'm not saying everything necessarily
would work with that), yet without a full log thereof I can't judge
whether your claim of that being the case is actually true. In any
event the "reboot=" command line option should allow to overcome
this problem, which is a secondary one only anyway.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Mon, Aug 1, 2016, at 11:36 PM, Jan Beulich wrote:
> yet without a full log thereof I can't judge

I've asked what that 'full log' should be

>>> Hmmm.... Could you provide full console dump from Xen and Linux kernel?
>
>Will serial console output with these options
>
>	kernel:     earlyprintk=xen,keep debug loglevel=8
>	hypervisor: loglvl=all guest_loglvl=all sync_console console_to_ring

got an ack, then provided the serial console output with that set of log options.

Like I said I'm happy to try to provide what's needed.

If THAT is not the full log output that's needed, then what specifically is?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Jan Beulich]

>>> On 02.08.16 at 15:13, <lists@ssl-mail.com> wrote:
> On Mon, Aug 1, 2016, at 11:36 PM, Jan Beulich wrote:
>> yet without a full log thereof I can't judge
> 
> I've asked what that 'full log' should be
> 
>>>> Hmmm.... Could you provide full console dump from Xen and Linux kernel?
>>
>>Will serial console output with these options
>>
>>	kernel:     earlyprintk=xen,keep debug loglevel=8
>>	hypervisor: loglvl=all guest_loglvl=all sync_console console_to_ring
> 
> got an ack, then provided the serial console output with that set of log 
> options.

Well, without going through the _full_ thread again, what I could
easily find is

"So full console output from boot -> crash now doesn't look any different than 

	https://lists.xen.org/archives/html/xen-devel/2016-07/msg02814.html 
"

which doesn't tell me at all which combination of /mapbs and/or
/noexitboot was used. Certainly in that run "efi=attr=uc" wasn't
used. It in particular seems highly questionable to me that the
reboot crash would look _exactly_ the same with /mapbs.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Tue, Aug 2, 2016, at 06:38 AM, Jan Beulich wrote:
> Well, without going through the _full_ thread again, what I could
> easily find is
> 
> "So full console output from boot -> crash now doesn't look any different than 
> 
> 	https://lists.xen.org/archives/html/xen-devel/2016-07/msg02814.html 
> "
> 
> which doesn't tell me at all which combination of /mapbs and/or
> /noexitboot was used. Certainly in that run "efi=attr=uc" wasn't
> used. It in particular seems highly questionable to me that the
> reboot crash would look _exactly_ the same with /mapbs.

You keep stating what you don't see.

Afaict, there are four options that seem to have been talked about 

/mapbs & /noexitboot on the EFI cmd line

and

efi=attr=uc and ef=no-rs on the xen cmd line

I really don't want to keep guessing and reposting unnecessary information, and am TRYING to provide the right information.

Can you simply say what output you want?  What combination of options of efi cmd line, xen cmd line, and log options?

I will provide THAT.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Jan Beulich]

>>> On 02.08.16 at 15:54, <lists@ssl-mail.com> wrote:

> 
> On Tue, Aug 2, 2016, at 06:38 AM, Jan Beulich wrote:
>> Well, without going through the _full_ thread again, what I could
>> easily find is
>> 
>> "So full console output from boot -> crash now doesn't look any different than 
> 
>> 
>> 	https://lists.xen.org/archives/html/xen-devel/2016-07/msg02814.html 
>> "
>> 
>> which doesn't tell me at all which combination of /mapbs and/or
>> /noexitboot was used. Certainly in that run "efi=attr=uc" wasn't
>> used. It in particular seems highly questionable to me that the
>> reboot crash would look _exactly_ the same with /mapbs.
> 
> You keep stating what you don't see.

Because you keep being vague...

> Afaict, there are four options that seem to have been talked about 
> 
> /mapbs & /noexitboot on the EFI cmd line
> 
> and
> 
> efi=attr=uc and ef=no-rs on the xen cmd line
> 
> I really don't want to keep guessing and reposting unnecessary information, 
> and am TRYING to provide the right information.
> 
> Can you simply say what output you want?  What combination of options of efi 
> cmd line, xen cmd line, and log options?
> 
> I will provide THAT.

Unless /mapbs really produces an _exactly_ identical crash, I'd like to
see that boot's output at maximum log level. I don't recall "efi=no-rs"
having been mentioned before on this thread, but yes, I'd expect
that one to help as much as the suggested "reboot=..." option, so
if either doesn't, logs thereof would also be of interest.

Jan

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Tue, Aug 2, 2016, at 07:13 AM, Jan Beulich wrote:
> > You keep stating what you don't see.
> 
> Because you keep being vague...

I have attempted to provide everything that's been asked of me.

If you don't like it that's fine.  State with specificity what it is you want.

> Unless /mapbs really produces an _exactly_ identical crash, I'd like to
> see that boot's output at maximum log level. I don't recall "efi=no-rs"
> having been mentioned before on this thread, but yes, I'd expect
> that one to help as much as the suggested "reboot=..." option, so
> if either doesn't, logs thereof would also be of interest.

You criticize MY vagueness then do this.

Identical crash to WHAT? WHICH boot's output?  WHAT do you consider maximum log level -- DIFFERENT than what I've already provided?

I am not a good mind reader, interpreter, guesser.

Just like I said I would, if & when you or someone else who cares to provides clear unequivocal set of options / cases that you want tested I will provide them.

Or do you really want a fishing expedition with every possible combination of every option?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Jan Beulich]

>>> On 02.08.16 at 16:25, <lists@ssl-mail.com> wrote:
> On Tue, Aug 2, 2016, at 07:13 AM, Jan Beulich wrote:
>> Unless /mapbs really produces an _exactly_ identical crash, I'd like to
>> see that boot's output at maximum log level. I don't recall "efi=no-rs"
>> having been mentioned before on this thread, but yes, I'd expect
>> that one to help as much as the suggested "reboot=..." option, so
>> if either doesn't, logs thereof would also be of interest.
> 
> You criticize MY vagueness then do this.

There's nothing vague in my above reply afaict:

> Identical crash to WHAT?

I do recall only a single full log that you had provided so far, so that's
the baseline to me.

> WHICH boot's output?

I've named them:
- one with /mapbs
- one with efi=no-rs
- one with some suitable variant of reboot=

>  WHAT do you consider maximum 
> log level -- DIFFERENT than what I've already provided?

"maximum log level" == "loglvl=all guest_loglvl=all"; I don't think
there's any other interpretation.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[lists]

On Tue, Aug 2, 2016, at 07:50 AM, Jan Beulich wrote:
> - one with some suitable variant of reboot=

What exactly is "some suitable variant of reboot" ?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.7.0 boot PANIC on kernel 4.7.0-4 + UEFI ?

[Jan Beulich]

>>> On 02.08.16 at 17:04, <lists@ssl-mail.com> wrote:
> On Tue, Aug 2, 2016, at 07:50 AM, Jan Beulich wrote:
>> - one with some suitable variant of reboot=
> 
> What exactly is "some suitable variant of reboot" ?

I can't really tell you which of the possible "reboot=" option values
would work on your system. "reboot=acpi" is a pretty safe bet,
though.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel