[Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10

All of lore.kernel.org
 help / color / mirror / Atom feed

* [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
@ 2019-09-24 18:20 Roman Shaposhnik
  2019-09-25  6:59 ` Roger Pau Monné
  2019-09-25  9:01 ` Jan Beulich
  0 siblings, 2 replies; 19+ messages in thread
From: Roman Shaposhnik @ 2019-09-24 18:20 UTC (permalink / raw)
  To: xen-devel

Hi!

I'm a bit at a loss of what's happening here, but it seems that
the latest Xen from master fails to boot on HP ProLiant DL20
GEN10 server (same Xen boots fine on every other piece of
hardware in my lab).

There are absolutely no signs of what's going wrong with it.
It just stops at

(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kb, 2MB, 1GB
...
(XEN) Adding cpu 1 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Adding cpu 2 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached

I guess the only clue is that your typical line of:

(XEN) Brought up X CPUs

never gets printed -- so perhaps there's something wonky
going on with CPU initialization.

Any advice on how to diagnose this further will be greatly appreciated.

Thanks,
Roman.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-24 18:20 [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10 Roman Shaposhnik
@ 2019-09-25  6:59 ` Roger Pau Monné
  2019-09-25  9:01 ` Jan Beulich
  1 sibling, 0 replies; 19+ messages in thread
From: Roger Pau Monné @ 2019-09-25  6:59 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel

On Tue, Sep 24, 2019 at 11:20:06AM -0700, Roman Shaposhnik wrote:
> Hi!
> 
> I'm a bit at a loss of what's happening here, but it seems that
> the latest Xen from master fails to boot on HP ProLiant DL20
> GEN10 server (same Xen boots fine on every other piece of
> hardware in my lab).
> 
> There are absolutely no signs of what's going wrong with it.
> It just stops at
> 
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kb, 2MB, 1GB
> ...
> (XEN) Adding cpu 1 to runqueue 0
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) Adding cpu 2 to runqueue 0
> (XEN) mwait-idle: max C-state count of 8 reached
> 
> I guess the only clue is that your typical line of:
> 
> (XEN) Brought up X CPUs
> 
> never gets printed -- so perhaps there's something wonky
> going on with CPU initialization.
> 
> Any advice on how to diagnose this further will be greatly appreciated.

Can you apply the debug patch below and try again? (you should get
more verbose output)

A bisection would also be helpful in order to identify the commit that
broke booting on this hardware, if older versions of Xen do boot fine.

Thanks, Roger.
---8<---
diff --git a/xen/include/asm-x86/apic.h b/xen/include/asm-x86/apic.h
index 4759279eb2..b33fce198b 100644
--- a/xen/include/asm-x86/apic.h
+++ b/xen/include/asm-x86/apic.h
@@ -5,7 +5,7 @@
 #include <asm/fixmap.h>
 #include <asm/msr.h>
 
-#define Dprintk(x...) do {} while (0)
+#define Dprintk printk
 
 /*
  * Debugging macros

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-24 18:20 [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10 Roman Shaposhnik
  2019-09-25  6:59 ` Roger Pau Monné
@ 2019-09-25  9:01 ` Jan Beulich
  2019-09-25 22:31   ` Roman Shaposhnik
  1 sibling, 1 reply; 19+ messages in thread
From: Jan Beulich @ 2019-09-25  9:01 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel

On 24.09.2019 20:20, Roman Shaposhnik wrote:
> I'm a bit at a loss of what's happening here, but it seems that
> the latest Xen from master fails to boot on HP ProLiant DL20
> GEN10 server (same Xen boots fine on every other piece of
> hardware in my lab).

First of all - is this known to be a regression, i.e. is older
Xen known to work on this particular hardware?

> There are absolutely no signs of what's going wrong with it.
> It just stops at
> 
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kb, 2MB, 1GB
> ...
> (XEN) Adding cpu 1 to runqueue 0
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) Adding cpu 2 to runqueue 0
> (XEN) mwait-idle: max C-state count of 8 reached
> 
> I guess the only clue is that your typical line of:
> 
> (XEN) Brought up X CPUs
> 
> never gets printed -- so perhaps there's something wonky
> going on with CPU initialization.
> 
> Any advice on how to diagnose this further will be greatly appreciated.

Second, as always, a complete log will already help, e.g. by allowing
us to see what CPU model it is that's in the system. Iirc there was a
similar report for a certain Atom variant, but I assume it's a Skylake
here. Maximum verbosity (loglvl=all) and extra CPU related output
("cpuinfo") should be enabled for this.

Furthermore, while I don't think the (bogus; I'll make a patch in a
minute) mwait-idle message is related, excluding there to be an effect
would be a helpful extra data point ("cpuidle=0" for simplicity).

Another potentially useful experiment would be to limit the number of
CPUs to boot ("nosmp" should boot fine in any case, so "maxcpus="
would be what you'd want to play with), and to override the default
of whether to park secondary hyperthreads ("smt=0" and "smt=1").

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-25  9:01 ` Jan Beulich
@ 2019-09-25 22:31   ` Roman Shaposhnik
  2019-09-26  7:44     ` Jan Beulich
  0 siblings, 1 reply; 19+ messages in thread
From: Roman Shaposhnik @ 2019-09-25 22:31 UTC (permalink / raw)
  To: Jan Beulich; +Cc: xen-devel

[-- Attachment #1: Type: text/plain, Size: 2843 bytes --]

Jan, Roger, thank you so much for the initial ideas. I tried a few of
those and here's where I am.

First of all, it is definitely related to CPU bring up. Adding
cpuidle=0 to xen command line made Xen boot.

Then, a good friend of mine (who you may know from ancient Xen days
;-)) suggested that this could be related to this:
     https://wiki.xenproject.org/wiki/Xen_power_management
so I went to the BIOS settings and quite to my surprise all of them
were grayed out (not tweakable).

The only one that wasn't was 2xAPIC support. So just for kicks -- I
disabled that.

That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.

So I guess at this point, you could say that I have a functional
system, but I'm curious whether you guys would be interested to look
into 2xAPIC situation.

Please let me know.

Thanks,
Roman.

On Wed, Sep 25, 2019 at 2:01 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 24.09.2019 20:20, Roman Shaposhnik wrote:
> > I'm a bit at a loss of what's happening here, but it seems that
> > the latest Xen from master fails to boot on HP ProLiant DL20
> > GEN10 server (same Xen boots fine on every other piece of
> > hardware in my lab).
>
> First of all - is this known to be a regression, i.e. is older
> Xen known to work on this particular hardware?
>
> > There are absolutely no signs of what's going wrong with it.
> > It just stops at
> >
> > (XEN) HVM: ASIDs enabled.
> > (XEN) HVM: VMX enabled
> > (XEN) HVM: Hardware Assisted Paging (HAP) detected
> > (XEN) HVM: HAP page sizes: 4kb, 2MB, 1GB
> > ...
> > (XEN) Adding cpu 1 to runqueue 0
> > (XEN) mwait-idle: max C-state count of 8 reached
> > (XEN) Adding cpu 2 to runqueue 0
> > (XEN) mwait-idle: max C-state count of 8 reached
> >
> > I guess the only clue is that your typical line of:
> >
> > (XEN) Brought up X CPUs
> >
> > never gets printed -- so perhaps there's something wonky
> > going on with CPU initialization.
> >
> > Any advice on how to diagnose this further will be greatly appreciated.
>
> Second, as always, a complete log will already help, e.g. by allowing
> us to see what CPU model it is that's in the system. Iirc there was a
> similar report for a certain Atom variant, but I assume it's a Skylake
> here. Maximum verbosity (loglvl=all) and extra CPU related output
> ("cpuinfo") should be enabled for this.
>
> Furthermore, while I don't think the (bogus; I'll make a patch in a
> minute) mwait-idle message is related, excluding there to be an effect
> would be a helpful extra data point ("cpuidle=0" for simplicity).
>
> Another potentially useful experiment would be to limit the number of
> CPUs to boot ("nosmp" should boot fine in any case, so "maxcpus="
> would be what you'd want to play with), and to override the default
> of whether to park secondary hyperthreads ("smt=0" and "smt=1").
>
> Jan

[-- Attachment #2: dmesg.linux --]
[-- Type: application/octet-stream, Size: 47841 bytes --]

[    0.000000] Linux version 4.19.5-linuxkit (root@dda7e419284b) (gcc version 6.3.0 (Alpine 6.3.0)) #1 SMP Sat Jul 13 05:48:19 UTC 2019
[    0.000000] Command line: console=tty0 rootdelay=3 root=PARTUUID=011e9e0a-bdbe-4e21-984d-e271e082de52 text
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[    0.000000] Released 0 page(s)
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] Xen: [mem 0x0000000000000000-0x000000000009efff] usable
[    0.000000] Xen: [mem 0x000000000009f000-0x00000000000fffff] reserved
[    0.000000] Xen: [mem 0x0000000000100000-0x0000000040060fff] usable
[    0.000000] Xen: [mem 0x0000000040061000-0x0000000065438fff] unusable
[    0.000000] Xen: [mem 0x0000000065439000-0x0000000065447fff] reserved
[    0.000000] Xen: [mem 0x0000000065448000-0x0000000065483fff] unusable
[    0.000000] Xen: [mem 0x0000000065484000-0x0000000065487fff] reserved
[    0.000000] Xen: [mem 0x0000000065488000-0x000000006e2b6fff] unusable
[    0.000000] Xen: [mem 0x000000006e2b7000-0x000000006e2c8fff] reserved
[    0.000000] Xen: [mem 0x000000006e2c9000-0x000000006e2c9fff] unusable
[    0.000000] Xen: [mem 0x000000006e2ca000-0x000000006f4ebfff] reserved
[    0.000000] Xen: [mem 0x000000006f4ec000-0x000000006f4ecfff] ACPI NVS
[    0.000000] Xen: [mem 0x000000006f4ed000-0x000000006f4fefff] reserved
[    0.000000] Xen: [mem 0x000000006f4ff000-0x0000000078cfefff] unusable
[    0.000000] Xen: [mem 0x0000000078cff000-0x00000000799fefff] reserved
[    0.000000] Xen: [mem 0x00000000799ff000-0x000000007bdfefff] ACPI NVS
[    0.000000] Xen: [mem 0x000000007bdff000-0x000000007bffefff] ACPI data
[    0.000000] Xen: [mem 0x000000007bfff000-0x000000007bffffff] unusable
[    0.000000] Xen: [mem 0x000000007c000000-0x000000007fffffff] reserved
[    0.000000] Xen: [mem 0x00000000fe010000-0x00000000fe010fff] reserved
[    0.000000] Xen: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] Xen: [mem 0x00000000fed91000-0x00000000fed91fff] reserved
[    0.000000] Xen: [mem 0x00000000fee00000-0x00000000feefffff] reserved
[    0.000000] Xen: [mem 0x0000000100000000-0x000000027effffff] unusable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] efi: EFI v2.60 by HPE
[    0.000000] efi:  SMBIOS=0x6e2cc000  SMBIOS 3.0=0x6e2ca000  ACPI=0x7bffe000  ACPI 2.0=0x7bffe014  MEMATTR=0x73b3d018 
[    0.000000] SMBIOS 3.1.1 present.
[    0.000000] DMI: HPE ProLiant DL20 Gen10/ProLiant DL20 Gen10, BIOS U43 02/02/2019
[    0.000000] Hypervisor detected: Xen PV
[    0.000521] tsc: Detected 3312.146 MHz processor
[    0.001360] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.001362] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.001365] last_pfn = 0x40061 max_arch_pfn = 0x400000000
[    0.001366] Disabled
[    0.001367] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[    0.001369] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WC  WP  UC  UC  
[    0.001384] Base memory trampoline at [(____ptrval____)] 99000 size 24576
[    0.001396] Kernel/User page tables isolation: disabled on XEN PV.
[    0.001735] BRK [0x02a01000, 0x02a01fff] PGTABLE
[    0.002355] BRK [0x02a02000, 0x02a02fff] PGTABLE
[    0.002639] BRK [0x02a03000, 0x02a03fff] PGTABLE
[    0.002923] BRK [0x02a04000, 0x02a04fff] PGTABLE
[    0.003208] BRK [0x02a05000, 0x02a05fff] PGTABLE
[    0.003493] BRK [0x02a06000, 0x02a06fff] PGTABLE
[    0.003777] BRK [0x02a07000, 0x02a07fff] PGTABLE
[    0.004062] BRK [0x02a08000, 0x02a08fff] PGTABLE
[    0.004345] BRK [0x02a09000, 0x02a09fff] PGTABLE
[    0.004630] BRK [0x02a0a000, 0x02a0afff] PGTABLE
[    0.004914] BRK [0x02a0b000, 0x02a0bfff] PGTABLE
[    0.005202] BRK [0x02a0c000, 0x02a0cfff] PGTABLE
[    0.143175] Secure boot disabled
[    0.143184] ACPI: Early table checksum verification disabled
[    0.143189] ACPI: RSDP 0x000000007BFFE014 000024 (v02 HPE   )
[    0.143195] ACPI: XSDT 0x000000007BFE7188 0000FC (v01 HPE    Server   00000001 INTL 20160422)
[    0.143223] ACPI: FACP 0x000000007BFF4000 00010C (v06 HPE    Server   00000001 1590 00000001)
[    0.143273] ACPI: DSDT 0x000000007BFE4000 0014F4 (v02 HPE    Server   00000002 HPE  00020000)
[    0.143281] ACPI: FACS 0x000000007BDD0000 000040
[    0.143289] ACPI: SSDT 0x000000007BFFC000 000197 (v01 HPE    CpuSsdt  00000001 INTL 20130328)
[    0.143298] ACPI: BOOT 0x000000007BFFB000 000028 (v01 HPE    Server   00000002 MSFT 01000013)
[    0.143306] ACPI: SSDT 0x000000007BFFA000 00003A (v01 HPE    PLATOPRG 00000001 INTL 20130328)
[    0.143314] ACPI: HEST 0x000000007BFF9000 0000A8 (v01 HPE    Server   00000001 INTL 00000001)
[    0.143323] ACPI: BERT 0x000000007BFF8000 000030 (v01 HPE    Server   00000001 INTL 00000001)
[    0.143331] ACPI: ERST 0x000000007BFF7000 000250 (v01 HPE    Server   00000001 INTL 00000001)
[    0.143339] ACPI: FPDT 0x000000007BFF6000 000034 (v01 HPE    Server   20170001 INTL 20160422)
[    0.143347] ACPI: BGRT 0x000000007BFF5000 000038 (v01 HPE    Server   20170001 1590 20160422)
[    0.143356] ACPI: HPET 0x000000007BFF3000 000038 (v01 HPE    Server   00000001 1590 00000001)
[    0.143364] ACPI: WDDT 0x000000007BFF2000 000040 (v01 HPE    Server   00000001 1590 00000001)
[    0.143372] ACPI: APIC 0x000000007BFF1000 000084 (v03 HPE    Server   00000001 1590 00000001)
[    0.143380] ACPI: MCFG 0x000000007BFF0000 00003C (v01 HPE    Server   00000001 1590 00000001)
[    0.143389] ACPI: SLIT 0x000000007BFEF000 00002D (v01 HPE    Server   00000001 1590 00000001)
[    0.143397] ACPI: SRAT 0x000000007BFEE000 000138 (v03 HPE    Server   00000001 1590 00000001)
[    0.143406] ACPI: SPMI 0x000000007BFED000 000041 (v05 HPE    Server   00000001 1590 00000001)
[    0.143414] ACPI: RASF 0x000000007BFEC000 000030 (v01 HPE    Server   00000001 1590 00000001)
[    0.143422] ACPI: SPCR 0x000000007BFEB000 000050 (v02 HPE    Server   00000001 1590 00000001)
[    0.143431] ACPI: MSCT 0x000000007BFEA000 00004E (v02 HPE    Server   00000001 1590 00000001)
[    0.143439] ACPI: BDAT 0x000000007BFE9000 000030 (v01 HPE    Server   00000001 1590 00000001)
[    0.143447] ACPI: WSMT 0x000000007BFE8000 000028 (v01 HPE    Server   00000001 1590 00000001)
[    0.143455] ACPI: PCCT 0x000000007BFFD000 000030 (v01 HPE    Server   00000001 1590 00000001)
[    0.143463] ACPI: RMAD 0x000000007BFE6000 0000B0 (v01 HPE    Server   00000001 1590 00000001)
[    0.143472] ACPI: SSDT 0x000000007BFE1000 002AA2 (v02 HPE    PCISSDT  00000002 HPE  00020000)
[    0.143480] ACPI: SSDT 0x000000007BFE0000 0001C4 (v02 HPE    TIMESSDT 00000002 HPE  00020000)
[    0.143488] ACPI: SSDT 0x000000007BFDF000 0002F2 (v01 HPE    pmab     00000001 INTL 20130328)
[    0.143497] ACPI: OEM1 0x000000007BFDE000 0000DF (v02 HPE    CPU  CPC 00000002 HPE  00020000)
[    0.143522] ACPI: Local APIC address 0xfee00000
[    0.143523] Setting APIC routing to Xen PV.
[    0.146326] Zone ranges:
[    0.146327]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.146328]   DMA32    [mem 0x0000000001000000-0x0000000040060fff]
[    0.146329]   Normal   empty
[    0.146330] Movable zone start for each node
[    0.146331] Early memory node ranges
[    0.146331]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.146332]   node   0: [mem 0x0000000000100000-0x0000000040060fff]
[    0.146645] Reserved but unavailable: 32769 pages
[    0.146646] Initmem setup node 0 [mem 0x0000000000001000-0x0000000040060fff]
[    0.146647] On node 0 totalpages: 262143
[    0.146648]   DMA zone: 64 pages used for memmap
[    0.146648]   DMA zone: 21 pages reserved
[    0.146649]   DMA zone: 3998 pages, LIFO batch:0
[    0.146693]   DMA32 zone: 4034 pages used for memmap
[    0.146693]   DMA32 zone: 258145 pages, LIFO batch:63
[    0.149385] p2m virtual area at (____ptrval____), size is 40000000
[    0.213850] Remapped 97 page(s)
[    0.214731] ACPI: PM-Timer IO Port: 0x508
[    0.214735] ACPI: Local APIC address 0xfee00000
[    0.214757] ACPI: LAPIC_NMI (acpi_id[0x00] high level lint[0x1])
[    0.214759] ACPI: LAPIC_NMI (acpi_id[0x02] high level lint[0x1])
[    0.214760] ACPI: LAPIC_NMI (acpi_id[0x04] high level lint[0x1])
[    0.214761] ACPI: LAPIC_NMI (acpi_id[0x06] high level lint[0x1])
[    0.214819] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
[    0.214826] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.214828] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.214832] ACPI: IRQ0 used by override.
[    0.214833] ACPI: IRQ9 used by override.
[    0.214839] Using ACPI (MADT) for SMP configuration information
[    0.214842] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.214848] ACPI: SPCR: console: uart,mmio,0x0,115200
[    0.214851] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[    0.214863] [mem 0x80000000-0xfe00ffff] available for PCI devices
[    0.214865] Booting paravirtualized kernel on Xen
[    0.214865] Xen version: 4.12.0 (preserve-AD)
[    0.214867] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.276409] random: get_random_bytes called from start_kernel+0x92/0x4be with crng_init=0
[    0.276412] setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:4 nr_node_ids:1
[    0.276548] percpu: Embedded 45 pages/cpu @(____ptrval____) s143768 r8192 d32360 u524288
[    0.276552] pcpu-alloc: s143768 r8192 d32360 u524288 alloc=1*2097152
[    0.276552] pcpu-alloc: [0] 0 1 2 3 
[    0.276575] xen: PV spinlocks disabled
[    0.276577] Built 1 zonelists, mobility grouping on.  Total pages: 258024
[    0.276578] Kernel command line: console=tty0 rootdelay=3 root=PARTUUID=011e9e0a-bdbe-4e21-984d-e271e082de52 text
[    0.276683] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.276708] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.293819] software IO TLB: mapped [mem 0x3a600000-0x3e600000] (64MB)
[    0.302886] Memory: 930940K/1048572K available (12300K kernel code, 1599K rwdata, 3084K rodata, 1592K init, 1924K bss, 117632K reserved, 0K cma-reserved)
[    0.302923] ftrace: allocating 43804 entries in 172 pages
[    0.312588] rcu: Hierarchical RCU implementation.
[    0.312590] rcu: 	RCU restricting CPUs from NR_CPUS=128 to nr_cpu_ids=1.
[    0.312591] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.314291] Using NULL legacy PIC
[    0.314292] NR_IRQS: 8448, nr_irqs: 256, preallocated irqs: 0
[    0.314301] xen:events: Using FIFO-based ABI
[    0.314311] xen: --> pirq=1 -> irq=1 (gsi=1)
[    0.314314] xen: --> pirq=2 -> irq=2 (gsi=2)
[    0.314316] xen: --> pirq=3 -> irq=3 (gsi=3)
[    0.314319] xen: --> pirq=4 -> irq=4 (gsi=4)
[    0.314322] xen: --> pirq=5 -> irq=5 (gsi=5)
[    0.314325] xen: --> pirq=6 -> irq=6 (gsi=6)
[    0.314328] xen: --> pirq=7 -> irq=7 (gsi=7)
[    0.314330] xen: --> pirq=8 -> irq=8 (gsi=8)
[    0.314334] xen: --> pirq=9 -> irq=9 (gsi=9)
[    0.314336] xen: --> pirq=10 -> irq=10 (gsi=10)
[    0.314339] xen: --> pirq=11 -> irq=11 (gsi=11)
[    0.314341] xen: --> pirq=12 -> irq=12 (gsi=12)
[    0.314344] xen: --> pirq=13 -> irq=13 (gsi=13)
[    0.314347] xen: --> pirq=14 -> irq=14 (gsi=14)
[    0.314350] xen: --> pirq=15 -> irq=15 (gsi=15)
[    0.314396] Console: colour dummy device 80x25
[    0.314499] console [tty0] enabled
[    0.314825] console [hvc0] enabled
[    0.314829] ACPI: Core revision 20180810
[    0.315345] ACPI BIOS Warning (bug): Incorrect checksum in table [BGRT] - 0x2B, should be 0x1E (20180810/tbprint-177)
[    0.316331] clocksource: xen: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.316342] Xen: using vcpuop timer interface
[    0.316344] installing Xen timer for CPU 0
[    0.316405] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x2fbe1e88e16, max_idle_ns: 440795327210 ns
[    0.316452] Calibrating delay loop (skipped), value calculated using timer frequency.. 6624.29 BogoMIPS (lpj=33121460)
[    0.316459] pid_max: default: 32768 minimum: 301
[    0.316505] Security Framework initialized
[    0.316508] Yama: becoming mindful.
[    0.316523] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
[    0.316528] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes)
[    0.316684] Last level iTLB entries: 4KB 128, 2MB 8, 4MB 8
[    0.316688] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[    0.316693] Spectre V2 : Vulnerable: Minimal generic ASM retpoline
[    0.316696] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.316700] Spectre V2 : Spectre v2 mitigation: Enabling Indirect Branch Prediction Barrier
[    0.316704] Spectre V2 : Enabling Restricted Speculation for firmware calls
[    0.316708] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[    0.352971] Freeing SMP alternatives memory: 24K
[    0.353691] VPMU disabled by hypervisor.
[    0.353799] Performance Events: unsupported p6 CPU model 158 no PMU driver, software events only.
[    0.353841] rcu: Hierarchical SRCU implementation.
[    0.353944] NMI watchdog: Perf NMI watchdog permanently disabled
[    0.353985] smp: Bringing up secondary CPUs ...
[    0.353988] smp: Brought up 1 node, 1 CPU
[    0.353991] smpboot: Max logical packages: 1
[    0.354076] devtmpfs: initialized
[    0.354105] x86/mm: Memory block size: 128MB
[    0.354300] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.354307] futex hash table entries: 256 (order: 2, 16384 bytes)
[    0.354332] pinctrl core: initialized pinctrl subsystem
[    0.354392] NET: Registered protocol family 16
[    0.354404] xen:grant_table: Grant tables using version 1 layout
[    0.354414] Grant table initialized
[    0.354488] audit: initializing netlink subsys (disabled)
[    0.354548] audit: type=2000 audit(1569419358.224:1): state=initialized audit_enabled=0 res=1
[    0.354591] Invalid PCCT: 0 PCC subspaces
[    0.354649] Simple Boot Flag at 0x47 set to 0x80
[    0.354673] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[    0.354677] ACPI: bus type PCI registered
[    0.354776] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.354787] PCI: not using MMCONFIG
[    0.354789] PCI: Using configuration type 1 for base access
[    0.355814] cryptd: max_cpu_qlen set to 1000
[    0.355886] ACPI: Added _OSI(Module Device)
[    0.355889] ACPI: Added _OSI(Processor Device)
[    0.355892] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.355895] ACPI: Added _OSI(Processor Aggregator Device)
[    0.355898] ACPI: Added _OSI(Linux-Dell-Video)
[    0.355901] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.357655] ACPI: 6 ACPI AML tables successfully acquired and loaded
[    0.357965] xen: registering gsi 9 triggering 0 polarity 0
[    0.359323] ACPI: Interpreter enabled
[    0.359329] ACPI: (supports S0 S5)
[    0.359332] ACPI: Using IOAPIC for interrupt routing
[    0.359346] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.359606] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI motherboard resources
[    0.400152] HEST: Table parsing has been initialized.
[    0.400157] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.400251] ACPI: Enabled 2 GPEs in block 00 to 7F
[    0.402568] APIC: NR_CPUS/possible_cpus limit of 1 reached. Processor 1/0x2 ignored.
[    0.402573] ACPI: Unable to map lapic to logical cpu number
[    0.402581] APIC: NR_CPUS/possible_cpus limit of 1 reached. Processor 2/0x4 ignored.
[    0.402585] ACPI: Unable to map lapic to logical cpu number
[    0.402592] APIC: NR_CPUS/possible_cpus limit of 1 reached. Processor 3/0x6 ignored.
[    0.402596] ACPI: Unable to map lapic to logical cpu number
[    0.402607] ACPI: PCI Root Bridge [PC00] (domain 0000 [bus 00-ff])
[    0.402615] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    0.402727] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug LTR]
[    0.402830] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability]
[    0.402834] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration
[    0.402856] PCI host bridge to bus 0000:00
[    0.402859] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.402863] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.402867] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.402871] pci_bus 0000:00: root bus resource [mem 0x80000000-0xfeafffff window]
[    0.402875] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.402894] pci 0000:00:00.0: [8086:3e33] type 00 class 0x060000
[    0.403090] pci 0000:00:01.0: [8086:1901] type 01 class 0x060400
[    0.403238] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[    0.403344] pci 0000:00:01.1: [8086:1905] type 01 class 0x060400
[    0.403490] pci 0000:00:01.1: PME# supported from D0 D3hot D3cold
[    0.403821] pci 0000:00:12.0: [8086:a379] type 00 class 0x118000
[    0.403877] pci 0000:00:12.0: reg 0x10: [mem 0x4000005000-0x4000005fff 64bit]
[    0.404196] pci 0000:00:14.0: [8086:a36d] type 00 class 0x0c0330
[    0.404246] pci 0000:00:14.0: reg 0x10: [mem 0x80100000-0x8010ffff 64bit]
[    0.404404] pci 0000:00:14.0: PME# supported from D3hot D3cold
[    0.404520] pci 0000:00:14.2: [8086:a36f] type 00 class 0x050000
[    0.404566] pci 0000:00:14.2: reg 0x10: [mem 0x4000000000-0x4000001fff 64bit]
[    0.404590] pci 0000:00:14.2: reg 0x18: [mem 0x4000004000-0x4000004fff 64bit]
[    0.404817] pci 0000:00:16.0: [8086:a360] type 00 class 0x078000
[    0.404872] pci 0000:00:16.0: reg 0x10: [mem 0x4000003000-0x4000003fff 64bit]
[    0.405032] pci 0000:00:16.0: PME# supported from D3hot
[    0.405208] pci 0000:00:17.0: [8086:a352] type 00 class 0x010601
[    0.405252] pci 0000:00:17.0: reg 0x10: [mem 0x80110000-0x80111fff]
[    0.405269] pci 0000:00:17.0: reg 0x14: [mem 0x80114000-0x801140ff]
[    0.405286] pci 0000:00:17.0: reg 0x18: [io  0x2040-0x2047]
[    0.405303] pci 0000:00:17.0: reg 0x1c: [io  0x2048-0x204b]
[    0.405320] pci 0000:00:17.0: reg 0x20: [io  0x2020-0x203f]
[    0.405337] pci 0000:00:17.0: reg 0x24: [mem 0x80113000-0x801137ff]
[    0.405432] pci 0000:00:17.0: PME# supported from D3hot
[    0.405560] pci 0000:00:1b.0: [8086:a32c] type 01 class 0x060400
[    0.405744] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    0.405933] pci 0000:00:1c.0: [8086:a338] type 01 class 0x060400
[    0.406109] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    0.406221] pci 0000:00:1c.1: [8086:a339] type 01 class 0x060400
[    0.406398] pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
[    0.406589] pci 0000:00:1d.0: [8086:a330] type 01 class 0x060400
[    0.406773] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[    0.406956] pci 0000:00:1f.0: [8086:a30a] type 00 class 0x060100
[    0.407216] pci 0000:00:1f.5: [8086:a324] type 00 class 0x0c8000
[    0.407251] pci 0000:00:1f.5: reg 0x10: [mem 0xfe010000-0xfe010fff]
[    0.407500] pci 0000:00:01.0: PCI bridge to [bus 05]
[    0.407576] pci 0000:00:01.1: PCI bridge to [bus 06]
[    0.407666] pci 0000:00:1b.0: PCI bridge to [bus 04]
[    0.407778] pci 0000:01:00.0: [103c:3306] type 00 class 0x088000
[    0.407821] pci 0000:01:00.0: reg 0x10: [io  0x1200-0x12ff]
[    0.407841] pci 0000:01:00.0: reg 0x14: [mem 0x85b9e000-0x85b9e3ff]
[    0.407861] pci 0000:01:00.0: reg 0x18: [io  0x1100-0x11ff]
[    0.407881] pci 0000:01:00.0: reg 0x1c: [mem 0x82000000-0x83ffffff]
[    0.407900] pci 0000:01:00.0: reg 0x20: [mem 0x85800000-0x859fffff]
[    0.408086] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5 GT/s x1 link at 0000:00:1c.0 (capable of 7.876 Gb/s with 8 GT/s x1 link)
[    0.408165] pci 0000:01:00.1: [102b:0538] type 00 class 0x030000
[    0.408208] pci 0000:01:00.1: reg 0x10: [mem 0x84000000-0x84ffffff pref]
[    0.408228] pci 0000:01:00.1: reg 0x14: [mem 0x85b98000-0x85b9bfff]
[    0.408248] pci 0000:01:00.1: reg 0x18: [mem 0x85000000-0x857fffff]
[    0.408349] pci 0000:01:00.1: BAR 0: assigned to efifb
[    0.408523] pci 0000:01:00.2: [103c:3307] type 00 class 0x088000
[    0.408565] pci 0000:01:00.2: reg 0x10: [io  0x1000-0x10ff]
[    0.408585] pci 0000:01:00.2: reg 0x14: [mem 0x85b9d000-0x85b9d0ff]
[    0.408605] pci 0000:01:00.2: reg 0x18: [mem 0x85a00000-0x85afffff]
[    0.408625] pci 0000:01:00.2: reg 0x1c: [mem 0x85b00000-0x85b7ffff]
[    0.408645] pci 0000:01:00.2: reg 0x20: [mem 0x85b90000-0x85b97fff]
[    0.408664] pci 0000:01:00.2: reg 0x24: [mem 0x85b80000-0x85b8ffff]
[    0.408685] pci 0000:01:00.2: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    0.408792] pci 0000:01:00.2: PME# supported from D0 D3hot D3cold
[    0.408884] pci 0000:01:00.4: [103c:22f6] type 00 class 0x0c0320
[    0.408926] pci 0000:01:00.4: reg 0x10: [mem 0x85b9c000-0x85b9c0ff]
[    0.409231] pci 0000:00:1c.0: PCI bridge to [bus 01]
[    0.409239] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
[    0.409244] pci 0000:00:1c.0:   bridge window [mem 0x81000000-0x85bfffff]
[    0.409352] pci 0000:02:00.0: [14e4:165f] type 00 class 0x020000
[    0.409410] pci 0000:02:00.0: reg 0x10: [mem 0x80030000-0x8003ffff 64bit pref]
[    0.409442] pci 0000:02:00.0: reg 0x18: [mem 0x80040000-0x8004ffff 64bit pref]
[    0.409474] pci 0000:02:00.0: reg 0x20: [mem 0x80050000-0x8005ffff 64bit pref]
[    0.409494] pci 0000:02:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref]
[    0.409672] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[    0.409739] pci 0000:02:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5 GT/s x1 link at 0000:00:1c.1 (capable of 15.752 Gb/s with 8 GT/s x2 link)
[    0.409839] pci 0000:02:00.1: [14e4:165f] type 00 class 0x020000
[    0.409897] pci 0000:02:00.1: reg 0x10: [mem 0x80000000-0x8000ffff 64bit pref]
[    0.409929] pci 0000:02:00.1: reg 0x18: [mem 0x80010000-0x8001ffff 64bit pref]
[    0.409961] pci 0000:02:00.1: reg 0x20: [mem 0x80020000-0x8002ffff 64bit pref]
[    0.409982] pci 0000:02:00.1: reg 0x30: [mem 0x00000000-0x0003ffff pref]
[    0.410159] pci 0000:02:00.1: PME# supported from D0 D3hot D3cold
[    0.410301] pci 0000:00:1c.1: PCI bridge to [bus 02]
[    0.410323] pci 0000:00:1c.1:   bridge window [mem 0x80000000-0x800fffff 64bit pref]
[    0.410392] pci 0000:00:1d.0: PCI bridge to [bus 03]
[    0.410487] pci_bus 0000:00: on NUMA node 0
[    0.410573] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 10 11 12 14 *15), disabled.
[    0.410635] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 10 11 12 14 *15), disabled.
[    0.410694] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 10 11 12 14 *15), disabled.
[    0.410756] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 11 12 14 *15), disabled.
[    0.410816] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 10 11 12 14 *15), disabled.
[    0.410876] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 10 11 12 14 *15), disabled.
[    0.410937] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 10 11 12 14 *15), disabled.
[    0.410996] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 10 11 12 14 *15), disabled.
[    0.411027] xen: registering gsi 13 triggering 1 polarity 0
[    0.411223] xen:balloon: Initialising balloon driver
[    0.411293] SCSI subsystem initialized
[    0.411324] libata version 3.00 loaded.
[    0.411328] ACPI: bus type USB registered
[    0.411340] usbcore: registered new interface driver usbfs
[    0.411347] usbcore: registered new interface driver hub
[    0.411360] usbcore: registered new device driver usb
[    0.411370] pps_core: LinuxPPS API ver. 1 registered
[    0.411373] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.411379] PTP clock support registered
[    0.411401] Registered efivars operations
[    0.430400] PCI: Using ACPI for IRQ routing
[    0.515313] PCI: pci_cache_line_size set to 64 bytes
[    0.515328] pci 0000:00:12.0: can't claim BAR 0 [mem 0x4000005000-0x4000005fff 64bit]: no compatible bridge window
[    0.515406] pci 0000:00:14.2: can't claim BAR 0 [mem 0x4000000000-0x4000001fff 64bit]: no compatible bridge window
[    0.515412] pci 0000:00:14.2: can't claim BAR 2 [mem 0x4000004000-0x4000004fff 64bit]: no compatible bridge window
[    0.515423] pci 0000:00:16.0: can't claim BAR 0 [mem 0x4000003000-0x4000003fff 64bit]: no compatible bridge window
[    0.515466] e820: reserve RAM buffer [mem 0x0009f000-0x0009ffff]
[    0.515467] e820: reserve RAM buffer [mem 0x40061000-0x43ffffff]
[    0.515529] NetLabel: Initializing
[    0.515533] NetLabel:  domain hash size = 128
[    0.515536] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.515549] NetLabel:  unlabeled traffic allowed by default
[    0.515668] clocksource: Switched to clocksource tsc-early
[    0.516543] VFS: Disk quotas dquot_6.6.0
[    0.516554] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.516584] FS-Cache: Loaded
[    0.516587] hugetlbfs: disabling because there are no supported hugepage sizes
[    0.516620] CacheFiles: Loaded
[    0.516630] pnp: PnP ACPI init
[    0.516812] xen: registering gsi 8 triggering 1 polarity 0
[    0.516876] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
[    0.517044] system 00:01: [io  0x0500-0x05fe] has been reserved
[    0.517050] system 00:01: [mem 0xfed10000-0xfed17fff] has been reserved
[    0.517054] system 00:01: [mem 0xfed18000-0xfed18fff] has been reserved
[    0.517058] system 00:01: [mem 0xfed19000-0xfed19fff] has been reserved
[    0.517062] system 00:01: [mem 0xe0000000-0xefffffff] has been reserved
[    0.517067] system 00:01: [mem 0xfed20000-0xfed3ffff] has been reserved
[    0.517071] system 00:01: [mem 0xfed90000-0xfed93fff] could not be reserved
[    0.517075] system 00:01: [mem 0xfed45000-0xfed8ffff] has been reserved
[    0.517079] system 00:01: [mem 0xff000000-0xffffffff] has been reserved
[    0.517083] system 00:01: [mem 0xfee00000-0xfeefffff] has been reserved
[    0.517087] system 00:01: [mem 0xfc800000-0xfcffffff] has been reserved
[    0.517092] system 00:01: [mem 0xfd000000-0xfd69ffff] has been reserved
[    0.517096] system 00:01: [mem 0xfd6c0000-0xfd6cffff] has been reserved
[    0.517100] system 00:01: [mem 0xfd6f0000-0xfdffffff] has been reserved
[    0.517104] system 00:01: [mem 0xfe000000-0xfe01ffff] could not be reserved
[    0.517109] system 00:01: [mem 0xfe036000-0xfe7fffff] has been reserved
[    0.517114] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.517120] xen: registering gsi 3 triggering 1 polarity 0
[    0.517162] pnp 00:02: Plug and Play ACPI device, IDs PNP0501 (active)
[    0.517162] xen: registering gsi 4 triggering 1 polarity 0
[    0.517162] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active)
[    0.517162] pnp: PnP ACPI: found 4 devices
[    0.527120] PM-Timer failed consistency check  (0xffffff) - aborting.
[    0.527226] pci 0000:00:1c.1: BAR 14: assigned [mem 0x80200000-0x802fffff]
[    0.527232] pci 0000:00:14.2: BAR 0: assigned [mem 0x80116000-0x80117fff 64bit]
[    0.527261] pci 0000:00:12.0: BAR 0: assigned [mem 0x80112000-0x80112fff 64bit]
[    0.527285] pci 0000:00:14.2: BAR 2: assigned [mem 0x80115000-0x80115fff 64bit]
[    0.527311] pci 0000:00:16.0: BAR 0: assigned [mem 0x80118000-0x80118fff 64bit]
[    0.527337] pci 0000:00:01.0: PCI bridge to [bus 05]
[    0.527359] pci 0000:00:01.1: PCI bridge to [bus 06]
[    0.527382] pci 0000:00:1b.0: PCI bridge to [bus 04]
[    0.527413] pci 0000:01:00.2: BAR 6: assigned [mem 0x81000000-0x8100ffff pref]
[    0.527418] pci 0000:00:1c.0: PCI bridge to [bus 01]
[    0.527426] pci 0000:00:1c.0:   bridge window [io  0x1000-0x1fff]
[    0.527438] pci 0000:00:1c.0:   bridge window [mem 0x81000000-0x85bfffff]
[    0.527457] pci 0000:02:00.0: BAR 6: assigned [mem 0x80200000-0x8023ffff pref]
[    0.527462] pci 0000:02:00.1: BAR 6: assigned [mem 0x80240000-0x8027ffff pref]
[    0.527466] pci 0000:00:1c.1: PCI bridge to [bus 02]
[    0.527481] pci 0000:00:1c.1:   bridge window [mem 0x80200000-0x802fffff]
[    0.527490] pci 0000:00:1c.1:   bridge window [mem 0x80000000-0x800fffff 64bit pref]
[    0.527504] pci 0000:00:1d.0: PCI bridge to [bus 03]
[    0.527531] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.527531] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.527532] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[    0.527533] pci_bus 0000:00: resource 7 [mem 0x80000000-0xfeafffff window]
[    0.527534] pci_bus 0000:01: resource 0 [io  0x1000-0x1fff]
[    0.527534] pci_bus 0000:01: resource 1 [mem 0x81000000-0x85bfffff]
[    0.527535] pci_bus 0000:02: resource 1 [mem 0x80200000-0x802fffff]
[    0.527536] pci_bus 0000:02: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[    0.527585] NET: Registered protocol family 2
[    0.527672] tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes)
[    0.527680] TCP established hash table entries: 8192 (order: 4, 65536 bytes)
[    0.527692] TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
[    0.527709] TCP: Hash tables configured (established 8192 bind 8192)
[    0.527724] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.527730] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.527767] NET: Registered protocol family 1
[    0.527911] pci 0000:01:00.1: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    0.528066] xen: registering gsi 16 triggering 0 polarity 1
[    0.528077] xen: --> pirq=16 -> irq=16 (gsi=16)
[    0.528148] xen: registering gsi 17 triggering 0 polarity 1
[    0.528155] xen: --> pirq=17 -> irq=17 (gsi=17)
[    0.528258] PCI: CLS 64 bytes, default 64
[    0.528330] RAPL PMU: API unit is 2^-32 Joules, 5 fixed counters, 655360 ms ovfl timer
[    0.528335] RAPL PMU: hw unit of domain pp0-core 2^-14 Joules
[    0.528339] RAPL PMU: hw unit of domain package 2^-14 Joules
[    0.528342] RAPL PMU: hw unit of domain dram 2^-14 Joules
[    0.528346] RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules
[    0.528349] RAPL PMU: hw unit of domain psys 2^-14 Joules
[    0.530444] Initialise system trusted keyrings
[    0.530475] workingset: timestamp_bits=46 max_order=18 bucket_order=0
[    0.530573] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.530640] FS-Cache: Netfs 'cifs' registered for caching
[    0.530655] fuse init (API version 7.27)
[    0.530697] SGI XFS with ACLs, security attributes, no debug enabled
[    0.530797] 9p: Installing v9fs 9p2000 file system support
[    0.530802] FS-Cache: Netfs '9p' registered for caching
[    0.530807] pstore: using deflate compression
[    0.532257] NET: Registered protocol family 38
[    0.532262] Key type asymmetric registered
[    0.532265] Asymmetric key parser 'x509' registered
[    0.532272] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)
[    0.532296] io scheduler noop registered
[    0.532299] io scheduler deadline registered (default)
[    0.532311] io scheduler cfq registered
[    0.532314] io scheduler mq-deadline registered (default)
[    0.532317] io scheduler kyber registered
[    0.532490] xen: registering gsi 16 triggering 0 polarity 1
[    0.532491] Already setup the GSI :16
[    0.532760] xen: registering gsi 16 triggering 0 polarity 1
[    0.532761] Already setup the GSI :16
[    0.533024] xen: registering gsi 16 triggering 0 polarity 1
[    0.533025] Already setup the GSI :16
[    0.533461] xen: registering gsi 17 triggering 0 polarity 1
[    0.533463] Already setup the GSI :17
[    0.533745] xen: registering gsi 16 triggering 0 polarity 1
[    0.533746] Already setup the GSI :16
[    0.533932] pcieport 0000:00:01.0: Signaling PME with IRQ 128
[    0.533962] pcieport 0000:00:01.1: Signaling PME with IRQ 129
[    0.533999] pcieport 0000:00:1b.0: Signaling PME with IRQ 130
[    0.534036] pcieport 0000:00:1c.0: Signaling PME with IRQ 131
[    0.534079] pcieport 0000:00:1c.1: Signaling PME with IRQ 132
[    0.534116] pcieport 0000:00:1d.0: Signaling PME with IRQ 133
[    0.534138] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    0.534147] hv_vmbus: registering driver hyperv_fb
[    0.534157] efifb: probing for efifb
[    0.534160] efi: EFI_MEMMAP is not enabled.
[    0.535042] efifb: framebuffer at 0x84000000, using 5120k, total 5120k
[    0.535047] efifb: mode is 1280x1024x32, linelength=5120, pages=1
[    0.535050] efifb: scrolling: redraw
[    0.535053] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
[    0.586199] Console: switching to colour frame buffer device 160x64
[    0.637641] fb0: EFI VGA frame buffer device
[    0.637790] intel_idle: MWAIT substates: 0x11142120
[    0.637790] intel_idle: v0.4.1 model 0x9E
[    0.637793] intel_idle: intel_idle yielding to none
[    0.637839] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    0.638119] ACPI: Power Button [PWRF]
[    0.638295] Monitor-Mwait will be used to enter C-1 state
[    0.638300] Monitor-Mwait will be used to enter C-2 state
[    0.638354] Warning: Processor Platform Limit not supported.
[    0.638465] ERST: Error Record Serialization Table (ERST) support is initialized.
[    0.638930] pstore: Registered erst as persistent store backend
[    0.639593] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC.
[    0.640046] xen:xen_evtchn: Event-channel device installed
[    0.640348] xen_pciback: backend is vpci
[    0.640696] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.661999] 00:02: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
[    0.683490] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    0.684160] Non-volatile memory driver v1.3
[    0.684387] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180 seconds, margin is 60 seconds).
[    0.685559] loop: module loaded
[    0.687073] Invalid max_queues (4), will use default max: 1.
[    0.687400] VMware PVSCSI driver - version 1.0.7.0-k
[    0.687642] hv_vmbus: registering driver hv_storvsc
[    0.687936] ahci 0000:00:17.0: version 3.0
[    0.688072] xen: registering gsi 16 triggering 0 polarity 1
[    0.688075] Already setup the GSI :16
[    0.706562] ahci 0000:00:17.0: AHCI 0001.0301 32 slots 6 ports 6 Gbps 0x3f impl SATA mode
[    0.706991] ahci 0000:00:17.0: flags: 64bit ncq sntf pm clo only pio slum part ems deso sadm sds apst 
[    0.857339] scsi host0: ahci
[    0.857649] scsi host1: ahci
[    0.857935] scsi host2: ahci
[    0.858220] scsi host3: ahci
[    0.858506] scsi host4: ahci
[    0.858790] scsi host5: ahci
[    0.858972] ata1: SATA max UDMA/133 abar m2048@0x80113000 port 0x80113100 irq 135
[    0.859402] ata2: SATA max UDMA/133 abar m2048@0x80113000 port 0x80113180 irq 135
[    0.859839] ata3: SATA max UDMA/133 abar m2048@0x80113000 port 0x80113200 irq 135
[    0.860266] ata4: SATA max UDMA/133 abar m2048@0x80113000 port 0x80113280 irq 135
[    0.860697] ata5: SATA max UDMA/133 abar m2048@0x80113000 port 0x80113300 irq 135
[    0.861127] ata6: SATA max UDMA/133 abar m2048@0x80113000 port 0x80113380 irq 135
[    0.861964] tun: Universal TUN/TAP device driver, 1.6
[    0.862292] VMware vmxnet3 virtual NIC driver - version 1.4.16.0-k-NAPI
[    0.862660] xen_netfront: Initialising Xen virtual ethernet driver
[    0.863025] hv_vmbus: registering driver hv_netvsc
[    0.863268] Fusion MPT base driver 3.04.20
[    0.863501] Copyright (c) 1999-2008 LSI Corporation
[    0.863785] Fusion MPT SPI Host driver 3.04.20
[    0.876614] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.889380] ehci-pci: EHCI PCI platform driver
[    0.902595] xen: registering gsi 17 triggering 0 polarity 1
[    0.902598] Already setup the GSI :17
[    0.915501] ehci-pci 0000:01:00.4: EHCI Host Controller
[    0.929838] ehci-pci 0000:01:00.4: new USB bus registered, assigned bus number 1
[    0.943160] ehci-pci 0000:01:00.4: irq 17, io mem 0x85b9c000
[    0.986534] ehci-pci 0000:01:00.4: USB 2.0 started, EHCI 1.00
[    0.999937] hub 1-0:1.0: USB hub found
[    1.013393] hub 1-0:1.0: 8 ports detected
[    1.027089] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.040868] ohci-pci: OHCI PCI platform driver
[    1.054876] uhci_hcd: USB Universal Host Controller Interface driver
[    1.069582] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    1.083935] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 2
[    1.101141] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000000009810
[    1.116330] xhci_hcd 0000:00:14.0: cache line size of 64 is not supported
[    1.116552] hub 2-0:1.0: USB hub found
[    1.131673] hub 2-0:1.0: 16 ports detected
[    1.146892] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    1.161799] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 3
[    1.177086] xhci_hcd 0000:00:14.0: Host supports USB 3.1 Enhanced SuperSpeed
[    1.192460] hub 3-0:1.0: USB hub found
[    1.207377] hub 3-0:1.0: 6 ports detected
[    1.222408] usbcore: registered new interface driver usb-storage
[    1.237698] i8042: PNP: No PS/2 controller found.
[    1.252848] i8042: Probing ports directly.
[    1.873419] i8042: Can't read CTR while initializing i8042
[    1.892463] i8042: probe of i8042 failed with error -5
[    1.910949] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    1.930038] hv_vmbus: registering driver hyperv_keyboard
[    1.948075] ata1.00: supports DRM functions and may not be fully accessible
[    1.966762] ata6: SATA link down (SStatus 4 SControl 300)
[    1.985119] ata5: SATA link down (SStatus 4 SControl 300)
[    2.003283] ata4: SATA link down (SStatus 4 SControl 300)
[    2.020906] ata3: SATA link down (SStatus 4 SControl 300)
[    2.038348] ata2: SATA link down (SStatus 4 SControl 300)
[    2.056105] mousedev: PS/2 mouse device common for all mice
[    2.073621] input: PC Speaker as /devices/platform/pcspkr/input/input1
[    2.090670] rtc_cmos 00:00: RTC can wake from S4
[    2.107558] ata1.00: ATA-11: Samsung SSD 860 EVO 250GB, RVT01B6Q, max UDMA/133
[    2.124935] ata1.00: 488397168 sectors, multi 1: LBA48 NCQ (depth 32), AA
[    2.143710] rtc_cmos 00:00: registered as rtc0
[    2.160973] rtc_cmos 00:00: alarms up to one month, y3k, 114 bytes nvram
[    2.179039] i2c /dev entries driver
[    2.196019] device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com
[    2.213930] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x2fbe1e88e16, max_idle_ns: 440795327210 ns
[    2.232168] sdhci: Secure Digital Host Controller Interface driver
[    2.249626] sdhci: Copyright(c) Pierre Ossman
[    2.267904] clocksource: Switched to clocksource tsc
[    2.284971] ata1.00: supports DRM functions and may not be fully accessible
[    2.302322] usb 2-3: new low-speed USB device number 2 using xhci_hcd
[    2.321510] sdhci-pltfm: SDHCI platform and OF driver helper
[    2.341002] EFI Variables Facility v0.08 2004-May-17
[    2.362759] ata1.00: configured for UDMA/133
[    2.382259] scsi 0:0:0:0: Direct-Access     ATA      Samsung SSD 860  1B6Q PQ: 0 ANSI: 5
[    2.421049] pstore: ignoring unexpected backend 'efi'
[    2.441375] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    2.461495] ata1.00: Enabling discard_zeroes_data
[    2.480169] sd 0:0:0:0: [sda] 488397168 512-byte logical blocks: (250 GB/233 GiB)
[    2.500290] usbcore: registered new interface driver usbhid
[    2.520950] usbhid: USB HID core driver
[    2.540103] sd 0:0:0:0: [sda] Write Protect is off
[    2.559928] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    2.559951] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[    2.576876] hv_utils: Registering HyperV Utility Driver
[    2.593626] hv_vmbus: registering driver hv_util
[    2.610515] hv_vmbus: registering driver hv_balloon
[    2.630914] ata1.00: Enabling discard_zeroes_data
[    2.650861] oprofile: using NMI timer interrupt.
[    2.671330] GACT probability on
[    2.691698] Mirror/redirect action on
[    2.712372] Simple TC action Loaded
[    2.733092] u32 classifier
[    2.752614]     Performance counters on
[    2.772749]     input device check on
[    2.791525]     Actions configured
[    2.808405] xt_time: kernel timezone is -0000
[    2.825744] IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
[    2.842989] IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
[    2.862535] IPVS: Each connection entry needs 288 bytes at least
[    2.862714] IPVS: ipvs loaded.
[    2.881263]  sda: sda1 sda2 sda3 sda4 sda9
[    2.899739] IPVS: [rr] scheduler registered.
[    2.917423] input: Dell KB216 Wired Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.0/0003:413C:2113.0001/input/input2
[    2.935411] ata1.00: Enabling discard_zeroes_data
[    2.953364] IPVS: [wrr] scheduler registered.
[    2.970692] sd 0:0:0:0: [sda] Attached SCSI disk
[    2.987668] IPVS: [lc] scheduler registered.
[    3.005210] IPVS: [wlc] scheduler registered.
[    3.023012] IPVS: [fo] scheduler registered.
[    3.039590] hid-generic 0003:413C:2113.0001: input: USB HID v1.11 Keyboard [Dell KB216 Wired Keyboard] on usb-0000:00:14.0-3/input0
[    3.056404] IPVS: [ovf] scheduler registered.
[    3.073124] IPVS: [lblc] scheduler registered.
[    3.089973] IPVS: [lblcr] scheduler registered.
[    3.106697] input: Dell KB216 Wired Keyboard System Control as /devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.1/0003:413C:2113.0002/input/input3
[    3.124078] IPVS: [dh] scheduler registered.
[    3.141350] IPVS: [sh] scheduler registered.
[    3.158346] IPVS: [sed] scheduler registered.
[    3.175496] IPVS: [nq] scheduler registered.
[    3.192820] IPVS: ftp: loaded support on port[0] = 21
[    3.210230] input: Dell KB216 Wired Keyboard Consumer Control as /devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.1/0003:413C:2113.0002/input/input4
[    3.228740] ipip: IPv4 and MPLS over IPv4 tunneling driver
[    3.247273] hid-generic 0003:413C:2113.0002: input: USB HID v1.11 Device [Dell KB216 Wired Keyboard] on usb-0000:00:14.0-3/input1
[    3.266440] gre: GRE over IPv4 demultiplexor driver
[    3.285608] ipt_CLUSTERIP: ClusterIP Version 0.8 loaded successfully
[    3.304204] Initializing XFRM netlink socket
[    3.323208] NET: Registered protocol family 10
[    3.342276] Segment Routing with IPv6
[    3.361851] NET: Registered protocol family 17
[    3.380998] NET: Registered protocol family 15
[    3.399290] Bridge firewalling registered
[    3.417263] 8021q: 802.1Q VLAN Support v1.8
[    3.434865] usb 2-4: new high-speed USB device number 3 using xhci_hcd
[    3.452732] 9pnet: Installing 9P2000 support
[    3.470321] Initialising Xen transport for 9pfs
[    3.487895] Key type dns_resolver registered
[    3.505435] AVX2 version of gcm_enc/dec engaged.
[    3.522945] AES CTR mode by8 optimization enabled
[    3.547554] sched_clock: Marking stable (3545011530, 2059674)->(3556866077, -9794873)
[    3.567325] registered taskstats version 1
[    3.586626] Loading compiled-in X.509 certificates
[    3.604901] Key type big_key registered
[    3.623646] Key type encrypted registered
[    3.644757] rtc_cmos 00:00: setting system clock to 2019-09-25 21:49:22 UTC (1569448162)
[    3.664338] Waiting 3 sec before mounting root device...
[    5.016483] random: fast init done
[    6.720766] VFS: Mounted root (squashfs filesystem) readonly on device 8:2.
[    6.739566] Freeing unused kernel image memory: 1592K
[    6.776530] Write protecting the kernel read-only data: 18432k
[    6.801514] Freeing unused kernel image memory: 2020K
[    6.820340] Freeing unused kernel image memory: 1012K
[    6.838965] Run /sbin/init as init process
[    7.102200] tg3.c:v3.137 (May 11, 2014)
[    7.120575] xen: registering gsi 17 triggering 0 polarity 1
[    7.120578] Already setup the GSI :17
[    7.184577] tg3 0000:02:00.0 eth0: Tigon3 [partno(N/A) rev 5720000] (PCI Express) MAC address 08:f1:ea:b1:ab:a4
[    7.203015] tg3 0000:02:00.0 eth0: attached PHY is 5720C (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[1])
[    7.222603] tg3 0000:02:00.0 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] TSOcap[1]
[    7.243111] tg3 0000:02:00.0 eth0: dma_rwctrl[00000001] dma_mask[64-bit]
[    7.269462] xen: registering gsi 18 triggering 0 polarity 1
[    7.269474] xen: --> pirq=18 -> irq=18 (gsi=18)
[    7.311556] tg3 0000:02:00.1 eth1: Tigon3 [partno(N/A) rev 5720000] (PCI Express) MAC address 08:f1:ea:b1:ab:a5
[    7.331861] tg3 0000:02:00.1 eth1: attached PHY is 5720C (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[1])
[    7.352657] tg3 0000:02:00.1 eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] TSOcap[1]
[    7.372782] tg3 0000:02:00.1 eth1: dma_rwctrl[00000001] dma_mask[64-bit]
[    8.743907] usb 2-4: device descriptor read/64, error -110
[    8.985080] w83627hf_wdt: Unsupported chip ID: 0x04
[    8.987068] w83627hf_wdt: Unsupported chip ID: 0x04
[    9.007438] IPVS: ftp: loaded support on port[0] = 21
[    9.037533] usb-storage 2-4:1.0: USB Mass Storage device detected
[    9.038884] scsi host6: usb-storage 2-4:1.0
[    9.086216] FAT-fs (sda4): utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!
[    9.149685] IPVS: ftp: loaded support on port[0] = 21
[    9.179912] random: crng init done
[    9.186435] usb 2-8: new high-speed USB device number 4 using xhci_hcd
[    9.367376] hub 2-8:1.0: USB hub found
[    9.367628] hub 2-8:1.0: 2 ports detected
[   10.077137] scsi 6:0:0:0: Direct-Access     General  UDisk            5.00 PQ: 0 ANSI: 2
[   10.077270] sd 6:0:0:0: Attached scsi generic sg1 type 0
[   10.077866] sd 6:0:0:0: [sdb] 15728640 512-byte logical blocks: (8.05 GB/7.50 GiB)
[   10.078182] sd 6:0:0:0: [sdb] Write Protect is off
[   10.078183] sd 6:0:0:0: [sdb] Mode Sense: 0b 00 00 08
[   10.078303] sd 6:0:0:0: [sdb] No Caching mode page found
[   10.078303] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[   10.134870]  sdb:
[   10.158686] sd 6:0:0:0: [sdb] Attached SCSI removable disk
[   10.879935] usbcore: registered new interface driver qcserial
[   10.904734] usbserial: USB Serial support registered for Qualcomm USB modem
[   10.934204] usbcore: registered new interface driver cdc_wdm
[   10.960548] usbcore: registered new interface driver qmi_wwan
[   12.118816] EXT4-fs (sda9): mounting ext3 file system using the ext4 subsystem
[   12.150070] EXT4-fs (sda9): mounted filesystem with ordered data mode. Opts: (null)
[  133.902114] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[  134.045963] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[  134.095029] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[  134.124102] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[  134.145963] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[  134.168944] cfg80211: failed to load regulatory.db
[  137.376648] tg3 0000:02:00.0 eth0: Link is up at 1000 Mbps, full duplex
[  137.395226] tg3 0000:02:00.0 eth0: Flow control is on for TX and on for RX
[  137.414684] tg3 0000:02:00.0 eth0: EEE is enabled
[  137.434416] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[  740.029099] FAT-fs (sdb): utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!
[  740.060295] FAT-fs (sdb): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.

[-- Attachment #3: dmesg.xen --]
[-- Type: application/octet-stream, Size: 14727 bytes --]

 Xen 4.12.0
(XEN) Xen version 4.12.0 (@) (gcc (Alpine 6.4.0) 6.4.0) debug=n  Wed Sep 25 16:44:37 UTC 2019
(XEN) Latest ChangeSet: 
(XEN) Bootloader: GRUB 2.03
(XEN) Command line: loglvl=all dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
(XEN) Xen image load base address: 0x71200000
(XEN) Video information:
(XEN)  VGA is graphics mode 1280x1024, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  0000000000000000 - 000000000009f000 (usable)
(XEN)  000000000009f000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000065439000 (usable)
(XEN)  0000000065439000 - 0000000065448000 (reserved)
(XEN)  0000000065448000 - 0000000065484000 (usable)
(XEN)  0000000065484000 - 0000000065488000 (reserved)
(XEN)  0000000065488000 - 000000006e2b7000 (usable)
(XEN)  000000006e2b7000 - 000000006e2c9000 (reserved)
(XEN)  000000006e2c9000 - 000000006e2ca000 (usable)
(XEN)  000000006e2ca000 - 000000006f4ec000 (reserved)
(XEN)  000000006f4ec000 - 000000006f4ed000 (ACPI NVS)
(XEN)  000000006f4ed000 - 000000006f4ff000 (reserved)
(XEN)  000000006f4ff000 - 0000000078cff000 (usable)
(XEN)  0000000078cff000 - 00000000799ff000 (reserved)
(XEN)  00000000799ff000 - 000000007bdff000 (ACPI NVS)
(XEN)  000000007bdff000 - 000000007bfff000 (ACPI data)
(XEN)  000000007bfff000 - 000000007c000000 (usable)
(XEN)  000000007c000000 - 0000000080000000 (reserved)
(XEN)  00000000fe010000 - 00000000fe011000 (reserved)
(XEN)  0000000100000000 - 000000027f000000 (usable)
(XEN) ACPI: RSDP 7BFFE014, 0024 (r2 HPE   )
(XEN) ACPI: XSDT 7BFE7188, 00FC (r1 HPE    Server          1 INTL 20160422)
(XEN) ACPI: FACP 7BFF4000, 010C (r6 HPE    Server          1 1590        1)
(XEN) ACPI: DSDT 7BFE4000, 14F4 (r2 HPE    Server          2 HPE     20000)
(XEN) ACPI: FACS 7BDD0000, 0040
(XEN) ACPI: SSDT 7BFFC000, 0197 (r1 HPE     CpuSsdt        1 INTL 20130328)
(XEN) ACPI: BOOT 7BFFB000, 0028 (r1 HPE    Server          2 MSFT  1000013)
(XEN) ACPI: SSDT 7BFFA000, 003A (r1 HPE    PLATOPRG        1 INTL 20130328)
(XEN) ACPI: HEST 7BFF9000, 00A8 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: BERT 7BFF8000, 0030 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: ERST 7BFF7000, 0250 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: FPDT 7BFF6000, 0034 (r1 HPE    Server   20170001 INTL 20160422)
(XEN) ACPI: BGRT 7BFF5000, 0038 (r1 HPE    Server   20170001 1590 20160422)
(XEN) ACPI: HPET 7BFF3000, 0038 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: WDDT 7BFF2000, 0040 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: APIC 7BFF1000, 0084 (r3 HPE    Server          1 1590        1)
(XEN) ACPI: MCFG 7BFF0000, 003C (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SLIT 7BFEF000, 002D (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SRAT 7BFEE000, 0138 (r3 HPE    Server          1 1590        1)
(XEN) ACPI: SPMI 7BFED000, 0041 (r5 HPE    Server          1 1590        1)
(XEN) ACPI: RASF 7BFEC000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SPCR 7BFEB000, 0050 (r2 HPE    Server          1 1590        1)
(XEN) ACPI: MSCT 7BFEA000, 004E (r2 HPE    Server          1 1590        1)
(XEN) ACPI: BDAT 7BFE9000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: WSMT 7BFE8000, 0028 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: PCCT 7BFFD000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: DMAR 7BFE6000, 00B0 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SSDT 7BFE1000, 2AA2 (r2 HPE    PCISSDT         2 HPE     20000)
(XEN) ACPI: SSDT 7BFE0000, 01C4 (r2 HPE    TIMESSDT        2 HPE     20000)
(XEN) ACPI: SSDT 7BFDF000, 02F2 (r1 HPE        pmab        1 INTL 20130328)
(XEN) ACPI: OEM1 7BFDE000, 00DF (r2 HPE    CPU  CPC        2 HPE     20000)
(XEN) System RAM: 8042MB (8235284kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 3f500000-7c000000
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-3f500000
(XEN) SRAT: Node 0 PXM 0 100000000-200000000
(XEN) SRAT: Node 0 PXM 0 200000000-27f000000
(XEN) NUMA: Allocated memnodemap from 278b58000 - 278b5b000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) vesafb: framebuffer at 0x84000000, mapped to 0xffff82c000201000, using 5120k, total 5120k
(XEN) vesafb: mode is 1280x1024x32, linelength=5120, font 8x16
(XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 158 (0x9e), Stepping 10 (raw 000906ea)
(XEN) SMBIOS 3.1 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x508 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[1:0], status[1:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:504,1:0], pm1x_evt[1:500,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 7bdd0000/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[7bdd000c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] enabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high level lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high level lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high level lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x06] high level lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
(XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) Xen ERST support is initialized.
(XEN) HEST: Table parsing has been initialized
(XEN) ACPI: BGRT: invalidating v1 image at 0x72701000
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
(XEN) Not enabling x2APIC (upon firmware request)
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware features: IBRS/IBPB STIBP L1D_FLUSH SSBD
(XEN)   Compiled-in support: SHADOW_PAGING
(XEN)   Xen settings: BTI-Thunk N/A, SPEC_CTRL: IBRS+ SSBD-, Other: IBPB L1D_FLUSH
(XEN)   L1TF: believed vulnerable, maxphysaddr L1D 46, CPUID 39, Safe address 8000000000
(XEN)   Support for HVM VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   Support for PV VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled (with PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU enabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Platform timer is 23.999MHz HPET
(XEN) Detected 3312.146 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000003fff type=2 attr=000000000000000f
(XEN)  0000000004000-000000008efff type=7 attr=000000000000000f
(XEN)  000000008f000-000000009efff type=2 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
(XEN)  0000000100000-00000008c5fff type=2 attr=000000000000000f
(XEN)  00000008c6000-0000014774fff type=7 attr=000000000000000f
(XEN)  0000014775000-000003f4fffff type=1 attr=000000000000000f
(XEN)  000003f500000-000003f57ffff type=4 attr=000000000000000f
(XEN)  000003f580000-000003fcfffff type=7 attr=000000000000000f
(XEN)  000003fd00000-000003fd00fff type=3 attr=000000000000000f
(XEN)  000003fd01000-00000454a4fff type=4 attr=000000000000000f
(XEN)  00000454a5000-0000065438fff type=7 attr=000000000000000f
(XEN)  0000065439000-0000065447fff type=6 attr=800000000000000f
(XEN)  0000065448000-0000065483fff type=3 attr=000000000000000f
(XEN)  0000065484000-0000065487fff type=6 attr=800000000000000f
(XEN)  0000065488000-00000660b5fff type=3 attr=000000000000000f
(XEN)  00000660b6000-000006e2b6fff type=4 attr=000000000000000f
(XEN)  000006e2b7000-000006e2c8fff type=6 attr=800000000000000f
(XEN)  000006e2c9000-000006e2c9fff type=7 attr=000000000000000f
(XEN)  000006e2ca000-000006f4ebfff type=6 attr=800000000000000f
(XEN)  000006f4ec000-000006f4ecfff type=10 attr=000000000000000f
(XEN)  000006f4ed000-000006f4fefff type=6 attr=800000000000000f
(XEN)  000006f4ff000-000006f8f7fff type=7 attr=000000000000000f
(XEN)  000006f8f8000-000006f8f8fff type=2 attr=000000000000000f
(XEN)  000006f8f9000-000006fa61fff type=7 attr=000000000000000f
(XEN)  000006fa62000-000006fcfefff type=1 attr=000000000000000f
(XEN)  000006fcff000-00000713fffff type=7 attr=000000000000000f
(XEN)  0000071400000-0000071788fff type=2 attr=000000000000000f
(XEN)  0000071789000-0000071899fff type=7 attr=000000000000000f
(XEN)  000007189a000-000007251cfff type=4 attr=000000000000000f
(XEN)  000007251d000-00000726acfff type=7 attr=000000000000000f
(XEN)  00000726ad000-0000073b34fff type=4 attr=000000000000000f
(XEN)  0000073b35000-0000073b3cfff type=7 attr=000000000000000f
(XEN)  0000073b3d000-0000073b3dfff type=4 attr=000000000000000f
(XEN)  0000073b3e000-0000073b40fff type=7 attr=000000000000000f
(XEN)  0000073b41000-0000077cfefff type=4 attr=000000000000000f
(XEN)  0000077cff000-0000077cfffff type=7 attr=000000000000000f
(XEN)  0000077d00000-0000078cfefff type=3 attr=000000000000000f
(XEN)  0000078cff000-00000794fefff type=5 attr=800000000000000f
(XEN)  00000794ff000-00000798fefff type=6 attr=800000000000000f
(XEN)  00000798ff000-00000799fefff type=0 attr=000000000000000f
(XEN)  00000799ff000-000007bdfefff type=10 attr=000000000000000f
(XEN)  000007bdff000-000007bffefff type=9 attr=000000000000000f
(XEN)  000007bfff000-000007bffffff type=4 attr=000000000000000f
(XEN)  0000100000000-000027effffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
(XEN)  000007c000000-000007fffffff type=0 attr=0000000000000000
(XEN)  00000fe010000-00000fe010fff type=11 attr=8000000000000001
(XEN) Initing memory sharing.
(XEN) alt table ffff82d08042ac70 -> ffff82d08042c97a
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC_DEADLINE disabled due to Errata; please update microcode to version 0x52 (or later)
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x11142120
(XEN) mwait-idle: v0.4.1 model 0x9e
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - VMCS shadowing
(XEN)  - VM Functions
(XEN)  - Virtualisation Exceptions
(XEN)  - Page Modification Logging
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Adding cpu 1 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Adding cpu 2 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Adding cpu 3 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Brought up 4 CPUs
(XEN) build-id: 52d6fb33eb68a4588dd69266fe7de74f0ed15686
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) Dom0 has maximum 312 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** Building a PV Dom0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2a2c000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000270000000->0000000274000000 (245760 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff82a2c000
(XEN)  Init. ramdisk: 0000000000000000->0000000000000000
(XEN)  Phys-Mach map: 0000008000000000->0000008000200000
(XEN)  Start info:    ffffffff82a2c000->ffffffff82a2c4b4
(XEN)  Xenstore ring: 0000000000000000->0000000000000000
(XEN)  Console ring:  0000000000000000->0000000000000000
(XEN)  Page tables:   ffffffff82a2d000->ffffffff82a46000
(XEN)  Boot stack:    ffffffff82a46000->ffffffff82a47000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82c00000
(XEN)  ENTRY ADDRESS: ffffffff826af180
(XEN) Dom0 has maximum 1 VCPUs
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) Freed 480kB init memory
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:01.0
(XEN) PCI add device 0000:00:01.1
(XEN) PCI add device 0000:00:12.0
(XEN) PCI add device 0000:00:14.0
(XEN) PCI add device 0000:00:14.2
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:17.0
(XEN) PCI add device 0000:00:1b.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.1
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.5
(XEN) PCI add device 0000:01:00.0
(XEN) PCI add device 0000:01:00.1
(XEN) PCI add device 0000:01:00.2
(XEN) PCI add device 0000:01:00.4
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:02:00.1
(XEN) PCI: Using MCFG for segment 0000 bus 00-ff

[-- Attachment #4: xl.info --]
[-- Type: application/octet-stream, Size: 1386 bytes --]

host                   : linuxkit-08f1eab1aba4
release                : 4.19.5-linuxkit
version                : #1 SMP Sat Jul 13 05:48:19 UTC 2019
machine                : x86_64
nr_cpus                : 4
max_cpu_id             : 3
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 1
cpu_mhz                : 3312.146
hw_caps                : bfebfbff:76faf3ff:2c100800:00000121:0000000f:009c6fbb:00000000:00000100
virt_caps              : pv hvm hvm_directio pv_directio
total_memory           : 8042
free_memory            : 6904
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 12
xen_extra              : .0
xen_version            : 4.12.0
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit2
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : 
xen_commandline        : loglvl=all dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
cc_compiler            : gcc (Alpine 6.4.0) 6.4.0
cc_compile_by          : 
cc_compile_domain      : 
cc_compile_date        : Wed Sep 25 16:44:37 UTC 2019
build_id               : 52d6fb33eb68a4588dd69266fe7de74f0ed15686
xend_config_format     : 4

[-- Attachment #5: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-25 22:31   ` Roman Shaposhnik
@ 2019-09-26  7:44     ` Jan Beulich
  2019-09-27  2:28       ` Roman Shaposhnik
  2019-09-28  3:07       ` Roman Shaposhnik
  0 siblings, 2 replies; 19+ messages in thread
From: Jan Beulich @ 2019-09-26  7:44 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel

On 26.09.2019 00:31, Roman Shaposhnik wrote:
> Jan, Roger, thank you so much for the initial ideas. I tried a few of
> those and here's where I am.
> 
> First of all, it is definitely related to CPU bring up. Adding
> cpuidle=0 to xen command line made Xen boot.
> 
> Then, a good friend of mine (who you may know from ancient Xen days
> ;-)) suggested that this could be related to this:
>      https://wiki.xenproject.org/wiki/Xen_power_management
> so I went to the BIOS settings and quite to my surprise all of them
> were grayed out (not tweakable).
> 
> The only one that wasn't was 2xAPIC support. So just for kicks -- I
> disabled that.
> 
> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.

Interesting, but unfortunately this particular log is of no real use
for investigation of the issue (other than knowing the CPU model). I
also notice it's a 4.12.0 log, when your original report was against
latest master.

> So I guess at this point, you could say that I have a functional
> system, but I'm curious whether you guys would be interested to look
> into 2xAPIC situation.

Of course we do. As a next step I'd suggest reverting the BIOS settings
change you did, and instead using the "x2apic=0" Xen command line option.

And then we of course need a complete boot log (as requested earlier) of
a problem case.

Further I'd suggest moving away from the black-and-white "cpuidle="
option, and instead limiting use of deep C states ("max_cstate="). I
wouldn't be surprised if this was the issue; we'd then have to first
of all go through errata for the part your system is using.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-26  7:44     ` Jan Beulich
@ 2019-09-27  2:28       ` Roman Shaposhnik
  2019-09-27  7:58         ` Jan Beulich
  2019-09-28  3:07       ` Roman Shaposhnik
  1 sibling, 1 reply; 19+ messages in thread
From: Roman Shaposhnik @ 2019-09-27  2:28 UTC (permalink / raw)
  To: Jan Beulich; +Cc: xen-devel

On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 26.09.2019 00:31, Roman Shaposhnik wrote:
> > Jan, Roger, thank you so much for the initial ideas. I tried a few of
> > those and here's where I am.
> >
> > First of all, it is definitely related to CPU bring up. Adding
> > cpuidle=0 to xen command line made Xen boot.
> >
> > Then, a good friend of mine (who you may know from ancient Xen days
> > ;-)) suggested that this could be related to this:
> >      https://wiki.xenproject.org/wiki/Xen_power_management
> > so I went to the BIOS settings and quite to my surprise all of them
> > were grayed out (not tweakable).
> >
> > The only one that wasn't was 2xAPIC support. So just for kicks -- I
> > disabled that.
> >
> > That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
>
> Interesting, but unfortunately this particular log is of no real use
> for investigation of the issue (other than knowing the CPU model). I
> also notice it's a 4.12.0 log, when your original report was against
> latest master.

Understood. But this brings us back to: if I don't get Xen booting,
I don't think I can capture the logs. This is a rackable server without
anything like a serial port, etc.

I suppose the best I can do is to capture a video of Xen failing to boot?

Any other ideas?

> > So I guess at this point, you could say that I have a functional
> > system, but I'm curious whether you guys would be interested to look
> > into 2xAPIC situation.
>
> Of course we do. As a next step I'd suggest reverting the BIOS settings
> change you did, and instead using the "x2apic=0" Xen command line option.
>
> And then we of course need a complete boot log (as requested earlier) of
> a problem case.
>
> Further I'd suggest moving away from the black-and-white "cpuidle="
> option, and instead limiting use of deep C states ("max_cstate="). I
> wouldn't be surprised if this was the issue; we'd then have to first
> of all go through errata for the part your system is using.

Will do these experiments tomorrow and report back.

Still -- please let me know how can I capture the log without serial, etc.

Thanks,
Roman.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-27  2:28       ` Roman Shaposhnik
@ 2019-09-27  7:58         ` Jan Beulich
  0 siblings, 0 replies; 19+ messages in thread
From: Jan Beulich @ 2019-09-27  7:58 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel

On 27.09.2019 04:28, Roman Shaposhnik wrote:
> On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 26.09.2019 00:31, Roman Shaposhnik wrote:
>>> Jan, Roger, thank you so much for the initial ideas. I tried a few of
>>> those and here's where I am.
>>>
>>> First of all, it is definitely related to CPU bring up. Adding
>>> cpuidle=0 to xen command line made Xen boot.
>>>
>>> Then, a good friend of mine (who you may know from ancient Xen days
>>> ;-)) suggested that this could be related to this:
>>>      https://wiki.xenproject.org/wiki/Xen_power_management
>>> so I went to the BIOS settings and quite to my surprise all of them
>>> were grayed out (not tweakable).
>>>
>>> The only one that wasn't was 2xAPIC support. So just for kicks -- I
>>> disabled that.
>>>
>>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
>>
>> Interesting, but unfortunately this particular log is of no real use
>> for investigation of the issue (other than knowing the CPU model). I
>> also notice it's a 4.12.0 log, when your original report was against
>> latest master.
> 
> Understood. But this brings us back to: if I don't get Xen booting,
> I don't think I can capture the logs. This is a rackable server without
> anything like a serial port, etc.
> 
> I suppose the best I can do is to capture a video of Xen failing to boot?

As a last resort - yes. But that'll be awful to work with. Are you
saying you can't even put in a PCI card providing a serial port?
(I realize that asking about using a possible EHCI debug port would
be a little much, as the necessary cable isn't really inexpensive.)

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-26  7:44     ` Jan Beulich
  2019-09-27  2:28       ` Roman Shaposhnik
@ 2019-09-28  3:07       ` Roman Shaposhnik
  2019-09-30  9:57         ` Jan Beulich
  2019-09-30 10:19         ` Andrew Cooper
  1 sibling, 2 replies; 19+ messages in thread
From: Roman Shaposhnik @ 2019-09-28  3:07 UTC (permalink / raw)
  To: Jan Beulich; +Cc: xen-devel

On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 26.09.2019 00:31, Roman Shaposhnik wrote:
> > Jan, Roger, thank you so much for the initial ideas. I tried a few of
> > those and here's where I am.
> >
> > First of all, it is definitely related to CPU bring up. Adding
> > cpuidle=0 to xen command line made Xen boot.
> >
> > Then, a good friend of mine (who you may know from ancient Xen days
> > ;-)) suggested that this could be related to this:
> >      https://wiki.xenproject.org/wiki/Xen_power_management
> > so I went to the BIOS settings and quite to my surprise all of them
> > were grayed out (not tweakable).
> >
> > The only one that wasn't was 2xAPIC support. So just for kicks -- I
> > disabled that.
> >
> > That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
>
> Interesting, but unfortunately this particular log is of no real use
> for investigation of the issue (other than knowing the CPU model). I
> also notice it's a 4.12.0 log, when your original report was against
> latest master.
>
> > So I guess at this point, you could say that I have a functional
> > system, but I'm curious whether you guys would be interested to look
> > into 2xAPIC situation.
>
> Of course we do. As a next step I'd suggest reverting the BIOS settings
> change you did, and instead using the "x2apic=0" Xen command line option.

Interestingly enough, this doesn't really solve the problem completely.
Specifying x2apic=0 certainly makes Xen go much further to a point
where it tries to load Dom0 and then the console VGA screen goes
blank (this is where that serial debug output would be very useful :-().

I suppose I can buy:
     https://www.cdw.com/product/hpe-dl20-ml30-gen10-m.2-dedicated-ilo-and-serial-port-kit/5348024
but that'll take some time to arrive.

> And then we of course need a complete boot log (as requested earlier) of
> a problem case.
>
> Further I'd suggest moving away from the black-and-white "cpuidle="
> option, and instead limiting use of deep C states ("max_cstate="). I
> wouldn't be surprised if this was the issue; we'd then have to first
> of all go through errata for the part your system is using.

Yup. max_cstate=1 makes it boot fine. max_cstate=2 though hangs
the system *exactly* in the same way as specifying x2apic=0
(which is different from the original problem as I've described above).

Can you please elaborate on "we'd then have to first of all go through
errata for the part your system is using"

Thanks,
Roman.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-28  3:07       ` Roman Shaposhnik
@ 2019-09-30  9:57         ` Jan Beulich
  2019-09-30 10:14           ` Jan Beulich
  2019-09-30 22:16           ` Roman Shaposhnik
  2019-09-30 10:19         ` Andrew Cooper
  1 sibling, 2 replies; 19+ messages in thread
From: Jan Beulich @ 2019-09-30  9:57 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel

On 28.09.2019 05:07, Roman Shaposhnik wrote:
> On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 26.09.2019 00:31, Roman Shaposhnik wrote:
>>> Jan, Roger, thank you so much for the initial ideas. I tried a few of
>>> those and here's where I am.
>>>
>>> First of all, it is definitely related to CPU bring up. Adding
>>> cpuidle=0 to xen command line made Xen boot.
>>>
>>> Then, a good friend of mine (who you may know from ancient Xen days
>>> ;-)) suggested that this could be related to this:
>>>      https://wiki.xenproject.org/wiki/Xen_power_management
>>> so I went to the BIOS settings and quite to my surprise all of them
>>> were grayed out (not tweakable).
>>>
>>> The only one that wasn't was 2xAPIC support. So just for kicks -- I
>>> disabled that.
>>>
>>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
>>
>> Interesting, but unfortunately this particular log is of no real use
>> for investigation of the issue (other than knowing the CPU model). I
>> also notice it's a 4.12.0 log, when your original report was against
>> latest master.
>>
>>> So I guess at this point, you could say that I have a functional
>>> system, but I'm curious whether you guys would be interested to look
>>> into 2xAPIC situation.
>>
>> Of course we do. As a next step I'd suggest reverting the BIOS settings
>> change you did, and instead using the "x2apic=0" Xen command line option.
> 
> Interestingly enough, this doesn't really solve the problem completely.
> Specifying x2apic=0 certainly makes Xen go much further to a point
> where it tries to load Dom0 and then the console VGA screen goes
> blank (this is where that serial debug output would be very useful :-().

Now that's again unexpected. In any event you could try "vga=keep".

>> And then we of course need a complete boot log (as requested earlier) of
>> a problem case.
>>
>> Further I'd suggest moving away from the black-and-white "cpuidle="
>> option, and instead limiting use of deep C states ("max_cstate="). I
>> wouldn't be surprised if this was the issue; we'd then have to first
>> of all go through errata for the part your system is using.
> 
> Yup. max_cstate=1 makes it boot fine. max_cstate=2 though hangs
> the system *exactly* in the same way as specifying x2apic=0
> (which is different from the original problem as I've described above).

"max_cstate=2" is much less of a "deep" C state than I had expected,
but well, so be it then. As to the hang - did you meanwhile figure
whether _any_ number of CPUs above 1 would result in a hang, or
whether instead there's a certain amount of them that would allow
boot to progress fine.

> Can you please elaborate on "we'd then have to first of all go through
> errata for the part your system is using"

Well, it wouldn't be the first time that hardware had issues with C
state handling. Therefore we'd need to (a) be sure you use up-to-date
microcode and (b) there are no errata documented for your CPU model
workarounds for which basically suggest to avoid use of deep C states.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-30  9:57         ` Jan Beulich
@ 2019-09-30 10:14           ` Jan Beulich
  2019-09-30 22:16           ` Roman Shaposhnik
  1 sibling, 0 replies; 19+ messages in thread
From: Jan Beulich @ 2019-09-30 10:14 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel

On 30.09.2019 11:57, Jan Beulich wrote:
> On 28.09.2019 05:07, Roman Shaposhnik wrote:
>> Can you please elaborate on "we'd then have to first of all go through
>> errata for the part your system is using"
> 
> Well, it wouldn't be the first time that hardware had issues with C
> state handling. Therefore we'd need to (a) be sure you use up-to-date
> microcode and (b) there are no errata documented for your CPU model
> workarounds for which basically suggest to avoid use of deep C states.

And I've just done (b) without finding anything that would suggest a
connection to the issue you observe.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-30  9:57         ` Jan Beulich
  2019-09-30 10:14           ` Jan Beulich
@ 2019-09-30 22:16           ` Roman Shaposhnik
  2019-09-30 22:38             ` Roman Shaposhnik
  1 sibling, 1 reply; 19+ messages in thread
From: Roman Shaposhnik @ 2019-09-30 22:16 UTC (permalink / raw)
  To: Jan Beulich; +Cc: xen-devel

On Mon, Sep 30, 2019 at 2:56 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 28.09.2019 05:07, Roman Shaposhnik wrote:
> > On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
> >>
> >> On 26.09.2019 00:31, Roman Shaposhnik wrote:
> >>> Jan, Roger, thank you so much for the initial ideas. I tried a few of
> >>> those and here's where I am.
> >>>
> >>> First of all, it is definitely related to CPU bring up. Adding
> >>> cpuidle=0 to xen command line made Xen boot.
> >>>
> >>> Then, a good friend of mine (who you may know from ancient Xen days
> >>> ;-)) suggested that this could be related to this:
> >>>      https://wiki.xenproject.org/wiki/Xen_power_management
> >>> so I went to the BIOS settings and quite to my surprise all of them
> >>> were grayed out (not tweakable).
> >>>
> >>> The only one that wasn't was 2xAPIC support. So just for kicks -- I
> >>> disabled that.
> >>>
> >>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
> >>
> >> Interesting, but unfortunately this particular log is of no real use
> >> for investigation of the issue (other than knowing the CPU model). I
> >> also notice it's a 4.12.0 log, when your original report was against
> >> latest master.
> >>
> >>> So I guess at this point, you could say that I have a functional
> >>> system, but I'm curious whether you guys would be interested to look
> >>> into 2xAPIC situation.
> >>
> >> Of course we do. As a next step I'd suggest reverting the BIOS settings
> >> change you did, and instead using the "x2apic=0" Xen command line option.
> >
> > Interestingly enough, this doesn't really solve the problem completely.
> > Specifying x2apic=0 certainly makes Xen go much further to a point
> > where it tries to load Dom0 and then the console VGA screen goes
> > blank (this is where that serial debug output would be very useful :-().
>
> Now that's again unexpected. In any event you could try "vga=keep".
>
> >> And then we of course need a complete boot log (as requested earlier) of
> >> a problem case.
> >>
> >> Further I'd suggest moving away from the black-and-white "cpuidle="
> >> option, and instead limiting use of deep C states ("max_cstate="). I
> >> wouldn't be surprised if this was the issue; we'd then have to first
> >> of all go through errata for the part your system is using.
> >
> > Yup. max_cstate=1 makes it boot fine. max_cstate=2 though hangs
> > the system *exactly* in the same way as specifying x2apic=0
> > (which is different from the original problem as I've described above).
>
> "max_cstate=2" is much less of a "deep" C state than I had expected,
> but well, so be it then. As to the hang - did you meanwhile figure
> whether _any_ number of CPUs above 1 would result in a hang, or
> whether instead there's a certain amount of them that would allow
> boot to progress fine.

Do you mean by tweaking maxcpus setting?

Anything above 1 doesn't work

maxcpus=1 boots fine tho

> > Can you please elaborate on "we'd then have to first of all go through
> > errata for the part your system is using"
>
> Well, it wouldn't be the first time that hardware had issues with C
> state handling. Therefore we'd need to (a) be sure you use up-to-date
> microcode and (b) there are no errata documented for your CPU model
> workarounds for which basically suggest to avoid use of deep C states.

It seems you've done that already -- thank you!

Thanks,
Roman.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-30 22:16           ` Roman Shaposhnik
@ 2019-09-30 22:38             ` Roman Shaposhnik
  2019-10-01  8:25               ` Jan Beulich
  2019-11-07 11:00               ` Jan Beulich
  0 siblings, 2 replies; 19+ messages in thread
From: Roman Shaposhnik @ 2019-09-30 22:38 UTC (permalink / raw)
  To: Jan Beulich; +Cc: xen-devel

[-- Attachment #1: Type: text/plain, Size: 3822 bytes --]

Btw, forgot to attach the patch with maxcpus=2 -- interestingly enough
Xen seems to hang much further down than before (basically after
attempting to build out Dom0)

On Mon, Sep 30, 2019 at 3:16 PM Roman Shaposhnik <roman@zededa.com> wrote:
>
> On Mon, Sep 30, 2019 at 2:56 AM Jan Beulich <jbeulich@suse.com> wrote:
> >
> > On 28.09.2019 05:07, Roman Shaposhnik wrote:
> > > On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
> > >>
> > >> On 26.09.2019 00:31, Roman Shaposhnik wrote:
> > >>> Jan, Roger, thank you so much for the initial ideas. I tried a few of
> > >>> those and here's where I am.
> > >>>
> > >>> First of all, it is definitely related to CPU bring up. Adding
> > >>> cpuidle=0 to xen command line made Xen boot.
> > >>>
> > >>> Then, a good friend of mine (who you may know from ancient Xen days
> > >>> ;-)) suggested that this could be related to this:
> > >>>      https://wiki.xenproject.org/wiki/Xen_power_management
> > >>> so I went to the BIOS settings and quite to my surprise all of them
> > >>> were grayed out (not tweakable).
> > >>>
> > >>> The only one that wasn't was 2xAPIC support. So just for kicks -- I
> > >>> disabled that.
> > >>>
> > >>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
> > >>
> > >> Interesting, but unfortunately this particular log is of no real use
> > >> for investigation of the issue (other than knowing the CPU model). I
> > >> also notice it's a 4.12.0 log, when your original report was against
> > >> latest master.
> > >>
> > >>> So I guess at this point, you could say that I have a functional
> > >>> system, but I'm curious whether you guys would be interested to look
> > >>> into 2xAPIC situation.
> > >>
> > >> Of course we do. As a next step I'd suggest reverting the BIOS settings
> > >> change you did, and instead using the "x2apic=0" Xen command line option.
> > >
> > > Interestingly enough, this doesn't really solve the problem completely.
> > > Specifying x2apic=0 certainly makes Xen go much further to a point
> > > where it tries to load Dom0 and then the console VGA screen goes
> > > blank (this is where that serial debug output would be very useful :-().
> >
> > Now that's again unexpected. In any event you could try "vga=keep".
> >
> > >> And then we of course need a complete boot log (as requested earlier) of
> > >> a problem case.
> > >>
> > >> Further I'd suggest moving away from the black-and-white "cpuidle="
> > >> option, and instead limiting use of deep C states ("max_cstate="). I
> > >> wouldn't be surprised if this was the issue; we'd then have to first
> > >> of all go through errata for the part your system is using.
> > >
> > > Yup. max_cstate=1 makes it boot fine. max_cstate=2 though hangs
> > > the system *exactly* in the same way as specifying x2apic=0
> > > (which is different from the original problem as I've described above).
> >
> > "max_cstate=2" is much less of a "deep" C state than I had expected,
> > but well, so be it then. As to the hang - did you meanwhile figure
> > whether _any_ number of CPUs above 1 would result in a hang, or
> > whether instead there's a certain amount of them that would allow
> > boot to progress fine.
>
> Do you mean by tweaking maxcpus setting?
>
> Anything above 1 doesn't work
>
> maxcpus=1 boots fine tho
>
> > > Can you please elaborate on "we'd then have to first of all go through
> > > errata for the part your system is using"
> >
> > Well, it wouldn't be the first time that hardware had issues with C
> > state handling. Therefore we'd need to (a) be sure you use up-to-date
> > microcode and (b) there are no errata documented for your CPU model
> > workarounds for which basically suggest to avoid use of deep C states.
>
> It seems you've done that already -- thank you!
>
> Thanks,
> Roman.

[-- Attachment #2: xen3.log --]
[-- Type: application/octet-stream, Size: 14307 bytes --]

0x0000:0x00:0x17.0x0: ROM: 0x5400 bytes at 0x73c86018
0x0000:0x01:0x00.0x1: ROM: 0x8000 bytes at 0x73c74018
 Xen 4.12.0
(XEN) Xen version 4.12.0 (@) (gcc (Alpine 6.4.0) 6.4.0) debug=n  Sat Jul 27 03:04:16 UTC 2019
(XEN) Latest ChangeSet:
(XEN) Bootloader: GRUB 2.03
(XEN) Command line: loglvl=all com1=115200,8n1 console=com1,vga vga=keep maxcpus=2 dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
(XEN) Xen image load base address: 0x71800000
(XEN) Video information:
(XEN)  VGA is graphics mode 1280x1024, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  0000000000000000 - 000000000009f000 (usable)
(XEN)  000000000009f000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000065439000 (usable)
(XEN)  0000000065439000 - 0000000065448000 (reserved)
(XEN)  0000000065448000 - 0000000065484000 (usable)
(XEN)  0000000065484000 - 0000000065488000 (reserved)
(XEN)  0000000065488000 - 000000006e2b7000 (usable)
(XEN)  000000006e2b7000 - 000000006e2c9000 (reserved)
(XEN)  000000006e2c9000 - 000000006e2ca000 (usable)
(XEN)  000000006e2ca000 - 000000006f4ec000 (reserved)
(XEN)  000000006f4ec000 - 000000006f4ed000 (ACPI NVS)
(XEN)  000000006f4ed000 - 000000006f4ff000 (reserved)
(XEN)  000000006f4ff000 - 0000000078cff000 (usable)
(XEN)  0000000078cff000 - 00000000799ff000 (reserved)
(XEN)  00000000799ff000 - 000000007bdff000 (ACPI NVS)
(XEN)  000000007bdff000 - 000000007bfff000 (ACPI data)
(XEN)  000000007bfff000 - 000000007c000000 (usable)
(XEN)  000000007c000000 - 0000000080000000 (reserved)
(XEN)  00000000fe010000 - 00000000fe011000 (reserved)
(XEN)  0000000100000000 - 000000027f000000 (usable)
(XEN) ACPI: RSDP 7BFFE014, 0024 (r2 HPE   )
(XEN) ACPI: XSDT 7BFE7188, 00FC (r1 HPE    Server          1 INTL 20160422)
(XEN) ACPI: FACP 7BFF4000, 010C (r6 HPE    Server          1 1590        1)
(XEN) ACPI: DSDT 7BFE4000, 1533 (r2 HPE    Server          2 HPE     20000)
(XEN) ACPI: FACS 7BDD0000, 0040
(XEN) ACPI: SSDT 7BFFC000, 0197 (r1 HPE     CpuSsdt        1 INTL 20130328)
(XEN) ACPI: BOOT 7BFFB000, 0028 (r1 HPE    Server          2 MSFT  1000013)
(XEN) ACPI: SSDT 7BFFA000, 003A (r1 HPE    PLATOPRG        1 INTL 20130328)
(XEN) ACPI: HEST 7BFF9000, 00A8 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: BERT 7BFF8000, 0030 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: ERST 7BFF7000, 0250 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: FPDT 7BFF6000, 0034 (r1 HPE    Server   20170001 INTL 20160422)
(XEN) ACPI: BGRT 7BFF5000, 0038 (r1 HPE    Server   20170001 1590 20160422)
(XEN) ACPI: HPET 7BFF3000, 0038 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: WDDT 7BFF2000, 0040 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: APIC 7BFF1000, 00BC (r3 HPE    Server          1 1590        1)
(XEN) ACPI: MCFG 7BFF0000, 003C (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SLIT 7BFEF000, 002D (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SRAT 7BFEE000, 0158 (r3 HPE    Server          1 1590        1)
(XEN) ACPI: SPMI 7BFED000, 0041 (r5 HPE    Server          1 1590        1)
(XEN) ACPI: RASF 7BFEC000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SPCR 7BFEB000, 0050 (r2 HPE    Server          1 1590        1)
(XEN) ACPI: MSCT 7BFEA000, 004E (r2 HPE    Server          1 1590        1)
(XEN) ACPI: BDAT 7BFE9000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: WSMT 7BFE8000, 0028 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: PCCT 7BFFD000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: DMAR 7BFE6000, 00B0 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SSDT 7BFE1000, 2AA2 (r2 HPE    PCISSDT         2 HPE     20000)
(XEN) ACPI: SSDT 7BFE0000, 01C4 (r2 HPE    TIMESSDT        2 HPE     20000)
(XEN) ACPI: SSDT 7BFDF000, 02F2 (r1 HPE        pmab        1 INTL 20130328)
(XEN) ACPI: OEM1 7BFDE000, 00DF (r2 HPE    CPU  CPC        2 HPE     20000)
(XEN) System RAM: 8042MB (8235284kB)
(XEN) SRAT: PXM 0 -> APIC 00000000 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000004 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000002 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000006 -> Node 0
(XEN) SRAT: Node 0 PXM 0 3f500000-7c000000
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-3f500000
(XEN) SRAT: Node 0 PXM 0 100000000-200000000
(XEN) SRAT: Node 0 PXM 0 200000000-27f000000
(XEN) NUMA: Allocated memnodemap from 278b58000 - 278b5b000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) vesafb: framebuffer at 0x84000000, mapped to 0xffff82c000201000, using 5120k, total 5120k
(XEN) vesafb: mode is 1280x1024x32, linelength=5120, font 8x16
(XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 158 (0x9e), Stepping 10 (raw 000906ea)
(XEN) SMBIOS 3.1 present.
(XEN) x2APIC mode is already enabled by BIOS.
(XEN) Using APIC driver x2apic_cluster
(XEN) ACPI: PM-Timer IO Port: 0x508 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[1:0], status[1:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:504,1:0], pm1x_evt[1:500,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 7bdd0000/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[7bdd000c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: X2APIC (apic_id[0x00] uid[0x00] enabled)
(XEN) ACPI: X2APIC (apic_id[0x04] uid[0x04] enabled)
(XEN) ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
(XEN) ACPI: X2APIC (apic_id[0x06] uid[0x06] enabled)
(XEN) ACPI: X2APIC_NMI (uid[0x00] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x02] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x04] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x06] high level lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
(XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) Xen ERST support is initialized.
(XEN) HEST: Table parsing has been initialized
(XEN) ACPI: BGRT: invalidating v1 image at 0x72ca8000
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 2 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware features: IBRS/IBPB STIBP L1D_FLUSH SSBD
(XEN)   Compiled-in support: SHADOW_PAGING
(XEN)   Xen settings: BTI-Thunk N/A, SPEC_CTRL: IBRS+ SSBD-, Other: IBPB L1D_FLUSH
(XEN)   L1TF: believed vulnerable, maxphysaddr L1D 46, CPUID 39, Safe address 8000000000
(XEN)   Support for HVM VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   Support for PV VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled (with PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU enabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Platform timer is 23.999MHz HPET
(XEN) Detected 3312.165 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000003fff type=2 attr=000000000000000f
(XEN)  0000000004000-000000008efff type=7 attr=000000000000000f
(XEN)  000000008f000-000000009efff type=2 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
(XEN)  0000000100000-00000008c5fff type=2 attr=000000000000000f
(XEN)  00000008c6000-000001452dfff type=7 attr=000000000000000f
(XEN)  000001452e000-000003f4fffff type=1 attr=000000000000000f
(XEN)  000003f500000-000003f57ffff type=4 attr=000000000000000f
(XEN)  000003f580000-000003fcfffff type=7 attr=000000000000000f
(XEN)  000003fd00000-000003fd00fff type=3 attr=000000000000000f
(XEN)  000003fd01000-00000454a4fff type=4 attr=000000000000000f
(XEN)  00000454a5000-0000065438fff type=7 attr=000000000000000f
(XEN)  0000065439000-0000065447fff type=6 attr=800000000000000f
(XEN)  0000065448000-0000065483fff type=3 attr=000000000000000f
(XEN)  0000065484000-0000065487fff type=6 attr=800000000000000f
(XEN)  0000065488000-00000660b5fff type=3 attr=000000000000000f
(XEN)  00000660b6000-000006e2b6fff type=4 attr=000000000000000f
(XEN)  000006e2b7000-000006e2c8fff type=6 attr=800000000000000f
(XEN)  000006e2c9000-000006e2c9fff type=7 attr=000000000000000f
(XEN)  000006e2ca000-000006f4ebfff type=6 attr=800000000000000f
(XEN)  000006f4ec000-000006f4ecfff type=10 attr=000000000000000f
(XEN)  000006f4ed000-000006f4fefff type=6 attr=800000000000000f
(XEN)  000006f4ff000-000006f8f7fff type=7 attr=000000000000000f
(XEN)  000006f8f8000-000006f8f8fff type=2 attr=000000000000000f
(XEN)  000006f8f9000-000006fa61fff type=7 attr=000000000000000f
(XEN)  000006fa62000-000006fcfefff type=1 attr=000000000000000f
(XEN)  000006fcff000-00000719fffff type=7 attr=000000000000000f
(XEN)  0000071a00000-0000071d88fff type=2 attr=000000000000000f
(XEN)  0000071d89000-0000071d95fff type=7 attr=000000000000000f
(XEN)  0000071d96000-0000072a18fff type=4 attr=000000000000000f
(XEN)  0000072a19000-0000072be2fff type=7 attr=000000000000000f
(XEN)  0000072be3000-0000073b76fff type=4 attr=000000000000000f
(XEN)  0000073b77000-0000073b7efff type=7 attr=000000000000000f
(XEN)  0000073b7f000-0000073b7ffff type=4 attr=000000000000000f
(XEN)  0000073b80000-0000073b82fff type=7 attr=000000000000000f
(XEN)  0000073b83000-0000077cfefff type=4 attr=000000000000000f
(XEN)  0000077cff000-0000077cfffff type=7 attr=000000000000000f
(XEN)  0000077d00000-0000078cfefff type=3 attr=000000000000000f
(XEN)  0000078cff000-00000794fefff type=5 attr=800000000000000f
(XEN)  00000794ff000-00000798fefff type=6 attr=800000000000000f
(XEN)  00000798ff000-00000799fefff type=0 attr=000000000000000f
(XEN)  00000799ff000-000007bdfefff type=10 attr=000000000000000f
(XEN)  000007bdff000-000007bffefff type=9 attr=000000000000000f
(XEN)  000007bfff000-000007bffffff type=4 attr=000000000000000f
(XEN)  0000100000000-000027effffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
(XEN)  000007c000000-000007fffffff type=0 attr=0000000000000000
(XEN)  00000fe010000-00000fe010fff type=11 attr=8000000000000001
(XEN) Initing memory sharing.
(XEN) alt table ffff82d08042ac70 -> ffff82d08042c97a
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC_DEADLINE disabled due to Errata; please update microcode to version 0x52 (or later)
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x11142120
(XEN) mwait-idle: v0.4.1 model 0x9e
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - VMCS shadowing
(XEN)  - VM Functions
(XEN)  - Virtualisation Exceptions
(XEN)  - Page Modification Logging
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Adding cpu 1 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Adding cpu 2 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Removing cpu 2 from runqueue 0
(XEN) Adding cpu 3 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Removing cpu 3 from runqueue 0
(XEN) Brought up 2 CPUs
(XEN) Parked 2 CPUs
(XEN) build-id: 97eb63b27af38eba068e90a79c3cf36ad6a4211b
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) Dom0 has maximum 312 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** Building a PV Dom0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2a2c000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000270000000->0000000274000000 (245760 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff82a2c000
(XEN)  Init. ramdisk: 0000000000000000->0000000000000000
(XEN)  Phys-Mach map: 0000008000000000->0000008000200000
(XEN)  Start info:    ffffffff82a2c000->ffffffff82a2c4b4
(XEN)  Xenstore ring: 0000000000000000->0000000000000000
(XEN)  Console ring:  0000000000000000->0000000000000000
(XEN)  Page tables:   ffffffff82a2d000->ffffffff82a46000
(XEN)  Boot stack:    ffffffff82a46000->ffffffff82a47000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82c00000
(XEN)  ENTRY ADDRESS: ffffffff826af180
(XEN) Dom0 has maximum 1 VCPUs
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is keeping VGA console.
(XEN) Boot video device 01:00.1
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)

[-- Attachment #3: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-30 22:38             ` Roman Shaposhnik
@ 2019-10-01  8:25               ` Jan Beulich
  2019-10-08  5:54                 ` Roman Shaposhnik
  2019-11-07 11:00               ` Jan Beulich
  1 sibling, 1 reply; 19+ messages in thread
From: Jan Beulich @ 2019-10-01  8:25 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel

On 01.10.2019 00:38, Roman Shaposhnik wrote:
> Btw, forgot to attach the patch with maxcpus=2 -- interestingly enough
> Xen seems to hang much further down than before (basically after
> attempting to build out Dom0)

All 3 logs contain

(XEN) TSC_DEADLINE disabled due to Errata; please update microcode to version 0x52 (or later)

Please load up-to-date microcode on the system and, preferably with
Andrew's suggestions also applied, re-post the logs. I notice that
even logs 1 and 2 have "Brought up 4 CPUs", other than you've
indicated in your initial report. This suggests something's broken
_after_ bringup of secondary CPUs, not while bringing them up. Log
3 effectively seems to confirm this.

Seeing that "max_cstate=1" did help, as another next step could you
try whether "mwait-idle=0" makes enough of a difference (it'll
likely make a difference initially, as it makes the system
effectively stay in a "max_cstate=1"-like mode until Dom0 has booted
up far enough; the question this is going to be whether a hang still
occurs one Dom0 has uploaded C-state data)?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-10-01  8:25               ` Jan Beulich
@ 2019-10-08  5:54                 ` Roman Shaposhnik
  2019-10-08  7:25                   ` Andrew Cooper
  0 siblings, 1 reply; 19+ messages in thread
From: Roman Shaposhnik @ 2019-10-08  5:54 UTC (permalink / raw)
  To: Jan Beulich; +Cc: xen-devel

Sorry -- was traveling last week, but I'm still very curious to get to
the bottom of this:

On Tue, Oct 1, 2019 at 1:25 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 01.10.2019 00:38, Roman Shaposhnik wrote:
> > Btw, forgot to attach the patch with maxcpus=2 -- interestingly enough
> > Xen seems to hang much further down than before (basically after
> > attempting to build out Dom0)
>
> All 3 logs contain
>
> (XEN) TSC_DEADLINE disabled due to Errata; please update microcode to version 0x52 (or later)

Ok. This makes some sense. Btw, in a situation like this, do we expect
Xen to cope with a broken microcode or we always expect microcode to
be updated?

> Please load up-to-date microcode on the system and,

I couldn't find any kind of HP guide on how to do this on this box.
Any chance someone here may have a pointer for me?

> preferably with Andrew's suggestions also applied, re-post the logs. I notice that
> even logs 1 and 2 have "Brought up 4 CPUs", other than you've
> indicated in your initial report. This suggests something's broken
> _after_ bringup of secondary CPUs, not while bringing them up. Log
> 3 effectively seems to confirm this.
>
> Seeing that "max_cstate=1" did help, as another next step could you
> try whether "mwait-idle=0" makes enough of a difference (it'll
> likely make a difference initially, as it makes the system
> effectively stay in a "max_cstate=1"-like mode until Dom0 has booted
> up far enough; the question this is going to be whether a hang still
> occurs one Dom0 has uploaded C-state data)?

Ok. Will do all these experiments tomorrow.

My plan is to use latest Xen master + build it with extra debug info
as was suggested earlier in this thread.

Thanks,
Roman.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-10-08  5:54                 ` Roman Shaposhnik
@ 2019-10-08  7:25                   ` Andrew Cooper
  0 siblings, 0 replies; 19+ messages in thread
From: Andrew Cooper @ 2019-10-08  7:25 UTC (permalink / raw)
  To: Roman Shaposhnik, Jan Beulich; +Cc: xen-devel

On 08/10/2019 06:54, Roman Shaposhnik wrote:
> Sorry -- was traveling last week, but I'm still very curious to get to
> the bottom of this:
>
> On Tue, Oct 1, 2019 at 1:25 AM Jan Beulich <jbeulich@suse.com> wrote:
>> On 01.10.2019 00:38, Roman Shaposhnik wrote:
>>> Btw, forgot to attach the patch with maxcpus=2 -- interestingly enough
>>> Xen seems to hang much further down than before (basically after
>>> attempting to build out Dom0)
>> All 3 logs contain
>>
>> (XEN) TSC_DEADLINE disabled due to Errata; please update microcode to version 0x52 (or later)
> Ok. This makes some sense. Btw, in a situation like this, do we expect
> Xen to cope with a broken microcode or we always expect microcode to
> be updated?

Intel consider you to be running out-of-spec if you don't have
up-to-date microcode.  As for functioning correctly, all bets are off.

It is quite possible the updated microcode doesn't have an impact here,
but it is not worth your time or ours trying to diagnose issues in the
face of an out-of-spec system.

>
>> Please load up-to-date microcode on the system and,
> I couldn't find any kind of HP guide on how to do this on this box.
> Any chance someone here may have a pointer for me?

Here is some documentation I prepared earlier:

https://xenbits.xen.org/docs/sphinx-unstable/admin-guide/microcode-loading.html

To a first approximation, install the intel-ucode package into dom0, and
it should DTRT.

If it doesn't,
https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files is
upstream for Intel's microcode, and you can manually arrange for Xen to
find and use it.

~Andrew

P.S. I'd also appreciate any feedback you have on the microcode
documentation, because that is the style/quality I'm aiming for across
the board in the Sphinx docs, and you are possibly the first user to use
it in anger.  In particular, if things are not clear, I'd like to see
about fixing that.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-30 22:38             ` Roman Shaposhnik
  2019-10-01  8:25               ` Jan Beulich
@ 2019-11-07 11:00               ` Jan Beulich
  1 sibling, 0 replies; 19+ messages in thread
From: Jan Beulich @ 2019-11-07 11:00 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel

On 01.10.2019 00:38, Roman Shaposhnik wrote:
> Btw, forgot to attach the patch with maxcpus=2 -- interestingly enough
> Xen seems to hang much further down than before (basically after
> attempting to build out Dom0)

Have you meanwhile had a chance to retry with latest microcode?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-28  3:07       ` Roman Shaposhnik
  2019-09-30  9:57         ` Jan Beulich
@ 2019-09-30 10:19         ` Andrew Cooper
  2019-09-30 22:01           ` Roman Shaposhnik
  1 sibling, 1 reply; 19+ messages in thread
From: Andrew Cooper @ 2019-09-30 10:19 UTC (permalink / raw)
  To: Roman Shaposhnik, Jan Beulich; +Cc: xen-devel

On 28/09/2019 04:07, Roman Shaposhnik wrote:
> On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
>> On 26.09.2019 00:31, Roman Shaposhnik wrote:
>>> Jan, Roger, thank you so much for the initial ideas. I tried a few of
>>> those and here's where I am.
>>>
>>> First of all, it is definitely related to CPU bring up. Adding
>>> cpuidle=0 to xen command line made Xen boot.
>>>
>>> Then, a good friend of mine (who you may know from ancient Xen days
>>> ;-)) suggested that this could be related to this:
>>>      https://wiki.xenproject.org/wiki/Xen_power_management
>>> so I went to the BIOS settings and quite to my surprise all of them
>>> were grayed out (not tweakable).
>>>
>>> The only one that wasn't was 2xAPIC support. So just for kicks -- I
>>> disabled that.
>>>
>>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
>> Interesting, but unfortunately this particular log is of no real use
>> for investigation of the issue (other than knowing the CPU model). I
>> also notice it's a 4.12.0 log, when your original report was against
>> latest master.
>>
>>> So I guess at this point, you could say that I have a functional
>>> system, but I'm curious whether you guys would be interested to look
>>> into 2xAPIC situation.
>> Of course we do. As a next step I'd suggest reverting the BIOS settings
>> change you did, and instead using the "x2apic=0" Xen command line option.
> Interestingly enough, this doesn't really solve the problem completely.
> Specifying x2apic=0 certainly makes Xen go much further to a point
> where it tries to load Dom0 and then the console VGA screen goes
> blank (this is where that serial debug output would be very useful :-().

Do you not even have IPMI SoL?  Its basically impossible to buy servers
without this base level of functionality.

It will be somewhere in the iLO configuration on an HP server.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-30 10:19         ` Andrew Cooper
@ 2019-09-30 22:01           ` Roman Shaposhnik
  2019-09-30 23:42             ` Andrew Cooper
  0 siblings, 1 reply; 19+ messages in thread
From: Roman Shaposhnik @ 2019-09-30 22:01 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: xen-devel, Jan Beulich

[-- Attachment #1: Type: text/plain, Size: 2183 bytes --]

On Mon, Sep 30, 2019 at 3:19 AM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>
> On 28/09/2019 04:07, Roman Shaposhnik wrote:
> > On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
> >> On 26.09.2019 00:31, Roman Shaposhnik wrote:
> >>> Jan, Roger, thank you so much for the initial ideas. I tried a few of
> >>> those and here's where I am.
> >>>
> >>> First of all, it is definitely related to CPU bring up. Adding
> >>> cpuidle=0 to xen command line made Xen boot.
> >>>
> >>> Then, a good friend of mine (who you may know from ancient Xen days
> >>> ;-)) suggested that this could be related to this:
> >>>      https://wiki.xenproject.org/wiki/Xen_power_management
> >>> so I went to the BIOS settings and quite to my surprise all of them
> >>> were grayed out (not tweakable).
> >>>
> >>> The only one that wasn't was 2xAPIC support. So just for kicks -- I
> >>> disabled that.
> >>>
> >>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
> >> Interesting, but unfortunately this particular log is of no real use
> >> for investigation of the issue (other than knowing the CPU model). I
> >> also notice it's a 4.12.0 log, when your original report was against
> >> latest master.
> >>
> >>> So I guess at this point, you could say that I have a functional
> >>> system, but I'm curious whether you guys would be interested to look
> >>> into 2xAPIC situation.
> >> Of course we do. As a next step I'd suggest reverting the BIOS settings
> >> change you did, and instead using the "x2apic=0" Xen command line option.
> > Interestingly enough, this doesn't really solve the problem completely.
> > Specifying x2apic=0 certainly makes Xen go much further to a point
> > where it tries to load Dom0 and then the console VGA screen goes
> > blank (this is where that serial debug output would be very useful :-().
>
> Do you not even have IPMI SoL?  Its basically impossible to buy servers
> without this base level of functionality.
>
> It will be somewhere in the iLO configuration on an HP server.

This is a super awesome suggestion!

Attached are two logs. Please take a look -- both result in Xen hanging.

Thanks,
Roman.

[-- Attachment #2: xen2.log --]
[-- Type: application/octet-stream, Size: 12631 bytes --]

0x0000:0x00:0x17.0x0: ROM: 0x5400 bytes at 0x73c86018
0x0000:0x01:0x00.0x1: ROM: 0x8000 bytes at 0x73c74018
 Xen 4.12.0
(XEN) Xen version 4.12.0 (@) (gcc (Alpine 6.4.0) 6.4.0) debug=n  Sat Jul 27 03:04:16 UTC 2019
(XEN) Latest ChangeSet:
(XEN) Bootloader: GRUB 2.03
(XEN) Command line: loglvl=all com1=115200,8n1 console=com1,vga vga=keep x2apic=0 dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
(XEN) Xen image load base address: 0x71800000
(XEN) Video information:
(XEN)  VGA is graphics mode 1280x1024, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  0000000000000000 - 000000000009f000 (usable)
(XEN)  000000000009f000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000065439000 (usable)
(XEN)  0000000065439000 - 0000000065448000 (reserved)
(XEN)  0000000065448000 - 0000000065484000 (usable)
(XEN)  0000000065484000 - 0000000065488000 (reserved)
(XEN)  0000000065488000 - 000000006e2b7000 (usable)
(XEN)  000000006e2b7000 - 000000006e2c9000 (reserved)
(XEN)  000000006e2c9000 - 000000006e2ca000 (usable)
(XEN)  000000006e2ca000 - 000000006f4ec000 (reserved)
(XEN)  000000006f4ec000 - 000000006f4ed000 (ACPI NVS)
(XEN)  000000006f4ed000 - 000000006f4ff000 (reserved)
(XEN)  000000006f4ff000 - 0000000078cff000 (usable)
(XEN)  0000000078cff000 - 00000000799ff000 (reserved)
(XEN)  00000000799ff000 - 000000007bdff000 (ACPI NVS)
(XEN)  000000007bdff000 - 000000007bfff000 (ACPI data)
(XEN)  000000007bfff000 - 000000007c000000 (usable)
(XEN)  000000007c000000 - 0000000080000000 (reserved)
(XEN)  00000000fe010000 - 00000000fe011000 (reserved)
(XEN)  0000000100000000 - 000000027f000000 (usable)
(XEN) ACPI: RSDP 7BFFE014, 0024 (r2 HPE   )
(XEN) ACPI: XSDT 7BFE7188, 00FC (r1 HPE    Server          1 INTL 20160422)
(XEN) ACPI: FACP 7BFF4000, 010C (r6 HPE    Server          1 1590        1)
(XEN) ACPI: DSDT 7BFE4000, 1533 (r2 HPE    Server          2 HPE     20000)
(XEN) ACPI: FACS 7BDD0000, 0040
(XEN) ACPI: SSDT 7BFFC000, 0197 (r1 HPE     CpuSsdt        1 INTL 20130328)
(XEN) ACPI: BOOT 7BFFB000, 0028 (r1 HPE    Server          2 MSFT  1000013)
(XEN) ACPI: SSDT 7BFFA000, 003A (r1 HPE    PLATOPRG        1 INTL 20130328)
(XEN) ACPI: HEST 7BFF9000, 00A8 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: BERT 7BFF8000, 0030 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: ERST 7BFF7000, 0250 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: FPDT 7BFF6000, 0034 (r1 HPE    Server   20170001 INTL 20160422)
(XEN) ACPI: BGRT 7BFF5000, 0038 (r1 HPE    Server   20170001 1590 20160422)
(XEN) ACPI: HPET 7BFF3000, 0038 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: WDDT 7BFF2000, 0040 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: APIC 7BFF1000, 00BC (r3 HPE    Server          1 1590        1)
(XEN) ACPI: MCFG 7BFF0000, 003C (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SLIT 7BFEF000, 002D (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SRAT 7BFEE000, 0158 (r3 HPE    Server          1 1590        1)
(XEN) ACPI: SPMI 7BFED000, 0041 (r5 HPE    Server          1 1590        1)
(XEN) ACPI: RASF 7BFEC000, 0030 (r1 HPE    Server          1 1590       ver          1 1590        1)
(XEN) ACPI: BDAT 7BFE9000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: WSMT 7BFE8000, 0028 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: PCCT 7BFFD000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: DMAR 7BFE6000, 00B0 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SSDT 7BFE1000, 2AA2 (r2 HPE    PCISSDT         2 HPE     20000)
(XEN) ACPI: SSDT 7BFE0000, 01C4 (r2 HPE    TIMESSDT        2 HPE     20000)
(XEN) ACPI: SSDT 7BFDF000, 02F2 (r1 HPE        pmab        1 INTL 20130328)
(XEN) ACPI: OEM1 7BFDE000, 00DF (r2 HPE    CPU  CPC        2 HPE     20000)
(XEN) System RAM: 8042MB (8235284kB)
(XEN) SRAT: PXM 0 -> APIC 00000000 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000004 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000002 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000006 -> Node 0
(XEN) SRAT: Node 0 PXM 0 3f500000-7c000000
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-3f500000
(XEN) SRAT: Node 0 PXM 0 100000000-200000000
(XEN) SRAT: Node 0 PXM 0 200000000-27f000000
(XEN) NUMA: Allocated memnodemap from 278b58000 - 278b5b000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) vesafb: framebuffer at 0x84000000, mapped to 0xffff82c000201000, using 5120k, total 5120k
(XEN) vesafb: mode is 1280x1024x32, linelength=5120, font 8x16
(XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 158 (0x9e), Stepping 10 (raw 000906ea)
(XEN) SMBIOS 3.1 present.
(XEN) x2APIC mode is already enabled by BIOS.
(XEN) Using APIC driver x2apic_cluster
(XEN) ACPI: PM-Timer IO Port: 0x508 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[1:0], status[1:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:504,1:0], pm1x_evt[1:500,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 7bdd0000/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[7bdd000c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: X2APIC (apic_id[0x00] uid[0x00] enabled)
(XEN) ACPI: X2APIC (apic_id[0x04] uid[0x04] enabled)
(XEN) ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
(XEN) ACPI: X2APIC (apic_id[0x06] uid[0x06] enabled)
(XEN) ACPI: X2APIC_NMI (uid[0x00] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x02] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x04] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x06] high level lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
(XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) Xen ERST support is initialized.
(XEN) HEST: Table parsing has been initialized
(XEN) ACPI: BGRT: invalidating v1 image at 0x72ca6000
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
(XEN) x2APIC: Already enabled by BIOS: Ignoring cmdline disable.
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware features: IBRS/IBPB STIBP L1D_FLUSH SSBD
(XEN)   Compiled-in support: SHADOW_PAGING
(XEN)   Xen settings: BTI-Thunk N/A, SPEC_CTRL: IBRS+ SSBD-, Other: IBPB L1D_FLUSH
(XEN)   L1TF: believed vulnerable, maxphysaddr L1D 46, CPUID 39, Safe address 8000000000
(XEN)   Support for HVM VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   Support for PV VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled (with PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU enabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Platform timer is 23.999MHz HPET
(XEN) Detected 3312.170 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000003fff type=2 attr=000000000000000f
(XEN)  0000000004000-000000008efff type=7 attr=000000000000000f
(XEN)  000000008f000-000000009efff type=2 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
(XEN)  0000000100000-00000008c5fff type=2 attr=000000000000000f
(XEN)  00000008c6000-000001452dfff type=7 attr=000000000000000f
(XEN)  000001452e000-000003f4fffff type=1 attr=000000000000000f
(XEN)  000003f500000-000003f57ffff type=4 attr=000000000000000f
(XEN)  000003f580000-000003fcfffff type=7 attr=000000000000000f
(XEN)  000003fd00000-000003fd00fff type=3 attr=000000000000000f
(XEN)  000003fd01000-00000454a4fff type=4 attr=000000000000000f
(XEN)  00000454a5000-0000065438fff type=7 attr=000000000000000f
(XEN)  0000065439000-0000065447fff type=6 attr=800000000000000f
(XEN)  0000065448000-0000065483fff type=3 attr=000000000000000f
(XEN)  0000065484000-0000065487fff type=6 attr=800000000000000f
(XEN)  0000065488000-00000660b5fff type=3 attr=000000000000000f
(XEN)  00000660b6000-000006e2b6fff type=4 attr=000000000000000f
(XEN)  000006e2b7000-000006e2c8fff type=6 attr=800000000000000f
(XEN)  000006e2c9000-000006e2c9fff type=7 attr=000000000000000f
(XEN)  000006e2ca000-000006f4ebfff type=6 attr=800000000000000f
(XEN)  000006f4ec000-000006f4ecfff type=10 attr=000000000000000f
(XEN)  000006f4ed000-000006f4fefff type=6 attr=800000000000000f
(XEN)  000006f4ff000-000006f8f7fff type=7 attr=000000000000000f
(XEN)  000006f8f8000-000006f8f8fff type=2 attr=000000000000000f
(XEN)  000006f8f9000-000006fa61fff type=7 attr=000000000000000f
(XEN)  000006fa62000-000006fcfefff type=1 attr=000000000000000f
(XEN)  000006fcff000-00000719fffff type=7 attr=000000000000000f
(XEN)  0000071a00000-0000071d88fff type=2 attr=000000000000000f
(XEN)  0000071d89000-0000071d95fff type=7 attr=000000000000000f
(XEN)  0000071d96000-0000072a18fff type=4 attr=000000000000000f
(XEN)  0000072a19000-0000072be1fff type=7 attr=000000000000000f
(XEN)  0000072be2000-0000073b76fff type=4 attr=000000000000000f
(XEN)  0000073b77000-0000073b7efff type=7 attr=000000000000000f
(XEN)  0000073b7f000-0000073b7ffff type=4 attr=000000000000000f
(XEN)  0000073b80000-0000073b82fff type=7 attr=000000000000000f
(XEN)  0000073b83000-0000077cfefff type=4 attr=000000000000000f
(XEN)  0000077cff000-0000077cfffff type=7 attr=000000000000000f
(XEN)  0000077d00000-0000078cfefff type=3 attr=000000000000000f
(XEN)  0000078cff000-00000794fefff type=5 attr=800000000000000f
(XEN)  00000794ff000-00000798fefff type=6 attr=800000000000000f
(XEN)  00000798ff000-00000799fefff type=0 attr=000000000000000f
(XEN)  00000799ff000-000007bdfefff type=10 attr=000000000000000f
(XEN)  000007bdff000-000007bffefff type=9 attr=000000000000000f
(XEN)  000007bfff000-000007bffffff type=4 attr=000000000000000f
(XEN)  0000100000000-000027effffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
(XEN)  000007c000000-000007fffffff type=0 attr=0000000000000000
(XEN)  00000fe010000-00000fe010fff type=11 attr=8000000000000001
(XEN) Initing memory sharing.
(XEN) alt table ffff82d08042ac70 -> ffff82d08042c97a
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC_DEADLINE disabled due to Errata; please update microcode to version 0x52 (or later)
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x11142120
(XEN) mwait-idle: v0.4.1 model 0x9e
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - VMCS shadowing
(XEN)  - VM Functions
(XEN)  - Virtualisation Exceptions
(XEN)  - Page Modification Logging
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Adding cpu 1 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Adding cpu 2 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Adding cpu 3 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Brought up 4 CPUs

[-- Attachment #3: xen1.log --]
[-- Type: application/octet-stream, Size: 12683 bytes --]

0x0000:0x00:0x17.0x0: ROM: 0x5400 bytes at 0x73c86018
0x0000:0x01:0x00.0x1: ROM: 0x8000 bytes at 0x73c74018
 Xen 4.12.0
(XEN) Xen version 4.12.0 (@) (gcc (Alpine 6.4.0) 6.4.0) debug=n  Sat Jul 27 03:04:16 UTC 2019
(XEN) Latest ChangeSet:
(XEN) Bootloader: GRUB 2.03
(XEN) Command line: loglvl=all com1=115200,8n1 console=com1,vga vga=keep dom0_mem=1024M,max:1024M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
(XEN) Xen image load base address: 0x71800000
(XEN) Video information:
(XEN)  VGA is graphics mode 1280x1024, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  0000000000000000 - 000000000009f000 (usable)
(XEN)  000000000009f000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000065439000 (usable)
(XEN)  0000000065439000 - 0000000065448000 (reserved)
(XEN)  0000000065448000 - 0000000065484000 (usable)
(XEN)  0000000065484000 - 0000000065488000 (reserved)
(XEN)  0000000065488000 - 000000006e2b7000 (usable)
(XEN)  000000006e2b7000 - 000000006e2c9000 (reserved)
(XEN)  000000006e2c9000 - 000000006e2ca000 (usable)
(XEN)  000000006e2ca000 - 000000006f4ec000 (reserved)
(XEN)  000000006f4ec000 - 000000006f4ed000 (ACPI NVS)
(XEN)  000000006f4ed000 - 000000006f4ff000 (reserved)
(XEN)  000000006f4ff000 - 0000000078cff000 (usable)
(XEN)  0000000078cff000 - 00000000799ff000 (reserved)
(XEN)  00000000799ff000 - 000000007bdff000 (ACPI NVS)
(XEN)  000000007bdff000 - 000000007bfff000 (ACPI data)
(XEN)  000000007bfff000 - 000000007c000000 (usable)
(XEN)  000000007c000000 - 0000000080000000 (reserved)
(XEN)  00000000fe010000 - 00000000fe011000 (reserved)
(XEN)  0000000100000000 - 000000027f000000 (usable)
(XEN) ACPI: RSDP 7BFFE014, 0024 (r2 HPE   )
(XEN) ACPI: XSDT 7BFE7188, 00FC (r1 HPE    Server          1 INTL 20160422)
(XEN) ACPI: FACP 7BFF4000, 010C (r6 HPE    Server          1 1590        1)
(XEN) ACPI: DSDT 7BFE4000, 1533 (r2 HPE    Server          2 HPE     20000)
(XEN) ACPI: FACS 7BDD0000, 0040
(XEN) ACPI: SSDT 7BFFC000, 0197 (r1 HPE     CpuSsdt        1 INTL 20130328)
(XEN) ACPI: BOOT 7BFFB000, 0028 (r1 HPE    Server          2 MSFT  1000013)
(XEN) ACPI: SSDT 7BFFA000, 003A (r1 HPE    PLATOPRG        1 INTL 20130328)
(XEN) ACPI: HEST 7BFF9000, 00A8 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: BERT 7BFF8000, 0030 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: ERST 7BFF7000, 0250 (r1 HPE    Server          1 INTL        1)
(XEN) ACPI: FPDT 7BFF6000, 0034 (r1 HPE    Server   20170001 INTL 20160422)
(XEN) ACPI: BGRT 7BFF5000, 0038 (r1 HPE    Server   20170001 1590 20160422)
(XEN) ACPI: HPET 7BFF3000, 0038 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: WDDT 7BFF2000, 0040 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: APIC 7BFF1000, 00BC (r3 HPE    Server          1 1590        1)
(XEN) ACPI: MCFG 7BFF0000, 003C (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SLIT 7BFEF000, 002D (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SRAT 7BFEE000, 0158 (r3 HPE    Server          1 1590        1)
(XEN) ACPI: SPMI 7BFED000, 0041 (r5 HPE    Server          1 1590        1)
(XEN) ACPI: RASF 7BFEC000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SPCR 7BFEB000, 0050 (r2 HPE    Server          1 1590        1)
(XEN) ACPI: MSCT 7BFEA000, 004E (r2 HPE    Server          1 1590        1)
(XEN) ACPI: BDAT 7BFE9000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: WSMT 7BFE8000, 0028 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: PCCT 7BFFD000, 0030 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: DMAR 7BFE6000, 00B0 (r1 HPE    Server          1 1590        1)
(XEN) ACPI: SSDT 7BFE1000, 2AA2 (r2 HPE    PCISSDT         2 HPE     20000)
(XEN) ACPI: SSDT 7BFE0000, 01C4 (r2 HPE    TIMESSDT        2 HPE     20000)
(XEN) ACPI: SSDT 7BFDF000, 02F2 (r1 HPE        pmab        1 INTL 20130328)
(XEN) ACPI: OEM1 7BFDE000, 00DF (r2 HPE    CPU  CPC        2 HPE     20000)
(XEN) System RAM: 8042MB (8235284kB)
(XEN) SRAT: PXM 0 -> APIC 00000000 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000004 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000002 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 00000006 -> Node 0
(XEN) SRAT: Node 0 PXM 0 3f500000-7c000000
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-3f500000
(XEN) SRAT: Node 0 PXM 0 100000000-200000000
(XEN) SRAT: Node 0 PXM 0 200000000-27f000000
(XEN) NUMA: Allocated memnodemap from 278b58000 - 278b5b000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) vesafb: framebuffer at 0x84000000, mapped to 0xffff82c000201000, using 5120k, total 5120k
(XEN) vesafb: mode is 1280x1024x32, linelength=5120, font 8x16
(XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 158 (0x9e), Stepping 10 (raw 000906ea)
(XEN) SMBIOS 3.1 present.
(XEN) x2APIC mode is already enabled by BIOS.
(XEN) Using APIC driver x2apic_cluster
(XEN) ACPI: PM-Timer IO Port: 0x508 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[1:0], status[1:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:504,1:0], pm1x_evt[1:500,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 7bdd0000/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[7bdd000c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: X2APIC (apic_id[0x00] uid[0x00] enabled)
(XEN) ACPI: X2APIC (apic_id[0x04] uid[0x04] enabled)
(XEN) ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
(XEN) ACPI: X2APIC (apic_id[0x06] uid[0x06] enabled)
(XEN) ACPI: X2APIC_NMI (uid[0x00] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x02] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x04] high level lint[0x1])
(XEN) ACPI: X2APIC_NMI (uid[0x06] high level lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
(XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
(XEN) Xen ERST support is initialized.
(XEN) HEST: Table parsing has been initialized
(XEN) ACPI: BGRT: invalidating v1 image at 0x72b8b000
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware features: IBRS/IBPB STIBP L1D_FLUSH SSBD
(XEN)   Compiled-in support: SHADOW_PAGING
(XEN)   Xen settings: BTI-Thunk N/A, SPEC_CTRL: IBRS+ SSBD-, Other: IBPB L1D_FLUSH
(XEN)   L1TF: believed vulnerable, maxphysaddr L1D 46, CPUID 39, Safe address 8000000000
(XEN)   Support for HVM VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   Support for PV VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled (with PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU enabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Platform timer is 23.999MHz HPET
(XEN) Detected 3312.297 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000003fff type=2 attr=000000000000000f
(XEN)  0000000004000-000000008efff type=7 attr=000000000000000f
(XEN)  000000008f000-000000009efff type=2 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
(XEN)  0000000100000-00000008c5fff type=2 attr=000000000000000f
(XEN)  00000008c6000-000001458cfff type=7 attr=000000000000000f
(XEN)  000001458d000-000003f4fffff type=1 attr=000000000000000f
(XEN)  000003f500000-000003f57ffff type=4 attr=000000000000000f
(XEN)  000003f580000-000003fcfffff type=7 attr=000000000000000f
(XEN)  000003fd00000-000003fd00fff type=3 attr=000000000000000f
(XEN)  000003fd01000-00000454a4fff type=4 attr=000000000000000f
(XEN)  00000454a5000-0000065438fff type=7 attr=000000000000000f
(XEN)  0000065439000-0000065447fff type=6 attr=800000000000000f
(XEN)  0000065448000-0000065483fff type=3 attr=000000000000000f
(XEN)  0000065484000-0000065487fff type=6 attr=800000000000000f
(XEN)  0000065488000-00000660b5fff type=3 attr=000000000000000f
(XEN)  00000660b6000-000006e2b6fff type=4 attr=000000000000000f
(XEN)  000006e2b7000-000006e2c8fff type=6 attr=800000000000000f
(XEN)  000006e2c9000-000006e2c9fff type=7 attr=000000000000000f
(XEN)  000006e2ca000-000006f4ebfff type=6 attr=800000000000000f
(XEN)  000006f4ec000-000006f4ecfff type=10 attr=000000000000000f
(XEN)  000006f4ed000-000006f4fefff type=6 attr=800000000000000f
(XEN)  000006f4ff000-000006f8f7fff type=7 attr=000000000000000f
(XEN)  000006f8f8000-000006f8f8fff type=2 attr=000000000000000f
(XEN)  000006f8f9000-000006fa61fff type=7 attr=000000000000000f
(XEN)  000006fa62000-000006fcfefff type=1 attr=000000000000000f
(XEN)  000006fcff000-00000719fffff type=7 attr=000000000000000f
(XEN)  0000071a00000-0000071d88fff type=2 attr=000000000000000f
(XEN)  0000071d89000-0000071d9bfff type=7 attr=000000000000000f
(XEN)  0000071d9c000-0000072a1efff type=4 attr=000000000000000f
(XEN)  0000072a1f000-0000072b0afff type=7 attr=000000000000000f
(XEN)  0000072b0b000-0000073b76fff type=4 attr=000000000000000f
(XEN)  0000073b77000-0000073b7efff type=7 attr=000000000000000f
(XEN)  0000073b7f000-0000073b7ffff type=4 attr=000000000000000f
(XEN)  0000073b80000-0000073b82fff type=7 attr=000000000000000f
(XEN)  0000073b83000-0000077cfefff type=4 attr=000000000000000f
(XEN)  0000077cff000-0000077cfffff type=7 attr=000000000000000f
(XEN)  0000077d00000-0000078cfefff type=3 attr=000000000000000f
(XEN)  0000078cff000-00000794fefff type=5 attr=800000000000000f
(XEN)  00000794ff000-00000798fefff type=6 attr=800000000000000f
(XEN)  00000798ff000-00000799fefff type=0 attr=000000000000000f
(XEN)  00000799ff000-000007bdfefff type=10 attr=000000000000000f
(XEN)  000007bdff000-000007bffefff type=9 attr=000000000000000f
(XEN)  000007bfff000-000007bffffff type=4 attr=000000000000000f
(XEN)  0000100000000-000027effffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
(XEN)  000007c000000-000007fffffff type=0 attr=0000000000000000
(XEN)  00000fe010000-00000fe010fff type=11 attr=8000000000000001
(XEN) Initing memory sharing.
(XEN) alt table ffff82d08042ac70 -> ffff82d08042c97a
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC_DEADLINE disabled due to Errata; please update microcode to version 0x52 (or later)
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x11142120
(XEN) mwait-idle: v0.4.1 model 0x9e
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - VMCS shadowing
(XEN)  - VM Functions
(XEN)  - Virtualisation Exceptions
(XEN)  - Page Modification Logging
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Adding cpu 1 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Adding cpu 2 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Adding cpu 3 to runqueue 0
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Brought up 4 CPUs

[-- Attachment #4: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10
  2019-09-30 22:01           ` Roman Shaposhnik
@ 2019-09-30 23:42             ` Andrew Cooper
  0 siblings, 0 replies; 19+ messages in thread
From: Andrew Cooper @ 2019-09-30 23:42 UTC (permalink / raw)
  To: Roman Shaposhnik; +Cc: xen-devel, Jan Beulich

On 30/09/2019 23:01, Roman Shaposhnik wrote:
> On Mon, Sep 30, 2019 at 3:19 AM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 28/09/2019 04:07, Roman Shaposhnik wrote:
>>> On Thu, Sep 26, 2019 at 12:44 AM Jan Beulich <jbeulich@suse.com> wrote:
>>>> On 26.09.2019 00:31, Roman Shaposhnik wrote:
>>>>> Jan, Roger, thank you so much for the initial ideas. I tried a few of
>>>>> those and here's where I am.
>>>>>
>>>>> First of all, it is definitely related to CPU bring up. Adding
>>>>> cpuidle=0 to xen command line made Xen boot.
>>>>>
>>>>> Then, a good friend of mine (who you may know from ancient Xen days
>>>>> ;-)) suggested that this could be related to this:
>>>>>      https://wiki.xenproject.org/wiki/Xen_power_management
>>>>> so I went to the BIOS settings and quite to my surprise all of them
>>>>> were grayed out (not tweakable).
>>>>>
>>>>> The only one that wasn't was 2xAPIC support. So just for kicks -- I
>>>>> disabled that.
>>>>>
>>>>> That, in turn, made Xen boot even without cpuidle=0. I'm attaching that log.
>>>> Interesting, but unfortunately this particular log is of no real use
>>>> for investigation of the issue (other than knowing the CPU model). I
>>>> also notice it's a 4.12.0 log, when your original report was against
>>>> latest master.
>>>>
>>>>> So I guess at this point, you could say that I have a functional
>>>>> system, but I'm curious whether you guys would be interested to look
>>>>> into 2xAPIC situation.
>>>> Of course we do. As a next step I'd suggest reverting the BIOS settings
>>>> change you did, and instead using the "x2apic=0" Xen command line option.
>>> Interestingly enough, this doesn't really solve the problem completely.
>>> Specifying x2apic=0 certainly makes Xen go much further to a point
>>> where it tries to load Dom0 and then the console VGA screen goes
>>> blank (this is where that serial debug output would be very useful :-().
>> Do you not even have IPMI SoL?  Its basically impossible to buy servers
>> without this base level of functionality.
>>
>> It will be somewhere in the iLO configuration on an HP server.
> This is a super awesome suggestion!
>
> Attached are two logs. Please take a look -- both result in Xen hanging.

That's good to hear.  Hopefully we can make some progress.

To begin with, try dropping "dom0_vcpus_pin smt=false" from the command
line.  Both may hit some corner cases.

Because of "(XEN) x2APIC mode is already enabled by BIOS.", the x2apic=0
part is useless and will be ignored.  It can be dropped too.


Can you rebase to using Xen 4.12.1?  It has a substantial number of
bugfixes over 4.12.0, and should strictly be used in preference.

Alternatively, staging-4.12 has a few fixes more than 4.12.1, notably
"x86/apic: enable x2APIC mode before doing any setup" which fixes an
issue with x2APIC initialisation.

If possible, can you use a debug build of Xen?

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2019-11-07 11:00 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-24 18:20 [Xen-devel] Latest development (master) Xen fails to boot on HP ProLiant DL20 GEN10 Roman Shaposhnik
2019-09-25  6:59 ` Roger Pau Monné
2019-09-25  9:01 ` Jan Beulich
2019-09-25 22:31   ` Roman Shaposhnik
2019-09-26  7:44     ` Jan Beulich
2019-09-27  2:28       ` Roman Shaposhnik
2019-09-27  7:58         ` Jan Beulich
2019-09-28  3:07       ` Roman Shaposhnik
2019-09-30  9:57         ` Jan Beulich
2019-09-30 10:14           ` Jan Beulich
2019-09-30 22:16           ` Roman Shaposhnik
2019-09-30 22:38             ` Roman Shaposhnik
2019-10-01  8:25               ` Jan Beulich
2019-10-08  5:54                 ` Roman Shaposhnik
2019-10-08  7:25                   ` Andrew Cooper
2019-11-07 11:00               ` Jan Beulich
2019-09-30 10:19         ` Andrew Cooper
2019-09-30 22:01           ` Roman Shaposhnik
2019-09-30 23:42             ` Andrew Cooper

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.