[Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Aaron Janse]

Hello all,

After attempting to install QubesOS on a new laptop, I've stumbled upon
a group of people with an assortment of laptops but all the same
problem: a Xen panic stating "IO-APIC + timer doesn't work!"

Many of us are on different stages of debugging this, so I'll cite all
our efforts here.

# Affected Xen versions
- 4.0.5-14.fc25
- 4.13.0
- probably other versions

# Affected hardware

- Dell XPS 7390 13" (i7-10710U) [1] [2] [3] [4]
- Dell XPS 7390 2-in-1 [1]
- Surface Laptop 3 Business Edition (i7-1065G7) [5] [6]
- Lenovo ThinkBook 13s (i7-8565U) [7]
- Mini-PcBarebone [8] [9]

[1]: https://www.reddit.com/r/Qubes/comments/edqrab/qubes_and_ice_lake/

[2]: https://www.reddit.com/r/Qubes/comments/dfv6jx/panic_on_cpu_0_ioapic_timer_doesnt_work_on_ice/
[3]: https://groups.google.com/forum/#!topic/qubes-users/W8mX-07xNZU
[4]: https://lists.xenproject.org/archives/html/xen-users/2019-12/msg00031.html

[5]: https://lists.xenproject.org/archives/html/xen-users/2019-12/msg00017.html
[6]: https://groups.google.com/forum/#!topic/qubes-users/4iswU7cfJHY

[7]: https://www.reddit.com/r/Qubes/comments/clk0eu/help_install_qubes_4_on_lenovo_thinkbook_13s/

[8]: https://groups.google.com/forum/#!topic/qubes-users/PIyz7BEV1mg
[9]: https://archive.is/RuiAD

# Excerpts from boot logs

Qubes on my XPS 7390 13"

    (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
    (XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
    (XEN) ...trying to set up timer (IRQ0) through 8259A ,,,
    (XEN) ...trying to set up timer as Virtual Wire IRQ... failed.
    (XEN) ...trying to set up timer as ExtINT IRQ...spurious 8259A interrupt: IRQ7.
    (XEN) CPU0: no irq handler for vector e7 (IRQ -8)
    (XEN) IRQ7 a=0001[0001,0000] v=60[ffffffff] t=IO-APIC-edge s=00000002
    (XEN)  failed :(.
    (XEN)
    (XEN) ***************************************
    (XEN) Panic on CPU 0:
    (XEN) IO-APIC + timer doesn't work! Boot with apic_verbosity=debug and send a report.  Then try booting with the 'noapic' option
    (XEN) ***************************************

Ubuntu (w/out Xen) on my XPS 7390 13"
(full logs: https://pastebin.com/SdRg87F8 https://pastebin.com/E3zCfb35)

    [    0.000000] microcode: microcode updated early to revision 0xca, date = 2019-10-03
    [    0.000000] Linux version 5.3.0-24-generic (buildd@lgw01-amd64-035) (gcc version 9.2.1 20191008 (Ubuntu 9.2.1-9ubuntu2)) #26-Ubuntu SMP Thu Nov 14 01:33:18 UTC 2019 (Ubuntu 5.3.0-24.26-generic 5.3.10)
    [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.3.0-24-generic root=UUID=062d0c69-31dc-4c7f-9915-731505fea81b ro quiet splash apic=debug vt.handoff=7
    [    ....    ]
    [    0.188468] x2apic enabled
    [    0.188492] Switched APIC routing to cluster x2apic.
    [    0.188493] masked ExtINT on CPU#0
    [    0.195266] ENABLING IO-APIC IRQs
    [    0.195267] init IO_APIC IRQs
    [    ....    ]
    [    0.195633] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
    [    0.213854] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x170fff30cc4, max_idle_ns: 440795237869 ns
    [    0.213857] Calibrating delay loop (skipped), value calculated using timer frequency.. 3199.92 BogoMIPS (lpj=6399840)
    [    0.213859] pid_max: default: 32768 minimum: 301
    [    0.215117] LSM: Security Framework initializing
    [    0.215124] Yama: becoming mindful.
    [    0.215172] AppArmor: AppArmor initialized

I'd like to note that Ubuntu, unlike Qubes, doesn't need to try
any `MP-BIOS bug` fallbacks.

# Relevant source code

Xen: https://github.com/xen-project/xen/blob/0cd791c499bdc698d14a24050ec56d60b45732e0/xen/arch/x86/io_apic.c#L1923-L1933
Linux: https://github.com/torvalds/linux/blob/fd6988496e79a6a4bdb514a4655d2920209eb85d/arch/x86/kernel/apic/io_apic.c#L2185-L2211

# Things that have been tried

Disabling APIC entirely (`noapic x2apic=off`)
- This is avoiding the problem, not fixing it
- QubeOS requires APIC anyway, so this is not an option for many of us

Switching the timer to HPET (via the `clocksource` flag)
- This didn't fix the panic (I've tried `acpi` and `pit`)
- On my XPS 13, this doesn't change any of the timer error output
- Ubuntu works on my laptop using HPET
- http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html#clocksource-x86

Updating to 5.4 Linux kernel [77]
- This didn't fix the panic
- https://www.reddit.com/r/Qubes/comments/edqrab/qubes_and_ice_lake/fcak799/

Reproducing the problem on Ubuntu
- I'm able to reproduce Xen crashing, but I'm unable to enable verbose
  logging
- https://lists.xenproject.org/archives/html/xen-users/2019-12/msg00031.html

# More verbose boot logs

I had to type these up by hand so please excuse the lack of detail.

    (XEN) [...]
    (XEN) CPU0: No irq handler for vector 40 (IRC -2147483648, LAPIC)
    (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
    (XEN) PCI: Not using MCFG for segment 0000 bus 00-ff
    (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
    (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
    (XEN) Intel VT-d Snoop Control not enabled.
    (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
    (XEN) Intel VT-d Queued Invalidation enabled.
    (XEN) Intel VT-d Posted Interrupt not enabled.
    (XEN) Intel VT-d Shared EPT tables enabled.
    (XEN) I/O virtualization enabled.
    (XEN)  - Dom0 mode: Relaxed

    (XEN) Interrupt remapping enabled
    (XEN) nr_sockets: 2
    (XEN) Getting VERSION: 1060015
    (XEN) Getting VERSION: 1060015

    (XEN) Enabled directed EOI with ioapic_ack_old on!
    (XEN) Getting ID: 0
    (XEN) Getting LVT0: 700
    (XEN) Getting LVT1: 400
    (XEN) Suppress EOI broadcast on CPU#0
    (XEN) enabled ExtINT on CPU#0

    (XEN) ENABLING IO_APIC IRQs
    (XEN)  -> Using old ACK method
    (XEN) init IO_APIC IRQs
    (XEN)  IO-APIC (apicid-pin) 2-0, 2-16, 2-17, 2-18, 2-19, 2-20, 2-21, 2-22, 2-23, 2-24, 2-25, [...], 2-115, 2-116, 2-117, 2-118, 2-119 not connected.

    (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
    (XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
    (XEN) ...trying to set up timer (IRQ0) through 8259A ,,,
    (XEN) ...trying to set up timer as Virtual Wire IRQ... failed.
    (XEN) ...trying to set up timer as ExtINT IRQ...spurious 8259A interrupt: IRQ7.
    (XEN) CPU0: no irq handler for vector e7 (IRQ -8)
    (XEN) IRQ7 a=0001[0001,0000] v=60[ffffffff] t=IO-APIC-edge s=00000002
    (XEN)  failed :(.
    (XEN)
    (XEN) ***************************************
    (XEN) Panic on CPU 0:
    (XEN) IO-APIC + timer doesn't work! Boot with apic_verbosity=debug and send a report.  Then try booting with the 'noapic' option
    (XEN) ***************************************
    (XEN)
    (XEN) Reboot in five seconds...


Cheers,
Aaron Janse

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Andrew Cooper]

On 31/12/2019 07:52, Aaron Janse wrote:
> Hello all,
>
> After attempting to install QubesOS on a new laptop, I've stumbled upon
> a group of people with an assortment of laptops but all the same
> problem: a Xen panic stating "IO-APIC + timer doesn't work!"
>
> Many of us are on different stages of debugging this, so I'll cite all
> our efforts here.
>

<snip>

> # Excerpts from boot logs
>
> Qubes on my XPS 7390 13"
>
>     (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
>     (XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
>     (XEN) ...trying to set up timer (IRQ0) through 8259A ,,,
>     (XEN) ...trying to set up timer as Virtual Wire IRQ... failed.
>     (XEN) ...trying to set up timer as ExtINT IRQ...spurious 8259A interrupt: IRQ7.
>     (XEN) CPU0: no irq handler for vector e7 (IRQ -8)
>     (XEN) IRQ7 a=0001[0001,0000] v=60[ffffffff] t=IO-APIC-edge s=00000002
>     (XEN)  failed :(.
>     (XEN)
>     (XEN) ***************************************
>     (XEN) Panic on CPU 0:
>     (XEN) IO-APIC + timer doesn't work! Boot with apic_verbosity=debug and send a report.  Then try booting with the 'noapic' option
>     (XEN) ***************************************

Is there any full boot log in the bad case?  Debugging via divination
isn't an effective way to get things done.

(Irrespective, I'm pretty sure this is a Grub2+EFI issue failing to pass
the ACPI tables to Xen, and this eventual panic is just cascade fallout.)

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Aaron Janse]

[-- Attachment #1: Type: text/plain, Size: 796 bytes --]

On Tue, Dec 31, 2019, at 12:27 AM, Andrew Cooper wrote:
> Is there any full boot log in the bad case?  Debugging via divination
> isn't an effective way to get things done.

Agreed. I included some more verbose logs towards the end of the email (typed up by hand).

Attached are pictures from a slow-motion video of my laptop booting. Note that I also included a picture of a stack trace that happens immediately before reboot. It doesn't look related, but I wanted to include it anyway.

I think the original email should have said "4.8.5" instead of "4.0.5." Regardless, everyone on this mailing list can now see all the boot logs that I've seen.

Attaching a serial console seems like it would be difficult to do on this laptop, otherwise I would have sent the logs as a txt file.

[-- Attachment #2: xen-20191231-1.jpeg --]
[-- Type: image/jpeg, Size: 335124 bytes --]

[-- Attachment #3: xen-20191231-2.jpeg --]
[-- Type: image/jpeg, Size: 364310 bytes --]

[-- Attachment #4: xen-20191231-3.jpeg --]
[-- Type: image/jpeg, Size: 261863 bytes --]

[-- Attachment #5: xen-20191231-4.jpeg --]
[-- Type: image/jpeg, Size: 400972 bytes --]

[-- Attachment #6: xen-20191231-5.jpeg --]
[-- Type: image/jpeg, Size: 362038 bytes --]

[-- Attachment #7: xen-20191231-6.jpeg --]
[-- Type: image/jpeg, Size: 463792 bytes --]

[-- Attachment #8: xen-stack-trace-4.13.0.jpeg --]
[-- Type: image/jpeg, Size: 297658 bytes --]

[-- Attachment #9: xen-stack-trace-4.8.5.jpeg --]
[-- Type: image/jpeg, Size: 360996 bytes --]

[-- Attachment #10: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

[-- Attachment #1: Type: text/plain, Size: 10565 bytes --]

enabling vecOn Tue, Dec 31, 2019 at 5:43 AM Aaron Janse <aaron@ajanse.me> wrote:
>
> On Tue, Dec 31, 2019, at 12:27 AM, Andrew Cooper wrote:
> > Is there any full boot log in the bad case?  Debugging via divination
> > isn't an effective way to get things done.
>
> Agreed. I included some more verbose logs towards the end of the email (typed up by hand).
>
> Attached are pictures from a slow-motion video of my laptop booting. Note that I also included a picture of a stack trace that happens immediately before reboot. It doesn't look related, but I wanted to include it anyway.
>
> I think the original email should have said "4.8.5" instead of "4.0.5." Regardless, everyone on this mailing list can now see all the boot logs that I've seen.
>
> Attaching a serial console seems like it would be difficult to do on this laptop, otherwise I would have sent the logs as a txt file.

I'm seeing Xen panic: "IO-APIC + timer doesn't work" on a Dell
Latitude 7200 2-in-1.  Fedora 31 Live USB image boots successfully.
No way to get serial output.  I manually recreated the output before
from the vga display.

Comparing the Linux and Xen, Xen does:
(XEN) I/O Virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Getting VERSION: 1060015
(XEN) Getting VERSION: 1060015
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) Getting ID: 0
(XEN) Getting LVT0: 700
(XEN) Getting LVT1: 400
(XEN) Suppress EOI broadcast on CPU#0
(XEN) enabled ExtINT on CPU#0
(XEN) ESR value before enabling vector: 0x40 after: 0
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) init IO_APIC IRQs
(XEN)  IO-APIC (apicid-pin) 2-0, 2-16, 2-17, ...<snip>... 2-119 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
(XEN) ...trying to set up timer (IRQ0) through 8259A ... failed
(XEN) ...trying to set up timer as Virtual Wire IRQ... failed.
(XEN) ...trying to set up timer as ExtINT IRQ...spurious 8259A interrupt: IRQ7.
(XEN) CPU0: no irq handler for vector e7 (IRQ -8)
(XEN) IRQ7 a=0001[0001,0000] v=60[ffffffff] t=IO-APIC-edge s=00000002
(XEN)  failed :(.

while linux apic=debug does:
kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
kernel: clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles:
0x1e44fb6c2ab, max_idle_ns: 440795206594 ns
kernel: Calibrating delay loop (skipped), value calculated using timer
frequency.. 4199.88 BogoMIPS (lpj=2099944)
.. and continues onward

Since linux doesn't print "...trying to set up timer (IRQ0) through
the 8259A ..." that seems to indicate Linux is seeing the timer
interrupt properly.
https://elixir.bootlin.com/linux/v5.3.7/source/arch/x86/kernel/apic/io_apic.c#L2198

I tested Linux with intel_iommu=on and that booted successfully.
Under Xen, this system sets iommu_x2apic_enabled = true, so
force_iommu is set and iommu=0 cannot disable the iommu.
fails.  Oh, I can disable x2apic and then disable iommu

x2apic=1 -> failure above
x2apic=0 iommu=0 -> failure above
clocksource=acpi -> doesn't help
clocksource=pit -> hangs after "load tracking window length 1073741824 ns"
noapic -> BUG in init_bsp_APIC

One other thing that might be noteworthy.  Linux only prints ACPI IRQ0
and IRQ9 used by override where Xen lists IRQ 0, 2 & 9.

Below is the re-constructed Xen console output.  The SMBIOS line is
the first thing displayed on the VGA output.  I skipped the full EFI
memory map dump since it is quite long.

I've also attached the Linux dmesg output.  Any pointers or
suggestions are most welcome.

Thanks,
Jason

(XEN) SMBIOS 3.2 present.
(XEN) APIC boot stats is `xapic`
(XEN) Using APIC driver default
(XEN) XSM Framework v1.0.0 initialized
(XEN) Flask: 128 avtab hash slots, 283 rules.
(XEN) Flask: 128 avtab hash slots, 283 rules.
(XEN) Flask:  4 users, 3 roles, 38 types, 2 bools
(XEN) Flask:  13 classes, 283 rules
(XEN) Flask:  Starting in enforcing mode.
(XEN) ACPI: PM-Timer IO Port: 0x1808 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[1:1804], status[1:1800]
(XEN) ACPI: Invalid sleep control/status register data: 0:0x8:0x3 0:0x8:0x3
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1804,1:0], pm1x_evt[1:1800,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
38c80c00/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[38c80c0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x08] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
(XEN) PCI: MCFG area at e0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-ff
(XEN) [VT-D]dmar.c:563:  Non-existent device (0000:00:16.7) is
reported in RMRR (386fa000, 38779fff)'s scope!
(XEN) [VT-D]dmar.c:579:   Ignore the RMRR (386fa000, 38779fff) due to
device under its scope are not PCI discoverable
(XEN) ACPI: BGRT: invalidating v1 image at 0x3329c018
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) mapped APIC to ffff82cfffffb000 (fee00000)
(XEN) mapped IOAPIC to ffff82cfffffa000 (fec00000)
(XEN) IRQ limits: 120 GSI, 1544 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) mce_intel.c:773: MCA Capability: firstbank 0, extended MCE MSR
0, BCAST, CMCI
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware features: IBRS/IBPB STIBP L1D_FLUSH SSBD MD_CLEAR
IBRS_ALL RDCL_NO SKIP_L1DFL MDS_NO
(XEN)   Compiled-in support: INDIRECT_THUNK SHADOW_PAGING
(XEN)   Xen settings: BTI-Thunk JMP, SPEC_CTRL: IBRS+ SSBD-, Other:
IBPB BRANCH_HARDEN
(XEN)   L1TF: believed vulnerable, maxphysaddr L1D 46, CPUID 39, Safe
address 8000000000
(XEN)   Support for HVM VMs: MSR_SPEC_CTRL RSB EAGER_FPU MD_CLEAR
(XEN)   Support for PV VMs: MSR_SPEC_CTRL RSB EAGER_FPU MD_CLEAR
(XEN)   XPTI (64-bit PV only): Dom0 disabled, DomU enabled (with PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU disabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Platform timer is 23.999MHz HPET
(XEN) Detected 2111.997 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000003fff type=2 attr=000000000000000f
(XEN)  0000000004000-000000008dfff type=7 attr=000000000000000f
(XEN)  000000008e000-000000009dfff type=2 attr=000000000000000f
(XEN)  000000009e000-000000009efff type=0 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=3 attr=000000000000000f
<snip>
(XEN)  0000037d0c000-000038779ffff type=0 attr=000000000000000f
(XEN)  000003877a000-0000387f6ffff type=9 attr=000000000000000f
(XEN)  00000387f7000-000038c81ffff type=10 attr=000000000000000f
<snip>
(XEN)  00000489f4000-00000489fffff type=7 attr=000000000000000f
(XEN)  0000100000000-00004ac7fffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
(XEN)  0000048a00000-000004f7fffff type=0 attr=0000000000000000
(XEN)  00000e0000000-00000efffffff type=11 attr=800000000000100d
(XEN)  00000fe000000-00000fe010fff type=11 attr=8000000000000001
(XEN)  00000fec00000-00000fec00fff type=11 attr=8000000000000001
(XEN)  00000fed20000-00000fed7ffff type=0 attr=0000000000000000
(XEN)  00000fee00000-00000fee00fff type=11 attr=8000000000000001
(XEN)  00000ff000000-00000ffffffff type=11 attr=800000000000100d
(XEN) alt table ffff82d080483030 -> ffff82d0804910d8
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Getting VERSION: 1060015
(XEN) Getting VERSION: 1060015
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) Getting ID: 0
(XEN) Getting LVT0: 700
(XEN) Getting LVT1: 400
(XEN) Suppress EOI broadcast on CPU#0
(XEN) enabled ExtINT on CPU#0
(XEN) ESR value before enabling vector: 0x40  after: 0
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) init IO_APIC IRQs
(XEN)  IO-APIC (apicid-pin) 2-0, 2-16, 2-17, ...<snip>... 2-119 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
(XEN) ...trying to set up timer (IRQ0) through 8259A ... failed.
(XEN) ...trying to set up timer as Virtual Wire IRQ... failed.
(XEN) ...trying to set up timer as ExtINT IRQ...spurious 8259A interrupt: IRQ7.
(XEN) CPU0: no irq handler for vector 27 (IRQ -8)
(XEN) IRQ7 a=ffffffffffffffff[0001,0000] v=68[ffffffff] t=IO-APIC-edge
s=00000002
(XEN)  failed :(.

[-- Attachment #2: dell-7200-dmesg.txt --]
[-- Type: text/plain, Size: 84229 bytes --]

[    0.000000] microcode: microcode updated early to revision 0xca, date = 2019-10-03
[    0.000000] Linux version 5.4.18-200.fc31.x86_64 (mockbuild@bkernel03.phx2.fedoraproject.org) (gcc version 9.2.1 20190827 (Red Hat 9.2.1-1) (GCC)) #1 SMP Fri Feb 7 14:50:05 UTC 2020
[    0.000000] Command line: BOOT_IMAGE=(hd0,gpt2)/vmlinuz-5.4.18-200.fc31.x86_64 root=/dev/mapper/fedora_localhost--live-root ro resume=/dev/mapper/fedora_localhost--live-swap rd.lvm.lv=fedora_localhost-live/root rd.lvm.lv=fedora_localhost-live/swap rhgb quiet apic=verbose
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64
[    0.000000] x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64
[    0.000000] x86/fpu: Enabled xstate features 0x1f, context size is 960 bytes, using 'compacted' format.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009efff] reserved
[    0.000000] BIOS-e820: [mem 0x000000000009f000-0x000000000009ffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000037d0bfff] usable
[    0.000000] BIOS-e820: [mem 0x0000000037d0c000-0x0000000038779fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000003877a000-0x00000000387f6fff] ACPI data
[    0.000000] BIOS-e820: [mem 0x00000000387f7000-0x0000000038c81fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000038c82000-0x0000000039d0bfff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000039d0c000-0x000000003c543fff] usable
[    0.000000] BIOS-e820: [mem 0x000000003c544000-0x000000003c544fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000003c545000-0x000000003c545fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000003c546000-0x000000003fffffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000040000000-0x00000000403fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000040400000-0x00000000489fffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000048a00000-0x000000004f7fffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fe000000-0x00000000fe010fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed20000-0x00000000fed7ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000004ac7fffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] e820: update [mem 0x2f87a018-0x2f88a057] usable ==> usable
[    0.000000] e820: update [mem 0x2f87a018-0x2f88a057] usable ==> usable
[    0.000000] extended physical RAM map:
[    0.000000] reserve setup_data: [mem 0x0000000000000000-0x000000000009dfff] usable
[    0.000000] reserve setup_data: [mem 0x000000000009e000-0x000000000009efff] reserved
[    0.000000] reserve setup_data: [mem 0x000000000009f000-0x000000000009ffff] usable
[    0.000000] reserve setup_data: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000000100000-0x000000002f87a017] usable
[    0.000000] reserve setup_data: [mem 0x000000002f87a018-0x000000002f88a057] usable
[    0.000000] reserve setup_data: [mem 0x000000002f88a058-0x0000000037d0bfff] usable
[    0.000000] reserve setup_data: [mem 0x0000000037d0c000-0x0000000038779fff] reserved
[    0.000000] reserve setup_data: [mem 0x000000003877a000-0x00000000387f6fff] ACPI data
[    0.000000] reserve setup_data: [mem 0x00000000387f7000-0x0000000038c81fff] ACPI NVS
[    0.000000] reserve setup_data: [mem 0x0000000038c82000-0x0000000039d0bfff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000039d0c000-0x000000003c543fff] usable
[    0.000000] reserve setup_data: [mem 0x000000003c544000-0x000000003c544fff] ACPI NVS
[    0.000000] reserve setup_data: [mem 0x000000003c545000-0x000000003c545fff] reserved
[    0.000000] reserve setup_data: [mem 0x000000003c546000-0x000000003fffffff] usable
[    0.000000] reserve setup_data: [mem 0x0000000040000000-0x00000000403fffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000040400000-0x00000000489fffff] usable
[    0.000000] reserve setup_data: [mem 0x0000000048a00000-0x000000004f7fffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000e0000000-0x00000000efffffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fe000000-0x00000000fe010fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fed20000-0x00000000fed7ffff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] reserve setup_data: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] reserve setup_data: [mem 0x0000000100000000-0x00000004ac7fffff] usable
[    0.000000] efi: EFI v2.70 by American Megatrends
[    0.000000] efi:  ACPI 2.0=0x38793000  ACPI=0x38793000  SMBIOS=0xf0000  SMBIOS 3.0=0xf0020  TPMFinalLog=0x38be0000  ESRT=0x39747018  MEMATTR=0x358db018  TPMEventLog=0x2f88b018 
[    0.000000] secureboot: Secure boot disabled
[    0.000000] SMBIOS 3.2.1 present.
[    0.000000] DMI: Dell Inc. Latitude 7200 2-in-1/0W2CYR, BIOS 1.3.4 08/12/2019
[    0.000000] tsc: Detected 2100.000 MHz processor
[    0.000450] tsc: Detected 2099.944 MHz TSC
[    0.000450] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000451] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000455] last_pfn = 0x4ac800 max_arch_pfn = 0x400000000
[    0.000458] MTRR default type: write-back
[    0.000458] MTRR fixed ranges enabled:
[    0.000459]   00000-9FFFF write-back
[    0.000459]   A0000-BFFFF uncachable
[    0.000460]   C0000-FFFFF write-protect
[    0.000460] MTRR variable ranges enabled:
[    0.000461]   0 base 0080000000 mask 7F80000000 uncachable
[    0.000462]   1 base 0060000000 mask 7FE0000000 uncachable
[    0.000462]   2 base 0050000000 mask 7FF0000000 write-combining
[    0.000463]   3 base 004C000000 mask 7FFC000000 uncachable
[    0.000463]   4 base 004B000000 mask 7FFF000000 uncachable
[    0.000463]   5 disabled
[    0.000463]   6 disabled
[    0.000464]   7 disabled
[    0.000464]   8 disabled
[    0.000464]   9 disabled
[    0.000932] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
[    0.001103] last_pfn = 0x48a00 max_arch_pfn = 0x400000000
[    0.001104] Scan for SMP in [mem 0x00000000-0x000003ff]
[    0.001117] Scan for SMP in [mem 0x0009fc00-0x0009ffff]
[    0.001128] Scan for SMP in [mem 0x000f0000-0x000fffff]
[    0.006340] esrt: Reserving ESRT space from 0x0000000039747018 to 0x0000000039747050.
[    0.006348] Using GB pages for direct mapping
[    0.006349] BRK [0x477601000, 0x477601fff] PGTABLE
[    0.006351] BRK [0x477602000, 0x477602fff] PGTABLE
[    0.006351] BRK [0x477603000, 0x477603fff] PGTABLE
[    0.006381] BRK [0x477604000, 0x477604fff] PGTABLE
[    0.006382] BRK [0x477605000, 0x477605fff] PGTABLE
[    0.006501] BRK [0x477606000, 0x477606fff] PGTABLE
[    0.006533] BRK [0x477607000, 0x477607fff] PGTABLE
[    0.006563] BRK [0x477608000, 0x477608fff] PGTABLE
[    0.006616] BRK [0x477609000, 0x477609fff] PGTABLE
[    0.006671] secureboot: Secure boot disabled
[    0.006672] RAMDISK: [mem 0x1e7b8000-0x20551fff]
[    0.006683] ACPI: Early table checksum verification disabled
[    0.006685] ACPI: RSDP 0x0000000038793000 000024 (v02 DELL  )
[    0.006687] ACPI: XSDT 0x00000000387930C0 0000FC (v01 DELL   CBX3     01072009 AMI  00010013)
[    0.006690] ACPI: FACP 0x00000000387D16E0 000114 (v06 DELL   CBX3     01072009 AMI  00010013)
[    0.006693] ACPI: DSDT 0x0000000038793248 03E494 (v02 DELL   CBX3     01072009 INTL 20160527)
[    0.006695] ACPI: FACS 0x0000000038C80C00 000040
[    0.006696] ACPI: APIC 0x00000000387D17F8 0000BC (v04 DELL   CBX3     01072009 AMI  00010013)
[    0.006698] ACPI: FPDT 0x00000000387D18B8 000044 (v01 DELL   CBX3     01072009 AMI  00010013)
[    0.006699] ACPI: FIDT 0x00000000387D1900 00009C (v01 DELL   CBX3     01072009 AMI  00010013)
[    0.006701] ACPI: MCFG 0x00000000387D19A0 00003C (v01 DELL   CBX3     01072009 MSFT 00000097)
[    0.006702] ACPI: SSDT 0x00000000387D19E0 001B1C (v02 CpuRef CpuSsdt  00003000 INTL 20160527)
[    0.006704] ACPI: SSDT 0x00000000387D3500 001663 (v02 SaSsdt SaSsdt   00003000 INTL 20160527)
[    0.006705] ACPI: HPET 0x00000000387D4B68 000038 (v01 DELL\x CBX3     00000002      01000013)
[    0.006707] ACPI: SSDT 0x00000000387D4BA0 001FC4 (v02 DELL\x WHL_Tbt_ 00001000 INTL 20160527)
[    0.006708] ACPI: SSDT 0x00000000387D6B68 003145 (v02 INTEL  xh_whlt4 00000000 INTL 20160527)
[    0.006709] ACPI: UEFI 0x00000000387D9CB0 000042 (v01 DELL\x CBX3     00000002      01000013)
[    0.006711] ACPI: LPIT 0x00000000387D9CF8 000094 (v01 DELL\x CBX3     00000002      01000013)
[    0.006712] ACPI: WSMT 0x00000000387D9D90 000028 (v01 DELL   CBX3     00000002      01000013)
[    0.006714] ACPI: SSDT 0x00000000387D9DB8 0027DE (v02 DELL\x PtidDevc 00001000 INTL 20160527)
[    0.006715] ACPI: SSDT 0x00000000387DC598 0014E2 (v02 DELL\x TbtTypeC 00000000 INTL 20160527)
[    0.006717] ACPI: DBGP 0x00000000387DDA80 000034 (v01 DELL\x CBX3     00000002      01000013)
[    0.006718] ACPI: DBG2 0x00000000387DDAB8 000054 (v00 DELL\x CBX3     00000002      01000013)
[    0.006719] ACPI: SSDT 0x00000000387DDB10 0007F0 (v02 DELL\x UsbCTabl 00001000 INTL 20160527)
[    0.006721] ACPI: SSDT 0x00000000387DE300 0053B8 (v02 DptfTa DptfTabl 00001000 INTL 20160527)
[    0.006722] ACPI: MSDM 0x00000000387E36B8 000055 (v03 DELL   CBX3     06222004 AMI  00010013)
[    0.006724] ACPI: SLIC 0x00000000387E3710 000176 (v03 DELL   CBX3     01072009 MSFT 00010013)
[    0.006725] ACPI: SSDT 0x00000000387E3888 000144 (v02 Intel  ADebTabl 00001000 INTL 20160527)
[    0.006727] ACPI: NHLT 0x00000000387E39D0 00002D (v00 INTEL  EDK2     00000002      01000013)
[    0.006728] ACPI: TPM2 0x00000000387E3A00 000034 (v04 DELL\x CBX3     00000001 AMI  00000000)
[    0.006729] ACPI: ASF! 0x00000000387E3A38 0000A0 (v32 DELL\x CBX3     00000002      01000013)
[    0.006731] ACPI: DMAR 0x00000000387E3AD8 0000C8 (v01 INTEL  EDK2     00000002      01000013)
[    0.006732] ACPI: BGRT 0x00000000387E3BA0 000038 (v00 DELL   CBX3     01072009 AMI  00010013)
[    0.006738] ACPI: Local APIC address 0xfee00000
[    0.006742] mapped APIC to ffffffffff5fc000 (        fee00000)
[    0.006996] No NUMA configuration found
[    0.006997] Faking a node at [mem 0x0000000000000000-0x00000004ac7fffff]
[    0.007011] NODE_DATA(0) allocated [mem 0x4ac7d5000-0x4ac7fffff]
[    0.052653] Zone ranges:
[    0.052654]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.052655]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.052656]   Normal   [mem 0x0000000100000000-0x00000004ac7fffff]
[    0.052656]   Device   empty
[    0.052657] Movable zone start for each node
[    0.052659] Early memory node ranges
[    0.052660]   node   0: [mem 0x0000000000001000-0x000000000009dfff]
[    0.052660]   node   0: [mem 0x000000000009f000-0x000000000009ffff]
[    0.052661]   node   0: [mem 0x0000000000100000-0x0000000037d0bfff]
[    0.052661]   node   0: [mem 0x0000000039d0c000-0x000000003c543fff]
[    0.052662]   node   0: [mem 0x000000003c546000-0x000000003fffffff]
[    0.052662]   node   0: [mem 0x0000000040400000-0x00000000489fffff]
[    0.052662]   node   0: [mem 0x0000000100000000-0x00000004ac7fffff]
[    0.053283] Zeroed struct page in unavailable ranges: 39524 pages
[    0.053284] Initmem setup node 0 [mem 0x0000000000001000-0x00000004ac7fffff]
[    0.053285] On node 0 totalpages: 4140444
[    0.053286]   DMA zone: 64 pages used for memmap
[    0.053286]   DMA zone: 22 pages reserved
[    0.053287]   DMA zone: 3998 pages, LIFO batch:0
[    0.053329]   DMA32 zone: 4440 pages used for memmap
[    0.053329]   DMA32 zone: 284158 pages, LIFO batch:63
[    0.060300]   Normal zone: 60192 pages used for memmap
[    0.060301]   Normal zone: 3852288 pages, LIFO batch:63
[    0.105709] Reserving Intel graphics memory at [mem 0x4b800000-0x4f7fffff]
[    0.106170] ACPI: PM-Timer IO Port: 0x1808
[    0.106171] ACPI: Local APIC address 0xfee00000
[    0.106176] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
[    0.106176] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
[    0.106177] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
[    0.106177] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
[    0.106177] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
[    0.106177] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])
[    0.106178] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])
[    0.106178] ACPI: LAPIC_NMI (acpi_id[0x08] high edge lint[0x1])
[    0.106237] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
[    0.106238] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.106239] Int: type 0, pol 0, trig 0, bus 00, IRQ 00, APIC ID 2, APIC INT 02
[    0.106240] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.106240] Int: type 0, pol 1, trig 3, bus 00, IRQ 09, APIC ID 2, APIC INT 09
[    0.106241] ACPI: IRQ0 used by override.
[    0.106242] Int: type 0, pol 0, trig 0, bus 00, IRQ 01, APIC ID 2, APIC INT 01
[    0.106242] Int: type 0, pol 0, trig 0, bus 00, IRQ 03, APIC ID 2, APIC INT 03
[    0.106243] Int: type 0, pol 0, trig 0, bus 00, IRQ 04, APIC ID 2, APIC INT 04
[    0.106243] Int: type 0, pol 0, trig 0, bus 00, IRQ 05, APIC ID 2, APIC INT 05
[    0.106244] Int: type 0, pol 0, trig 0, bus 00, IRQ 06, APIC ID 2, APIC INT 06
[    0.106244] Int: type 0, pol 0, trig 0, bus 00, IRQ 07, APIC ID 2, APIC INT 07
[    0.106244] Int: type 0, pol 0, trig 0, bus 00, IRQ 08, APIC ID 2, APIC INT 08
[    0.106245] ACPI: IRQ9 used by override.
[    0.106245] Int: type 0, pol 0, trig 0, bus 00, IRQ 0a, APIC ID 2, APIC INT 0a
[    0.106246] Int: type 0, pol 0, trig 0, bus 00, IRQ 0b, APIC ID 2, APIC INT 0b
[    0.106246] Int: type 0, pol 0, trig 0, bus 00, IRQ 0c, APIC ID 2, APIC INT 0c
[    0.106246] Int: type 0, pol 0, trig 0, bus 00, IRQ 0d, APIC ID 2, APIC INT 0d
[    0.106247] Int: type 0, pol 0, trig 0, bus 00, IRQ 0e, APIC ID 2, APIC INT 0e
[    0.106247] Int: type 0, pol 0, trig 0, bus 00, IRQ 0f, APIC ID 2, APIC INT 0f
[    0.106248] Using ACPI (MADT) for SMP configuration information
[    0.106249] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.106253] e820: update [mem 0x3329d000-0x3335afff] usable ==> reserved
[    0.106260] smpboot: Allowing 8 CPUs, 0 hotplug CPUs
[    0.106262] mapped IOAPIC to ffffffffff5fb000 (fec00000)
[    0.106275] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.106276] PM: Registered nosave memory: [mem 0x0009e000-0x0009efff]
[    0.106277] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
[    0.106278] PM: Registered nosave memory: [mem 0x2f87a000-0x2f87afff]
[    0.106279] PM: Registered nosave memory: [mem 0x2f88a000-0x2f88afff]
[    0.106280] PM: Registered nosave memory: [mem 0x3329d000-0x3335afff]
[    0.106281] PM: Registered nosave memory: [mem 0x37d0c000-0x38779fff]
[    0.106281] PM: Registered nosave memory: [mem 0x3877a000-0x387f6fff]
[    0.106281] PM: Registered nosave memory: [mem 0x387f7000-0x38c81fff]
[    0.106282] PM: Registered nosave memory: [mem 0x38c82000-0x39d0bfff]
[    0.106283] PM: Registered nosave memory: [mem 0x3c544000-0x3c544fff]
[    0.106283] PM: Registered nosave memory: [mem 0x3c545000-0x3c545fff]
[    0.106284] PM: Registered nosave memory: [mem 0x40000000-0x403fffff]
[    0.106285] PM: Registered nosave memory: [mem 0x48a00000-0x4f7fffff]
[    0.106285] PM: Registered nosave memory: [mem 0x4f800000-0xdfffffff]
[    0.106286] PM: Registered nosave memory: [mem 0xe0000000-0xefffffff]
[    0.106286] PM: Registered nosave memory: [mem 0xf0000000-0xfdffffff]
[    0.106286] PM: Registered nosave memory: [mem 0xfe000000-0xfe010fff]
[    0.106286] PM: Registered nosave memory: [mem 0xfe011000-0xfebfffff]
[    0.106287] PM: Registered nosave memory: [mem 0xfec00000-0xfec00fff]
[    0.106287] PM: Registered nosave memory: [mem 0xfec01000-0xfed1ffff]
[    0.106287] PM: Registered nosave memory: [mem 0xfed20000-0xfed7ffff]
[    0.106287] PM: Registered nosave memory: [mem 0xfed80000-0xfedfffff]
[    0.106288] PM: Registered nosave memory: [mem 0xfee00000-0xfee00fff]
[    0.106288] PM: Registered nosave memory: [mem 0xfee01000-0xfeffffff]
[    0.106288] PM: Registered nosave memory: [mem 0xff000000-0xffffffff]
[    0.106289] [mem 0x4f800000-0xdfffffff] available for PCI devices
[    0.106290] Booting paravirtualized kernel on bare hardware
[    0.106293] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[    0.185070] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.185312] percpu: Embedded 52 pages/cpu s176128 r8192 d28672 u262144
[    0.185317] pcpu-alloc: s176128 r8192 d28672 u262144 alloc=1*2097152
[    0.185318] pcpu-alloc: [0] 0 1 2 3 4 5 6 7 
[    0.185348] Built 1 zonelists, mobility grouping on.  Total pages: 4075726
[    0.185353] Policy zone: Normal
[    0.185360] Kernel command line: BOOT_IMAGE=(hd0,gpt2)/vmlinuz-5.4.18-200.fc31.x86_64 root=/dev/mapper/fedora_localhost--live-root ro resume=/dev/mapper/fedora_localhost--live-swap rd.lvm.lv=fedora_localhost-live/root rd.lvm.lv=fedora_localhost-live/swap rhgb quiet apic=verbose
[    0.186902] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
[    0.187856] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.187909] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.242158] Memory: 16045300K/16561776K available (14339K kernel code, 2270K rwdata, 4720K rodata, 2552K init, 4320K bss, 516476K reserved, 0K cma-reserved)
[    0.242259] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
[    0.242269] ftrace: allocating 41128 entries in 161 pages
[    0.254759] rcu: Hierarchical RCU implementation.
[    0.254761] rcu: 	RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8.
[    0.254761] 	Tasks RCU enabled.
[    0.254762] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[    0.254763] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8
[    0.256749] NR_IRQS: 524544, nr_irqs: 2048, preallocated irqs: 16
[    0.257149] random: crng done (trusting CPU's manufacturer)
[    0.257172] Console: colour dummy device 80x25
[    0.257176] printk: console [tty0] enabled
[    0.257189] ACPI: Core revision 20190816
[    0.257544] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635855245 ns
[    0.257660] APIC: Switch to symmetric I/O mode setup
[    0.257662] DMAR: Host address width 39
[    0.257663] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.257667] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.257668] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.257670] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.257671] DMAR: RMRR base: 0x00000038356000 end: 0x00000038375fff
[    0.257672] DMAR: RMRR base: 0x0000004b000000 end: 0x0000004f7fffff
[    0.257672] DMAR: RMRR base: 0x000000386fa000 end: 0x00000038779fff
[    0.257673] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.257674] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.257674] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.260223] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.260224] x2apic enabled
[    0.260248] Switched APIC routing to cluster x2apic.
[    0.260250] masked ExtINT on CPU#0
[    0.265781] ESR value before enabling vector: 0x00000040  after: 0x00000000
[    0.265794] ENABLING IO-APIC IRQs
[    0.265796] init IO_APIC IRQs
[    0.265796]  apic 2 pin 0 not connected
[    0.265800] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265811] IOAPIC[0]: Set routing entry (2-1 -> 0xef -> IRQ 1 Mode:0 Active:0 Dest:1)
[    0.265812] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:30 Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265822] IOAPIC[0]: Set routing entry (2-2 -> 0x30 -> IRQ 0 Mode:0 Active:0 Dest:1)
[    0.265824] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265833] IOAPIC[0]: Set routing entry (2-3 -> 0xef -> IRQ 3 Mode:0 Active:0 Dest:1)
[    0.265835] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265845] IOAPIC[0]: Set routing entry (2-4 -> 0xef -> IRQ 4 Mode:0 Active:0 Dest:1)
[    0.265846] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265856] IOAPIC[0]: Set routing entry (2-5 -> 0xef -> IRQ 5 Mode:0 Active:0 Dest:1)
[    0.265858] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265867] IOAPIC[0]: Set routing entry (2-6 -> 0xef -> IRQ 6 Mode:0 Active:0 Dest:1)
[    0.265869] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265879] IOAPIC[0]: Set routing entry (2-7 -> 0xef -> IRQ 7 Mode:0 Active:0 Dest:1)
[    0.265880] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265890] IOAPIC[0]: Set routing entry (2-8 -> 0xef -> IRQ 8 Mode:0 Active:0 Dest:1)
[    0.265891] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265902] IOAPIC[0]: Set routing entry (2-9 -> 0xef -> IRQ 9 Mode:1 Active:0 Dest:1)
[    0.265906] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265915] IOAPIC[0]: Set routing entry (2-10 -> 0xef -> IRQ 10 Mode:0 Active:0 Dest:1)
[    0.265916] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265926] IOAPIC[0]: Set routing entry (2-11 -> 0xef -> IRQ 11 Mode:0 Active:0 Dest:1)
[    0.265927] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265938] IOAPIC[0]: Set routing entry (2-12 -> 0xef -> IRQ 12 Mode:0 Active:0 Dest:1)
[    0.265939] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265949] IOAPIC[0]: Set routing entry (2-13 -> 0xef -> IRQ 13 Mode:0 Active:0 Dest:1)
[    0.265956] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265964] IOAPIC[0]: Set routing entry (2-14 -> 0xef -> IRQ 14 Mode:0 Active:0 Dest:1)
[    0.265965] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.265975] IOAPIC[0]: Set routing entry (2-15 -> 0xef -> IRQ 15 Mode:0 Active:0 Dest:1)
[    0.265976]  apic 2 pin 16 not connected
[    0.265976]  apic 2 pin 17 not connected
[    0.265976]  apic 2 pin 18 not connected
[    0.265977]  apic 2 pin 19 not connected
[    0.265977]  apic 2 pin 20 not connected
[    0.265977]  apic 2 pin 21 not connected
[    0.265977]  apic 2 pin 22 not connected
[    0.265978]  apic 2 pin 23 not connected
[    0.265978]  apic 2 pin 24 not connected
[    0.265978]  apic 2 pin 25 not connected
[    0.265978]  apic 2 pin 26 not connected
[    0.265979]  apic 2 pin 27 not connected
[    0.265979]  apic 2 pin 28 not connected
[    0.265979]  apic 2 pin 29 not connected
[    0.265979]  apic 2 pin 30 not connected
[    0.265980]  apic 2 pin 31 not connected
[    0.265980]  apic 2 pin 32 not connected
[    0.265980]  apic 2 pin 33 not connected
[    0.265980]  apic 2 pin 34 not connected
[    0.265981]  apic 2 pin 35 not connected
[    0.265981]  apic 2 pin 36 not connected
[    0.265981]  apic 2 pin 37 not connected
[    0.265981]  apic 2 pin 38 not connected
[    0.265982]  apic 2 pin 39 not connected
[    0.265982]  apic 2 pin 40 not connected
[    0.265982]  apic 2 pin 41 not connected
[    0.265982]  apic 2 pin 42 not connected
[    0.265983]  apic 2 pin 43 not connected
[    0.265983]  apic 2 pin 44 not connected
[    0.265983]  apic 2 pin 45 not connected
[    0.265983]  apic 2 pin 46 not connected
[    0.265984]  apic 2 pin 47 not connected
[    0.265984]  apic 2 pin 48 not connected
[    0.265984]  apic 2 pin 49 not connected
[    0.265984]  apic 2 pin 50 not connected
[    0.265985]  apic 2 pin 51 not connected
[    0.265985]  apic 2 pin 52 not connected
[    0.265985]  apic 2 pin 53 not connected
[    0.265985]  apic 2 pin 54 not connected
[    0.265986]  apic 2 pin 55 not connected
[    0.265986]  apic 2 pin 56 not connected
[    0.265986]  apic 2 pin 57 not connected
[    0.265986]  apic 2 pin 58 not connected
[    0.265987]  apic 2 pin 59 not connected
[    0.265987]  apic 2 pin 60 not connected
[    0.265987]  apic 2 pin 61 not connected
[    0.265987]  apic 2 pin 62 not connected
[    0.265988]  apic 2 pin 63 not connected
[    0.265988]  apic 2 pin 64 not connected
[    0.265988]  apic 2 pin 65 not connected
[    0.265988]  apic 2 pin 66 not connected
[    0.265989]  apic 2 pin 67 not connected
[    0.265989]  apic 2 pin 68 not connected
[    0.265989]  apic 2 pin 69 not connected
[    0.265989]  apic 2 pin 70 not connected
[    0.265990]  apic 2 pin 71 not connected
[    0.265990]  apic 2 pin 72 not connected
[    0.265990]  apic 2 pin 73 not connected
[    0.265990]  apic 2 pin 74 not connected
[    0.265991]  apic 2 pin 75 not connected
[    0.265991]  apic 2 pin 76 not connected
[    0.265991]  apic 2 pin 77 not connected
[    0.265991]  apic 2 pin 78 not connected
[    0.265992]  apic 2 pin 79 not connected
[    0.265992]  apic 2 pin 80 not connected
[    0.265992]  apic 2 pin 81 not connected
[    0.265992]  apic 2 pin 82 not connected
[    0.265993]  apic 2 pin 83 not connected
[    0.265993]  apic 2 pin 84 not connected
[    0.265993]  apic 2 pin 85 not connected
[    0.265993]  apic 2 pin 86 not connected
[    0.265994]  apic 2 pin 87 not connected
[    0.265994]  apic 2 pin 88 not connected
[    0.265994]  apic 2 pin 89 not connected
[    0.265994]  apic 2 pin 90 not connected
[    0.265995]  apic 2 pin 91 not connected
[    0.265995]  apic 2 pin 92 not connected
[    0.265995]  apic 2 pin 93 not connected
[    0.265995]  apic 2 pin 94 not connected
[    0.265996]  apic 2 pin 95 not connected
[    0.265996]  apic 2 pin 96 not connected
[    0.265996]  apic 2 pin 97 not connected
[    0.265996]  apic 2 pin 98 not connected
[    0.265997]  apic 2 pin 99 not connected
[    0.265997]  apic 2 pin 100 not connected
[    0.265997]  apic 2 pin 101 not connected
[    0.265997]  apic 2 pin 102 not connected
[    0.265998]  apic 2 pin 103 not connected
[    0.265998]  apic 2 pin 104 not connected
[    0.265998]  apic 2 pin 105 not connected
[    0.265998]  apic 2 pin 106 not connected
[    0.265999]  apic 2 pin 107 not connected
[    0.265999]  apic 2 pin 108 not connected
[    0.265999]  apic 2 pin 109 not connected
[    0.265999]  apic 2 pin 110 not connected
[    0.266000]  apic 2 pin 111 not connected
[    0.266000]  apic 2 pin 112 not connected
[    0.266000]  apic 2 pin 113 not connected
[    0.266000]  apic 2 pin 114 not connected
[    0.266001]  apic 2 pin 115 not connected
[    0.266001]  apic 2 pin 116 not connected
[    0.266001]  apic 2 pin 117 not connected
[    0.266001]  apic 2 pin 118 not connected
[    0.266002]  apic 2 pin 119 not connected
[    0.266175] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.270696] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x1e44fb6c2ab, max_idle_ns: 440795206594 ns
[    0.270719] Calibrating delay loop (skipped), value calculated using timer frequency.. 4199.88 BogoMIPS (lpj=2099944)
[    0.270721] pid_max: default: 32768 minimum: 301
[    0.273688] LSM: Security Framework initializing
[    0.273695] Yama: becoming mindful.
[    0.273702] SELinux:  Initializing.
[    0.273711] *** VALIDATE SELinux ***
[    0.273762] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.273805] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.273816] *** VALIDATE tmpfs ***
[    0.273947] *** VALIDATE proc ***
[    0.273984] *** VALIDATE cgroup1 ***
[    0.273985] *** VALIDATE cgroup2 ***
[    0.274027] mce: CPU0: Thermal monitoring enabled (TM1)
[    0.274041] process: using mwait in idle threads
[    0.274044] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
[    0.274044] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[    0.274047] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.274048] Spectre V2 : Mitigation: Enhanced IBRS
[    0.274049] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.274050] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[    0.274050] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[    0.274051] TAA: Mitigation: TSX disabled
[    0.274353] Freeing SMP alternatives memory: 36K
[    0.276795] TSC deadline timer enabled
[    0.276800] smpboot: CPU0: Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz (family: 0x6, model: 0x8e, stepping: 0xc)
[    0.276867] Performance Events: PEBS fmt3+, Skylake events, 32-deep LBR, full-width counters, Intel PMU driver.
[    0.276873] ... version:                4
[    0.276873] ... bit width:              48
[    0.276873] ... generic registers:      4
[    0.276873] ... value mask:             0000ffffffffffff
[    0.276874] ... max period:             00007fffffffffff
[    0.276874] ... fixed-purpose events:   3
[    0.276874] ... event mask:             000000070000000f
[    0.276902] rcu: Hierarchical SRCU implementation.
[    0.277501] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[    0.277573] smp: Bringing up secondary CPUs ...
[    0.277619] x86: Booting SMP configuration:
[    0.277621] .... node  #0, CPUs:      #1
[    0.006605] masked ExtINT on CPU#1
[    0.279262]  #2
[    0.006605] masked ExtINT on CPU#2
[    0.280782]  #3
[    0.006605] masked ExtINT on CPU#3
[    0.282222]  #4
[    0.006605] masked ExtINT on CPU#4
[    0.282786]  #5
[    0.006605] masked ExtINT on CPU#5
[    0.283121]  #6
[    0.006605] masked ExtINT on CPU#6
[    0.283793]  #7
[    0.006605] masked ExtINT on CPU#7
[    0.284124] smp: Brought up 1 node, 8 CPUs
[    0.284124] smpboot: Max logical packages: 1
[    0.284124] smpboot: Total of 8 processors activated (33599.10 BogoMIPS)
[    0.285188] devtmpfs: initialized
[    0.285188] x86/mm: Memory block size: 128MB
[    0.286187] PM: Registering ACPI NVS region [mem 0x387f7000-0x38c81fff] (4763648 bytes)
[    0.286187] PM: Registering ACPI NVS region [mem 0x3c544000-0x3c544fff] (4096 bytes)
[    0.286187] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[    0.286187] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[    0.286187] pinctrl core: initialized pinctrl subsystem
[    0.286187] PM: RTC time: 15:33:52, date: 2020-02-17
[    0.286187] NET: Registered protocol family 16
[    0.286762] audit: initializing netlink subsys (disabled)
[    0.286765] audit: type=2000 audit(1581953632.024:1): state=initialized audit_enabled=0 res=1
[    0.286765] cpuidle: using governor menu
[    0.286790] ACPI: bus type PCI registered
[    0.286790] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.286804] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.286806] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
[    0.286815] PCI: Using configuration type 1 for base access
[    0.287349] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.288414] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.288414] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.370829] cryptd: max_cpu_qlen set to 1000
[    0.371779] alg: No test for 842 (842-generic)
[    0.371779] alg: No test for 842 (842-scomp)
[    0.372812] ACPI: Added _OSI(Module Device)
[    0.372812] ACPI: Added _OSI(Processor Device)
[    0.372812] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.372812] ACPI: Added _OSI(Processor Aggregator Device)
[    0.372812] ACPI: Added _OSI(Linux-Dell-Video)
[    0.372812] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.372812] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    0.410535] ACPI: 10 ACPI AML tables successfully acquired and loaded
[    0.469289] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
[    0.477279] ACPI: Dynamic OEM Table Load:
[    0.477286] ACPI: SSDT 0xFFFF9FBF59408100 0000F4 (v02 PmRef  Cpu0Psd  00003000 INTL 20160527)
[    0.479827] ACPI: \_SB_.PR00: _OSC native thermal LVT Acked
[    0.480980] ACPI: Dynamic OEM Table Load:
[    0.480986] ACPI: SSDT 0xFFFF9FBF59415800 000400 (v02 PmRef  Cpu0Cst  00003001 INTL 20160527)
[    0.482638] ACPI: Dynamic OEM Table Load:
[    0.482644] ACPI: SSDT 0xFFFF9FBF59FB8800 000560 (v02 PmRef  Cpu0Ist  00003000 INTL 20160527)
[    0.484234] ACPI: Dynamic OEM Table Load:
[    0.484238] ACPI: SSDT 0xFFFF9FBF5940C800 00011B (v02 PmRef  Cpu0Hwp  00003000 INTL 20160527)
[    0.485473] ACPI: Dynamic OEM Table Load:
[    0.485479] ACPI: SSDT 0xFFFF9FBF59FBA800 000724 (v02 PmRef  HwpLvt   00003000 INTL 20160527)
[    0.487027] ACPI: Dynamic OEM Table Load:
[    0.487033] ACPI: SSDT 0xFFFF9FBF59FBB000 0005FC (v02 PmRef  ApIst    00003000 INTL 20160527)
[    0.488467] ACPI: Dynamic OEM Table Load:
[    0.488471] ACPI: SSDT 0xFFFF9FBF59415C00 000317 (v02 PmRef  ApHwp    00003000 INTL 20160527)
[    0.489676] ACPI: Dynamic OEM Table Load:
[    0.489681] ACPI: SSDT 0xFFFF9FBF5911D000 000AB0 (v02 PmRef  ApPsd    00003000 INTL 20160527)
[    0.491326] ACPI: Dynamic OEM Table Load:
[    0.491330] ACPI: SSDT 0xFFFF9FBF59416800 00030A (v02 PmRef  ApCst    00003000 INTL 20160527)
[    0.494840] ACPI: EC: EC started
[    0.494840] ACPI: EC: interrupt blocked
[    0.503812] ACPI: \_SB_.PCI0.LPCB.ECDV: Used as first EC
[    0.503814] ACPI: \_SB_.PCI0.LPCB.ECDV: GPE=0x6e, EC_CMD/EC_SC=0x934, EC_DATA=0x930
[    0.503815] ACPI: \_SB_.PCI0.LPCB.ECDV: Boot DSDT EC used to handle transactions
[    0.503815] ACPI: Interpreter enabled
[    0.503857] ACPI: (supports S0 S3 S4 S5)
[    0.503858] ACPI: Using IOAPIC for interrupt routing
[    0.503919] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.504314] ACPI: Enabled 9 GPEs in block 00 to 7F
[    0.509260] ACPI: Power Resource [PXP] (on)
[    0.514415] ACPI: Power Resource [BTPR] (off)
[    0.515308] ACPI: Power Resource [USBC] (on)
[    0.515408] ACPI: Power Resource [PAUD] (on)
[    0.517386] ACPI: Power Resource [V0PR] (on)
[    0.517472] ACPI: Power Resource [V1PR] (on)
[    0.517555] ACPI: Power Resource [V2PR] (on)
[    0.520200] ACPI: Power Resource [WRST] (on)
[    0.528227] ACPI: Power Resource [PIN] (off)
[    0.528484] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe])
[    0.528488] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    0.529575] acpi PNP0A08:00: _OSC: platform does not support [AER]
[    0.531640] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug SHPCHotplug PME PCIeCapability LTR]
[    0.532253] PCI host bridge to bus 0000:00
[    0.532254] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.532255] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.532255] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.532256] pci_bus 0000:00: root bus resource [mem 0x000c0000-0x000c3fff window]
[    0.532256] pci_bus 0000:00: root bus resource [mem 0x000c4000-0x000c7fff window]
[    0.532257] pci_bus 0000:00: root bus resource [mem 0x000c8000-0x000cbfff window]
[    0.532258] pci_bus 0000:00: root bus resource [mem 0x000cc000-0x000cffff window]
[    0.532258] pci_bus 0000:00: root bus resource [mem 0x000d0000-0x000d3fff window]
[    0.532259] pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000d7fff window]
[    0.532259] pci_bus 0000:00: root bus resource [mem 0x000d8000-0x000dbfff window]
[    0.532260] pci_bus 0000:00: root bus resource [mem 0x000dc000-0x000dffff window]
[    0.532260] pci_bus 0000:00: root bus resource [mem 0x4f800000-0xdfffffff window]
[    0.532261] pci_bus 0000:00: root bus resource [mem 0xfc800000-0xfe7fffff window]
[    0.532262] pci_bus 0000:00: root bus resource [bus 00-fe]
[    0.532269] pci 0000:00:00.0: [8086:3e34] type 00 class 0x060000
[    0.532372] pci 0000:00:02.0: [8086:3ea0] type 00 class 0x030000
[    0.532380] pci 0000:00:02.0: reg 0x10: [mem 0xdb000000-0xdbffffff 64bit]
[    0.532385] pci 0000:00:02.0: reg 0x18: [mem 0x50000000-0x5fffffff 64bit pref]
[    0.532388] pci 0000:00:02.0: reg 0x20: [io  0x3000-0x303f]
[    0.532397] pci 0000:00:02.0: BAR 2: assigned to efifb
[    0.532486] pci 0000:00:04.0: [8086:1903] type 00 class 0x118000
[    0.532496] pci 0000:00:04.0: reg 0x10: [mem 0xdc310000-0xdc317fff 64bit]
[    0.532618] pci 0000:00:08.0: [8086:1911] type 00 class 0x088000
[    0.532628] pci 0000:00:08.0: reg 0x10: [mem 0xdc32a000-0xdc32afff 64bit]
[    0.532751] pci 0000:00:12.0: [8086:9df9] type 00 class 0x118000
[    0.532772] pci 0000:00:12.0: reg 0x10: [mem 0xdc329000-0xdc329fff 64bit]
[    0.532925] pci 0000:00:13.0: [8086:9dfc] type 00 class 0x070000
[    0.532946] pci 0000:00:13.0: reg 0x10: [mem 0xdc322000-0xdc323fff 64bit]
[    0.533002] pci 0000:00:13.0: PME# supported from D0 D3hot
[    0.533109] pci 0000:00:14.0: [8086:9ded] type 00 class 0x0c0330
[    0.533125] pci 0000:00:14.0: reg 0x10: [mem 0xdc300000-0xdc30ffff 64bit]
[    0.533176] pci 0000:00:14.0: PME# supported from D3hot D3cold
[    0.533273] pci 0000:00:14.2: [8086:9def] type 00 class 0x050000
[    0.533290] pci 0000:00:14.2: reg 0x10: [mem 0xdc320000-0xdc321fff 64bit]
[    0.533300] pci 0000:00:14.2: reg 0x18: [mem 0xdc328000-0xdc328fff 64bit]
[    0.533416] pci 0000:00:14.3: [8086:9df0] type 00 class 0x028000
[    0.533451] pci 0000:00:14.3: reg 0x10: [mem 0xdc31c000-0xdc31ffff 64bit]
[    0.533545] pci 0000:00:14.3: PME# supported from D0 D3hot D3cold
[    0.533695] pci 0000:00:15.0: [8086:9de8] type 00 class 0x0c8000
[    0.533778] pci 0000:00:15.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
[    0.534057] pci 0000:00:16.0: [8086:9de0] type 00 class 0x078000
[    0.534079] pci 0000:00:16.0: reg 0x10: [mem 0xdc326000-0xdc326fff 64bit]
[    0.534145] pci 0000:00:16.0: PME# supported from D3hot
[    0.534246] pci 0000:00:16.3: [8086:9de3] type 00 class 0x070002
[    0.534260] pci 0000:00:16.3: reg 0x10: [io  0x3060-0x3067]
[    0.534265] pci 0000:00:16.3: reg 0x14: [mem 0xdc325000-0xdc325fff]
[    0.534416] pci 0000:00:1c.0: [8086:9dbc] type 01 class 0x060400
[    0.534513] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    0.534527] pci 0000:00:1c.0: PTM enabled (root), 4ns granularity
[    0.534676] pci 0000:00:1d.0: [8086:9db3] type 01 class 0x060400
[    0.534747] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[    0.534761] pci 0000:00:1d.0: PTM enabled (root), 4ns granularity
[    0.534886] pci 0000:00:1d.4: [8086:9db4] type 01 class 0x060400
[    0.534964] pci 0000:00:1d.4: PME# supported from D0 D3hot D3cold
[    0.534980] pci 0000:00:1d.4: PTM enabled (root), 4ns granularity
[    0.535105] pci 0000:00:1f.0: [8086:9d84] type 00 class 0x060100
[    0.535264] pci 0000:00:1f.3: [8086:9dc8] type 00 class 0x040380
[    0.535304] pci 0000:00:1f.3: reg 0x10: [mem 0xdc318000-0xdc31bfff 64bit]
[    0.535340] pci 0000:00:1f.3: reg 0x20: [mem 0xdc000000-0xdc0fffff 64bit]
[    0.535415] pci 0000:00:1f.3: PME# supported from D3hot D3cold
[    0.535556] pci 0000:00:1f.4: [8086:9da3] type 00 class 0x0c0500
[    0.535581] pci 0000:00:1f.4: reg 0x10: [mem 0xdc324000-0xdc3240ff 64bit]
[    0.535609] pci 0000:00:1f.4: reg 0x20: [io  0xefa0-0xefbf]
[    0.535709] pci 0000:00:1f.5: [8086:9da4] type 00 class 0x0c8000
[    0.535723] pci 0000:00:1f.5: reg 0x10: [mem 0xfe010000-0xfe010fff]
[    0.535907] pci 0000:01:00.0: [8086:15d3] type 01 class 0x060400
[    0.535962] pci 0000:01:00.0: enabling Extended Tags
[    0.536036] pci 0000:01:00.0: supports D1 D2
[    0.536036] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.536136] pci 0000:00:1c.0: PCI bridge to [bus 01-6b]
[    0.536139] pci 0000:00:1c.0:   bridge window [mem 0xac000000-0xda0fffff]
[    0.536142] pci 0000:00:1c.0:   bridge window [mem 0x60000000-0xa9ffffff 64bit pref]
[    0.536192] pci 0000:02:00.0: [8086:15d3] type 01 class 0x060400
[    0.536251] pci 0000:02:00.0: enabling Extended Tags
[    0.536328] pci 0000:02:00.0: supports D1 D2
[    0.536329] pci 0000:02:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.536401] pci 0000:02:01.0: [8086:15d3] type 01 class 0x060400
[    0.536460] pci 0000:02:01.0: enabling Extended Tags
[    0.536538] pci 0000:02:01.0: supports D1 D2
[    0.536538] pci 0000:02:01.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.536610] pci 0000:02:02.0: [8086:15d3] type 01 class 0x060400
[    0.536669] pci 0000:02:02.0: enabling Extended Tags
[    0.536744] pci 0000:02:02.0: supports D1 D2
[    0.536744] pci 0000:02:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.536826] pci 0000:02:04.0: [8086:15d3] type 01 class 0x060400
[    0.536885] pci 0000:02:04.0: enabling Extended Tags
[    0.536964] pci 0000:02:04.0: supports D1 D2
[    0.536964] pci 0000:02:04.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.537055] pci 0000:01:00.0: PCI bridge to [bus 02-6b]
[    0.537062] pci 0000:01:00.0:   bridge window [mem 0xac000000-0xda0fffff]
[    0.537067] pci 0000:01:00.0:   bridge window [mem 0x60000000-0xa9ffffff 64bit pref]
[    0.537112] pci 0000:03:00.0: [8086:15d2] type 00 class 0x088000
[    0.537142] pci 0000:03:00.0: reg 0x10: [mem 0xda000000-0xda03ffff]
[    0.537152] pci 0000:03:00.0: reg 0x14: [mem 0xda040000-0xda040fff]
[    0.537207] pci 0000:03:00.0: enabling Extended Tags
[    0.537301] pci 0000:03:00.0: supports D1 D2
[    0.537302] pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.537426] pci 0000:02:00.0: PCI bridge to [bus 03]
[    0.537433] pci 0000:02:00.0:   bridge window [mem 0xda000000-0xda0fffff]
[    0.537480] pci 0000:02:01.0: PCI bridge to [bus 04-36]
[    0.537487] pci 0000:02:01.0:   bridge window [mem 0xac000000-0xc3efffff]
[    0.537492] pci 0000:02:01.0:   bridge window [mem 0x60000000-0x7fffffff 64bit pref]
[    0.537543] pci 0000:37:00.0: [8086:15d4] type 00 class 0x0c0330
[    0.537576] pci 0000:37:00.0: reg 0x10: [mem 0xc3f00000-0xc3f0ffff]
[    0.537647] pci 0000:37:00.0: enabling Extended Tags
[    0.537744] pci 0000:37:00.0: supports D1 D2
[    0.537744] pci 0000:37:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.537804] pci 0000:37:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x4 link at 0000:02:02.0 (capable of 31.504 Gb/s with 8 GT/s x4 link)
[    0.537890] pci 0000:02:02.0: PCI bridge to [bus 37]
[    0.537897] pci 0000:02:02.0:   bridge window [mem 0xc3f00000-0xc3ffffff]
[    0.537933] pci 0000:02:04.0: PCI bridge to [bus 38-6b]
[    0.537940] pci 0000:02:04.0:   bridge window [mem 0xc4000000-0xd9ffffff]
[    0.537945] pci 0000:02:04.0:   bridge window [mem 0x80000000-0xa9ffffff 64bit pref]
[    0.538051] pci 0000:6c:00.0: [10ec:525a] type 00 class 0xff0000
[    0.538080] pci 0000:6c:00.0: reg 0x14: [mem 0xdc200000-0xdc200fff]
[    0.538176] pci 0000:6c:00.0: supports D1 D2
[    0.538177] pci 0000:6c:00.0: PME# supported from D1 D2 D3hot D3cold
[    0.538285] pci 0000:00:1d.0: PCI bridge to [bus 6c]
[    0.538288] pci 0000:00:1d.0:   bridge window [mem 0xdc200000-0xdc2fffff]
[    0.538465] pci 0000:6d:00.0: [1179:0113] type 00 class 0x010802
[    0.538492] pci 0000:6d:00.0: reg 0x10: [mem 0xdc100000-0xdc103fff 64bit]
[    0.538598] pci 0000:6d:00.0: PME# supported from D0 D3hot
[    0.538708] pci 0000:00:1d.4: PCI bridge to [bus 6d]
[    0.538712] pci 0000:00:1d.4:   bridge window [mem 0xdc100000-0xdc1fffff]
[    0.539710] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 10 11 12 14 15) *0
[    0.539710] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 10 11 12 14 15) *1
[    0.539710] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 10 11 12 14 15) *0
[    0.539710] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 11 12 14 15) *0
[    0.539710] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 10 11 12 14 15) *0
[    0.539710] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 10 11 12 14 15) *0
[    0.539710] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 10 11 12 14 15) *0
[    0.539710] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 10 11 12 14 15) *0
[    0.553890] ACPI: EC: interrupt unblocked
[    0.553901] ACPI: EC: event unblocked
[    0.553918] ACPI: \_SB_.PCI0.LPCB.ECDV: GPE=0x6e, EC_CMD/EC_SC=0x934, EC_DATA=0x930
[    0.553918] ACPI: \_SB_.PCI0.LPCB.ECDV: Boot DSDT EC used to handle transactions and events
[    0.553994] iommu: Default domain type: Translated 
[    0.553994] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.553994] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    0.553994] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.553994] vgaarb: loaded
[    0.553994] SCSI subsystem initialized
[    0.553994] libata version 3.00 loaded.
[    0.553994] ACPI: bus type USB registered
[    0.553994] usbcore: registered new interface driver usbfs
[    0.553994] usbcore: registered new interface driver hub
[    0.553994] usbcore: registered new device driver usb
[    0.553994] pps_core: LinuxPPS API ver. 1 registered
[    0.553994] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.553994] PTP clock support registered
[    0.553994] EDAC MC: Ver: 3.0.0
[    0.554728] Registered efivars operations
[    0.554740] PCI: Using ACPI for IRQ routing
[    0.585549] PCI: pci_cache_line_size set to 64 bytes
[    0.585797] e820: reserve RAM buffer [mem 0x0009e000-0x0009ffff]
[    0.585798] e820: reserve RAM buffer [mem 0x2f87a018-0x2fffffff]
[    0.585798] e820: reserve RAM buffer [mem 0x3329d000-0x33ffffff]
[    0.585799] e820: reserve RAM buffer [mem 0x37d0c000-0x37ffffff]
[    0.585799] e820: reserve RAM buffer [mem 0x3c544000-0x3fffffff]
[    0.585799] e820: reserve RAM buffer [mem 0x48a00000-0x4bffffff]
[    0.585800] e820: reserve RAM buffer [mem 0x4ac800000-0x4afffffff]
[    0.585879] NetLabel: Initializing
[    0.585879] NetLabel:  domain hash size = 128
[    0.585880] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.585888] NetLabel:  unlabeled traffic allowed by default
[    0.585890] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    0.585890] hpet0: 8 comparators, 64-bit 24.000000 MHz counter
[    0.588742] clocksource: Switched to clocksource tsc-early
[    0.598043] *** VALIDATE bpf ***
[    0.598088] VFS: Disk quotas dquot_6.6.0
[    0.598098] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.598115] *** VALIDATE ramfs ***
[    0.598117] *** VALIDATE hugetlbfs ***
[    0.598135] pnp: PnP ACPI init
[    0.598190] system 00:00: [mem 0x40000000-0x403fffff] has been reserved
[    0.598193] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.598303] system 00:01: [io  0x0680-0x069f] has been reserved
[    0.598303] system 00:01: [io  0x164e-0x164f] has been reserved
[    0.598305] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.598350] pnp 00:02: Plug and Play ACPI device, IDs PNP0b00 (active)
[    0.598404] system 00:03: [io  0x1854-0x1857] has been reserved
[    0.598405] system 00:03: Plug and Play ACPI device, IDs INT3f0d PNP0c02 (active)
[    0.598470] pnp 00:04: Plug and Play ACPI device, IDs PNP0303 (active)
[    0.599218] pnp 00:05: Plug and Play ACPI device, IDs PNP0401 (disabled)
[    0.599326] system 00:06: [mem 0xfed10000-0xfed17fff] has been reserved
[    0.599327] system 00:06: [mem 0xfed18000-0xfed18fff] has been reserved
[    0.599328] system 00:06: [mem 0xfed19000-0xfed19fff] has been reserved
[    0.599328] system 00:06: [mem 0xe0000000-0xefffffff] has been reserved
[    0.599329] system 00:06: [mem 0xfed20000-0xfed3ffff] has been reserved
[    0.599330] system 00:06: [mem 0xfed90000-0xfed93fff] could not be reserved
[    0.599330] system 00:06: [mem 0xfed45000-0xfed8ffff] could not be reserved
[    0.599331] system 00:06: [mem 0xfee00000-0xfeefffff] could not be reserved
[    0.599333] system 00:06: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.599470] system 00:07: [io  0x1800-0x18fe] could not be reserved
[    0.599471] system 00:07: [mem 0xfd000000-0xfd69ffff] has been reserved
[    0.599471] system 00:07: [mem 0xfd6b0000-0xfd6cffff] has been reserved
[    0.599472] system 00:07: [mem 0xfd6f0000-0xfdffffff] has been reserved
[    0.599473] system 00:07: [mem 0xfe000000-0xfe01ffff] could not be reserved
[    0.599473] system 00:07: [mem 0xfe200000-0xfe7fffff] has been reserved
[    0.599474] system 00:07: [mem 0xff000000-0xffffffff] has been reserved
[    0.599476] system 00:07: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.599658] system 00:08: [io  0x2000-0x20fe] has been reserved
[    0.599659] system 00:08: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.600266] system 00:09: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.604297] pnp: PnP ACPI: found 10 devices
[    0.604949] thermal_sys: Registered thermal governor 'fair_share'
[    0.604949] thermal_sys: Registered thermal governor 'bang_bang'
[    0.604950] thermal_sys: Registered thermal governor 'step_wise'
[    0.604950] thermal_sys: Registered thermal governor 'user_space'
[    0.609471] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.609506] pci 0000:02:01.0: bridge window [io  0x1000-0x0fff] to [bus 04-36] add_size 1000
[    0.609507] pci 0000:02:04.0: bridge window [io  0x1000-0x0fff] to [bus 38-6b] add_size 1000
[    0.609508] pci 0000:01:00.0: bridge window [io  0x1000-0x0fff] to [bus 02-6b] add_size 2000
[    0.609509] pci 0000:00:1c.0: bridge window [io  0x1000-0x0fff] to [bus 01-6b] add_size 3000
[    0.609517] pci 0000:00:15.0: BAR 0: assigned [mem 0x4f800000-0x4f800fff 64bit]
[    0.609568] pci 0000:00:1c.0: BAR 13: assigned [io  0x4000-0x6fff]
[    0.609569] pci 0000:01:00.0: BAR 13: assigned [io  0x4000-0x5fff]
[    0.609570] pci 0000:02:01.0: BAR 13: assigned [io  0x4000-0x4fff]
[    0.609571] pci 0000:02:04.0: BAR 13: assigned [io  0x5000-0x5fff]
[    0.609572] pci 0000:02:00.0: PCI bridge to [bus 03]
[    0.609576] pci 0000:02:00.0:   bridge window [mem 0xda000000-0xda0fffff]
[    0.609583] pci 0000:02:01.0: PCI bridge to [bus 04-36]
[    0.609585] pci 0000:02:01.0:   bridge window [io  0x4000-0x4fff]
[    0.609589] pci 0000:02:01.0:   bridge window [mem 0xac000000-0xc3efffff]
[    0.609592] pci 0000:02:01.0:   bridge window [mem 0x60000000-0x7fffffff 64bit pref]
[    0.609597] pci 0000:02:02.0: PCI bridge to [bus 37]
[    0.609600] pci 0000:02:02.0:   bridge window [mem 0xc3f00000-0xc3ffffff]
[    0.609607] pci 0000:02:04.0: PCI bridge to [bus 38-6b]
[    0.609609] pci 0000:02:04.0:   bridge window [io  0x5000-0x5fff]
[    0.609613] pci 0000:02:04.0:   bridge window [mem 0xc4000000-0xd9ffffff]
[    0.609615] pci 0000:02:04.0:   bridge window [mem 0x80000000-0xa9ffffff 64bit pref]
[    0.609620] pci 0000:01:00.0: PCI bridge to [bus 02-6b]
[    0.609622] pci 0000:01:00.0:   bridge window [io  0x4000-0x5fff]
[    0.609625] pci 0000:01:00.0:   bridge window [mem 0xac000000-0xda0fffff]
[    0.609628] pci 0000:01:00.0:   bridge window [mem 0x60000000-0xa9ffffff 64bit pref]
[    0.609633] pci 0000:00:1c.0: PCI bridge to [bus 01-6b]
[    0.609634] pci 0000:00:1c.0:   bridge window [io  0x4000-0x6fff]
[    0.609636] pci 0000:00:1c.0:   bridge window [mem 0xac000000-0xda0fffff]
[    0.609638] pci 0000:00:1c.0:   bridge window [mem 0x60000000-0xa9ffffff 64bit pref]
[    0.609641] pci 0000:00:1d.0: PCI bridge to [bus 6c]
[    0.609643] pci 0000:00:1d.0:   bridge window [mem 0xdc200000-0xdc2fffff]
[    0.609647] pci 0000:00:1d.4: PCI bridge to [bus 6d]
[    0.609650] pci 0000:00:1d.4:   bridge window [mem 0xdc100000-0xdc1fffff]
[    0.609655] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.609655] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.609656] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[    0.609656] pci_bus 0000:00: resource 7 [mem 0x000c0000-0x000c3fff window]
[    0.609657] pci_bus 0000:00: resource 8 [mem 0x000c4000-0x000c7fff window]
[    0.609657] pci_bus 0000:00: resource 9 [mem 0x000c8000-0x000cbfff window]
[    0.609658] pci_bus 0000:00: resource 10 [mem 0x000cc000-0x000cffff window]
[    0.609659] pci_bus 0000:00: resource 11 [mem 0x000d0000-0x000d3fff window]
[    0.609659] pci_bus 0000:00: resource 12 [mem 0x000d4000-0x000d7fff window]
[    0.609660] pci_bus 0000:00: resource 13 [mem 0x000d8000-0x000dbfff window]
[    0.609660] pci_bus 0000:00: resource 14 [mem 0x000dc000-0x000dffff window]
[    0.609661] pci_bus 0000:00: resource 15 [mem 0x4f800000-0xdfffffff window]
[    0.609661] pci_bus 0000:00: resource 16 [mem 0xfc800000-0xfe7fffff window]
[    0.609662] pci_bus 0000:01: resource 0 [io  0x4000-0x6fff]
[    0.609663] pci_bus 0000:01: resource 1 [mem 0xac000000-0xda0fffff]
[    0.609663] pci_bus 0000:01: resource 2 [mem 0x60000000-0xa9ffffff 64bit pref]
[    0.609664] pci_bus 0000:02: resource 0 [io  0x4000-0x5fff]
[    0.609664] pci_bus 0000:02: resource 1 [mem 0xac000000-0xda0fffff]
[    0.609665] pci_bus 0000:02: resource 2 [mem 0x60000000-0xa9ffffff 64bit pref]
[    0.609666] pci_bus 0000:03: resource 1 [mem 0xda000000-0xda0fffff]
[    0.609666] pci_bus 0000:04: resource 0 [io  0x4000-0x4fff]
[    0.609667] pci_bus 0000:04: resource 1 [mem 0xac000000-0xc3efffff]
[    0.609667] pci_bus 0000:04: resource 2 [mem 0x60000000-0x7fffffff 64bit pref]
[    0.609668] pci_bus 0000:37: resource 1 [mem 0xc3f00000-0xc3ffffff]
[    0.609669] pci_bus 0000:38: resource 0 [io  0x5000-0x5fff]
[    0.609669] pci_bus 0000:38: resource 1 [mem 0xc4000000-0xd9ffffff]
[    0.609670] pci_bus 0000:38: resource 2 [mem 0x80000000-0xa9ffffff 64bit pref]
[    0.609670] pci_bus 0000:6c: resource 1 [mem 0xdc200000-0xdc2fffff]
[    0.609671] pci_bus 0000:6d: resource 1 [mem 0xdc100000-0xdc1fffff]
[    0.609761] NET: Registered protocol family 2
[    0.609849] tcp_listen_portaddr_hash hash table entries: 8192 (order: 5, 131072 bytes, linear)
[    0.609871] TCP established hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.610065] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    0.610239] TCP: Hash tables configured (established 131072 bind 65536)
[    0.610261] UDP hash table entries: 8192 (order: 6, 262144 bytes, linear)
[    0.610303] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes, linear)
[    0.610395] NET: Registered protocol family 1
[    0.610398] NET: Registered protocol family 44
[    0.610404] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    0.610497] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.610499] IOAPIC[0]: Set routing entry (2-16 -> 0xef -> IRQ 16 Mode:1 Active:1 Dest:1)
[    0.610654] pci 0000:01:00.0: CLS mismatch (64 != 128), using 64 bytes
[    0.610724] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.610724] IOAPIC[0]: Set routing entry (2-16 -> 0xef -> IRQ 16 Mode:1 Active:1 Dest:1)
[    0.610734] pci 0000:01:00.0: enabling device (0002 -> 0003)
[    0.610797] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.610798] IOAPIC[0]: Set routing entry (2-18 -> 0xef -> IRQ 18 Mode:1 Active:1 Dest:1)
[    0.610899] Trying to unpack rootfs image as initramfs...
[    0.906939] Freeing initrd memory: 30312K
[    0.907012] DMAR: Intel-IOMMU force enabled due to platform opt in
[    0.907051] DMAR: No ATSR found
[    0.907092] DMAR: dmar0: Using Queued invalidation
[    0.907095] DMAR: dmar1: Using Queued invalidation
[    0.907358] pci 0000:00:00.0: Adding to iommu group 0
[    0.907379] pci 0000:00:00.0: Using iommu direct mapping
[    0.912158] pci 0000:00:02.0: Adding to iommu group 1
[    0.912278] pci 0000:00:02.0: Using iommu direct mapping
[    0.912302] pci 0000:00:04.0: Adding to iommu group 2
[    0.912309] pci 0000:00:04.0: Using iommu direct mapping
[    0.912335] pci 0000:00:08.0: Adding to iommu group 3
[    0.912341] pci 0000:00:08.0: Using iommu direct mapping
[    0.912364] pci 0000:00:12.0: Adding to iommu group 4
[    0.912371] pci 0000:00:12.0: Using iommu direct mapping
[    0.912390] pci 0000:00:13.0: Adding to iommu group 5
[    0.912397] pci 0000:00:13.0: Using iommu direct mapping
[    0.912445] pci 0000:00:14.0: Adding to iommu group 6
[    0.912453] pci 0000:00:14.0: Using iommu direct mapping
[    0.912459] pci 0000:00:14.2: Adding to iommu group 6
[    0.912474] pci 0000:00:14.3: Adding to iommu group 6
[    0.912495] pci 0000:00:15.0: Adding to iommu group 7
[    0.912502] pci 0000:00:15.0: Using iommu direct mapping
[    0.912525] pci 0000:00:16.0: Adding to iommu group 8
[    0.912531] pci 0000:00:16.0: Using iommu direct mapping
[    0.912539] pci 0000:00:16.3: Adding to iommu group 8
[    0.912563] pci 0000:00:1c.0: Adding to iommu group 9
[    0.912564] pci 0000:00:1c.0: DMAR: Use Intel IOMMU bounce page dma_ops
[    0.912604] pci 0000:00:1d.0: Adding to iommu group 10
[    0.912612] pci 0000:00:1d.0: Using iommu direct mapping
[    0.912624] pci 0000:00:1d.4: Adding to iommu group 10
[    0.913715] pci 0000:00:1f.0: Adding to iommu group 11
[    0.913723] pci 0000:00:1f.0: Using iommu direct mapping
[    0.913732] pci 0000:00:1f.3: Adding to iommu group 11
[    0.913738] pci 0000:00:1f.4: Adding to iommu group 11
[    0.913744] pci 0000:00:1f.5: Adding to iommu group 11
[    0.913753] pci 0000:01:00.0: Adding to iommu group 9
[    0.913753] pci 0000:01:00.0: DMAR: Use Intel IOMMU bounce page dma_ops
[    0.913760] pci 0000:02:00.0: Adding to iommu group 9
[    0.913760] pci 0000:02:00.0: DMAR: Use Intel IOMMU bounce page dma_ops
[    0.913765] pci 0000:02:01.0: Adding to iommu group 9
[    0.913766] pci 0000:02:01.0: DMAR: Use Intel IOMMU bounce page dma_ops
[    0.913771] pci 0000:02:02.0: Adding to iommu group 9
[    0.913771] pci 0000:02:02.0: DMAR: Use Intel IOMMU bounce page dma_ops
[    0.913777] pci 0000:02:04.0: Adding to iommu group 9
[    0.913777] pci 0000:02:04.0: DMAR: Use Intel IOMMU bounce page dma_ops
[    0.913787] pci 0000:03:00.0: Adding to iommu group 9
[    0.913788] pci 0000:03:00.0: DMAR: Use Intel IOMMU bounce page dma_ops
[    0.913797] pci 0000:37:00.0: Adding to iommu group 9
[    0.913798] pci 0000:37:00.0: DMAR: Use Intel IOMMU bounce page dma_ops
[    0.913804] pci 0000:6c:00.0: Adding to iommu group 10
[    0.913811] pci 0000:6d:00.0: Adding to iommu group 10
[    0.913910] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    0.913911] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.913912] software IO TLB: mapped [mem 0x44a00000-0x48a00000] (64MB)
[    0.914582] Initialise system trusted keyrings
[    0.914590] Key type blacklist registered
[    0.914627] workingset: timestamp_bits=36 max_order=22 bucket_order=0
[    0.915453] zbud: loaded
[    0.915800] Platform Keyring initialized
[    0.918716] NET: Registered protocol family 38
[    0.918718] Key type asymmetric registered
[    0.918719] Asymmetric key parser 'x509' registered
[    0.918724] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.918754] io scheduler mq-deadline registered
[    0.918755] io scheduler kyber registered
[    0.918770] io scheduler bfq registered
[    0.918814] atomic64_test: passed for x86-64 platform with CX8 and with SSE
[    0.918947] pcieport 0000:00:1c.0: PME: Signaling with IRQ 122
[    0.918965] pcieport 0000:00:1c.0: pciehp: Slot #8 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl+ LLActRep+
[    0.919184] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.919186] IOAPIC[0]: Set routing entry (2-19 -> 0xef -> IRQ 19 Mode:1 Active:1 Dest:1)
[    0.919223] pcieport 0000:00:1d.0: PME: Signaling with IRQ 123
[    0.919363] pcieport 0000:00:1d.4: PME: Signaling with IRQ 124
[    0.919617] pcieport 0000:02:01.0: enabling device (0002 -> 0003)
[    0.919647] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.919648] IOAPIC[0]: Set routing entry (2-17 -> 0xef -> IRQ 17 Mode:1 Active:1 Dest:1)
[    0.919699] pcieport 0000:02:01.0: pciehp: Slot #1 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl+ LLActRep+
[    0.919906] pcieport 0000:02:04.0: enabling device (0002 -> 0003)
[    0.919983] pcieport 0000:02:04.0: pciehp: Slot #4 AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Interlock- NoCompl+ LLActRep+
[    0.920089] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    0.920115] efifb: probing for efifb
[    0.920130] efifb: showing boot graphics
[    0.921577] efifb: framebuffer at 0x50000000, using 9600k, total 9600k
[    0.921577] efifb: mode is 1920x1280x32, linelength=7680, pages=1
[    0.921577] efifb: scrolling: redraw
[    0.921578] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
[    0.921602] fbcon: Deferring console take-over
[    0.921602] fb0: EFI VGA frame buffer device
[    0.921606] intel_idle: MWAIT substates: 0x11142120
[    0.921607] intel_idle: v0.4.1 model 0x8E
[    0.921874] intel_idle: lapic_timer_reliable_states 0xffffffff
[    0.922790] ACPI: AC Adapter [AC] (on-line)
[    0.922833] input: Lid Switch as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0D:00/input/input0
[    0.922852] ACPI: Lid Switch [LID0]
[    0.922882] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input1
[    0.922894] ACPI: Power Button [PBTN]
[    0.922906] input: Sleep Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0E:00/input/input2
[    0.922913] ACPI: Sleep Button [SBTN]
[    0.924551] thermal LNXTHERM:00: registered as thermal_zone0
[    0.924552] ACPI: Thermal Zone [THM] (25 C)
[    0.924702] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    0.925674] serial 0000:00:13.0: enabling device (0000 -> 0002)
[    0.925778] IOAPIC[2]: Set IRTE entry (P:1 FPD:0 Dst_Mode:1 Redir_hint:1 Trig_Mode:0 Dlvry_Mode:0 Avail:0 Vector:EF Dest:00000001 SID:00F7 SQ:0 SVT:1)
[    0.925779] IOAPIC[0]: Set routing entry (2-20 -> 0xef -> IRQ 20 Mode:1 Active:1 Dest:1)
[    0.925901] serial 0000:00:16.3: enabling device (0000 -> 0003)
[    0.947052] 0000:00:16.3: ttyS4 at I/O 0x3060 (irq = 19, base_baud = 115200) is a 16550A
[    0.947703] Non-volatile memory driver v1.3
[    0.947748] Linux agpgart interface v0.103
[    0.949935] tpm_tis MSFT0101:00: 2.0 TPM (device-id 0x0, rev-id 78)
[    0.952910] battery: ACPI: Battery Slot [BAT0] (battery present)
[    0.961861] intel-lpss 0000:00:15.0: enabling device (0000 -> 0002)
[    0.971013] libphy: Fixed MDIO Bus: probed
[    0.971124] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.971126] ehci-pci: EHCI PCI platform driver
[    0.971134] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.971135] ohci-pci: OHCI PCI platform driver
[    0.971140] uhci_hcd: USB Universal Host Controller Interface driver
[    0.971223] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    0.971294] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 1
[    0.972362] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000000009810
[    0.972364] xhci_hcd 0000:00:14.0: cache line size of 64 is not supported
[    0.972491] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.04
[    0.972492] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.972493] usb usb1: Product: xHCI Host Controller
[    0.972494] usb usb1: Manufacturer: Linux 5.4.18-200.fc31.x86_64 xhci-hcd
[    0.972494] usb usb1: SerialNumber: 0000:00:14.0
[    0.972595] hub 1-0:1.0: USB hub found
[    0.972607] hub 1-0:1.0: 12 ports detected
[    0.973370] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    0.973422] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 2
[    0.973423] xhci_hcd 0000:00:14.0: Host supports USB 3.1 Enhanced SuperSpeed
[    0.973441] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.04
[    0.973442] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.973442] usb usb2: Product: xHCI Host Controller
[    0.973443] usb usb2: Manufacturer: Linux 5.4.18-200.fc31.x86_64 xhci-hcd
[    0.973443] usb usb2: SerialNumber: 0000:00:14.0
[    0.973506] hub 2-0:1.0: USB hub found
[    0.973515] hub 2-0:1.0: 6 ports detected
[    0.973719] usb: port power management may be unreliable
[    0.973877] xhci_hcd 0000:37:00.0: xHCI Host Controller
[    0.973929] xhci_hcd 0000:37:00.0: new USB bus registered, assigned bus number 3
[    0.975061] xhci_hcd 0000:37:00.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000200009810
[    0.975220] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.04
[    0.975221] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.975221] usb usb3: Product: xHCI Host Controller
[    0.975222] usb usb3: Manufacturer: Linux 5.4.18-200.fc31.x86_64 xhci-hcd
[    0.975222] usb usb3: SerialNumber: 0000:37:00.0
[    0.975304] hub 3-0:1.0: USB hub found
[    0.975309] hub 3-0:1.0: 2 ports detected
[    0.975684] xhci_hcd 0000:37:00.0: xHCI Host Controller
[    0.975763] xhci_hcd 0000:37:00.0: new USB bus registered, assigned bus number 4
[    0.975764] xhci_hcd 0000:37:00.0: Host supports USB 3.1 Enhanced SuperSpeed
[    0.975783] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.04
[    0.975784] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.975784] usb usb4: Product: xHCI Host Controller
[    0.975785] usb usb4: Manufacturer: Linux 5.4.18-200.fc31.x86_64 xhci-hcd
[    0.975785] usb usb4: SerialNumber: 0000:37:00.0
[    0.975875] hub 4-0:1.0: USB hub found
[    0.975881] hub 4-0:1.0: 2 ports detected
[    0.976165] usbcore: registered new interface driver usbserial_generic
[    0.976167] usbserial: USB Serial support registered for generic
[    0.976179] i8042: PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1
[    0.976179] i8042: PNP: PS/2 appears to have AUX port disabled, if this is incorrect please boot with i8042.nopnp
[    0.976781] i8042: Warning: Keylock active
[    0.977021] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.977093] mousedev: PS/2 mouse device common for all mice
[    0.977323] rtc_cmos 00:02: RTC can wake from S4
[    0.978172] rtc_cmos 00:02: registered as rtc0
[    0.978173] rtc_cmos 00:02: alarms up to one month, y3k, 242 bytes nvram
[    0.978196] device-mapper: uevent: version 1.0.3
[    0.978285] device-mapper: ioctl: 4.41.0-ioctl (2019-09-16) initialised: dm-devel@redhat.com
[    0.978344] intel_pstate: Intel P-state driver initializing
[    0.978846] intel_pstate: HWP enabled
[    0.979013] hidraw: raw HID events driver (C) Jiri Kosina
[    0.979032] usbcore: registered new interface driver usbhid
[    0.979033] usbhid: USB HID core driver
[    0.979079] intel_pmc_core INT33A1:00:  initialized
[    0.979153] drop_monitor: Initializing network drop monitor service
[    0.979195] Initializing XFRM netlink socket
[    0.979272] NET: Registered protocol family 10
[    0.980792] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input3
[    0.982339] Segment Routing with IPv6
[    0.982352] mip6: Mobile IPv6
[    0.982353] NET: Registered protocol family 17
[    0.982989] RAS: Correctable Errors collector initialized.
[    0.983035] microcode: sig=0x806ec, pf=0x80, revision=0xca
[    0.983208] microcode: Microcode Update Driver: v2.2.
[    0.983212] IPI shorthand broadcast: enabled
[    0.983327] ... APIC ID:      00000000 (0)
[    0.983327] ... APIC VERSION: 01060015
[    0.983328] 0000000000000000000000000000000000000000000000000000000000000000
[    0.983329] 0000000000000000000000000000000000000000000000000000000000000000
[    0.983330] 0000000000000000000000000000000000000000000000000000000000000000
[    0.983332] number of MP IRQ sources: 15.
[    0.983332] number of IO-APIC #2 registers: 120.
[    0.983332] testing the IO APIC.......................
[    0.983372] IO APIC #2......
[    0.983372] .... register #00: 02000000
[    0.983372] .......    : physical APIC id: 02
[    0.983372] .......    : Delivery Type: 0
[    0.983373] .......    : LTS          : 0
[    0.983373] .... register #01: 00770020
[    0.983373] .......     : max redirection entries: 77
[    0.983373] .......     : PRQ implemented: 0
[    0.983374] .......     : IO APIC version: 20
[    0.983374] .... register #02: 00000000
[    0.983374] .......     : arbitration: 00
[    0.983374] .... IRQ redirection table:
[    0.983375] IOAPIC 0:
[    0.983397]  pin00, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983415]  pin01, enabled , edge , high, V(01), IRR(0), S(0), remapped, I(0000),  Z(0)
[    0.983439]  pin02, enabled , edge , high, V(02), IRR(0), S(0), remapped, I(0001),  Z(0)
[    0.983456]  pin03, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983474]  pin04, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983497]  pin05, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983520]  pin06, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983543]  pin07, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983560]  pin08, enabled , edge , high, V(08), IRR(0), S(0), remapped, I(0007),  Z(0)
[    0.983579]  pin09, enabled , level, high, V(09), IRR(0), S(0), remapped, I(0008),  Z(0)
[    0.983597]  pin0a, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983621]  pin0b, disabled, edge , high, V(FF), IRR(0), S(0), remapped, I(BFFB),  Z(2)
[    0.983643]  pin0c, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983666]  pin0d, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983690]  pin0e, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983708]  pin0f, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983728]  pin10, enabled , level, low , V(10), IRR(0), S(0), remapped, I(000F),  Z(0)
[    0.983766]  pin11, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983803]  pin12, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983826]  pin13, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983849]  pin14, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983872]  pin15, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983889]  pin16, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983906]  pin17, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983929]  pin18, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983952]  pin19, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983975]  pin1a, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.983999]  pin1b, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984018]  pin1c, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984036]  pin1d, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984060]  pin1e, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984082]  pin1f, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984100]  pin20, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984118]  pin21, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984136]  pin22, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984159]  pin23, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984182]  pin24, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984205]  pin25, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984228]  pin26, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984250]  pin27, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984273]  pin28, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984291]  pin29, disabled, edge , high, V(FB), IRR(0), S(0), remapped, I(BEE3),  Z(2)
[    0.984310]  pin2a, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984334]  pin2b, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984357]  pin2c, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984380]  pin2d, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984397]  pin2e, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984415]  pin2f, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984438]  pin30, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984461]  pin31, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984485]  pin32, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984508]  pin33, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984531]  pin34, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984549]  pin35, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984567]  pin36, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984585]  pin37, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984602]  pin38, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984625]  pin39, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984649]  pin3a, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984673]  pin3b, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984691]  pin3c, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984709]  pin3d, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984729]  pin3e, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984771]  pin3f, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984793]  pin40, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984831]  pin41, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984855]  pin42, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984873]  pin43, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984892]  pin44, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984910]  pin45, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984932]  pin46, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984956]  pin47, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984980]  pin48, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.984999]  pin49, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985017]  pin4a, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985041]  pin4b, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985060]  pin4c, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985077]  pin4d, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985095]  pin4e, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985119]  pin4f, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985142]  pin50, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985166]  pin51, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985185]  pin52, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985203]  pin53, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985225]  pin54, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985249]  pin55, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985272]  pin56, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985295]  pin57, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985318]  pin58, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985335]  pin59, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985353]  pin5a, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985376]  pin5b, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985400]  pin5c, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985418]  pin5d, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985442]  pin5e, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985465]  pin5f, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985487]  pin60, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985511]  pin61, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985535]  pin62, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985559]  pin63, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985581]  pin64, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985599]  pin65, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985618]  pin66, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985642]  pin67, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985665]  pin68, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985688]  pin69, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985705]  pin6a, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985730]  pin6b, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985771]  pin6c, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985804]  pin6d, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985827]  pin6e, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985850]  pin6f, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985873]  pin70, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985896]  pin71, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985915]  pin72, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985933]  pin73, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985956]  pin74, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.985979]  pin75, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.986002]  pin76, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.986025]  pin77, disabled, edge , high, V(00), IRR(0), S(0), physical, D(00), M(0)
[    0.986025] IRQ to pin mappings:
[    0.986026] IRQ0 -> 0:2
[    0.986027] IRQ1 -> 0:1
[    0.986027] IRQ3 -> 0:3
[    0.986028] IRQ4 -> 0:4
[    0.986028] IRQ5 -> 0:5
[    0.986029] IRQ6 -> 0:6
[    0.986029] IRQ7 -> 0:7
[    0.986030] IRQ8 -> 0:8
[    0.986030] IRQ9 -> 0:9
[    0.986031] IRQ10 -> 0:10
[    0.986032] IRQ11 -> 0:11
[    0.986032] IRQ12 -> 0:12
[    0.986033] IRQ13 -> 0:13
[    0.986033] IRQ14 -> 0:14
[    0.986034] IRQ15 -> 0:15
[    0.986034] IRQ16 -> 0:16
[    0.986035] IRQ17 -> 0:17
[    0.986035] IRQ18 -> 0:18
[    0.986036] IRQ19 -> 0:19
[    0.986037] .................................... done.
[    0.986042] AVX2 version of gcm_enc/dec engaged.
[    0.986042] AES CTR mode by8 optimization enabled
[    1.010906] sched_clock: Marking stable (1005297016, 5605563)->(1030913916, -20011337)
[    1.011083] registered taskstats version 1
[    1.011093] Loading compiled-in X.509 certificates
[    1.031648] Loaded X.509 cert 'Fedora kernel signing key: 29b28ec934e49fbfc335d0341def5f4b743c2c18'
[    1.031667] zswap: loaded using pool lzo/zbud
[    1.031889] Key type ._fscrypt registered
[    1.031890] Key type .fscrypt registered
[    1.036983] Key type big_key registered
[    1.039081] Key type encrypted registered
[    1.039996] integrity: Loading X.509 certificate: UEFI:db
[    1.040011] integrity: Loaded X.509 cert 'Dell Inc. UEFI DB: 5ddb772dc880660055ba0bc131886bb630a639e7'
[    1.040011] integrity: Loading X.509 certificate: UEFI:db
[    1.040024] integrity: Loaded X.509 cert 'Microsoft Corporation UEFI CA 2011: 13adbf4309bd82709c8cd54f316ed522988a1bd4'
[    1.040024] integrity: Loading X.509 certificate: UEFI:db
[    1.040035] integrity: Loaded X.509 cert 'Microsoft Windows Production PCA 2011: a92902398e16c49778cd90f99e4f9ae17c55af53'
[    1.041139] ima: Allocated hash algorithm: sha256
[    1.059927] ima: No architecture policies found
[    1.060648] PM:   Magic number: 8:871:588
[    1.060671] pci_express 0000:02:00.0:pcie210: hash matches
[    1.060968] rtc_cmos 00:02: setting system clock to 2020-02-17T15:33:53 UTC (1581953633)
[    1.061785] Freeing unused decrypted memory: 2040K
[    1.062171] Freeing unused kernel image memory: 2552K
[    1.062173] Write protecting the kernel read-only data: 22528k
[    1.062597] Freeing unused kernel image memory: 2016K
[    1.062768] Freeing unused kernel image memory: 1424K
[    1.070011] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    1.070035] rodata_test: all tests were successful
[    1.070038] Run /init as init process

[-- Attachment #3: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Andrew Cooper]

On 17/02/2020 19:19, Jason Andryuk wrote:
> enabling vecOn Tue, Dec 31, 2019 at 5:43 AM Aaron Janse <aaron@ajanse.me> wrote:
>> On Tue, Dec 31, 2019, at 12:27 AM, Andrew Cooper wrote:
>>> Is there any full boot log in the bad case?  Debugging via divination
>>> isn't an effective way to get things done.
>> Agreed. I included some more verbose logs towards the end of the email (typed up by hand).
>>
>> Attached are pictures from a slow-motion video of my laptop booting. Note that I also included a picture of a stack trace that happens immediately before reboot. It doesn't look related, but I wanted to include it anyway.
>>
>> I think the original email should have said "4.8.5" instead of "4.0.5." Regardless, everyone on this mailing list can now see all the boot logs that I've seen.
>>
>> Attaching a serial console seems like it would be difficult to do on this laptop, otherwise I would have sent the logs as a txt file.
> I'm seeing Xen panic: "IO-APIC + timer doesn't work" on a Dell
> Latitude 7200 2-in-1.  Fedora 31 Live USB image boots successfully.
> No way to get serial output.  I manually recreated the output before
> from the vga display.

We have multiple bugs.

First and foremost, Xen seems totally broken when running in ExtINT
mode.  This needs addressing, and ought to be sufficient to let Xen
boot, at which point we can try to figure out why it is trying to fall
back into 486(ish) compatibility mode.

> I tested Linux with intel_iommu=on and that booted successfully.
> Under Xen, this system sets iommu_x2apic_enabled = true, so
> force_iommu is set and iommu=0 cannot disable the iommu.
> fails.  Oh, I can disable x2apic and then disable iommu
>
> x2apic=1 -> failure above
> x2apic=0 iommu=0 -> failure above
> clocksource=acpi -> doesn't help
> clocksource=pit -> hangs after "load tracking window length 1073741824 ns"

None of these are surprising, given that Xen can't make any interrupts
work at all.

> noapic -> BUG in init_bsp_APIC

This is a surprise.  Its clearly a bug in Xen.  (OTOH, I've been
threatening to rip all of that logic out, because there is no such thing
as a 64bit capable system without an integrated APIC.)

> One other thing that might be noteworthy.  Linux only prints ACPI IRQ0
> and IRQ9 used by override where Xen lists IRQ 0, 2 & 9.

Huh - this is supposed to come directly from the ACPI tables, so Linux
and Xen should be using the same source of information.

>
> Below is the re-constructed Xen console output.  The SMBIOS line is
> the first thing displayed on the VGA output.

Yes - it is the first thing printed after vesa_init() which I think is a
manifestation of a previous EFI bug I've reported.  Does booting with
-basevideo help?  (No need to transcribe the output, manually.  Just
need to know if it lets you see the full log.)

>   I skipped the full EFI
> memory map dump since it is quite long.
>
> I've also attached the Linux dmesg output.  Any pointers or
> suggestions are most welcome.

Lets start with getting Xen able to limp along to a full boot.  After
that, we can figure out how to stop it making silly decisions during boot.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Mon, Feb 17, 2020 at 2:46 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>
> On 17/02/2020 19:19, Jason Andryuk wrote:
> > enabling vecOn Tue, Dec 31, 2019 at 5:43 AM Aaron Janse <aaron@ajanse.me> wrote:
> >> On Tue, Dec 31, 2019, at 12:27 AM, Andrew Cooper wrote:
> >>> Is there any full boot log in the bad case?  Debugging via divination
> >>> isn't an effective way to get things done.
> >> Agreed. I included some more verbose logs towards the end of the email (typed up by hand).
> >>
> >> Attached are pictures from a slow-motion video of my laptop booting. Note that I also included a picture of a stack trace that happens immediately before reboot. It doesn't look related, but I wanted to include it anyway.
> >>
> >> I think the original email should have said "4.8.5" instead of "4.0.5." Regardless, everyone on this mailing list can now see all the boot logs that I've seen.
> >>
> >> Attaching a serial console seems like it would be difficult to do on this laptop, otherwise I would have sent the logs as a txt file.
> > I'm seeing Xen panic: "IO-APIC + timer doesn't work" on a Dell
> > Latitude 7200 2-in-1.  Fedora 31 Live USB image boots successfully.
> > No way to get serial output.  I manually recreated the output before
> > from the vga display.
>
> We have multiple bugs.
>
> First and foremost, Xen seems totally broken when running in ExtINT
> mode.  This needs addressing, and ought to be sufficient to let Xen
> boot, at which point we can try to figure out why it is trying to fall
> back into 486(ish) compatibility mode.
>
> > I tested Linux with intel_iommu=on and that booted successfully.
> > Under Xen, this system sets iommu_x2apic_enabled = true, so
> > force_iommu is set and iommu=0 cannot disable the iommu.
> > fails.  Oh, I can disable x2apic and then disable iommu
> >
> > x2apic=1 -> failure above
> > x2apic=0 iommu=0 -> failure above
> > clocksource=acpi -> doesn't help
> > clocksource=pit -> hangs after "load tracking window length 1073741824 ns"
>
> None of these are surprising, given that Xen can't make any interrupts
> work at all.
>
> > noapic -> BUG in init_bsp_APIC
>
> This is a surprise.  Its clearly a bug in Xen.  (OTOH, I've been
> threatening to rip all of that logic out, because there is no such thing
> as a 64bit capable system without an integrated APIC.)

It's a GPF [error_code=0000] at init_bsp_APIC+0x53 which is
   0xffff82d080428f86 <+64>:    je     0xffff82d080428fc9 <init_bsp_APIC+131>
   0xffff82d080428f88 <+66>:    or     $0xff,%al
   0xffff82d080428f8a <+68>:    test   %sil,%sil
   0xffff82d080428f8d <+71>:    je     0xffff82d080428fd8 <init_bsp_APIC+146>
   0xffff82d080428f8f <+73>:    mov    $0x80f,%ecx
   0xffff82d080428f94 <+78>:    mov    $0x0,%edx
   0xffff82d080428f99 <+83>:    wrmsr

RAX is 0x3ff

This is immediately after Xen prints "Switched to APIC driver x2apic_cluster"

> > One other thing that might be noteworthy.  Linux only prints ACPI IRQ0
> > and IRQ9 used by override where Xen lists IRQ 0, 2 & 9.
>
> Huh - this is supposed to come directly from the ACPI tables, so Linux
> and Xen should be using the same source of information.
>
> >
> > Below is the re-constructed Xen console output.  The SMBIOS line is
> > the first thing displayed on the VGA output.
>
> Yes - it is the first thing printed after vesa_init() which I think is a
> manifestation of a previous EFI bug I've reported.  Does booting with
> -basevideo help?  (No need to transcribe the output, manually.  Just
> need to know if it lets you see the full log.)

I'm booting grub->xen.gz so -basevideo isn't directly applicable.  My
attempt at setting a boot entry failed, so I'll have to try that
again.

> >   I skipped the full EFI
> > memory map dump since it is quite long.
> >
> > I've also attached the Linux dmesg output.  Any pointers or
> > suggestions are most welcome.
>
> Lets start with getting Xen able to limp along to a full boot.  After
> that, we can figure out how to stop it making silly decisions during boot.
>
> ~Andrew

Thanks for taking a look.

-Jason

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Andrew Cooper]

On 17/02/2020 20:41, Jason Andryuk wrote:
> On Mon, Feb 17, 2020 at 2:46 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 17/02/2020 19:19, Jason Andryuk wrote:
>>> enabling vecOn Tue, Dec 31, 2019 at 5:43 AM Aaron Janse <aaron@ajanse.me> wrote:
>>>> On Tue, Dec 31, 2019, at 12:27 AM, Andrew Cooper wrote:
>>>>> Is there any full boot log in the bad case?  Debugging via divination
>>>>> isn't an effective way to get things done.
>>>> Agreed. I included some more verbose logs towards the end of the email (typed up by hand).
>>>>
>>>> Attached are pictures from a slow-motion video of my laptop booting. Note that I also included a picture of a stack trace that happens immediately before reboot. It doesn't look related, but I wanted to include it anyway.
>>>>
>>>> I think the original email should have said "4.8.5" instead of "4.0.5." Regardless, everyone on this mailing list can now see all the boot logs that I've seen.
>>>>
>>>> Attaching a serial console seems like it would be difficult to do on this laptop, otherwise I would have sent the logs as a txt file.
>>> I'm seeing Xen panic: "IO-APIC + timer doesn't work" on a Dell
>>> Latitude 7200 2-in-1.  Fedora 31 Live USB image boots successfully.
>>> No way to get serial output.  I manually recreated the output before
>>> from the vga display.
>> We have multiple bugs.
>>
>> First and foremost, Xen seems totally broken when running in ExtINT
>> mode.  This needs addressing, and ought to be sufficient to let Xen
>> boot, at which point we can try to figure out why it is trying to fall
>> back into 486(ish) compatibility mode.
>>
>>> I tested Linux with intel_iommu=on and that booted successfully.
>>> Under Xen, this system sets iommu_x2apic_enabled = true, so
>>> force_iommu is set and iommu=0 cannot disable the iommu.
>>> fails.  Oh, I can disable x2apic and then disable iommu
>>>
>>> x2apic=1 -> failure above
>>> x2apic=0 iommu=0 -> failure above
>>> clocksource=acpi -> doesn't help
>>> clocksource=pit -> hangs after "load tracking window length 1073741824 ns"
>> None of these are surprising, given that Xen can't make any interrupts
>> work at all.
>>
>>> noapic -> BUG in init_bsp_APIC
>> This is a surprise.  Its clearly a bug in Xen.  (OTOH, I've been
>> threatening to rip all of that logic out, because there is no such thing
>> as a 64bit capable system without an integrated APIC.)
> It's a GPF [error_code=0000] at init_bsp_APIC+0x53 which is
>     0xffff82d080428f86 <+64>:    je     0xffff82d080428fc9 <init_bsp_APIC+131>
>     0xffff82d080428f88 <+66>:    or     $0xff,%al
>     0xffff82d080428f8a <+68>:    test   %sil,%sil
>     0xffff82d080428f8d <+71>:    je     0xffff82d080428fd8 <init_bsp_APIC+146>
>     0xffff82d080428f8f <+73>:    mov    $0x80f,%ecx
>     0xffff82d080428f94 <+78>:    mov    $0x0,%edx
>     0xffff82d080428f99 <+83>:    wrmsr
>
> RAX is 0x3ff
>
> This is immediately after Xen prints "Switched to APIC driver x2apic_cluster"

Hmm, in which case it isn't a BUG specifically, but merely a crash. 
0x3ff to SPIV is trying to set reserved bits, so it is no surprise that 
there is a #GP.

In which case this can safely be filed under "even more collateral 
damage from failing to set up any kind of interrupt handling".

>>> One other thing that might be noteworthy.  Linux only prints ACPI IRQ0
>>> and IRQ9 used by override where Xen lists IRQ 0, 2 & 9.
>> Huh - this is supposed to come directly from the ACPI tables, so Linux
>> and Xen should be using the same source of information.
>>
>>> Below is the re-constructed Xen console output.  The SMBIOS line is
>>> the first thing displayed on the VGA output.
>> Yes - it is the first thing printed after vesa_init() which I think is a
>> manifestation of a previous EFI bug I've reported.  Does booting with
>> -basevideo help?  (No need to transcribe the output, manually.  Just
>> need to know if it lets you see the full log.)
> I'm booting grub->xen.gz so -basevideo isn't directly applicable.  My
> attempt at setting a boot entry failed, so I'll have to try that
> again.

Ah ok.  One thing which Xen(.gz) needs to do is to take video details 
from the bootloader rather than trying to figure them out itself.

By default, Xen.gz will try and write into the legacy vga range which 
most likely isn't working in an EFI system.

(As a slight tangent, It is possible to test xen.efi via grub with a 
suitable chainloader stanza, but xen.efi is deficient in enough 
important ways that I'd avoid it unless absolutely necessary.)

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Mon, Feb 17, 2020, 8:22 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>
> On 17/02/2020 20:41, Jason Andryuk wrote:
> > On Mon, Feb 17, 2020 at 2:46 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> >> On 17/02/2020 19:19, Jason Andryuk wrote:
> >>> enabling vecOn Tue, Dec 31, 2019 at 5:43 AM Aaron Janse <aaron@ajanse.me> wrote:
> >>>> On Tue, Dec 31, 2019, at 12:27 AM, Andrew Cooper wrote:
> >>>>> Is there any full boot log in the bad case?  Debugging via divination
> >>>>> isn't an effective way to get things done.
> >>>> Agreed. I included some more verbose logs towards the end of the email (typed up by hand).
> >>>>
> >>>> Attached are pictures from a slow-motion video of my laptop booting. Note that I also included a picture of a stack trace that happens immediately before reboot. It doesn't look related, but I wanted to include it anyway.
> >>>>
> >>>> I think the original email should have said "4.8.5" instead of "4.0.5." Regardless, everyone on this mailing list can now see all the boot logs that I've seen.
> >>>>
> >>>> Attaching a serial console seems like it would be difficult to do on this laptop, otherwise I would have sent the logs as a txt file.
> >>> I'm seeing Xen panic: "IO-APIC + timer doesn't work" on a Dell
> >>> Latitude 7200 2-in-1.  Fedora 31 Live USB image boots successfully.
> >>> No way to get serial output.  I manually recreated the output before
> >>> from the vga display.
> >> We have multiple bugs.
> >>
> >> First and foremost, Xen seems totally broken when running in ExtINT
> >> mode.  This needs addressing, and ought to be sufficient to let Xen
> >> boot, at which point we can try to figure out why it is trying to fall
> >> back into 486(ish) compatibility mode.

Xen has "enabled ExtINT on CPU#0" while linux has "masked ExtINT on
CPU#0" so linux isn't using ExtINT?

I copy and pasted the linux setup_local_APIC() into Xen and then
massaged it into compiling,  Now Xen reports masked ExtINT, but still
fails to enable the timer.

> >>> I tested Linux with intel_iommu=on and that booted successfully.
> >>> Under Xen, this system sets iommu_x2apic_enabled = true, so
> >>> force_iommu is set and iommu=0 cannot disable the iommu.
> >>> fails.  Oh, I can disable x2apic and then disable iommu
> >>>
> >>> x2apic=1 -> failure above
> >>> x2apic=0 iommu=0 -> failure above
> >>> clocksource=acpi -> doesn't help
> >>> clocksource=pit -> hangs after "load tracking window length 1073741824 ns"
> >> None of these are surprising, given that Xen can't make any interrupts
> >> work at all.
> >>
> >>> noapic -> BUG in init_bsp_APIC
> >> This is a surprise.  Its clearly a bug in Xen.  (OTOH, I've been
> >> threatening to rip all of that logic out, because there is no such thing
> >> as a 64bit capable system without an integrated APIC.)
> > It's a GPF [error_code=0000] at init_bsp_APIC+0x53 which is
> >     0xffff82d080428f86 <+64>:    je     0xffff82d080428fc9 <init_bsp_APIC+131>
> >     0xffff82d080428f88 <+66>:    or     $0xff,%al
> >     0xffff82d080428f8a <+68>:    test   %sil,%sil
> >     0xffff82d080428f8d <+71>:    je     0xffff82d080428fd8 <init_bsp_APIC+146>
> >     0xffff82d080428f8f <+73>:    mov    $0x80f,%ecx
> >     0xffff82d080428f94 <+78>:    mov    $0x0,%edx
> >     0xffff82d080428f99 <+83>:    wrmsr
> >
> > RAX is 0x3ff
> >
> > This is immediately after Xen prints "Switched to APIC driver x2apic_cluster"
>
> Hmm, in which case it isn't a BUG specifically, but merely a crash.
> 0x3ff to SPIV is trying to set reserved bits, so it is no surprise that
> there is a #GP.

Yeah, I used the wrong word.  There was a backtrace and it rebooted
quickly, so I didn't have details when I wrote the first email.  I
re-ran afterward to capture the info.

> In which case this can safely be filed under "even more collateral
> damage from failing to set up any kind of interrupt handling".
>
> >>> One other thing that might be noteworthy.  Linux only prints ACPI IRQ0
> >>> and IRQ9 used by override where Xen lists IRQ 0, 2 & 9.
> >> Huh - this is supposed to come directly from the ACPI tables, so Linux
> >> and Xen should be using the same source of information.
> >>
> >>> Below is the re-constructed Xen console output.  The SMBIOS line is
> >>> the first thing displayed on the VGA output.
> >> Yes - it is the first thing printed after vesa_init() which I think is a
> >> manifestation of a previous EFI bug I've reported.  Does booting with
> >> -basevideo help?  (No need to transcribe the output, manually.  Just
> >> need to know if it lets you see the full log.)
> > I'm booting grub->xen.gz so -basevideo isn't directly applicable.  My
> > attempt at setting a boot entry failed, so I'll have to try that
> > again.
>
> Ah ok.  One thing which Xen(.gz) needs to do is to take video details
> from the bootloader rather than trying to figure them out itself.
>
> By default, Xen.gz will try and write into the legacy vga range which
> most likely isn't working in an EFI system.
>
> (As a slight tangent, It is possible to test xen.efi via grub with a
> suitable chainloader stanza, but xen.efi is deficient in enough
> important ways that I'd avoid it unless absolutely necessary.)

I think I tried chainloader at some point and received an "Unsupported
relocation type" error.

This Dell doesn't want to boot my xen.efi.  After selecting a boot
entry, there is a 3-4 second pause and then EFI prints  "Press
F1/VolumeUp key to retry boot."

-Jason

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Andrew Cooper]

On 18/02/2020 18:43, Jason Andryuk wrote:
> On Mon, Feb 17, 2020, 8:22 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 17/02/2020 20:41, Jason Andryuk wrote:
>>> On Mon, Feb 17, 2020 at 2:46 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>>> On 17/02/2020 19:19, Jason Andryuk wrote:
>>>>> enabling vecOn Tue, Dec 31, 2019 at 5:43 AM Aaron Janse <aaron@ajanse.me> wrote:
>>>>>> On Tue, Dec 31, 2019, at 12:27 AM, Andrew Cooper wrote:
>>>>>>> Is there any full boot log in the bad case?  Debugging via divination
>>>>>>> isn't an effective way to get things done.
>>>>>> Agreed. I included some more verbose logs towards the end of the email (typed up by hand).
>>>>>>
>>>>>> Attached are pictures from a slow-motion video of my laptop booting. Note that I also included a picture of a stack trace that happens immediately before reboot. It doesn't look related, but I wanted to include it anyway.
>>>>>>
>>>>>> I think the original email should have said "4.8.5" instead of "4.0.5." Regardless, everyone on this mailing list can now see all the boot logs that I've seen.
>>>>>>
>>>>>> Attaching a serial console seems like it would be difficult to do on this laptop, otherwise I would have sent the logs as a txt file.
>>>>> I'm seeing Xen panic: "IO-APIC + timer doesn't work" on a Dell
>>>>> Latitude 7200 2-in-1.  Fedora 31 Live USB image boots successfully.
>>>>> No way to get serial output.  I manually recreated the output before
>>>>> from the vga display.
>>>> We have multiple bugs.
>>>>
>>>> First and foremost, Xen seems totally broken when running in ExtINT
>>>> mode.  This needs addressing, and ought to be sufficient to let Xen
>>>> boot, at which point we can try to figure out why it is trying to fall
>>>> back into 486(ish) compatibility mode.
> Xen has "enabled ExtINT on CPU#0" while linux has "masked ExtINT on
> CPU#0" so linux isn't using ExtINT?

It would appear not.  Even more concerningly, on my Kabylake box,

# xl dmesg | grep ExtINT
(XEN) enabled ExtINT on CPU#0
(XEN) masked ExtINT on CPU#1
(XEN) masked ExtINT on CPU#2
(XEN) masked ExtINT on CPU#3
(XEN) masked ExtINT on CPU#4
(XEN) masked ExtINT on CPU#5
(XEN) masked ExtINT on CPU#6
(XEN) masked ExtINT on CPU#7

which at first glance suggests that we have something asymmetric being
set up.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jan Beulich]

On 18.02.2020 22:45, Andrew Cooper wrote:
> On 18/02/2020 18:43, Jason Andryuk wrote:
>> On Mon, Feb 17, 2020, 8:22 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>> On 17/02/2020 20:41, Jason Andryuk wrote:
>>>> On Mon, Feb 17, 2020 at 2:46 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>>>> On 17/02/2020 19:19, Jason Andryuk wrote:
>>>>>> enabling vecOn Tue, Dec 31, 2019 at 5:43 AM Aaron Janse <aaron@ajanse.me> wrote:
>>>>>>> On Tue, Dec 31, 2019, at 12:27 AM, Andrew Cooper wrote:
>>>>>>>> Is there any full boot log in the bad case?  Debugging via divination
>>>>>>>> isn't an effective way to get things done.
>>>>>>> Agreed. I included some more verbose logs towards the end of the email (typed up by hand).
>>>>>>>
>>>>>>> Attached are pictures from a slow-motion video of my laptop booting. Note that I also included a picture of a stack trace that happens immediately before reboot. It doesn't look related, but I wanted to include it anyway.
>>>>>>>
>>>>>>> I think the original email should have said "4.8.5" instead of "4.0.5." Regardless, everyone on this mailing list can now see all the boot logs that I've seen.
>>>>>>>
>>>>>>> Attaching a serial console seems like it would be difficult to do on this laptop, otherwise I would have sent the logs as a txt file.
>>>>>> I'm seeing Xen panic: "IO-APIC + timer doesn't work" on a Dell
>>>>>> Latitude 7200 2-in-1.  Fedora 31 Live USB image boots successfully.
>>>>>> No way to get serial output.  I manually recreated the output before
>>>>>> from the vga display.
>>>>> We have multiple bugs.
>>>>>
>>>>> First and foremost, Xen seems totally broken when running in ExtINT
>>>>> mode.  This needs addressing, and ought to be sufficient to let Xen
>>>>> boot, at which point we can try to figure out why it is trying to fall
>>>>> back into 486(ish) compatibility mode.
>> Xen has "enabled ExtINT on CPU#0" while linux has "masked ExtINT on
>> CPU#0" so linux isn't using ExtINT?
> 
> It would appear not.  Even more concerningly, on my Kabylake box,
> 
> # xl dmesg | grep ExtINT
> (XEN) enabled ExtINT on CPU#0
> (XEN) masked ExtINT on CPU#1
> (XEN) masked ExtINT on CPU#2
> (XEN) masked ExtINT on CPU#3
> (XEN) masked ExtINT on CPU#4
> (XEN) masked ExtINT on CPU#5
> (XEN) masked ExtINT on CPU#6
> (XEN) masked ExtINT on CPU#7
> 
> which at first glance suggests that we have something asymmetric being
> set up.

That's perfectly normal - ExtINT may be enabled on just one CPU,
and that's CPU0 in our case (until such time that we would want
to be able to offline CPU0).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Wed, Feb 19, 2020 at 3:25 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 18.02.2020 22:45, Andrew Cooper wrote:
> > On 18/02/2020 18:43, Jason Andryuk wrote:
> >> On Mon, Feb 17, 2020, 8:22 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> >>> On 17/02/2020 20:41, Jason Andryuk wrote:
> >>>> On Mon, Feb 17, 2020 at 2:46 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> >>>>> We have multiple bugs.
> >>>>>
> >>>>> First and foremost, Xen seems totally broken when running in ExtINT
> >>>>> mode.  This needs addressing, and ought to be sufficient to let Xen
> >>>>> boot, at which point we can try to figure out why it is trying to fall
> >>>>> back into 486(ish) compatibility mode.
> >> Xen has "enabled ExtINT on CPU#0" while linux has "masked ExtINT on
> >> CPU#0" so linux isn't using ExtINT?
> >
> > It would appear not.  Even more concerningly, on my Kabylake box,
> >
> > # xl dmesg | grep ExtINT
> > (XEN) enabled ExtINT on CPU#0
> > (XEN) masked ExtINT on CPU#1
> > (XEN) masked ExtINT on CPU#2
> > (XEN) masked ExtINT on CPU#3
> > (XEN) masked ExtINT on CPU#4
> > (XEN) masked ExtINT on CPU#5
> > (XEN) masked ExtINT on CPU#6
> > (XEN) masked ExtINT on CPU#7
> >
> > which at first glance suggests that we have something asymmetric being
> > set up.
>
> That's perfectly normal - ExtINT may be enabled on just one CPU,
> and that's CPU0 in our case (until such time that we would want
> to be able to offline CPU0).

Thanks, Jan.  Linux prints masked ExtINT for all 8 CPU threads.

I inserted __print_IO_APIC() before the "IO-APIC + timer doesn't work" panic.

Using vector-based indexing
IRQ to ping mappings:
IRQ240 -> 0:2

where Linux prints
IRQ0 -> 0:2

That may just be the difference between Xen printing the Vector vs.
Linux printing the IRQ number.

Any pointers to what I should investigate?

Regards,
Jason

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Wed, Mar 4, 2020 at 11:06 AM Jason Andryuk <jandryuk@gmail.com> wrote:
>
> On Wed, Feb 19, 2020 at 3:25 AM Jan Beulich <jbeulich@suse.com> wrote:
> >
> > On 18.02.2020 22:45, Andrew Cooper wrote:
> > > On 18/02/2020 18:43, Jason Andryuk wrote:
> > >> On Mon, Feb 17, 2020, 8:22 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> > >>> On 17/02/2020 20:41, Jason Andryuk wrote:
> > >>>> On Mon, Feb 17, 2020 at 2:46 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> > >>>>> We have multiple bugs.
> > >>>>>
> > >>>>> First and foremost, Xen seems totally broken when running in ExtINT
> > >>>>> mode.  This needs addressing, and ought to be sufficient to let Xen
> > >>>>> boot, at which point we can try to figure out why it is trying to fall
> > >>>>> back into 486(ish) compatibility mode.
> > >> Xen has "enabled ExtINT on CPU#0" while linux has "masked ExtINT on
> > >> CPU#0" so linux isn't using ExtINT?
> > >
> > > It would appear not.  Even more concerningly, on my Kabylake box,
> > >
> > > # xl dmesg | grep ExtINT
> > > (XEN) enabled ExtINT on CPU#0
> > > (XEN) masked ExtINT on CPU#1
> > > (XEN) masked ExtINT on CPU#2
> > > (XEN) masked ExtINT on CPU#3
> > > (XEN) masked ExtINT on CPU#4
> > > (XEN) masked ExtINT on CPU#5
> > > (XEN) masked ExtINT on CPU#6
> > > (XEN) masked ExtINT on CPU#7
> > >
> > > which at first glance suggests that we have something asymmetric being
> > > set up.
> >
> > That's perfectly normal - ExtINT may be enabled on just one CPU,
> > and that's CPU0 in our case (until such time that we would want
> > to be able to offline CPU0).
>
> Thanks, Jan.  Linux prints masked ExtINT for all 8 CPU threads.
>
> I inserted __print_IO_APIC() before the "IO-APIC + timer doesn't work" panic.
>
> Using vector-based indexing
> IRQ to ping mappings:
> IRQ240 -> 0:2
>
> where Linux prints
> IRQ0 -> 0:2
>
> That may just be the difference between Xen printing the Vector vs.
> Linux printing the IRQ number.
>
> Any pointers to what I should investigate?

I got it to boot past "IO-APIC + timer doesn't work".  I programmed
the HPET to provide a periodic timer in hpet_resume() on T0.  When I
actually got it programmed properly, it worked to increment
pit0_ticks.  I also made timer_interrupt() unconditionally
pit0_ticks++ though that may not matter.

Now it panics in pv_destroy_gdt() when it fails "ASSERT(v == current
|| !vcpu_cpu_dirty(v));" when building dom0.  I haven't investigated
that yet.

Regards,
Jason

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Tue, Mar 17, 2020 at 9:48 AM Jason Andryuk <jandryuk@gmail.com> wrote:
> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
> actually got it programmed properly, it worked to increment
> pit0_ticks.  I also made timer_interrupt() unconditionally
> pit0_ticks++ though that may not matter.

Also, HPET_CFG_LEGACY is enabled for the HPET.

Regards,
Jason

> Now it panics in pv_destroy_gdt() when it fails "ASSERT(v == current
> || !vcpu_cpu_dirty(v));" when building dom0.  I haven't investigated
> that yet.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jan Beulich]

On 17.03.2020 15:08, Jason Andryuk wrote:
> On Tue, Mar 17, 2020 at 9:48 AM Jason Andryuk <jandryuk@gmail.com> wrote:
>> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
>> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
>> actually got it programmed properly, it worked to increment
>> pit0_ticks.  I also made timer_interrupt() unconditionally
>> pit0_ticks++ though that may not matter.
> 
> Also, HPET_CFG_LEGACY is enabled for the HPET.

Which we clear in hpet_resume(), much like Linux does in
hpet_enable().

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jan Beulich]

On 17.03.2020 14:48, Jason Andryuk wrote:
> On Wed, Mar 4, 2020 at 11:06 AM Jason Andryuk <jandryuk@gmail.com> wrote:
>>
>> On Wed, Feb 19, 2020 at 3:25 AM Jan Beulich <jbeulich@suse.com> wrote:
>>>
>>> On 18.02.2020 22:45, Andrew Cooper wrote:
>>>> On 18/02/2020 18:43, Jason Andryuk wrote:
>>>>> On Mon, Feb 17, 2020, 8:22 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>>>>> On 17/02/2020 20:41, Jason Andryuk wrote:
>>>>>>> On Mon, Feb 17, 2020 at 2:46 PM Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>>>>>>> We have multiple bugs.
>>>>>>>>
>>>>>>>> First and foremost, Xen seems totally broken when running in ExtINT
>>>>>>>> mode.  This needs addressing, and ought to be sufficient to let Xen
>>>>>>>> boot, at which point we can try to figure out why it is trying to fall
>>>>>>>> back into 486(ish) compatibility mode.
>>>>> Xen has "enabled ExtINT on CPU#0" while linux has "masked ExtINT on
>>>>> CPU#0" so linux isn't using ExtINT?
>>>>
>>>> It would appear not.  Even more concerningly, on my Kabylake box,
>>>>
>>>> # xl dmesg | grep ExtINT
>>>> (XEN) enabled ExtINT on CPU#0
>>>> (XEN) masked ExtINT on CPU#1
>>>> (XEN) masked ExtINT on CPU#2
>>>> (XEN) masked ExtINT on CPU#3
>>>> (XEN) masked ExtINT on CPU#4
>>>> (XEN) masked ExtINT on CPU#5
>>>> (XEN) masked ExtINT on CPU#6
>>>> (XEN) masked ExtINT on CPU#7
>>>>
>>>> which at first glance suggests that we have something asymmetric being
>>>> set up.
>>>
>>> That's perfectly normal - ExtINT may be enabled on just one CPU,
>>> and that's CPU0 in our case (until such time that we would want
>>> to be able to offline CPU0).
>>
>> Thanks, Jan.  Linux prints masked ExtINT for all 8 CPU threads.
>>
>> I inserted __print_IO_APIC() before the "IO-APIC + timer doesn't work" panic.
>>
>> Using vector-based indexing
>> IRQ to ping mappings:
>> IRQ240 -> 0:2
>>
>> where Linux prints
>> IRQ0 -> 0:2
>>
>> That may just be the difference between Xen printing the Vector vs.
>> Linux printing the IRQ number.
>>
>> Any pointers to what I should investigate?
> 
> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
> actually got it programmed properly, it worked to increment
> pit0_ticks.  I also made timer_interrupt() unconditionally
> pit0_ticks++ though that may not matter.

Hmm, at the first glance I would imply the system gets handed to Xen
with a HPET state that we don't (and probably also shouldn't) expect.
Could you provide HPET_CFG as well as all HPET_Tn_CFG and
HPET_Tn_ROUTE values as hpet_resume() finds them before doing any
adjustments to them? What are the components / parties involved in
getting Xen loaded and started?

> Now it panics in pv_destroy_gdt() when it fails "ASSERT(v == current
> || !vcpu_cpu_dirty(v));" when building dom0.  I haven't investigated
> that yet.

This would seem entirely unrelated to me.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jan Beulich]

On 17.03.2020 15:08, Jan Beulich wrote:
> On 17.03.2020 14:48, Jason Andryuk wrote:
>> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
>> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
>> actually got it programmed properly, it worked to increment
>> pit0_ticks.  I also made timer_interrupt() unconditionally
>> pit0_ticks++ though that may not matter.
> 
> Hmm, at the first glance I would imply the system gets handed to Xen
> with a HPET state that we don't (and probably also shouldn't) expect.
> Could you provide HPET_CFG as well as all HPET_Tn_CFG and
> HPET_Tn_ROUTE values as hpet_resume() finds them before doing any
> adjustments to them? What are the components / parties involved in
> getting Xen loaded and started?

Of course much depends on what exactly you mean you've done to
the HPET by saying "I programmed the HPET to provide ...".

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On 17.03.2020 15:08, Jan Beulich wrote:
>On 17.03.2020 15:08, Jan Beulich wrote:
>> On 17.03.2020 14:48, Jason Andryuk wrote:
>>> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
>>> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
>>> actually got it programmed properly, it worked to increment
>>> pit0_ticks.  I also made timer_interrupt() unconditionally
>>> pit0_ticks++ though that may not matter.
>> 
>> Hmm, at the first glance I would imply the system gets handed to Xen
>> with a HPET state that we don't (and probably also shouldn't) expect.
>> Could you provide HPET_CFG as well as all HPET_Tn_CFG and
>> HPET_Tn_ROUTE values as hpet_resume() finds them before doing any
>> adjustments to them? What are the components / parties involved in
>> getting Xen loaded and started?
>
>Of course much depends on what exactly you mean you've done to
>the HPET by saying "I programmed the HPET to provide ...".

Below is the diff.  It was messier and I tidied it up some.

It's mainly the change to hpet_resume() to mimic Linux's legacy HPET
setup on T0.  It turns on HPET_CFG_LEGACY to ensure the timer interrupt
is running.  And it also includes the printing of the initial HPET
config:
HPET_CFG 00000001
HPET_T0_CFG 00008030
HPET_T0_ROUTE 0000016c
HPET_T1_CFG 00008000
HPET_T1_ROUTE 00000000
HPET_T2_CFG 00008000
HPET_T2_ROUTE 00000000
HPET_T3_CFG 00008000
HPET_T3_ROUTE 00000000
HPET_T4_CFG 0000c000
HPET_T4_ROUTE 00000000
HPET_T5_CFG 0000c000
HPET_T5_ROUTE 00000000
HPET_T6_CFG 0000c000
HPET_T6_ROUTE 00000000
HPET_T7_CFG 0000c000
HPET_T7_ROUTE 00000000

Other changes are to try to prevent Xen from clobbering T0 as a periodic
timer.  I had some printks and didn't see Xen call any of them though.

Regards,
Jason

diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index 86929b9ba1..f39aafda7d 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -585,16 +585,27 @@ void __init hpet_broadcast_init(void)
             pv_rtc_handler = handle_rtc_once;
     }
 
+    printk(XENLOG_INFO "%s cfg %d\n", __func__, cfg);
     hpet_write32(cfg, HPET_CFG);
 
     for ( i = 0; i < n; i++ )
     {
-        if ( i == 0 && (cfg & HPET_CFG_LEGACY) )
+        printk(XENLOG_INFO "hpet cfg %d legacy %d\n", i, cfg & HPET_CFG_LEGACY);
+        if ( i == 1 && (cfg & HPET_CFG_LEGACY) )
         {
             /* set HPET T0 as oneshot */
-            cfg = hpet_read32(HPET_Tn_CFG(0));
+            cfg = hpet_read32(HPET_Tn_CFG(1));
             cfg &= ~(HPET_TN_LEVEL | HPET_TN_PERIODIC);
             cfg |= HPET_TN_ENABLE | HPET_TN_32BIT;
+            hpet_write32(cfg, HPET_Tn_CFG(1));
+        }
+
+        if ( i == 0 && (cfg & HPET_CFG_LEGACY) )
+        {
+            /* set HPET T0 as periodic */
+            cfg = hpet_read32(HPET_Tn_CFG(0));
+            cfg |= (HPET_TN_LEVEL | HPET_TN_PERIODIC);
+            cfg |= HPET_TN_ENABLE | HPET_TN_32BIT;
             hpet_write32(cfg, HPET_Tn_CFG(0));
         }
 
@@ -645,6 +656,7 @@ void hpet_broadcast_resume(void)
         n = 1;
     }
 
+    printk(XENLOG_INFO "%s cfg %d\n", __func__, cfg);
     hpet_write32(cfg, HPET_CFG);
 
     for ( i = 0; i < n; i++ )
@@ -652,6 +664,7 @@ void hpet_broadcast_resume(void)
         if ( hpet_events[i].msi.irq >= 0 )
             __hpet_setup_msi_irq(irq_to_desc(hpet_events[i].msi.irq));
 
+	if (i != 0) {
         /* set HPET Tn as oneshot */
         cfg = hpet_read32(HPET_Tn_CFG(hpet_events[i].idx));
         cfg &= ~(HPET_TN_LEVEL | HPET_TN_PERIODIC);
@@ -659,6 +672,7 @@ void hpet_broadcast_resume(void)
         if ( !(hpet_events[i].flags & HPET_EVT_LEGACY) )
             cfg |= HPET_TN_FSB;
         hpet_write32(cfg, HPET_Tn_CFG(hpet_events[i].idx));
+	}
 
         hpet_events[i].next_event = STIME_MAX;
     }
@@ -684,6 +698,7 @@ void hpet_disable_legacy_broadcast(void)
     /* Stop HPET legacy interrupts */
     cfg = hpet_read32(HPET_CFG);
     cfg &= ~HPET_CFG_LEGACY;
+    printk(XENLOG_INFO "%s cfg %d\n", __func__, cfg);
     hpet_write32(cfg, HPET_CFG);
 
     spin_unlock_irqrestore(&hpet_events->lock, flags);
@@ -759,6 +774,7 @@ int hpet_legacy_irq_tick(void)
          (hpet_events->flags & (HPET_EVT_DISABLE|HPET_EVT_LEGACY)) !=
          HPET_EVT_LEGACY )
         return 0;
+
     hpet_events->event_handler(hpet_events);
     return 1;
 }
@@ -804,6 +820,8 @@ u64 __init hpet_setup(void)
     return hpet_rate + (last * 2 > hpet_period);
 }
 
+#include <asm/delay.h>
+
 void hpet_resume(u32 *boot_cfg)
 {
     static u32 system_reset_latch;
@@ -815,6 +833,7 @@ void hpet_resume(u32 *boot_cfg)
     system_reset_latch = system_reset_counter;
 
     cfg = hpet_read32(HPET_CFG);
+    printk(XENLOG_INFO "%s HPET_CFG %08x\n", __func__, cfg);
     if ( boot_cfg )
         *boot_cfg = cfg;
     cfg &= ~(HPET_CFG_ENABLE | HPET_CFG_LEGACY);
@@ -825,13 +844,18 @@ void hpet_resume(u32 *boot_cfg)
                cfg);
         cfg = 0;
     }
+    printk(XENLOG_INFO "%s cfg %d\n", __func__, cfg);
     hpet_write32(cfg, HPET_CFG);
 
     hpet_id = hpet_read32(HPET_ID);
     last = (hpet_id & HPET_ID_NUMBER) >> HPET_ID_NUMBER_SHIFT;
     for ( i = 0; i <= last; ++i )
     {
+        u32 tmp;
         cfg = hpet_read32(HPET_Tn_CFG(i));
+        printk(XENLOG_INFO "%s HPET_T%d_CFG %08x\n", __func__, i, cfg);
+        tmp = hpet_read32(HPET_Tn_ROUTE(i));
+        printk(XENLOG_INFO "%s HPET_T%d_ROUTE %08x\n", __func__, i, tmp);
         if ( boot_cfg )
             boot_cfg[i + 1] = cfg;
         cfg &= ~HPET_TN_ENABLE;
@@ -842,11 +866,34 @@ void hpet_resume(u32 *boot_cfg)
                    cfg & HPET_TN_RESERVED, i);
             cfg &= ~HPET_TN_RESERVED;
         }
+        if (i == 0) {
+            cfg |= HPET_TN_ENABLE | HPET_TN_PERIODIC | HPET_TN_SETVAL |
+                   HPET_TN_32BIT;
+        }
         hpet_write32(cfg, HPET_Tn_CFG(i));
+        if (i == 0) {
+#define NSEC_PER_SEC    1000000000L
+            uint64_t delta;
+            unsigned int now;
+            unsigned int cmp;
+            u64 hpet_rate = hpet_setup();
+            uint32_t mult = div_sc((unsigned long)hpet_rate,
+                                     1000000000ul, 32);
+            uint32_t shift = 32;
+            printk(XENLOG_INFO "hpet mult %d shift %d\n", mult, shift);
+            delta = ((uint64_t)(NSEC_PER_SEC / HZ)) * mult;
+            delta >>= shift;
+            now = hpet_read32(HPET_COUNTER);
+            cmp = now + (unsigned int)delta;
+            hpet_write32(cmp, HPET_Tn_CMP(i));
+            udelay(1);
+            hpet_write32(delta, HPET_Tn_CMP(i));
+        }
     }
 
     cfg = hpet_read32(HPET_CFG);
-    cfg |= HPET_CFG_ENABLE;
+    cfg |= HPET_CFG_ENABLE | HPET_CFG_LEGACY;
+    printk(XENLOG_INFO "%s cfg %d\n", __func__, cfg);
     hpet_write32(cfg, HPET_CFG);
 }
 
@@ -862,6 +909,7 @@ void hpet_disable(void)
         return;
     }
 
+    printk(XENLOG_INFO "%s cfg %d\n", __func__, *hpet_boot_cfg);
     hpet_write32(*hpet_boot_cfg & ~HPET_CFG_ENABLE, HPET_CFG);
 
     id = hpet_read32(HPET_ID);
@@ -869,5 +917,6 @@ void hpet_disable(void)
         hpet_write32(hpet_boot_cfg[i + 1], HPET_Tn_CFG(i));
 
     if ( *hpet_boot_cfg & HPET_CFG_ENABLE )
+        printk(XENLOG_INFO "%s cfg %d\n", __func__, *hpet_boot_cfg);
         hpet_write32(*hpet_boot_cfg, HPET_CFG);
 }

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Tue, Mar 17, 2020 at 11:23 AM Jason Andryuk <jandryuk@gmail.com> wrote:
>
> On 17.03.2020 15:08, Jan Beulich wrote:
> >On 17.03.2020 15:08, Jan Beulich wrote:
> >> On 17.03.2020 14:48, Jason Andryuk wrote:
> >>> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
> >>> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
> >>> actually got it programmed properly, it worked to increment
> >>> pit0_ticks.  I also made timer_interrupt() unconditionally
> >>> pit0_ticks++ though that may not matter.
> >>
> >> Hmm, at the first glance I would imply the system gets handed to Xen
> >> with a HPET state that we don't (and probably also shouldn't) expect.
> >> Could you provide HPET_CFG as well as all HPET_Tn_CFG and
> >> HPET_Tn_ROUTE values as hpet_resume() finds them before doing any
> >> adjustments to them? What are the components / parties involved in
> >> getting Xen loaded and started?

I forgot to mention the boot sequence:
EFI -> grub2-efi -> xen.gz
grub2 is using multiboot2 & module2 commands.

Thanks for taking a look.

Regards,
Jason

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jan Beulich]

On 17.03.2020 16:23, Jason Andryuk wrote:
> On 17.03.2020 15:08, Jan Beulich wrote:
>> On 17.03.2020 15:08, Jan Beulich wrote:
>>> On 17.03.2020 14:48, Jason Andryuk wrote:
>>>> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
>>>> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
>>>> actually got it programmed properly, it worked to increment
>>>> pit0_ticks.  I also made timer_interrupt() unconditionally
>>>> pit0_ticks++ though that may not matter.
>>>
>>> Hmm, at the first glance I would imply the system gets handed to Xen
>>> with a HPET state that we don't (and probably also shouldn't) expect.
>>> Could you provide HPET_CFG as well as all HPET_Tn_CFG and
>>> HPET_Tn_ROUTE values as hpet_resume() finds them before doing any
>>> adjustments to them? What are the components / parties involved in
>>> getting Xen loaded and started?
>>
>> Of course much depends on what exactly you mean you've done to
>> the HPET by saying "I programmed the HPET to provide ...".
> 
> Below is the diff.  It was messier and I tidied it up some.
> 
> It's mainly the change to hpet_resume() to mimic Linux's legacy HPET
> setup on T0.  It turns on HPET_CFG_LEGACY to ensure the timer interrupt
> is running.  And it also includes the printing of the initial HPET
> config:
> HPET_CFG 00000001
> HPET_T0_CFG 00008030
> HPET_T0_ROUTE 0000016c
> HPET_T1_CFG 00008000
> HPET_T1_ROUTE 00000000
> HPET_T2_CFG 00008000
> HPET_T2_ROUTE 00000000
> HPET_T3_CFG 00008000
> HPET_T3_ROUTE 00000000
> HPET_T4_CFG 0000c000
> HPET_T4_ROUTE 00000000
> HPET_T5_CFG 0000c000
> HPET_T5_ROUTE 00000000
> HPET_T6_CFG 0000c000
> HPET_T6_ROUTE 00000000
> HPET_T7_CFG 0000c000
> HPET_T7_ROUTE 00000000
> 
> Other changes are to try to prevent Xen from clobbering T0 as a periodic
> timer.

Why "clobbering"? According to the values above T0 is neither enabled
nor set to periodic.

> --- a/xen/arch/x86/hpet.c
> +++ b/xen/arch/x86/hpet.c
> @@ -585,16 +585,27 @@ void __init hpet_broadcast_init(void)
>               pv_rtc_handler = handle_rtc_once;
>       }
>   
> +    printk(XENLOG_INFO "%s cfg %d\n", __func__, cfg);
>       hpet_write32(cfg, HPET_CFG);
>   
>       for ( i = 0; i < n; i++ )
>       {
> -        if ( i == 0 && (cfg & HPET_CFG_LEGACY) )
> +        printk(XENLOG_INFO "hpet cfg %d legacy %d\n", i, cfg & HPET_CFG_LEGACY);
> +        if ( i == 1 && (cfg & HPET_CFG_LEGACY) )
>           {
>               /* set HPET T0 as oneshot */
> -            cfg = hpet_read32(HPET_Tn_CFG(0));
> +            cfg = hpet_read32(HPET_Tn_CFG(1));
>               cfg &= ~(HPET_TN_LEVEL | HPET_TN_PERIODIC);
>               cfg |= HPET_TN_ENABLE | HPET_TN_32BIT;
> +            hpet_write32(cfg, HPET_Tn_CFG(1));
> +        }
> +
> +        if ( i == 0 && (cfg & HPET_CFG_LEGACY) )
> +        {
> +            /* set HPET T0 as periodic */
> +            cfg = hpet_read32(HPET_Tn_CFG(0));
> +            cfg |= (HPET_TN_LEVEL | HPET_TN_PERIODIC);

A change like this of course won't be acceptable outside of
your own repo, but I assume you're clear about this.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Wed, Mar 18, 2020 at 6:38 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 17.03.2020 16:23, Jason Andryuk wrote:
> > On 17.03.2020 15:08, Jan Beulich wrote:
> >> On 17.03.2020 15:08, Jan Beulich wrote:
> >>> On 17.03.2020 14:48, Jason Andryuk wrote:
> >>>> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
> >>>> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
> >>>> actually got it programmed properly, it worked to increment
> >>>> pit0_ticks.  I also made timer_interrupt() unconditionally
> >>>> pit0_ticks++ though that may not matter.
> >>>
> >>> Hmm, at the first glance I would imply the system gets handed to Xen
> >>> with a HPET state that we don't (and probably also shouldn't) expect.
> >>> Could you provide HPET_CFG as well as all HPET_Tn_CFG and
> >>> HPET_Tn_ROUTE values as hpet_resume() finds them before doing any
> >>> adjustments to them? What are the components / parties involved in
> >>> getting Xen loaded and started?
> >>
> >> Of course much depends on what exactly you mean you've done to
> >> the HPET by saying "I programmed the HPET to provide ...".
> >
> > Below is the diff.  It was messier and I tidied it up some.
> >
> > It's mainly the change to hpet_resume() to mimic Linux's legacy HPET
> > setup on T0.  It turns on HPET_CFG_LEGACY to ensure the timer interrupt
> > is running.  And it also includes the printing of the initial HPET
> > config:
> > HPET_CFG 00000001
> > HPET_T0_CFG 00008030
> > HPET_T0_ROUTE 0000016c
> > HPET_T1_CFG 00008000
> > HPET_T1_ROUTE 00000000
> > HPET_T2_CFG 00008000
> > HPET_T2_ROUTE 00000000
> > HPET_T3_CFG 00008000
> > HPET_T3_ROUTE 00000000
> > HPET_T4_CFG 0000c000
> > HPET_T4_ROUTE 00000000
> > HPET_T5_CFG 0000c000
> > HPET_T5_ROUTE 00000000
> > HPET_T6_CFG 0000c000
> > HPET_T6_ROUTE 00000000
> > HPET_T7_CFG 0000c000
> > HPET_T7_ROUTE 00000000
> >
> > Other changes are to try to prevent Xen from clobbering T0 as a periodic
> > timer.
>
> Why "clobbering"? According to the values above T0 is neither enabled
> nor set to periodic.

I was trying to indicated the changes in hpet_broadcast_init() to
preserve T0 as a periodic timer after it was set up in hpet_resume().

> > --- a/xen/arch/x86/hpet.c
> > +++ b/xen/arch/x86/hpet.c
> > @@ -585,16 +585,27 @@ void __init hpet_broadcast_init(void)
> >               pv_rtc_handler = handle_rtc_once;
> >       }
> >
> > +    printk(XENLOG_INFO "%s cfg %d\n", __func__, cfg);
> >       hpet_write32(cfg, HPET_CFG);
> >
> >       for ( i = 0; i < n; i++ )
> >       {
> > -        if ( i == 0 && (cfg & HPET_CFG_LEGACY) )
> > +        printk(XENLOG_INFO "hpet cfg %d legacy %d\n", i, cfg & HPET_CFG_LEGACY);
> > +        if ( i == 1 && (cfg & HPET_CFG_LEGACY) )
> >           {
> >               /* set HPET T0 as oneshot */
> > -            cfg = hpet_read32(HPET_Tn_CFG(0));
> > +            cfg = hpet_read32(HPET_Tn_CFG(1));
> >               cfg &= ~(HPET_TN_LEVEL | HPET_TN_PERIODIC);
> >               cfg |= HPET_TN_ENABLE | HPET_TN_32BIT;
> > +            hpet_write32(cfg, HPET_Tn_CFG(1));
> > +        }
> > +
> > +        if ( i == 0 && (cfg & HPET_CFG_LEGACY) )
> > +        {
> > +            /* set HPET T0 as periodic */
> > +            cfg = hpet_read32(HPET_Tn_CFG(0));
> > +            cfg |= (HPET_TN_LEVEL | HPET_TN_PERIODIC);
>
> A change like this of course won't be acceptable outside of
> your own repo, but I assume you're clear about this.

Of course.  I was just providing the example that passes
check_timer().  I'm not familiar with the Xen timer code or HPETs, so
I was hoping this provides useful information to clarify the problem
and find a cleaner solution.

Locally, I minimized the HPET changes to just enable it during
check_timer() and then disable it afterwards.  That diff is below.

Xen is still having issues booting dom0 with the HPET changes, so this
change may be incorrect and break something else.  Previously, I wrote
about a failed assert in pv_destroy_gdt() during dom0 construction.  I
added a printk before the assert, and that issue disappeared and is
also gone after removing it again.  Since this is a tablet form
factor, serial output is impossible.  I added a delay to printk so I
could more easily capture screen output without it scrolling by.  I
have since removed that delay which may have been shifted the problem
as there is now a pagefault in emulate_forced_invalid_op().

r12 is NULL in
testb  $0x1,0x4(%r12)
which is:
if ( msrs->misc_features_enables.cpuid_faulting &&

So msrs is NULL?  msrs = current->arch.msrs ealier in the function.

The pv_destroy_gdt failed assert was:
ASSERT(v == current || !vcpu_cpu_dirty(v));

I wonder if the timer interrupt could be messing with current somehow?

Thanks,
Jason

diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
index 86929b9ba1..93a34792b2 100644
--- a/xen/arch/x86/hpet.c
+++ b/xen/arch/x86/hpet.c
@@ -765,6 +765,15 @@ int hpet_legacy_irq_tick(void)

 static u32 *hpet_boot_cfg;

+void hpet_disable_legacy(void)
+{
+    u32 cfg = hpet_read32(HPET_CFG);
+    printk(XENLOG_INFO "%s HPET_CFG %08x\n", __func__, cfg);
+    cfg &= ~HPET_CFG_LEGACY;
+    printk(XENLOG_INFO "%s HPET_CFG %08x\n", __func__, cfg);
+    hpet_write32(cfg, HPET_CFG);
+}
+
 u64 __init hpet_setup(void)
 {
     static u64 __initdata hpet_rate;
@@ -804,6 +813,8 @@ u64 __init hpet_setup(void)
     return hpet_rate + (last * 2 > hpet_period);
 }

+#include <asm/delay.h>
+
 void hpet_resume(u32 *boot_cfg)
 {
     static u32 system_reset_latch;
@@ -842,11 +853,33 @@ void hpet_resume(u32 *boot_cfg)
                    cfg & HPET_TN_RESERVED, i);
             cfg &= ~HPET_TN_RESERVED;
         }
+        if (i == 0) {
+            cfg |= HPET_TN_ENABLE | HPET_TN_PERIODIC | HPET_TN_SETVAL |
+                   HPET_TN_32BIT;
+        }
         hpet_write32(cfg, HPET_Tn_CFG(i));
+        if (i == 0) {
+#define NSEC_PER_SEC    1000000000L
+            uint64_t delta;
+            unsigned int now;
+            unsigned int cmp;
+            u64 hpet_rate = hpet_setup();
+            uint32_t mult = div_sc((unsigned long)hpet_rate,
+                                     1000000000ul, 32);
+            uint32_t shift = 32;
+            printk(XENLOG_INFO "hpet mult %d shift %d\n", mult, shift);
+            delta = ((uint64_t)(NSEC_PER_SEC / HZ)) * mult;
+            delta >>= shift;
+            now = hpet_read32(HPET_COUNTER);
+            cmp = now + (unsigned int)delta;
+            hpet_write32(cmp, HPET_Tn_CMP(i));
+            udelay(1);
+            hpet_write32(delta, HPET_Tn_CMP(i));
+        }
     }

     cfg = hpet_read32(HPET_CFG);
-    cfg |= HPET_CFG_ENABLE;
+    cfg |= HPET_CFG_ENABLE | HPET_CFG_LEGACY;
     hpet_write32(cfg, HPET_CFG);
 }

diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index e98e08e9c8..b62dea190a 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -34,6 +34,7 @@
 #include <asm/desc.h>
 #include <asm/msi.h>
 #include <asm/setup.h>
+#include <asm/hpet.h>
 #include <mach_apic.h>
 #include <io_ports.h>
 #include <irq_vectors.h>
@@ -2047,6 +2048,7 @@ void __init setup_IO_APIC(void)
     setup_IO_APIC_irqs();
     init_IO_APIC_traps();
     check_timer();
+    hpet_disable_legacy();
     print_IO_APIC();
     ioapic_pm_state_alloc();

diff --git a/xen/include/asm-x86/hpet.h b/xen/include/asm-x86/hpet.h
index fb6bf05065..531e94e904 100644
--- a/xen/include/asm-x86/hpet.h
+++ b/xen/include/asm-x86/hpet.h
@@ -82,6 +82,7 @@ void hpet_broadcast_enter(void);
 void hpet_broadcast_exit(void);
 int hpet_broadcast_is_available(void);
 void hpet_disable_legacy_broadcast(void);
+void hpet_disable_legacy(void);

 extern void (*pv_rtc_handler)(uint8_t reg, uint8_t value);

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Wed, Mar 18, 2020 at 10:04 AM Jason Andryuk <jandryuk@gmail.com> wrote:
> On Wed, Mar 18, 2020 at 6:38 AM Jan Beulich <jbeulich@suse.com> wrote:
> > On 17.03.2020 16:23, Jason Andryuk wrote:
> > > On 17.03.2020 15:08, Jan Beulich wrote:
> > >> On 17.03.2020 15:08, Jan Beulich wrote:
> > >>> On 17.03.2020 14:48, Jason Andryuk wrote:
> > >>>> I got it to boot past "IO-APIC + timer doesn't work".  I programmed
> > >>>> the HPET to provide a periodic timer in hpet_resume() on T0.  When I
> > >>>> actually got it programmed properly, it worked to increment
> > >>>> pit0_ticks.  I also made timer_interrupt() unconditionally
> > >>>> pit0_ticks++ though that may not matter.
> > >>>
> > >>> Hmm, at the first glance I would imply the system gets handed to Xen
> > >>> with a HPET state that we don't (and probably also shouldn't) expect.
> > >>> Could you provide HPET_CFG as well as all HPET_Tn_CFG and
> > >>> HPET_Tn_ROUTE values as hpet_resume() finds them before doing any
> > >>> adjustments to them? What are the components / parties involved in
> > >>> getting Xen loaded and started?
> > >>
> > >> Of course much depends on what exactly you mean you've done to
> > >> the HPET by saying "I programmed the HPET to provide ...".
> > >
> > > Below is the diff.  It was messier and I tidied it up some.
> > >
> > > It's mainly the change to hpet_resume() to mimic Linux's legacy HPET
> > > setup on T0.  It turns on HPET_CFG_LEGACY to ensure the timer interrupt
> > > is running.  And it also includes the printing of the initial HPET
> > > config:
> > > HPET_CFG 00000001
> > > HPET_T0_CFG 00008030
> > > HPET_T0_ROUTE 0000016c
> > > HPET_T1_CFG 00008000
> > > HPET_T1_ROUTE 00000000
> > > HPET_T2_CFG 00008000
> > > HPET_T2_ROUTE 00000000
> > > HPET_T3_CFG 00008000
> > > HPET_T3_ROUTE 00000000
> > > HPET_T4_CFG 0000c000
> > > HPET_T4_ROUTE 00000000
> > > HPET_T5_CFG 0000c000
> > > HPET_T5_ROUTE 00000000
> > > HPET_T6_CFG 0000c000
> > > HPET_T6_ROUTE 00000000
> > > HPET_T7_CFG 0000c000
> > > HPET_T7_ROUTE 00000000
> > >
> > > Other changes are to try to prevent Xen from clobbering T0 as a periodic
> > > timer.
> >
> > Why "clobbering"? According to the values above T0 is neither enabled
> > nor set to periodic.
>
> I was trying to indicated the changes in hpet_broadcast_init() to
> preserve T0 as a periodic timer after it was set up in hpet_resume().
>
> > > --- a/xen/arch/x86/hpet.c
> > > +++ b/xen/arch/x86/hpet.c
> > > @@ -585,16 +585,27 @@ void __init hpet_broadcast_init(void)
> > >               pv_rtc_handler = handle_rtc_once;
> > >       }
> > >
> > > +    printk(XENLOG_INFO "%s cfg %d\n", __func__, cfg);
> > >       hpet_write32(cfg, HPET_CFG);
> > >
> > >       for ( i = 0; i < n; i++ )
> > >       {
> > > -        if ( i == 0 && (cfg & HPET_CFG_LEGACY) )
> > > +        printk(XENLOG_INFO "hpet cfg %d legacy %d\n", i, cfg & HPET_CFG_LEGACY);
> > > +        if ( i == 1 && (cfg & HPET_CFG_LEGACY) )
> > >           {
> > >               /* set HPET T0 as oneshot */
> > > -            cfg = hpet_read32(HPET_Tn_CFG(0));
> > > +            cfg = hpet_read32(HPET_Tn_CFG(1));
> > >               cfg &= ~(HPET_TN_LEVEL | HPET_TN_PERIODIC);
> > >               cfg |= HPET_TN_ENABLE | HPET_TN_32BIT;
> > > +            hpet_write32(cfg, HPET_Tn_CFG(1));
> > > +        }
> > > +
> > > +        if ( i == 0 && (cfg & HPET_CFG_LEGACY) )
> > > +        {
> > > +            /* set HPET T0 as periodic */
> > > +            cfg = hpet_read32(HPET_Tn_CFG(0));
> > > +            cfg |= (HPET_TN_LEVEL | HPET_TN_PERIODIC);
> >
> > A change like this of course won't be acceptable outside of
> > your own repo, but I assume you're clear about this.
>
> Of course.  I was just providing the example that passes
> check_timer().  I'm not familiar with the Xen timer code or HPETs, so
> I was hoping this provides useful information to clarify the problem
> and find a cleaner solution.
>
> Locally, I minimized the HPET changes to just enable it during
> check_timer() and then disable it afterwards.  That diff is below.
>
> Xen is still having issues booting dom0 with the HPET changes, so this
> change may be incorrect and break something else.  Previously, I wrote
> about a failed assert in pv_destroy_gdt() during dom0 construction.  I
> added a printk before the assert, and that issue disappeared and is
> also gone after removing it again.  Since this is a tablet form
> factor, serial output is impossible.  I added a delay to printk so I
> could more easily capture screen output without it scrolling by.  I
> have since removed that delay which may have been shifted the problem
> as there is now a pagefault in emulate_forced_invalid_op().
>
> r12 is NULL in
> testb  $0x1,0x4(%r12)
> which is:
> if ( msrs->misc_features_enables.cpuid_faulting &&
>
> So msrs is NULL?  msrs = current->arch.msrs ealier in the function.
>
> The pv_destroy_gdt failed assert was:
> ASSERT(v == current || !vcpu_cpu_dirty(v));
>
> I wonder if the timer interrupt could be messing with current somehow?
>

Something was stale in my build tree.  After cleaning and re-build
Xen, it boots into dom0 with the patch below.

Regards,
Jason

> diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c
> index 86929b9ba1..93a34792b2 100644
> --- a/xen/arch/x86/hpet.c
> +++ b/xen/arch/x86/hpet.c
> @@ -765,6 +765,15 @@ int hpet_legacy_irq_tick(void)
>
>  static u32 *hpet_boot_cfg;
>
> +void hpet_disable_legacy(void)
> +{
> +    u32 cfg = hpet_read32(HPET_CFG);
> +    printk(XENLOG_INFO "%s HPET_CFG %08x\n", __func__, cfg);
> +    cfg &= ~HPET_CFG_LEGACY;
> +    printk(XENLOG_INFO "%s HPET_CFG %08x\n", __func__, cfg);
> +    hpet_write32(cfg, HPET_CFG);
> +}
> +
>  u64 __init hpet_setup(void)
>  {
>      static u64 __initdata hpet_rate;
> @@ -804,6 +813,8 @@ u64 __init hpet_setup(void)
>      return hpet_rate + (last * 2 > hpet_period);
>  }
>
> +#include <asm/delay.h>
> +
>  void hpet_resume(u32 *boot_cfg)
>  {
>      static u32 system_reset_latch;
> @@ -842,11 +853,33 @@ void hpet_resume(u32 *boot_cfg)
>                     cfg & HPET_TN_RESERVED, i);
>              cfg &= ~HPET_TN_RESERVED;
>          }
> +        if (i == 0) {
> +            cfg |= HPET_TN_ENABLE | HPET_TN_PERIODIC | HPET_TN_SETVAL |
> +                   HPET_TN_32BIT;
> +        }
>          hpet_write32(cfg, HPET_Tn_CFG(i));
> +        if (i == 0) {
> +#define NSEC_PER_SEC    1000000000L
> +            uint64_t delta;
> +            unsigned int now;
> +            unsigned int cmp;
> +            u64 hpet_rate = hpet_setup();
> +            uint32_t mult = div_sc((unsigned long)hpet_rate,
> +                                     1000000000ul, 32);
> +            uint32_t shift = 32;
> +            printk(XENLOG_INFO "hpet mult %d shift %d\n", mult, shift);
> +            delta = ((uint64_t)(NSEC_PER_SEC / HZ)) * mult;
> +            delta >>= shift;
> +            now = hpet_read32(HPET_COUNTER);
> +            cmp = now + (unsigned int)delta;
> +            hpet_write32(cmp, HPET_Tn_CMP(i));
> +            udelay(1);
> +            hpet_write32(delta, HPET_Tn_CMP(i));
> +        }
>      }
>
>      cfg = hpet_read32(HPET_CFG);
> -    cfg |= HPET_CFG_ENABLE;
> +    cfg |= HPET_CFG_ENABLE | HPET_CFG_LEGACY;
>      hpet_write32(cfg, HPET_CFG);
>  }
>
> diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
> index e98e08e9c8..b62dea190a 100644
> --- a/xen/arch/x86/io_apic.c
> +++ b/xen/arch/x86/io_apic.c
> @@ -34,6 +34,7 @@
>  #include <asm/desc.h>
>  #include <asm/msi.h>
>  #include <asm/setup.h>
> +#include <asm/hpet.h>
>  #include <mach_apic.h>
>  #include <io_ports.h>
>  #include <irq_vectors.h>
> @@ -2047,6 +2048,7 @@ void __init setup_IO_APIC(void)
>      setup_IO_APIC_irqs();
>      init_IO_APIC_traps();
>      check_timer();
> +    hpet_disable_legacy();
>      print_IO_APIC();
>      ioapic_pm_state_alloc();
>
> diff --git a/xen/include/asm-x86/hpet.h b/xen/include/asm-x86/hpet.h
> index fb6bf05065..531e94e904 100644
> --- a/xen/include/asm-x86/hpet.h
> +++ b/xen/include/asm-x86/hpet.h
> @@ -82,6 +82,7 @@ void hpet_broadcast_enter(void);
>  void hpet_broadcast_exit(void);
>  int hpet_broadcast_is_available(void);
>  void hpet_disable_legacy_broadcast(void);
> +void hpet_disable_legacy(void);
>
>  extern void (*pv_rtc_handler)(uint8_t reg, uint8_t value);

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jan Beulich]

On 17.03.2020 16:23, Jason Andryuk wrote:
> Below is the diff.  It was messier and I tidied it up some.

I've looked into this some more. I can see how what we currently
do is not in line with firmware handing off with LegacyReplacement
mode enabled. However, this case doesn't look to apply here:

> It's mainly the change to hpet_resume() to mimic Linux's legacy HPET
> setup on T0.  It turns on HPET_CFG_LEGACY to ensure the timer interrupt
> is running.  And it also includes the printing of the initial HPET
> config:
> HPET_CFG 00000001

While HPET_CFG_ENABLE is set, HPET_CFG_LEGACY is clear.

> HPET_T0_CFG 00008030
> HPET_T0_ROUTE 0000016c

And while firmware must have setup FSB routing for T0, its enable
bits (both HPET_TN_ENABLE and HPET_TN_FSB) are also clear.
Therefore we have, afaics, no indication whatsoever that we ought
to enable LegacyReplacement mode. Of course the spec also says
"Assuming platform does not have 8254/RTC hardware or does not
want to support this legacy timer hardware, for this case, System
BIOS should set the LegacyReplacement Route bit and report IRQ0 &
IRQ8 as being consumed by the HPET block in system name space:"
(followed by a table). What is referred to as "system name space"
is, I assume, ACPI DSDT/SSDTs, which we have no access to this
early (and Linux doesn't either, aiui), so also can't be used as
indicator.

Otoh I also don't think it is correct to blindly enable
LegacyReplacement mode, like - afaics - Linux does, the more with
our split brain model as far as affected devices go (Xen "owns"
the PIT [and of course also the HPET], while Linux "owns" the
RTC). This is because of the effect of this setting on what
actually drives IRQ8. In theory we might be able do so when
ACPI_FADT_NO_CMOS_RTC is set, but Linux may use the CMOS RTC
even when that flag is set.

So right now the only possible approach I see to address your
problem is to add yet another fallback mode to check_timer(),
forcing LegacyReplacement mode to be enabled. But between /
after which step(s) to put this there isn't at all obvious to me.

Jan

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

On Fri, Apr 17, 2020 at 5:31 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 17.03.2020 16:23, Jason Andryuk wrote:
> > Below is the diff.  It was messier and I tidied it up some.
>
> I've looked into this some more. I can see how what we currently
> do is not in line with firmware handing off with LegacyReplacement
> mode enabled. However, this case doesn't look to apply here:

Thanks for taking a look again.  Sorry for the slow response.  I
wanted to dig through the linux history to see if there was a reason
for enabling the periodic timer there, but I haven't gotten around to
it.

> So right now the only possible approach I see to address your
> problem is to add yet another fallback mode to check_timer(),
> forcing LegacyReplacement mode to be enabled. But between /
> after which step(s) to put this there isn't at all obvious to me.

I don't really have a good suggestion here.

Aaron, I'm curious to know if you've tried this patch on your hardware
and if it helped.

Thanks again,
Jason

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jason Andryuk]

> > >>> One other thing that might be noteworthy.  Linux only prints ACPI IRQ0
> > >>> and IRQ9 used by override where Xen lists IRQ 0, 2 & 9.
> > >> Huh - this is supposed to come directly from the ACPI tables, so Linux
> > >> and Xen should be using the same source of information.

Both Xen and Linux only see two ACPI overrides (0 & 9) from the
tables.  However the Xen logic in mp_config_acpi_legacy_irqs() thinks
IRQ2 is an override
irq 2: irq->mpc_srcbus 0, irq->mpc_srcbusirq 0, irq->mpc_dstapic 2,
intsrc.mpc_dstapic 2
Matches
((irq->mpc_dstapic == intsrc.mpc_dstapic) &&
                                (irq->mpc_dstirq == i))

i is 2, so irq->mpc_dstirq must be as well.

Regards,
Jason

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jan Beulich]

On 31.12.2019 08:52,  Aaron Janse  wrote:
> I'd like to note that Ubuntu, unlike Qubes, doesn't need to try
> any `MP-BIOS bug` fallbacks.

"Doesn't need to try" is supposed to mean what? That it gets past
the timer interrupt initialization, meaning if it crashes another
way, it's a different problem? Or instead meaning it works
(contrary to information found elsewhere), suggesting there's a
Qubes side change involved?

> # Things that have been tried
> 
> Disabling APIC entirely (`noapic x2apic=off`)
> - This is avoiding the problem, not fixing it
> - QubeOS requires APIC anyway, so this is not an option for many of us
> 
> Switching the timer to HPET (via the `clocksource` flag)
> - This didn't fix the panic (I've tried `acpi` and `pit`)
> - On my XPS 13, this doesn't change any of the timer error output
> - Ubuntu works on my laptop using HPET
> - http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html#clocksource-x86

Did you try disabling use of the IOMMU ("iommu=0" on the Xen
command line)?

> Updating to 5.4 Linux kernel [77]
> - This didn't fix the panic

As long as you don't even reach Dom0 initialization, no change
whatsoever to the Dom0 kernel will possibly help.

If this is as common a problem as you say, it's hard to believe
this has never worked on any of these systems. Hence it would be
helpful to know starting from which version this has been
regressed.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Aaron Janse]

On Fri, Jan 3, 2020, at 4:51 AM, Jan Beulich wrote:
> On 31.12.2019 08:52,  Aaron Janse  wrote:
> > I'd like to note that Ubuntu, unlike Qubes, doesn't need to try
> > any `MP-BIOS bug` fallbacks.
> 
> "Doesn't need to try" is supposed to mean what? That it gets past
> the timer interrupt initialization, meaning if it crashes another
> way, it's a different problem? Or instead meaning it works
> (contrary to information found elsewhere), suggesting there's a
> Qubes side change involved?

I originally thought that the problem was the timer specifically, but
based on what you and Andrew Cooper have said, it sounds like the root
cause is somewhere else.

Andrew Cooper wrote:
> (Irrespective, I'm pretty sure this is a Grub2+EFI issue failing to pass
> the ACPI tables to Xen, and this eventual panic is just cascade fallout.)

I tried to get Xen working via legacy boot, but I haven't been able to get
my laptop to boot anything but UEFI. The BIOS even states "UEFI only."

> Did you try disabling use of the IOMMU ("iommu=0" on the Xen
> command line)?

Unfortunately, Qubes requires iommu. Setting "iommu=0" results in a panic:

```
Couldn't enable IOMMU and iommu=required/force
```

I also (unsuccessfully) tried iommu=no-igfx and iommu=soft (both resulted
in the timer panic).

I couldn't find anywhere to disable the flag (even though it would break
Qubes, at least the flag could help minimize the scope of the cause of the
timer crash).

I installed Xen on Arch Linux in order to test this flag, but I'm having
the same problem I had on Ubuntu: booting to Xen hangs on loading
initramfs. [1]

> If this is as common a problem as you say, it's hard to believe
> this has never worked on any of these systems. Hence it would be
> helpful to know starting from which version this has been
> regressed.

That makes sense. I've tried to reproduce the problem on both Arch and
Ubuntu (both hang, and I'm not sure why or how to debug that). Because
Qubes is the only OS I've been able to boot verbose Xen from, I installed
a 2016 release to try out. However, I couldn't get past a hang showing
the Dell logo. I had this same issue on NixOS, described by someone else
with the same laptop on the NixOS Discourse [4]. The solution for NixOS
was to use a newer version of the distro.

If I can get past the boot hang on Ubuntu or Arch, I'd be happy to go about
bisecting the issue, comparing Ubuntu/Arch vs Qubes, compiling with new
printf statements, etc.

As a side note, the XPS 7390 2-in-1 user was able to get Xen to boot
using the acpi=noirq flag [2]. My understanding is that needing this flag
indicates that something's still wrong [3].

[1] https://lists.xenproject.org/archives/html/xen-users/2019-12/msg00031.html
[2] https://www.reddit.com/r/Qubes/comments/edqrab/qubes_and_ice_lake/fcresld/
[3] http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html#acpi
[4] https://discourse.nixos.org/t/nixos-stable-wont-boot-from-usb-on-xps-7390/4776

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [BUG] panic: "IO-APIC + timer doesn't work" - several people have reproduced

[Jan Beulich]

On 06.01.2020 01:35,  Aaron Janse  wrote:
> On Fri, Jan 3, 2020, at 4:51 AM, Jan Beulich wrote:
>> Did you try disabling use of the IOMMU ("iommu=0" on the Xen
>> command line)?
> 
> Unfortunately, Qubes requires iommu. Setting "iommu=0" results in a panic:
> 
> ```
> Couldn't enable IOMMU and iommu=required/force
> ```

Since this isn't the upstream default, there are two options: Either
you simply have e.g. "iommu=force" elsewhere on the command line - in
this case simply delete the option for this experimenting. Or they've
patched their sources to this effect, in which case you'll want to
undo that source change.

> I couldn't find anywhere to disable the flag (even though it would break
> Qubes, at least the flag could help minimize the scope of the cause of the
> timer crash).
> 
> I installed Xen on Arch Linux in order to test this flag, but I'm having
> the same problem I had on Ubuntu: booting to Xen hangs on loading
> initramfs. [1]

Booting which exact version of Xen? Iirc these initramfs issues
(with LZ4 compression) have been fixed on 4.13.0 as well as
4.12.2. Also you may not have realized that _any_ initramfs (or
Dom0 kernel image) issue could be avoided for the purposes here
by simply omitting them from the directives issued to grub -
after all you don't get as far as booting Dom0.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel