From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Thimo E." Subject: Re: cpuidle and un-eoid interrupts at the local apic Date: Sun, 11 Aug 2013 19:46:52 +0200 Message-ID: <5207CE0C.1000502@digithi.de> References: <51A908CA.7050604@citrix.com><51F8CB15.1070608@digithi.de><51F8DD40.2090207@citrix.com><51FC37A9.9090809@digithi.de><51FC418D.8020708@citrix.com><51FFBA8502000078000E9462@nat28.tlf.novell.com><51FFBC08.6070804@citrix.com> <52055EC9.8030207@digithi.de><520561E1.8020809@citrix.com> <520562C8.8080703@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------070108090104060700070201" Return-path: In-Reply-To: <520562C8.8080703@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Andrew Cooper Cc: Keir Fraser , Jan Beulich , Eddie Dong , Xen-develList , Jun Nakajima , Xiantao Zhang List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --------------070108090104060700070201 Content-Type: multipart/alternative; boundary="------------070508080003040409050706" --------------070508080003040409050706 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hello again, attached you'll find another crash dump from today. Don't know if it gives you more information than the last one. Just FYI, this is a system with an Intel Mainboard (H87 chipset) and a Core i5-4670 CPU. Best regards Thimo Am 09.08.2013 23:44, schrieb Andrew Cooper: > On 09/08/13 22:40, Andrew Cooper wrote: >> >> So according to my debugging, we really have just pushed the same irq >> which we have subsequently seen again unexpectedly. >> >> This bug has only ever been seen on Haswell hardware, and appears >> linked to running HVM guests. >> >> So either there is an erroneous ACK the LAPIC which is clearing the >> ISR before the PEOI stack is expecting (which I > > "can't" > > Apologies for the confusion. > > ~Andrew > >> obviously see, looking at the code), or something more funky is going >> on with the hardware. >> >> CC'ing in the Intel maintainers: Do you have any ideas? Could this >> be related to APICv? >> >> ~Andrew > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel --------------070508080003040409050706 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
Hello again,

attached you'll find another crash dump from today. Don't know if it gives you more information than the last one.

Just FYI, this is a system with an Intel Mainboard (H87 chipset) and a Core i5-4670 CPU.

Best regards
  Thimo

Am 09.08.2013 23:44, schrieb Andrew Cooper:
On 09/08/13 22:40, Andrew Cooper wrote:

So according to my debugging, we really have just pushed the same irq which we have subsequently seen again unexpectedly.

This bug has only ever been seen on Haswell hardware, and appears linked to running HVM guests.

So either there is an erroneous ACK the LAPIC which is clearing the ISR before the PEOI stack is expecting (which I

"can't"

Apologies for the confusion.

~Andrew

obviously see, looking at the code), or something more funky is going on with the hardware.

CC'ing in the Intel maintainers:  Do you have any ideas?  Could this be related to APICv?

~Andrew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

--------------070508080003040409050706-- --------------070108090104060700070201 Content-Type: text/plain; charset=windows-1252; name="crash20130811.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="crash20130811.txt" (XEN) **Pending EOI error (XEN) irq 29, vector 0x26 (XEN) s[0] irq 29, vec 0x26, ready 0, ISR 00000001, TMR 00000000, IRR 00000001 (XEN) All LAPIC state: (XEN) [vector] ISR TMR IRR (XEN) [1f:00] 00000000 00000000 00000000 (XEN) [3f:20] 00000040 8c334526 00000040 (XEN) [5f:40] 00000000 006c000a 00000000 (XEN) [7f:60] 00000000 00240278 00000000 (XEN) [9f:80] 00000000 40000a10 00000000 (XEN) [bf:a0] 00000000 060d708a 00000000 (XEN) [df:c0] 00000000 42054000 00000000 (XEN) [ff:e0] 00000000 00000000 00000000 (XEN) Peoi stack trace records: (XEN) Pushed {sp 0, irq 29, vec 0x26} (XEN) Poped entry {sp 1, irq 29, vec 0x26} (XEN) Marked {sp 0, irq 29, vec 0x26} ready (XEN) Pushed {sp 0, irq 29, vec 0x26} (XEN) Poped entry {sp 1, irq 29, vec 0x26} (XEN) Marked {sp 0, irq 29, vec 0x26} ready (XEN) Pushed {sp 0, irq 29, vec 0x26} (XEN) Poped entry {sp 1, irq 29, vec 0x26} (XEN) Marked {sp 0, irq 29, vec 0x26} ready (XEN) Pushed {sp 0, irq 29, vec 0x26} (XEN) Poped entry {sp 1, irq 29, vec 0x26} (XEN) Marked {sp 0, irq 29, vec 0x26} ready (XEN) Pushed {sp 0, irq 29, vec 0x26} (XEN) Poped entry {sp 1, irq 29, vec 0xc7} (XEN) Marked {sp 0, irq 29, vec 0x31} ready (XEN) Pushed {sp 0, irq 29, vec 0xc7} (XEN) Poped entry {sp 1, irq 29, vec 0xc7} (XEN) Marked {sp 0, irq 29, vec 0xc7} ready (XEN) Pushed {sp 0, irq 29, vec 0xc7} (XEN) Poped entry {sp 1, irq 29, vec 0xc7} (XEN) Marked {sp 0, irq 29, vec 0xc7} ready (XEN) Pushed {sp 0, irq 29, vec 0xc7} (XEN) Poped entry {sp 1, irq 29, vec 0xc7} (XEN) Marked {sp 0, irq 29, vec 0xc7} ready (XEN) Pushed {sp 0, irq 29, vec 0xc7} (XEN) Poped entry {sp 1, irq 29, vec 0xcd} (XEN) Marked {sp 0, irq 29, vec 0x9e} ready (XEN) Pushed {sp 0, irq 29, vec 0xcd} (XEN) Poped entry {sp 1, irq 29, vec 0xcd} (XEN) Marked {sp 0, irq 29, vec 0xcd} ready (XEN) Pushed {sp 0, irq 29, vec 0xcd} (XEN) Poped entry {sp 1, irq 29, vec 0xcd} (XEN) Guest interrupt information: (XEN) IRQ: 0 affinity:1 vec:f0 type=IO-APIC-edge status=00000000 mapped, unbound (XEN) IRQ: 1 affinity:1 vec:38 type=IO-APIC-edge status=00000054 in-flight=0 domain-list=0: 1(----), (XEN) IRQ: 2 affinity:f vec:00 type=XT-PIC status=00000000 mapped, unbound (XEN) IRQ: 3 affinity:1 vec:40 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 4 affinity:1 vec:48 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 5 affinity:1 vec:50 type=IO-APIC-edge status=00000050 in-flight=0 domain-list=0: 5(----), (XEN) IRQ: 6 affinity:1 vec:58 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 7 affinity:1 vec:60 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 8 affinity:1 vec:68 type=IO-APIC-edge status=00000050 in-flight=0 domain-list=0: 8(----), (XEN) IRQ: 9 affinity:1 vec:70 type=IO-APIC-level status=00000050 in-flight=0 domain-list=0: 9(----), (XEN) IRQ: 10 affinity:1 vec:78 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 11 affinity:1 vec:88 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 12 affinity:1 vec:90 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 13 affinity:1 vec:98 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 14 affinity:1 vec:a0 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 15 affinity:1 vec:a8 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 16 affinity:1 vec:36 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 16(----), (XEN) IRQ: 18 affinity:4 vec:b5 type=IO-APIC-level status=00000050 in-flight=0 domain-list=0: 18(----), (XEN) IRQ: 19 affinity:1 vec:39 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 20 affinity:1 vec:29 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 22 affinity:8 vec:66 type=IO-APIC-level status=00000050 in-flight=0 domain-list=0: 22(----), (XEN) IRQ: 23 affinity:1 vec:d8 type=IO-APIC-level status=00000050 in-flight=0 domain-list=0: 23(----), (XEN) IRQ: 24 affinity:1 vec:28 type=DMA_MSI status=00000000 mapped, unbound (XEN) IRQ: 25 affinity:1 vec:30 type=DMA_MSI status=00000000 mapped, unbound (XEN) IRQ: 26 affinity:f vec:c0 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 27 affinity:f vec:c8 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 28 affinity:f vec:d0 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 29 affinity:8 vec:26 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:276(----), (XEN) IRQ: 30 affinity:2 vec:9b type=PCI-MSI status=00000050 in-flight=0 domain-list=0:275(----), (XEN) IRQ: 31 affinity:2 vec:72 type=PCI-MSI status=00000050 in-flight=0 domain-list=0:274(----), (XEN) IRQ: 32 affinity:8 vec:3e type=PCI-MSI status=00000050 in-flight=0 domain-list=0:273(----), (XEN) IRQ: 33 affinity:8 vec:5e type=PCI-MSI status=00000050 in-flight=0 domain-list=0:272(----), (XEN) IRQ: 34 affinity:1 vec:59 type=PCI-MSI status=00000050 in-flight=0 domain-list=0:271(----), (XEN) IO-APIC interrupt information: (XEN) IRQ 0 Vec240: (XEN) Apic 0x00, Pin 2: vec=f0 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 1 Vec 56: (XEN) Apic 0x00, Pin 1: vec=38 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 3 Vec 64: (XEN) Apic 0x00, Pin 3: vec=40 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 4 Vec 72: (XEN) Apic 0x00, Pin 4: vec=48 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 5 Vec 80: (XEN) Apic 0x00, Pin 5: vec=50 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 6 Vec 88: (XEN) Apic 0x00, Pin 6: vec=58 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 7 Vec 96: (XEN) Apic 0x00, Pin 7: vec=60 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 8 Vec104: (XEN) Apic 0x00, Pin 8: vec=68 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 9 Vec112: (XEN) Apic 0x00, Pin 9: vec=70 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=L mask=0 dest_id:0 (XEN) IRQ 10 Vec120: (XEN) Apic 0x00, Pin 10: vec=78 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 11 Vec136: (XEN) Apic 0x00, Pin 11: vec=88 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 12 Vec144: (XEN) Apic 0x00, Pin 12: vec=90 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 13 Vec152: (XEN) Apic 0x00, Pin 13: vec=98 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 14 Vec160: (XEN) Apic 0x00, Pin 14: vec=a0 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 15 Vec168: (XEN) Apic 0x00, Pin 15: vec=a8 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 16 Vec 54: (XEN) Apic 0x00, Pin 16: vec=36 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0 (XEN) IRQ 18 Vec181: (XEN) Apic 0x00, Pin 18: vec=b5 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0 (XEN) IRQ 19 Vec 57: (XEN) Apic 0x00, Pin 19: vec=39 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=1 dest_id:0 (XEN) IRQ 20 Vec 41: (XEN) Apic 0x00, Pin 20: vec=29 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=1 dest_id:0 (XEN) IRQ 22 Vec102: (XEN) Apic 0x00, Pin 22: vec=66 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0 (XEN) IRQ 23 Vec216: (XEN) Apic 0x00, Pin 23: vec=d8 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0 (XEN) number of MP IRQ sources: 15. (XEN) number of IO-APIC #2 registers: 24. (XEN) testing the IO APIC....................... (XEN) IO APIC #2...... (XEN) .... register #00: 02000000 (XEN) ....... : physical APIC id: 02 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 000 00 0 0 0 0 0 1 1 38 (XEN) 02 000 00 0 0 0 0 0 1 1 F0 (XEN) 03 000 00 0 0 0 0 0 1 1 40 (XEN) 04 000 00 0 0 0 0 0 1 1 48 (XEN) 05 000 00 0 0 0 0 0 1 1 50 (XEN) 06 000 00 0 0 0 0 0 1 1 58 (XEN) 07 000 00 0 0 0 0 0 1 1 60 (XEN) 08 000 00 0 0 0 0 0 1 1 68 (XEN) 09 000 00 0 1 0 0 0 1 1 70 (XEN) 0a 000 00 0 0 0 0 0 1 1 78 (XEN) 0b 000 00 0 0 0 0 0 1 1 88 (XEN) 0c 000 00 0 0 0 0 0 1 1 90 (XEN) 0d 000 00 0 0 0 0 0 1 1 98 (XEN) 0e 000 00 0 0 0 0 0 1 1 A0 (XEN) 0f 000 00 0 0 0 0 0 1 1 A8 (XEN) 10 000 00 0 1 0 1 0 1 1 36 (XEN) 11 000 00 1 0 0 0 0 0 0 00 (XEN) 12 000 00 0 1 0 1 0 1 1 B5 (XEN) 13 000 00 1 1 0 1 0 1 1 39 (XEN) 14 000 00 1 1 0 1 0 1 1 29 (XEN) 15 07A 0A 1 0 0 0 0 0 2 B4 (XEN) 16 000 00 0 1 0 1 0 1 1 66 (XEN) 17 000 00 0 1 0 1 0 1 1 D8 (XEN) Using vector-based indexing (XEN) IRQ to pin mappings: (XEN) IRQ240 -> 0:2 (XEN) IRQ56 -> 0:1 (XEN) IRQ64 -> 0:3 (XEN) IRQ72 -> 0:4 (XEN) IRQ80 -> 0:5 (XEN) IRQ88 -> 0:6 (XEN) IRQ96 -> 0:7 (XEN) IRQ104 -> 0:8 (XEN) IRQ112 -> 0:9 (XEN) IRQ120 -> 0:10 (XEN) IRQ136 -> 0:11 (XEN) IRQ144 -> 0:12 (XEN) IRQ152 -> 0:13 (XEN) IRQ160 -> 0:14 (XEN) IRQ168 -> 0:15 (XEN) IRQ54 -> 0:16 (XEN) IRQ181 -> 0:18 (XEN) IRQ57 -> 0:19 (XEN) IRQ41 -> 0:20 (XEN) IRQ102 -> 0:22 (XEN) IRQ216 -> 0:23 (XEN) .................................... done. (XEN) (XEN) **************************************** (XEN) Panic on CPU 3: (XEN) CA-107844**************************************** (XEN) (XEN) Reboot in five seconds... (XEN) Executing crash image --------------070108090104060700070201 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --------------070108090104060700070201--