From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Thimo E." Subject: Re: cpuidle and un-eoid interrupts at the local apic Date: Sun, 08 Sep 2013 12:24:16 +0200 Message-ID: <522C5050.9030508@digithi.de> References: <51A908CA.7050604@citrix.com><51F8CB15.1070608@digithi.de><51F8DD40.2090207@citrix.com><51FC37A9.9090809@digithi.de><51FC418D.8020708@citrix.com><51FFBA8502000078000E9462@nat28.tlf.novell.com><51FFBC08.6070804@citrix.com><52055EC9.8030207@digithi.de><5208B6DE02000078000EB08E@nat28.tlf.novell.com><5208AACF.7050901@citrix.com><5208CF9702000078000EB1F2@nat28.tlf.novell.com><5208B88F.1030002@citrix.com><520B5345.4000508@citrix.com> <522B29BC.3010003@digithi.de> <522B5C1A.1050100@citrix.com> <522BB8BE.8060800@digithi.de> <522C491B.6000004@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------050105070007080103010705" Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1VIcAU-0003jc-39 for xen-devel@lists.xenproject.org; Sun, 08 Sep 2013 10:24:34 +0000 In-Reply-To: <522C491B.6000004@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Andrew Cooper Cc: "Zhang, Yang Z" , xen-devel , Keir Fraser , Jan Beulich List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --------------050105070007080103010705 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Ah, sorry. Output is attached. Am 08.09.2013 11:53, schrieb Andrew Cooper: > On 08/09/2013 00:37, Thimo E. wrote: >> Hello Andrew, >> >> ok, thanks. This is what I assumed. >> >> The output of "xl debug-keys iMQ" is empty. > Sorry - I should have been more clear. `xl debug-keys` dumps its > information into the xen dmesg buffer, so `xl dmesg` will capture the > results. > > ~Andrew > >> [root@localhost ~]# dmesg |grep arcmsr >> [ 8.159321] arcmsr 0000:01:00.0: PCI INT A -> GSI 16 (level, low) >> -> IRQ 16 >> [ 8.159413] arcmsr 0000:01:00.0: setting latency timer to 64 >> [ 8.170316] arcmsr 0000:01:00.0: get owner: 7ff0 >> [ 8.170414] arcmsr 0000:01:00.0: irq 1276 (276) for MSI/MSI-X >> [ 8.170421] IRQ 1276/arcmsr: IRQF_DISABLED is not guaranteed on >> shared IRQs >> [ 8.170654] arcmsr0: msi enabled >> >> [root@localhost /]# cat /proc/irq/1276/spurious >> count 61007 >> unhandled 8 >> last_unhandled 36736990 ms >> >> arcmsr is the driver of the Areca Storage Raid Controller. Used it >> already before with Xenserver 6.0.2 for years, no problems. >> >> THe messages "IRQF_DISABLED is not guaranteed...." and "8 unhandled >> interrupts" look interesting. I am not a kernel hacker but what I >> interpret from >> http://lxr.free-electrons.com/source/kernel/irq/manage.c?v=2.6.32: >> >> 1025 if ((irqflags & (IRQF_SHARED|IRQF_DISABLED)) == >> 1026 (IRQF_SHARED|IRQF_DISABLED)) { >> 1027 pr_warning( >> 1028 "IRQ %d/%s: IRQF_DISABLED is not guaranteed on >> shared IRQs\n", >> 1029 irq, devname); >> ... >> 738 * Force MSI interrupts to run with interrupts >> 739 * disabled. The multi vector cards can cause stack >> 740 * overflows due to nested interrupts when enough of >> 741 * them are directed to a core and fire at the same >> 742 * time. >> 743 */ >> 744 if (desc->msi_desc) >> 745 new->flags |= IRQF_DISABLED; >> >> --> "IRQF_DISABLED is not guaranteed on shared IRQs" warning is only >> printed when irqflags IRQF_SHARED and IRQF_DISABLED are set >> --> Is that what we see in the kernel oops the stack overflow the >> comment in lines 738-742 is talking about ?! >> --> IRQF_SHARED is set, so MSI interrupt 1276 is shared ?! I thought >> that it is not possible that MSI interrupts are shared. Attached >> you'll see my /proc/interrupts >> >> So what I do now is disabling MSI for the arcmsr driver. Could this be >> the source of the problem ?! But why is 1276 shared ?! >> >> Best regards >> Thimo >> >> Am 07.09.2013 19:02, schrieb Andrew Cooper: >>> irq 29 is just an internal Xen number for accounting all interrupts. It >>> doesn't mean anything specific regarding hardware etc. The vector and >>> affinity would expect to change as dom0s vcpus are moved around by the >>> scheduler. >>> >>> domain-list=0 means that this interrupt is targeted at dom0 (It is a >>> list because certain interrupts have to be shared my more than 1 >>> domain). Helpfully, the keyhandler truncates the pirq field, so 276 is >>> unlikely to be correct. As it is a dom0 MSI, I am guessing it actually >>> matches up with interrupt 1276 in /proc/interrupts, if there is one. >>> >>> Can you provide the results of `xl debug-keys iMQ`, and attach >>> /proc/interrupts to this email (just in case the setup has changed after >>> playing with your BIOS) >>> >>> ~Andrew >>> --------------050105070007080103010705 Content-Type: text/plain; charset=windows-1252; name="xl_debus_keys20130908.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xl_debus_keys20130908.txt" (XEN) Guest interrupt information: (XEN) IRQ: 0 affinity:1 vec:f0 type=IO-APIC-edge status=00000000 mapped, unbound (XEN) IRQ: 1 affinity:1 vec:38 type=IO-APIC-edge status=00000014 in-flight=0 domain-list=0: 1(----), (XEN) IRQ: 2 affinity:f vec:00 type=XT-PIC status=00000000 mapped, unbound (XEN) IRQ: 3 affinity:1 vec:40 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 4 affinity:1 vec:48 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 5 affinity:1 vec:50 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 5(----), (XEN) IRQ: 6 affinity:1 vec:58 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 7 affinity:1 vec:60 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 8 affinity:1 vec:68 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 8(----), (XEN) IRQ: 9 affinity:1 vec:70 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 9(----), (XEN) IRQ: 10 affinity:1 vec:78 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 11 affinity:1 vec:88 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 12 affinity:1 vec:90 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 13 affinity:1 vec:98 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 14 affinity:1 vec:a0 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 15 affinity:1 vec:a8 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 16 affinity:1 vec:b0 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 16(----), (XEN) IRQ: 17 affinity:1 vec:39 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 18 affinity:1 vec:31 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 18(----), (XEN) IRQ: 22 affinity:1 vec:29 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 22(----), (XEN) IRQ: 23 affinity:1 vec:d0 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 23(----), (XEN) IRQ: 24 affinity:1 vec:28 type=DMA_MSI status=00000000 mapped, unbound (XEN) IRQ: 25 affinity:1 vec:30 type=DMA_MSI status=00000000 mapped, unbound (XEN) IRQ: 26 affinity:f vec:b8 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 27 affinity:f vec:c0 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 28 affinity:f vec:c8 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 29 affinity:f vec:d8 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:276(----), (XEN) IRQ: 30 affinity:f vec:51 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:275(----), (XEN) IRQ: 31 affinity:f vec:61 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:274(----), (XEN) IO-APIC interrupt information: (XEN) IRQ 0 Vec240: (XEN) Apic 0x00, Pin 2: vec=f0 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 1 Vec 56: (XEN) Apic 0x00, Pin 1: vec=38 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 3 Vec 64: (XEN) Apic 0x00, Pin 3: vec=40 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 4 Vec 72: (XEN) Apic 0x00, Pin 4: vec=48 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 5 Vec 80: (XEN) Apic 0x00, Pin 5: vec=50 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 6 Vec 88: (XEN) Apic 0x00, Pin 6: vec=58 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 7 Vec 96: (XEN) Apic 0x00, Pin 7: vec=60 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 8 Vec104: (XEN) Apic 0x00, Pin 8: vec=68 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 9 Vec112: (XEN) Apic 0x00, Pin 9: vec=70 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=L mask=0 dest_id:0 (XEN) IRQ 10 Vec120: (XEN) Apic 0x00, Pin 10: vec=78 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 11 Vec136: (XEN) Apic 0x00, Pin 11: vec=88 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 12 Vec144: (XEN) Apic 0x00, Pin 12: vec=90 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 13 Vec152: (XEN) Apic 0x00, Pin 13: vec=98 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 14 Vec160: (XEN) Apic 0x00, Pin 14: vec=a0 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 15 Vec168: (XEN) Apic 0x00, Pin 15: vec=a8 delivery=LoPri dest=L status=0 polarity=0 irr=0 trig=E mask=0 dest_id:0 (XEN) IRQ 16 Vec176: (XEN) Apic 0x00, Pin 16: vec=b0 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0 (XEN) IRQ 17 Vec 57: (XEN) Apic 0x00, Pin 17: vec=39 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=1 dest_id:0 (XEN) IRQ 18 Vec 49: (XEN) Apic 0x00, Pin 18: vec=31 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0 (XEN) IRQ 22 Vec 41: (XEN) Apic 0x00, Pin 22: vec=29 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0 (XEN) Apic 0x00, Pin 23: vec=d0 delivery=LoPri dest=L status=0 polarity=1 irr=0 trig=L mask=0 dest_id:0 (XEN) PCI-MSI interrupt information: (XEN) MSI 26 vec=b8 lowest edge assert log lowest dest=00000001 mask=0/1/-1 (XEN) MSI 27 vec=c0 lowest edge assert log lowest dest=00000001 mask=0/1/-1 (XEN) MSI 28 vec=c8 lowest edge assert log lowest dest=00000001 mask=0/1/-1 (XEN) MSI 29 vec=d8 lowest edge assert log lowest dest=00000001 mask=0/0/-1 (XEN) MSI 30 vec=51 lowest edge assert log lowest dest=00000001 mask=0/0/-1 (XEN) MSI 31 vec=61 lowest edge assert log lowest dedmes st=00000001 mask=0/0/-1 (XEN) ==== PCI devices ==== (XEN) 04:00.1 - dom 0 - MSIs < 31 > (XEN) 04:00.0 - dom 0 - MSIs < 30 > (XEN) 03:02.0 - dom 0 - MSIs < > (XEN) 02:00.0 - dom 0 - MSIs < > (XEN) 01:00.0 - dom 0 - MSIs < 29 > (XEN) 00:1f.3 - dom 0 - MSIs < > (XEN) 00:1f.0 - dom 0 - MSIs < > (XEN) 00:1d.0 - dom 0 - MSIs < > (XEN) 00:1c.4 - dom 0 - MSIs < 28 > (XEN) 00:1c.0 - dom 0 - MSIs < 27 > (XEN) 00:1b.0 - dom 0 - MSIs < > (XEN) 00:1a.0 - dom 0 - MSIs < > (XEN) 00:16.0 - dom 0 - MSIs < > (XEN) 00:14.0 - dom 0 - MSIs < > (XEN) 00:02.0 - dom 0 - MSIs < > (XEN) 00:01.0 - dom 0 - MSIs < 26 > (XEN) 00:00.0 - dom 0 - MSIs < > --------------050105070007080103010705 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --------------050105070007080103010705--