Re: cpuidle and un-eoid interrupts at the local apic

From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Thimo E <abc@digithi.de>
Cc: Keir Fraser <keir@xen.org>, Jan Beulich <JBeulich@suse.com>,
	"Dong, Eddie" <eddie.dong@intel.com>,
	Xen-develList <xen-devel@lists.xen.org>,
	"Nakajima, Jun" <jun.nakajima@intel.com>,
	"Zhang, Yang Z" <yang.z.zhang@intel.com>,
	"Zhang, Xiantao" <xiantao.zhang@intel.com>
Subject: Re: cpuidle and un-eoid interrupts at the local apic
Date: Mon, 12 Aug 2013 13:04:59 +0100	[thread overview]
Message-ID: <5208CF6B.7030505@citrix.com> (raw)
In-Reply-To: <5208CC8A.7070703@digithi.de>


[-- Attachment #1.1: Type: text/plain, Size: 2916 bytes --]

On 12/08/13 12:52, Thimo E wrote:
> Hello Yang,
>
> attached you'll find the kernel dmesg, xen dmesg, lspci and output of
> /proc/interrupts. If you want to see further logfiles, please let me know.
>
> The processor is a Core i5-4670. The board is an Intel  DH87MC
> Mainboard. I am really not sure if it supports APICv, but VT-d is
> supported enabled enabled.
>
>
>> 4.       The status of IRQ 29 is 10 which means the guest already
>> issues the EOI because the bit IRQ_GUEST_EOI_PENDING is cleared, so
>> there should be no pending EOI in the EOI stack. If possible, can you
>> add some debug message in the guest EOI code path(like
>> _irq_guest_eoi())) to track the EOI?
>>
> I don't see the IRQ29 in /proc/interrupts, what I see is:
> cat xen-dmesg.txt |grep "29": (XEN) allocated vector 29 for irq 20
> cat dmesg.txt | grep "eth0": [   23.152355] e1000e 0000:00:19.0: PCI
> INT A -> GSI 20 (level, low) -> IRQ 20
>                                                   [   23.330408]
> e1000e 0000:00:19.0: eth0: Intel(R) PRO/1000 Network Connection
>
> So is the ethernet irq the bad one ? That is an Onboard Intel network
> adapter.

That would be consistent with the crash seen with our hardware in XenServer

>
>> 6.       I guess the interrupt remapping is enabled in your machine.
>> Can you try to disable IR to see whether it still reproduceable?
>>
>>  
>>
> Just to be sure, your proposal is to try the parameter "no-intremap" ?

specifically, iommu=no-intremap

>
> Best regards
>   Thimo

~Andrew

>
> Am 12.08.2013 10:49, schrieb Zhang, Yang Z:
>>
>> Hi Thimo,
>>
>> From your previous experience and log, it shows:
>>
>> 1.       The interrupt that triggers the issue is a MSI.
>>
>> 2.       MSI are treated as edge-triggered interrupts nomally, except
>> when there is no way to mask the device. In this case, your previous
>> log indicates the device is unmaskable(What special device are you
>> using?Modern PCI devcie should be maskable).
>>
>> 3.       The IRQ 29 is belong to dom0, it seems it is not a HVM
>> related issue.
>>
>> 4.       The status of IRQ 29 is 10 which means the guest already
>> issues the EOI because the bit IRQ_GUEST_EOI_PENDING is cleared, so
>> there should be no pending EOI in the EOI stack. If possible, can you
>> add some debug message in the guest EOI code path(like
>> _irq_guest_eoi())) to track the EOI?
>>
>> 5.       Both of the log show when the issue occured, most of the
>> other interrupts which owned by dom0 were in IRQ_MOVE_PENDING status.
>> Is it a coincidence? Or it happened only on the special condition
>> like heavy of IRQ migration?Perhaps you can disable irq balance in
>> dom0 and pin the IRQ manually.
>>
> |6.       I guess the interrupt remapping is enabled in your machine.
> Can you try to disable IR to see whether it still reproduceable?
>>
>> Also, please provide the whole Xen log.
>>
>>  
>>
>> Best regards,
>>
>> Yang
>>
>


[-- Attachment #1.2: Type: text/html, Size: 16253 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel