* pci-passthrough in pvops causing offline raid @ 2010-11-11 10:24 Mark Adams 2010-11-11 11:13 ` Olivier Hanesse 2010-11-11 16:53 ` [Xen-devel] " Konrad Rzeszutek Wilk 0 siblings, 2 replies; 48+ messages in thread From: Mark Adams @ 2010-11-11 10:24 UTC (permalink / raw) To: xen-devel, xen-users Hi All, Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. In a voip setup, where I have forwarded the onboard NIC interfaces through to domU using the following grub config: module /vmlinuz-2.6.32-5-xen-amd64 placeholder root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) pci=resource_alignment=02:00.0;03:00.0 I'm having a serious issue where the raid card goes offline after an indefinate period of time. Sometimes runs fine for a week, other times 1 day before I get "offline device" errors. Rebooting the machine fixes it straight away, and everything is back online. What in the Xen pciback is causing the raid card to go offline? The only devices hidden are the 2 onboard NIC's. I know that this issue is with Xen, as I had this running on a different server (same xen setup) and it had the same issues, which I initially thought were to do with the raid card. Is there known issues in this kernel and xen version with pciback? I'm going to update to the current package versions this evening (4.0.1-1 and 2.6.32-27) however would appreciate if anyone has any other insight into this issue, or even just a note to say it is a bug that has been fixed in current versions! Thanks, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: pci-passthrough in pvops causing offline raid 2010-11-11 10:24 pci-passthrough in pvops causing offline raid Mark Adams @ 2010-11-11 11:13 ` Olivier Hanesse 2010-11-11 12:03 ` Re: [Xen-users] " Mark Adams 2010-11-11 16:53 ` [Xen-devel] " Konrad Rzeszutek Wilk 1 sibling, 1 reply; 48+ messages in thread From: Olivier Hanesse @ 2010-11-11 11:13 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users [-- Attachment #1.1: Type: text/plain, Size: 1677 bytes --] Hello, What is the module of your raid card ? If it is "megaraid_sas", please try upgrading the version of that module (see others post in the ML). Regards 2010/11/11 Mark Adams <mark@campbell-lange.net> > Hi All, > > Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. > > In a voip setup, where I have forwarded the onboard NIC interfaces > through to domU using the following grub config: > > module /vmlinuz-2.6.32-5-xen-amd64 placeholder > root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet > xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) > pci=resource_alignment=02:00.0;03:00.0 > > I'm having a serious issue where the raid card goes offline after an > indefinate period of time. Sometimes runs fine for a week, other times 1 > day before I get "offline device" errors. Rebooting the machine fixes it > straight away, and everything is back online. > > What in the Xen pciback is causing the raid card to go offline? The > only devices hidden are the 2 onboard NIC's. > > I know that this issue is with Xen, as I had this running on a different > server (same xen setup) and it had the same issues, which I initially > thought were to do with the raid card. > > Is there known issues in this kernel and xen version with pciback? I'm > going to update to the current package versions this evening (4.0.1-1 > and 2.6.32-27) however would appreciate if anyone has any other insight > into this issue, or even just a note to say it is a bug that has been > fixed in current versions! > > Thanks, > Mark > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > [-- Attachment #1.2: Type: text/html, Size: 2215 bytes --] [-- Attachment #2: Type: text/plain, Size: 137 bytes --] _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-users] pci-passthrough in pvops causing offline raid 2010-11-11 11:13 ` Olivier Hanesse @ 2010-11-11 12:03 ` Mark Adams 0 siblings, 0 replies; 48+ messages in thread From: Mark Adams @ 2010-11-11 12:03 UTC (permalink / raw) To: Olivier Hanesse; +Cc: xen-devel, xen-users Hi - It's not megaraid, its an Areca card (arcmsr) This is definately something to do with the pciback. Anyone else got any ideas or views on this? The domU is an HVM debian-squeeze instance. On Thu, Nov 11, 2010 at 12:13:31PM +0100, Olivier Hanesse wrote: > Hello, > > What is the module of your raid card ? > If it is "megaraid_sas", please try upgrading the version of that module > (see others post in the ML). > > Regards > > 2010/11/11 Mark Adams <mark@campbell-lange.net> > > > Hi All, > > > > Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. > > > > In a voip setup, where I have forwarded the onboard NIC interfaces > > through to domU using the following grub config: > > > > module /vmlinuz-2.6.32-5-xen-amd64 placeholder > > root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet > > xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) > > pci=resource_alignment=02:00.0;03:00.0 > > > > I'm having a serious issue where the raid card goes offline after an > > indefinate period of time. Sometimes runs fine for a week, other times 1 > > day before I get "offline device" errors. Rebooting the machine fixes it > > straight away, and everything is back online. > > > > What in the Xen pciback is causing the raid card to go offline? The > > only devices hidden are the 2 onboard NIC's. > > > > I know that this issue is with Xen, as I had this running on a different > > server (same xen setup) and it had the same issues, which I initially > > thought were to do with the raid card. > > > > Is there known issues in this kernel and xen version with pciback? I'm > > going to update to the current package versions this evening (4.0.1-1 > > and 2.6.32-27) however would appreciate if anyone has any other insight > > into this issue, or even just a note to say it is a bug that has been > > fixed in current versions! > > > > Thanks, > > Mark > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-11 10:24 pci-passthrough in pvops causing offline raid Mark Adams 2010-11-11 11:13 ` Olivier Hanesse @ 2010-11-11 16:53 ` Konrad Rzeszutek Wilk 2010-11-11 17:38 ` Mark Adams 2010-11-11 17:40 ` Re: [Xen-devel] " Richie 1 sibling, 2 replies; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-11 16:53 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users On Thu, Nov 11, 2010 at 10:24:17AM +0000, Mark Adams wrote: > Hi All, > > Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. > > In a voip setup, where I have forwarded the onboard NIC interfaces > through to domU using the following grub config: > > module /vmlinuz-2.6.32-5-xen-amd64 placeholder root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) pci=resource_alignment=02:00.0;03:00.0 > > I'm having a serious issue where the raid card goes offline after an > indefinate period of time. Sometimes runs fine for a week, other times 1 > day before I get "offline device" errors. Rebooting the machine fixes it > straight away, and everything is back online. > > What in the Xen pciback is causing the raid card to go offline? The > only devices hidden are the 2 onboard NIC's. You need to give more details. Is the RAID card a 3Ware? An LSI? Do you run with an IOMMU? When the RAID card goes offline, do you see a stop of IRQs going to the device? Are the IRQs for the RAID card sent to all of your CPUs or just a specific one? Are you pinning your guests to specific CPUs? Does the issue disappear if you don't passthrough the NIC interfaces? If so have you run this setup for "a week" to make sure? > > I know that this issue is with Xen, as I had this running on a different > server (same xen setup) and it had the same issues, which I initially > thought were to do with the raid card. So you never ran this setup on this kernel (2.6.32-5) without the Xen hypervisor? > > Is there known issues in this kernel and xen version with pciback? I'm No. It all works perfectly :-) > going to update to the current package versions this evening (4.0.1-1 > and 2.6.32-27) however would appreciate if anyone has any other insight > into this issue, or even just a note to say it is a bug that has been > fixed in current versions! Well, there were issues with the LSI cards having a hidden PCI device. But those are pretty obvious as you can't even use it correctly. There is also a problem with 3Ware 9506 IDE card - which on my box stops sending IRQs on the IOAPIC it has been assigned (28) and instead uses another one (17). Not sure if this is just the PCI card using the wrong PCI interrupt pin on the card and it ends up poking the wrong IOAPIC. ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-11 16:53 ` [Xen-devel] " Konrad Rzeszutek Wilk @ 2010-11-11 17:38 ` Mark Adams 2010-11-11 17:58 ` Konrad Rzeszutek Wilk 2010-11-11 17:40 ` Re: [Xen-devel] " Richie 1 sibling, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-11 17:38 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users On Thu, Nov 11, 2010 at 11:53:40AM -0500, Konrad Rzeszutek Wilk wrote: > On Thu, Nov 11, 2010 at 10:24:17AM +0000, Mark Adams wrote: > > Hi All, > > > > Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. > > > > In a voip setup, where I have forwarded the onboard NIC interfaces > > through to domU using the following grub config: > > > > module /vmlinuz-2.6.32-5-xen-amd64 placeholder root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) pci=resource_alignment=02:00.0;03:00.0 > > > > I'm having a serious issue where the raid card goes offline after an > > indefinate period of time. Sometimes runs fine for a week, other times 1 > > day before I get "offline device" errors. Rebooting the machine fixes it > > straight away, and everything is back online. > > > > What in the Xen pciback is causing the raid card to go offline? The > > only devices hidden are the 2 onboard NIC's. > > You need to give more details. Is the RAID card a 3Ware? An LSI? Do you > run with an IOMMU? When the RAID card goes offline, do you see a stop of > IRQs going to the device? Are the IRQs for the RAID card sent to all of your > CPUs or just a specific one? Are you pinning your guests to specific CPUs? > Does the issue disappear if you don't passthrough the NIC interfaces? If so have > you run this setup for "a week" to make sure? It is an Areca 1220. I can't see anything when the device goes offline apart from [77324.264270] sd 0:0:0:1: rejecting I/O to offline device [77334.005854] sd 0:0:0:0: rejecting I/O to offline device Unfortunately nothing get's logged because there is nothing to write to anymore. I'm not sure how I can see the IRQs otherwise. There is no pinning being done at all, and the machine was running for a few months OK before the pciback was added. Is my kernel module line correct above? are the xen-pciback.permissive and resource_alignment options required? Also I am passing through the onboard NIC's - is this something that should be avoided or is it ok to do? > > > > I know that this issue is with Xen, as I had this running on a different > > server (same xen setup) and it had the same issues, which I initially > > thought were to do with the raid card. > > So you never ran this setup on this kernel (2.6.32-5) without the Xen hypervisor? no, its always had the hypervisor - but it was running ok before the pciback options were added. This week, it's seemed to happen approximately every 24 hours. > > > > > Is there known issues in this kernel and xen version with pciback? I'm > > No. It all works perfectly :-) > > > going to update to the current package versions this evening (4.0.1-1 > > and 2.6.32-27) however would appreciate if anyone has any other insight > > into this issue, or even just a note to say it is a bug that has been > > fixed in current versions! > > Well, there were issues with the LSI cards having a hidden PCI device. But those > are pretty obvious as you can't even use it correctly. There is also > a problem with 3Ware 9506 IDE card - which on my box stops sending IRQs > on the IOAPIC it has been assigned (28) and instead uses another one (17). > Not sure if this is just the PCI card using the wrong PCI interrupt pin on the > card and it ends up poking the wrong IOAPIC. Thanks, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: pci-passthrough in pvops causing offline raid 2010-11-11 17:38 ` Mark Adams @ 2010-11-11 17:58 ` Konrad Rzeszutek Wilk 2010-11-11 18:13 ` [Xen-devel] " Mark Adams 0 siblings, 1 reply; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-11 17:58 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users On Thu, Nov 11, 2010 at 05:38:50PM +0000, Mark Adams wrote: > On Thu, Nov 11, 2010 at 11:53:40AM -0500, Konrad Rzeszutek Wilk wrote: > > On Thu, Nov 11, 2010 at 10:24:17AM +0000, Mark Adams wrote: > > > Hi All, > > > > > > Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. > > > > > > In a voip setup, where I have forwarded the onboard NIC interfaces > > > through to domU using the following grub config: > > > > > > module /vmlinuz-2.6.32-5-xen-amd64 placeholder root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) pci=resource_alignment=02:00.0;03:00.0 > > > > > > I'm having a serious issue where the raid card goes offline after an > > > indefinate period of time. Sometimes runs fine for a week, other times 1 > > > day before I get "offline device" errors. Rebooting the machine fixes it > > > straight away, and everything is back online. > > > > > > What in the Xen pciback is causing the raid card to go offline? The > > > only devices hidden are the 2 onboard NIC's. > > > > You need to give more details. Is the RAID card a 3Ware? An LSI? Do you > > run with an IOMMU? When the RAID card goes offline, do you see a stop of > > IRQs going to the device? Are the IRQs for the RAID card sent to all of your > > CPUs or just a specific one? Are you pinning your guests to specific CPUs? > > Does the issue disappear if you don't passthrough the NIC interfaces? If so have > > you run this setup for "a week" to make sure? > > It is an Areca 1220. I can't see anything when the device goes offline > apart from > > [77324.264270] sd 0:0:0:1: rejecting I/O to offline device > [77334.005854] sd 0:0:0:0: rejecting I/O to offline device That is it? No other details from the driver? Did you poke at the driver (modinfo) to see if there are any options to increase its verbosity. > > Unfortunately nothing get's logged because there is nothing to write to > anymore. I'm not sure how I can see the IRQs otherwise. There is no cat /proc/interrupts > pinning being done at all, and the machine was running for a few months > OK before the pciback was added. Ok, what about your NICs? Are they on-board? Are they sharing the IRQ with the card? You should be able to see this by looking at /proc/interrupts. Which NICs are they? lspci can you help you there. As of matter of fact, run lspci -vvv and send that. > > Is my kernel module line correct above? are the xen-pciback.permissive > and resource_alignment options required? Also I am passing through the Not always. The resource_alignment only if the BARs (look at lspci output) are not page-aligned. If you have no idea what I am talking about then the answer is yes. > onboard NIC's - is this something that should be avoided or is it ok to > do? It is fine. That is the first thing I test.. > > > > > > > I know that this issue is with Xen, as I had this running on a different > > > server (same xen setup) and it had the same issues, which I initially > > > thought were to do with the raid card. > > > > So you never ran this setup on this kernel (2.6.32-5) without the Xen hypervisor? > > no, its always had the hypervisor - but it was running ok before the > pciback options were added. This week, it's seemed to happen > approximately every 24 hours. When this hang occurs, can you do 'xm debug-key Q', 'xm debug-key i', 'xm debug-key z'. Then run 'xm dmesg' and provide that to me? Is your boot disk on the same disk as the RAID? ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-11 17:58 ` Konrad Rzeszutek Wilk @ 2010-11-11 18:13 ` Mark Adams 2010-11-11 18:47 ` Mark Adams 2010-11-11 18:57 ` Konrad Rzeszutek Wilk 0 siblings, 2 replies; 48+ messages in thread From: Mark Adams @ 2010-11-11 18:13 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users [-- Attachment #1: Type: text/plain, Size: 4363 bytes --] On Thu, Nov 11, 2010 at 12:58:09PM -0500, Konrad Rzeszutek Wilk wrote: > On Thu, Nov 11, 2010 at 05:38:50PM +0000, Mark Adams wrote: > > On Thu, Nov 11, 2010 at 11:53:40AM -0500, Konrad Rzeszutek Wilk wrote: > > > On Thu, Nov 11, 2010 at 10:24:17AM +0000, Mark Adams wrote: > > > > Hi All, > > > > > > > > Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. > > > > > > > > In a voip setup, where I have forwarded the onboard NIC interfaces > > > > through to domU using the following grub config: > > > > > > > > module /vmlinuz-2.6.32-5-xen-amd64 placeholder root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) pci=resource_alignment=02:00.0;03:00.0 > > > > > > > > I'm having a serious issue where the raid card goes offline after an > > > > indefinate period of time. Sometimes runs fine for a week, other times 1 > > > > day before I get "offline device" errors. Rebooting the machine fixes it > > > > straight away, and everything is back online. > > > > > > > > What in the Xen pciback is causing the raid card to go offline? The > > > > only devices hidden are the 2 onboard NIC's. > > > > > > You need to give more details. Is the RAID card a 3Ware? An LSI? Do you > > > run with an IOMMU? When the RAID card goes offline, do you see a stop of > > > IRQs going to the device? Are the IRQs for the RAID card sent to all of your > > > CPUs or just a specific one? Are you pinning your guests to specific CPUs? > > > Does the issue disappear if you don't passthrough the NIC interfaces? If so have > > > you run this setup for "a week" to make sure? > > > > It is an Areca 1220. I can't see anything when the device goes offline > > apart from > > > > [77324.264270] sd 0:0:0:1: rejecting I/O to offline device > > [77334.005854] sd 0:0:0:0: rejecting I/O to offline device > > That is it? No other details from the driver? Did you poke at the driver (modinfo) > to see if there are any options to increase its verbosity. I can't do anything once its happened, everything is offline so I have no utils... > > > > > Unfortunately nothing get's logged because there is nothing to write to > > anymore. I'm not sure how I can see the IRQs otherwise. There is no > > cat /proc/interrupts > > > pinning being done at all, and the machine was running for a few months > > OK before the pciback was added. > > Ok, what about your NICs? Are they on-board? Are they sharing the IRQ > with the card? You should be able to see this by looking at /proc/interrupts. > Which NICs are they? lspci can you help you there. As of matter of fact, run > lspci -vvv and send that. It is the onboard nics, they are Intel 82574L. I can see the arcmsr line, but not anything for the NICS (because they are hidden?) 39: 1126249 0 0 0 0 0 0 0 xen-pirq-ioapic-level arcmsr Nothing else is on 1126249 see lspci.txt attached. > > > > Is my kernel module line correct above? are the xen-pciback.permissive > > and resource_alignment options required? Also I am passing through the > > Not always. The resource_alignment only if the BARs (look at lspci output) are > not page-aligned. If you have no idea what I am talking about then the answer > is yes. > > > onboard NIC's - is this something that should be avoided or is it ok to > > do? > > It is fine. That is the first thing I test.. > > > > > > > > > > > I know that this issue is with Xen, as I had this running on a different > > > > server (same xen setup) and it had the same issues, which I initially > > > > thought were to do with the raid card. > > > > > > So you never ran this setup on this kernel (2.6.32-5) without the Xen hypervisor? > > > > no, its always had the hypervisor - but it was running ok before the > > pciback options were added. This week, it's seemed to happen > > approximately every 24 hours. > > When this hang occurs, can you do 'xm debug-key Q', 'xm debug-key i', 'xm debug-key z'. > Then run 'xm dmesg' and provide that to me? I can try this, but It probably won't work as the device is will not be readable. > > Is your boot disk on the same disk as the RAID? There are 2 raids, a Raid1 for the OS (/boot / /var /tmp /usr) and a raid5 for VM's - They both dissapear at the same time so it appears the card is dissapearing.. [-- Attachment #2: lspci.txt --] [-- Type: text/plain, Size: 94196 bytes --] 00:00.0 Host bridge: Intel Corporation 5520 I/O Hub to ESI Port (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Capabilities: [60] MSI: Enable- Count=1/2 Maskable+ 64bit- Address: 00000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [90] Express (v2) Root Port (Slot-), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <64us ClockPM- Surprise+ LLActRep+ BwNot+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Range BCD, TimeoutDis+ ARIFwd+ DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis- ARIFwd- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?> 00:01.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev 13) (prog-if 00 [Normal decode]) Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=00, secondary=0f, subordinate=0f, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Subsystem: Intel Corporation Device 0000 Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit- Address: fee005f8 Data: 0000 Masking: 00000003 Pending: 00000000 Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <64us ClockPM- Surprise+ LLActRep+ BwNot+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #0, PowerLimit 0.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd Off, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- Changed: MRL- PresDet+ LinkState- RootCtl: ErrCorrectable+ ErrNon-Fatal+ ErrFatal+ PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Range BCD, TimeoutDis+ ARIFwd+ DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis- ARIFwd- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?> Kernel driver in use: pcieport Kernel modules: shpchp 00:03.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev 13) (prog-if 00 [Normal decode]) Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=00, secondary=0e, subordinate=0e, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Subsystem: Intel Corporation Device 0000 Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit- Address: fee00618 Data: 0000 Masking: 00000003 Pending: 00000000 Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <512ns, L1 <64us ClockPM- Surprise+ LLActRep+ BwNot+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #4, PowerLimit 25.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd Off, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- Changed: MRL- PresDet+ LinkState- RootCtl: ErrCorrectable+ ErrNon-Fatal+ ErrFatal+ PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Range BCD, TimeoutDis+ ARIFwd+ DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis- ARIFwd- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?> Kernel driver in use: pcieport Kernel modules: shpchp 00:05.0 PCI bridge: Intel Corporation 5520/X58 I/O Hub PCI Express Root Port 5 (rev 13) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=00, secondary=08, subordinate=0d, sec-latency=0 I/O behind bridge: 0000d000-0000efff Memory behind bridge: f9f00000-fbefffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Subsystem: Intel Corporation Device 0000 Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit- Address: fee00638 Data: 0000 Masking: 00000003 Pending: 00000000 Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <512ns, L1 <64us ClockPM- Surprise+ LLActRep+ BwNot+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #3, PowerLimit 25.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd Off, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet+ LinkState+ RootCtl: ErrCorrectable+ ErrNon-Fatal+ ErrFatal+ PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Range BCD, TimeoutDis+ ARIFwd+ DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis- ARIFwd- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Kernel driver in use: pcieport Kernel modules: shpchp 00:07.0 PCI bridge: Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev 13) (prog-if 00 [Normal decode]) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=00, secondary=05, subordinate=07, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: f9e00000-f9efffff Prefetchable memory behind bridge: 00000000f7c00000-00000000f7ffffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Subsystem: Intel Corporation Device 0000 Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit- Address: fee00658 Data: 0000 Masking: 00000003 Pending: 00000000 Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <512ns, L1 <64us ClockPM- Surprise+ LLActRep+ BwNot+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #5, PowerLimit 25.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd Off, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet+ LinkState+ RootCtl: ErrCorrectable+ ErrNon-Fatal+ ErrFatal+ PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Range BCD, TimeoutDis+ ARIFwd+ DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis- ARIFwd- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [150 v1] Access Control Services ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans- Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?> Kernel driver in use: pcieport Kernel modules: shpchp 00:13.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub I/OxAPIC Interrupt Controller (rev 13) (prog-if 20 [IO(X)-APIC]) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Region 0: Memory at fec8a000 (32-bit, non-prefetchable) [size=4K] Capabilities: [6c] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- 00:14.0 PIC: Intel Corporation 5520/5500/X58 I/O Hub System Management Registers (rev 13) (prog-if 00 [8259]) Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0 unlimited, L1 unlimited ClockPM- Surprise+ LLActRep+ BwNot+ LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB 00:14.1 PIC: Intel Corporation 5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers (rev 13) (prog-if 00 [8259]) Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0 unlimited, L1 unlimited ClockPM- Surprise+ LLActRep+ BwNot+ LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB 00:14.2 PIC: Intel Corporation 5520/5500/X58 I/O Hub Control Status and RAS Registers (rev 13) (prog-if 00 [8259]) Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0 unlimited, L1 unlimited ClockPM- Surprise+ LLActRep+ BwNot+ LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB 00:14.3 PIC: Intel Corporation 5520/5500/X58 I/O Hub Throttle Registers (rev 13) (prog-if 00 [8259]) Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- 00:16.0 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 43 Region 0: Memory at f8ed4000 (64-bit, non-prefetchable) [size=16K] Capabilities: [80] MSI-X: Enable+ Count=1 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: ioatdma Kernel modules: ioatdma 00:16.1 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin B routed to IRQ 44 Region 0: Memory at f8ed8000 (64-bit, non-prefetchable) [size=16K] Capabilities: [80] MSI-X: Enable+ Count=1 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: ioatdma Kernel modules: ioatdma 00:16.2 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin C routed to IRQ 45 Region 0: Memory at f8edc000 (64-bit, non-prefetchable) [size=16K] Capabilities: [80] MSI-X: Enable+ Count=1 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: ioatdma Kernel modules: ioatdma 00:16.3 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin D routed to IRQ 46 Region 0: Memory at f8ee0000 (64-bit, non-prefetchable) [size=16K] Capabilities: [80] MSI-X: Enable+ Count=1 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: ioatdma Kernel modules: ioatdma 00:16.4 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 43 Region 0: Memory at f8ee4000 (64-bit, non-prefetchable) [size=16K] Capabilities: [80] MSI-X: Enable+ Count=1 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: ioatdma Kernel modules: ioatdma 00:16.5 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin B routed to IRQ 44 Region 0: Memory at f8ee8000 (64-bit, non-prefetchable) [size=16K] Capabilities: [80] MSI-X: Enable+ Count=1 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: ioatdma Kernel modules: ioatdma 00:16.6 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin C routed to IRQ 45 Region 0: Memory at f8eec000 (64-bit, non-prefetchable) [size=16K] Capabilities: [80] MSI-X: Enable+ Count=1 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: ioatdma Kernel modules: ioatdma 00:16.7 System peripheral: Intel Corporation 5520/5500/X58 Chipset QuickData Technology Device (rev 13) Subsystem: Intel Corporation Device 0000 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin D routed to IRQ 46 Region 0: Memory at f8ef0000 (64-bit, non-prefetchable) [size=16K] Capabilities: [80] MSI-X: Enable+ Count=1 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00003000 Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Not Supported, TimeoutDis+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [e0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: ioatdma Kernel modules: ioatdma 00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 (prog-if 00 [UHCI]) Subsystem: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 16 Region 4: I/O ports at 7400 [size=32] Capabilities: [50] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: uhci_hcd Kernel modules: uhci-hcd 00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 (prog-if 00 [UHCI]) Subsystem: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 21 Region 4: I/O ports at 7480 [size=32] Capabilities: [50] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: uhci_hcd Kernel modules: uhci-hcd 00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 (prog-if 00 [UHCI]) Subsystem: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin D routed to IRQ 19 Region 4: I/O ports at 7800 [size=32] Capabilities: [50] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: uhci_hcd Kernel modules: uhci-hcd 00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 (prog-if 20 [EHCI]) Subsystem: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin C routed to IRQ 18 Region 0: Memory at f8ef6000 (32-bit, non-prefetchable) [size=1K] Capabilities: [50] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [58] Debug port: BAR=1 offset=00a0 Capabilities: [98] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: ehci_hcd Kernel modules: ehci-hcd 00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller Subsystem: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 22 Region 0: Memory at f8ef8000 (64-bit, non-prefetchable) [size=16K] Capabilities: [50] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [70] Express (v1) Root Complex Integrated Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE- FLReset+ DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed unknown, Width x0, ASPM unknown, Latency L0 <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- VC1: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable- ID=0 ArbSelect=Fixed TC/VC=00 Status: NegoPending- InProgress- Capabilities: [130 v1] Root Complex Link Desc: PortNumber=0f ComponentID=00 EltType=Config Link0: Desc: TargetPort=00 TargetComponent=00 AssocRCRB- LinkType=MemMapped LinkValid+ Addr: 00000000fed1c000 Kernel driver in use: HDA Intel Kernel modules: snd-hda-intel 00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=00, secondary=04, subordinate=04, sec-latency=0 I/O behind bridge: 00001000-00001fff Memory behind bridge: 80000000-801fffff Prefetchable memory behind bridge: 0000000080200000-00000000803fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v1) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #1, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <256ns, L1 <4us ClockPM- Surprise- LLActRep+ BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Slot #0, PowerLimit 10.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee00678 Data: 0000 Capabilities: [90] Subsystem: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1 Capabilities: [a0] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed+ WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01 Status: NegoPending- InProgress- Capabilities: [180 v1] Root Complex Link Desc: PortNumber=01 ComponentID=00 EltType=Config Link0: Desc: TargetPort=00 TargetComponent=00 AssocRCRB- LinkType=MemMapped LinkValid+ Addr: 00000000fed1c000 Kernel driver in use: pcieport Kernel modules: shpchp 00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=00, secondary=03, subordinate=03, sec-latency=0 I/O behind bridge: 0000c000-0000cfff Memory behind bridge: f9d00000-f9dfffff Prefetchable memory behind bridge: 0000000080400000-00000000805fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v1) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #5, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 <4us ClockPM- Surprise- LLActRep+ BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Slot #0, PowerLimit 10.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet+ LinkState+ RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee00698 Data: 0000 Capabilities: [90] Subsystem: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5 Capabilities: [a0] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed+ WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01 Status: NegoPending- InProgress- Capabilities: [180 v1] Root Complex Link Desc: PortNumber=05 ComponentID=00 EltType=Config Link0: Desc: TargetPort=00 TargetComponent=00 AssocRCRB- LinkType=MemMapped LinkValid+ Addr: 00000000fed1c000 Kernel driver in use: pcieport Kernel modules: shpchp 00:1c.5 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6 (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=00, secondary=02, subordinate=02, sec-latency=0 I/O behind bridge: 0000b000-0000bfff Memory behind bridge: f9c00000-f9cfffff Prefetchable memory behind bridge: 0000000080600000-00000000807fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v1) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #6, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 <4us ClockPM- Surprise- LLActRep+ BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug+ Surprise+ Slot #0, PowerLimit 10.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet+ LinkState+ RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee006b8 Data: 0000 Capabilities: [90] Subsystem: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6 Capabilities: [a0] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed+ WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01 Status: NegoPending- InProgress- Capabilities: [180 v1] Root Complex Link Desc: PortNumber=06 ComponentID=00 EltType=Config Link0: Desc: TargetPort=00 TargetComponent=00 AssocRCRB- LinkType=MemMapped LinkValid+ Addr: 00000000fed1c000 Kernel driver in use: pcieport Kernel modules: shpchp 00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 (prog-if 00 [UHCI]) Subsystem: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 23 Region 4: I/O ports at 7880 [size=32] Capabilities: [50] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: uhci_hcd Kernel modules: uhci-hcd 00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 (prog-if 00 [UHCI]) Subsystem: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 19 Region 4: I/O ports at 7c00 [size=32] Capabilities: [50] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: uhci_hcd Kernel modules: uhci-hcd 00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 (prog-if 00 [UHCI]) Subsystem: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin C routed to IRQ 18 Region 4: I/O ports at 8000 [size=32] Capabilities: [50] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: uhci_hcd Kernel modules: uhci-hcd 00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 (prog-if 20 [EHCI]) Subsystem: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 23 Region 0: Memory at f8efc000 (32-bit, non-prefetchable) [size=1K] Capabilities: [50] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [58] Debug port: BAR=1 offset=00a0 Capabilities: [98] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: ehci_hcd Kernel modules: ehci-hcd 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) (prog-if 01 [Subtractive decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Bus: primary=00, secondary=01, subordinate=01, sec-latency=32 I/O behind bridge: 0000a000-0000afff Memory behind bridge: f8f00000-f9bfffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR+ BridgeCtl: Parity- SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [50] Subsystem: Intel Corporation 82801 PCI Bridge 00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller Subsystem: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Capabilities: [e0] Vendor Specific Information: Len=0c <?> Kernel modules: iTCO_wdt 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller #1 (prog-if 8f [Master SecP SecO PriP PriO]) Subsystem: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller #1 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 19 Region 0: I/O ports at 8c00 [size=8] Region 1: I/O ports at 8880 [size=4] Region 2: I/O ports at 8800 [size=8] Region 3: I/O ports at 8480 [size=4] Region 4: I/O ports at 8400 [size=16] Region 5: I/O ports at 8080 [size=16] Capabilities: [70] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [b0] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: ata_piix Kernel modules: ata_generic, ata_piix 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller Subsystem: Intel Corporation 82801JI (ICH10 Family) SMBus Controller Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin C routed to IRQ 18 Region 0: Memory at f8efe000 (64-bit, non-prefetchable) [size=256] Region 4: I/O ports at 0400 [size=32] Kernel driver in use: i801_smbus Kernel modules: i2c-i801 00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller #2 (prog-if 85 [Master SecO PriO]) Subsystem: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller #2 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 19 Region 0: I/O ports at 9c00 [size=8] Region 1: I/O ports at 9880 [size=4] Region 2: I/O ports at 9800 [size=8] Region 3: I/O ports at 9480 [size=4] Region 4: I/O ports at 9400 [size=16] Region 5: I/O ports at 9080 [size=16] Capabilities: [70] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [b0] PCI Advanced Features AFCap: TP+ FLR+ AFCtrl: FLR- AFStatus: TP- Kernel driver in use: ata_piix Kernel modules: ata_generic, ata_piix 01:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 10) (prog-if 00 [VGA controller]) Subsystem: ASPEED Technology, Inc. ASPEED Graphics Family Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin A routed to IRQ 10 Region 0: Memory at f9000000 (32-bit, non-prefetchable) [size=8M] Region 1: Memory at f8fe0000 (32-bit, non-prefetchable) [size=128K] Region 2: I/O ports at a880 [size=128] Expansion ROM at <unassigned> [disabled] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- 01:05.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0) (prog-if 10 [OHCI]) Subsystem: VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 64 (8000ns max), Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 21 Region 0: Memory at f9bfe000 (32-bit, non-prefetchable) [size=2K] Region 1: I/O ports at ac00 [size=128] Capabilities: [50] Power Management version 2 Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0-,D1-,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: firewire_ohci Kernel modules: firewire-ohci 02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection Subsystem: Intel Corporation Device 0000 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 17 Region 0: Memory at f9c00000 (32-bit, non-prefetchable) [size=128K] Region 2: I/O ports at bc00 [size=32] Region 3: Memory at f9c20000 (32-bit, non-prefetchable) [size=16K] Capabilities: [c8] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 00000000fee00c98 Data: 0000 Capabilities: [e0] Express (v1) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <128ns, L1 <64us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Capabilities: [a0] MSI-X: Enable+ Count=5 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn- Kernel driver in use: pciback Kernel modules: e1000e 03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection Subsystem: Intel Corporation Device 0000 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 16 Region 0: Memory at f9de0000 (32-bit, non-prefetchable) [size=128K] Region 2: I/O ports at cc00 [size=32] Region 3: Memory at f9ddc000 (32-bit, non-prefetchable) [size=16K] Capabilities: [c8] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 00000000fee00cb8 Data: 0000 Capabilities: [e0] Express (v1) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <128ns, L1 <64us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Capabilities: [a0] MSI-X: Enable+ Count=5 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Kernel driver in use: pciback Kernel modules: e1000e 05:00.0 PCI bridge: Intel Corporation 80333 Segment-A PCI Express-to-PCI Express Bridge (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=05, secondary=07, subordinate=07, sec-latency=64 I/O behind bridge: 0000f000-00000fff Memory behind bridge: f9e00000-f9efffff Prefetchable memory behind bridge: 00000000f7c00000-00000000f7ffffff Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [44] Express (v1) PCI/PCI-X Bridge, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x8, ASPM L0s, Latency L0 unlimited, L1 unlimited ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [5c] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [6c] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [d8] PCI-X bridge device Secondary Status: 64bit+ 133MHz+ SCD- USC- SCO- SRD- Freq=133MHz Status: Dev=05:00.0 64bit- 133MHz- SCD- USC- SCO- SRD- Upstream: Capacity=65535 CommitmentLimit=65535 Downstream: Capacity=65535 CommitmentLimit=65535 Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [300 v1] Power Budgeting <?> Kernel modules: shpchp 05:00.2 PCI bridge: Intel Corporation 80333 Segment-B PCI Express-to-PCI Express Bridge (prog-if 00 [Normal decode]) Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=05, secondary=06, subordinate=06, sec-latency=64 I/O behind bridge: 0000f000-00000fff Memory behind bridge: fff00000-000fffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [44] Express (v1) PCI/PCI-X Bridge, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x8, ASPM L0s, Latency L0 unlimited, L1 unlimited ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- Capabilities: [5c] MSI: Enable- Count=1/1 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [6c] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [d8] PCI-X bridge device Secondary Status: 64bit+ 133MHz+ SCD- USC- SCO- SRD- Freq=133MHz Status: Dev=05:00.2 64bit- 133MHz- SCD- USC- SCO- SRD- Upstream: Capacity=65535 CommitmentLimit=65535 Downstream: Capacity=65535 CommitmentLimit=65535 Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [300 v1] Power Budgeting <?> Kernel modules: shpchp 07:0e.0 RAID bus controller: Areca Technology Corp. ARC-1220 8-Port PCI-Express to SATA RAID Controller Subsystem: Areca Technology Corp. ARC-1220 8-Port PCI-Express to SATA RAID Controller Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr+ Stepping+ SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 64 (32000ns min), Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 39 Region 0: Memory at f9eff000 (32-bit, non-prefetchable) [size=4K] Region 2: Memory at f7c00000 (32-bit, prefetchable) [size=4M] Expansion ROM at f9ee0000 [disabled] [size=64K] Capabilities: [c0] Power Management version 2 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [d0] MSI: Enable- Count=1/2 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [e0] PCI-X non-bridge device Command: DPERE+ ERO- RBC=1024 OST=8 Status: Dev=07:0e.0 64bit+ 133MHz+ SCD- USC- DC=bridge DMMRBC=1024 DMOST=4 DMCRS=32 RSCEM- 266MHz- 533MHz- Kernel driver in use: arcmsr Kernel modules: arcmsr 08:00.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=08, secondary=09, subordinate=0d, sec-latency=0 I/O behind bridge: 0000d000-0000efff Memory behind bridge: f9f00000-fbefffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v1) Upstream Port, MSI 00 DevCap: MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-SlotPowerLimit 25.000W DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <4us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Capabilities: [c0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 05, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [200 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=4 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=02 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32+ WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- Port Arbitration Table <?> Kernel driver in use: pcieport Kernel modules: shpchp 09:02.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=09, secondary=0c, subordinate=0d, sec-latency=0 I/O behind bridge: 0000e000-0000efff Memory behind bridge: faf00000-fbefffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v1) Downstream Port (Slot-), MSI 00 DevCap: MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr+ UncorrErr- FatalErr+ UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #2, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <4us ClockPM- Surprise+ LLActRep+ BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- Capabilities: [c0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000fee006d8 Data: 0000 Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES+ TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 05, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [200 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- Kernel driver in use: pcieport Kernel modules: shpchp 09:04.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Bus: primary=09, secondary=0a, subordinate=0b, sec-latency=0 I/O behind bridge: 0000d000-0000dfff Memory behind bridge: f9f00000-faefffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity+ SERR+ NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v1) Downstream Port (Slot-), MSI 00 DevCap: MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag+ RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr+ UncorrErr- FatalErr+ UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #4, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <4us ClockPM- Surprise+ LLActRep+ BwNot- LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt- Capabilities: [c0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+ Address: 00000000fee006f8 Data: 0000 Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES+ TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 05, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [200 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- Kernel driver in use: pcieport Kernel modules: shpchp 0a:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) Subsystem: Intel Corporation Gigabit ET Quad Port Server Adapter Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 26 Region 0: Memory at fa780000 (32-bit, non-prefetchable) [size=128K] Region 1: Memory at fa000000 (32-bit, non-prefetchable) [size=4M] Region 2: I/O ports at d880 [size=32] Region 3: Memory at f9ffc000 (32-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [70] MSI-X: Enable+ Count=10 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #4, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-5d-54-44 Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 1 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy- IOVSta: Migration- Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 00 VF offset: 384, stride: 2, Device ID: 10ca Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000f9f00000 (64-bit, non-prefetchable) Region 3: Memory at 00000000f9f20000 (64-bit, non-prefetchable) VF Migration: offset: 00000000, BIR: 0 Kernel driver in use: igb Kernel modules: igb 0a:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) Subsystem: Intel Corporation Gigabit ET Quad Port Server Adapter Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin B routed to IRQ 25 Region 0: Memory at faee0000 (32-bit, non-prefetchable) [size=128K] Region 1: Memory at fa800000 (32-bit, non-prefetchable) [size=4M] Region 2: I/O ports at dc00 [size=32] Region 3: Memory at fa7fc000 (32-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [70] MSI-X: Enable+ Count=10 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend- LnkCap: Port #4, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-5d-54-44 Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy- IOVSta: Migration- Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 01 VF offset: 384, stride: 2, Device ID: 10ca Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000f9f40000 (64-bit, non-prefetchable) Region 3: Memory at 00000000f9f60000 (64-bit, non-prefetchable) VF Migration: offset: 00000000, BIR: 0 Kernel driver in use: igb Kernel modules: igb 0c:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) Subsystem: Intel Corporation Gigabit ET Quad Port Server Adapter Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin A routed to IRQ 27 Region 0: Memory at fb780000 (32-bit, non-prefetchable) [size=128K] Region 1: Memory at fb000000 (32-bit, non-prefetchable) [size=4M] Region 2: I/O ports at e880 [size=32] Region 3: Memory at faffc000 (32-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [70] MSI-X: Enable+ Count=10 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend- LnkCap: Port #2, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-5d-54-40 Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 1 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy- IOVSta: Migration- Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 00 VF offset: 384, stride: 2, Device ID: 10ca Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000faf00000 (64-bit, non-prefetchable) Region 3: Memory at 00000000faf20000 (64-bit, non-prefetchable) VF Migration: offset: 00000000, BIR: 0 Kernel driver in use: igb Kernel modules: igb 0c:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) Subsystem: Intel Corporation Gigabit ET Quad Port Server Adapter Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 256 bytes Interrupt: pin B routed to IRQ 29 Region 0: Memory at fbee0000 (32-bit, non-prefetchable) [size=128K] Region 1: Memory at fb800000 (32-bit, non-prefetchable) [size=4M] Region 2: I/O ports at ec00 [size=32] Region 3: Memory at fb7fc000 (32-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME- Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [70] MSI-X: Enable+ Count=10 Masked- Vector table: BAR=3 offset=00000000 PBA: BAR=3 offset=00002000 Capabilities: [a0] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend- LnkCap: Port #2, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us ClockPM- Surprise- LLActRep- BwNot- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn- Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-5d-54-40 Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy- IOVSta: Migration- Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 01 VF offset: 384, stride: 2, Device ID: 10ca Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000faf40000 (64-bit, non-prefetchable) Region 3: Memory at 00000000faf60000 (64-bit, non-prefetchable) VF Migration: offset: 00000000, BIR: 0 Kernel driver in use: igb Kernel modules: igb [-- Attachment #3: Type: text/plain, Size: 137 bytes --] _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: pci-passthrough in pvops causing offline raid 2010-11-11 18:13 ` [Xen-devel] " Mark Adams @ 2010-11-11 18:47 ` Mark Adams 2010-11-11 19:06 ` Konrad Rzeszutek Wilk 2010-11-11 18:57 ` Konrad Rzeszutek Wilk 1 sibling, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-11 18:47 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users On Thu, Nov 11, 2010 at 06:13:29PM +0000, Mark Adams wrote: > On Thu, Nov 11, 2010 at 12:58:09PM -0500, Konrad Rzeszutek Wilk wrote: > > On Thu, Nov 11, 2010 at 05:38:50PM +0000, Mark Adams wrote: > > > On Thu, Nov 11, 2010 at 11:53:40AM -0500, Konrad Rzeszutek Wilk wrote: > > > > On Thu, Nov 11, 2010 at 10:24:17AM +0000, Mark Adams wrote: > > > > > Hi All, > > > > > > > > > > Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. > > > > > > > > > > In a voip setup, where I have forwarded the onboard NIC interfaces > > > > > through to domU using the following grub config: > > > > > > > > > > module /vmlinuz-2.6.32-5-xen-amd64 placeholder root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) pci=resource_alignment=02:00.0;03:00.0 > > > > > > > > > > I'm having a serious issue where the raid card goes offline after an > > > > > indefinate period of time. Sometimes runs fine for a week, other times 1 > > > > > day before I get "offline device" errors. Rebooting the machine fixes it > > > > > straight away, and everything is back online. > > > > > > > > > > What in the Xen pciback is causing the raid card to go offline? The > > > > > only devices hidden are the 2 onboard NIC's. > > > > > > > > You need to give more details. Is the RAID card a 3Ware? An LSI? Do you > > > > run with an IOMMU? When the RAID card goes offline, do you see a stop of > > > > IRQs going to the device? Are the IRQs for the RAID card sent to all of your > > > > CPUs or just a specific one? Are you pinning your guests to specific CPUs? > > > > Does the issue disappear if you don't passthrough the NIC interfaces? If so have > > > > you run this setup for "a week" to make sure? > > > > > > It is an Areca 1220. I can't see anything when the device goes offline > > > apart from > > > > > > [77324.264270] sd 0:0:0:1: rejecting I/O to offline device > > > [77334.005854] sd 0:0:0:0: rejecting I/O to offline device > > > > That is it? No other details from the driver? Did you poke at the driver (modinfo) > > to see if there are any options to increase its verbosity. > > I can't do anything once its happened, everything is offline so I have > no utils... > > > > > > > > Unfortunately nothing get's logged because there is nothing to write to > > > anymore. I'm not sure how I can see the IRQs otherwise. There is no > > > > cat /proc/interrupts > > > > > pinning being done at all, and the machine was running for a few months > > > OK before the pciback was added. > > > > Ok, what about your NICs? Are they on-board? Are they sharing the IRQ > > with the card? You should be able to see this by looking at /proc/interrupts. > > Which NICs are they? lspci can you help you there. As of matter of fact, run > > lspci -vvv and send that. > > It is the onboard nics, they are Intel 82574L. I can see the arcmsr > line, but not anything for the NICS (because they are hidden?) > > 39: 1126249 0 0 0 0 0 0 0 xen-pirq-ioapic-level arcmsr > > Nothing else is on 1126249 > > see lspci.txt attached. > I've just noticed this at the end of xm dmesg (XEN) msi.c:715: MSI is already in use on device 02:00.0 (XEN) msi.c:715: MSI is already in use on device 02:00.0 (XEN) msi.c:715: MSI is already in use on device 02:00.0 Something else trying to use the device being exported? (the nics are 02:00.0 and 03:00.0) ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: pci-passthrough in pvops causing offline raid 2010-11-11 18:47 ` Mark Adams @ 2010-11-11 19:06 ` Konrad Rzeszutek Wilk 2010-11-11 19:22 ` [Xen-users] " Mark Adams 2010-11-12 17:10 ` [Xen-users] " Mark Adams 0 siblings, 2 replies; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-11 19:06 UTC (permalink / raw) To: Mark Adams, JBeulich; +Cc: xen-devel, xen-users > I've just noticed this at the end of xm dmesg > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > Something else trying to use the device being exported? (the nics are > 02:00.0 and 03:00.0) Hmm, looks like it, but it should not have happend. Can you attach the output of 'xm dmesg' and also do the 'xm debug-keys ..' that I asked for in the previous e-mail? Jan, the fixes for the MSI you did, they weren't for 4.0.1 right? Just for unstable? ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-11 19:06 ` Konrad Rzeszutek Wilk @ 2010-11-11 19:22 ` Mark Adams 2010-11-11 19:42 ` Re: [Xen-devel] " Mark Adams 2010-11-12 17:10 ` [Xen-users] " Mark Adams 1 sibling, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-11 19:22 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich [-- Attachment #1: Type: text/plain, Size: 846 bytes --] See attached On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > I've just noticed this at the end of xm dmesg > > > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > Something else trying to use the device being exported? (the nics are > > 02:00.0 and 03:00.0) > > Hmm, looks like it, but it should not have happend. Can you attach > the output of 'xm dmesg' and also do the 'xm debug-keys ..' that I > asked for in the previous e-mail? > > Jan, the fixes for the MSI you did, they weren't for 4.0.1 right? Just > for unstable? > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users [-- Attachment #2: dmesg_debugi.txt --] [-- Type: text/plain, Size: 16384 bytes --] ype=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 8 affinity:00000000,00000000,00000000,00000001 vec:60 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 8(----), (XEN) IRQ: 9 affinity:00000000,00000000,00000000,00000001 vec:68 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 9(----), (XEN) IRQ: 10 affinity:00000000,00000000,00000000,00000001 vec:70 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 11 affinity:00000000,00000000,00000000,00000001 vec:78 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 12 affinity:00000000,00000000,00000000,00000001 vec:88 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 12(----), (XEN) IRQ: 13 affinity:00000000,00000000,00000000,000000ff vec:90 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 14 affinity:00000000,00000000,00000000,00000001 vec:98 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 15 affinity:00000000,00000000,00000000,00000001 vec:a0 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 16 affinity:00000000,00000000,00000000,00000001 vec:b0 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 16(----),3: 16(--M-), (XEN) IRQ: 17 affinity:00000000,00000000,00000000,00000008 vec:a8 type=IO-APIC-level status=00000010 in-flight=0 domain-list=3: 17(--M-), (XEN) IRQ: 18 affinity:00000000,00000000,00000000,00000001 vec:91 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 18(----), (XEN) IRQ: 19 affinity:00000000,00000000,00000000,00000001 vec:a9 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 19(----), (XEN) IRQ: 21 affinity:00000000,00000000,00000000,00000001 vec:a1 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 21(----), (XEN) IRQ: 22 affinity:00000000,00000000,00000000,00000010 vec:93 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 22(----), (XEN) IRQ: 23 affinity:00000000,00000000,00000000,00000001 vec:b1 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 23(----), (XEN) IRQ: 25 affinity:00000000,00000000,00000000,000000ff vec:a2 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 26 affinity:00000000,00000000,00000000,000000ff vec:4a type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 27 affinity:00000000,00000000,00000000,000000ff vec:41 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 29 affinity:00000000,00000000,00000000,000000ff vec:b9 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 39 affinity:00000000,00000000,00000000,00000001 vec:99 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 39(----), (XEN) IRQ: 43 affinity:00000000,00000000,00000000,000000ff vec:33 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 44 affinity:00000000,00000000,00000000,000000ff vec:43 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 45 affinity:00000000,00000000,00000000,000000ff vec:53 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 46 affinity:00000000,00000000,00000000,000000ff vec:63 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 48 affinity:00000000,00000000,00000000,00000001 vec:28 type=DMA_MSI status=00000000 mapped, unbound (XEN) IRQ: 49 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:b8 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 50 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c0 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 51 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c8 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 52 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d0 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 53 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d8 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 54 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:21 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 55 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:29 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 56 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:31 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 57 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:39 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 58 affinity:00000000,00000000,00000000,00000002 vec:49 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:294(----), (XEN) IRQ: 59 affinity:00000000,00000000,00000000,00000002 vec:51 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:293(----), (XEN) IRQ: 60 affinity:00000000,00000000,00000000,00000002 vec:59 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:292(----), (XEN) IRQ: 61 affinity:00000000,00000000,00000000,00000002 vec:61 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:291(----), (XEN) IRQ: 62 affinity:00000000,00000000,00000000,00000002 vec:69 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:290(----), (XEN) IRQ: 63 affinity:00000000,00000000,00000000,00000002 vec:71 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:289(----), (XEN) IRQ: 64 affinity:00000000,00000000,00000000,00000002 vec:79 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:288(----), (XEN) IRQ: 65 affinity:00000000,00000000,00000000,00000002 vec:81 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:287(----), (XEN) IRQ: 66 affinity:00000000,00000000,00000000,00000002 vec:89 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:286(----), (XEN) IRQ: 67 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c1 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 68 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c9 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 69 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d1 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 70 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d9 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 71 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:22 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 72 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:2a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 73 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:32 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 74 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:3a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 75 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:42 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 76 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:52 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 77 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:5a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 78 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:62 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 79 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:6a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 80 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:72 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 81 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:7a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 82 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:8a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 83 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:92 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 84 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:9a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 85 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:aa type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 86 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:b2 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 87 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:ba type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 88 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c2 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 89 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:ca type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 90 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d2 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 91 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:da type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 92 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:23 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 93 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:2b type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 94 affinity:00000000,00000000,00000000,00000008 vec:3b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:258(----), (XEN) IRQ: 95 affinity:00000000,00000000,00000000,00000008 vec:4b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:257(----), (XEN) IRQ: 96 affinity:00000000,00000000,00000000,00000008 vec:5b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:256(----), (XEN) IRQ: 97 affinity:00000000,00000000,00000000,00000008 vec:6b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:255(----), (XEN) IRQ: 98 affinity:00000000,00000000,00000000,00000008 vec:73 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:254(----), (XEN) IRQ: 99 affinity:00000000,00000000,00000000,00000008 vec:7b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:253(----), (XEN) IRQ: 100 affinity:00000000,00000000,00000000,00000008 vec:83 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:252(----), (XEN) IRQ: 101 affinity:00000000,00000000,00000000,00000008 vec:8b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:251(----), (XEN) IRQ: 102 affinity:00000000,00000000,00000000,00000001 vec:9b type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 103 affinity:00000000,00000000,00000000,00000001 vec:a3 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 104 affinity:00000000,00000000,00000000,00000002 vec:ab type=PCI-MSI status=00000010 in-flight=0 domain-list=3: 77(--M-), (XEN) IRQ: 105 affinity:00000000,00000000,00000000,00000002 vec:b3 type=PCI-MSI status=00000010 in-flight=0 domain-list=3: 76(--M-), (XEN) IRQ: 106 affinity:00000000,00000000,00000000,00000008 vec:bb type=PCI-MSI status=00000050 in-flight=0 domain-list=3: 75(--M-), (XEN) IO-APIC interrupt information: (XEN) IRQ 0 Vec240: (XEN) Apic 0x00, Pin 2: vector=240, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:255 (XEN) IRQ 1 Vec 48: (XEN) Apic 0x00, Pin 1: vector=48, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 3 Vec 56: (XEN) Apic 0x00, Pin 3: vector=56, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 4 Vec 64: (XEN) Apic 0x00, Pin 4: vector=64, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 5 Vec 72: (XEN) Apic 0x00, Pin 5: vector=72, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 6 Vec 80: (XEN) Apic 0x00, Pin 6: vector=80, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 7 Vec 88: (XEN) Apic 0x00, Pin 7: vector=88, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 8 Vec 96: (XEN) Apic 0x00, Pin 8: vector=96, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 9 Vec104: (XEN) Apic 0x00, Pin 9: vector=104, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 10 Vec112: (XEN) Apic 0x00, Pin 10: vector=112, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 11 Vec120: (XEN) Apic 0x00, Pin 11: vector=120, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 12 Vec136: (XEN) Apic 0x00, Pin 12: vector=136, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 13 Vec144: (XEN) Apic 0x00, Pin 13: vector=144, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 14 Vec152: (XEN) Apic 0x00, Pin 14: vector=152, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 15 Vec160: (XEN) Apic 0x00, Pin 15: vector=160, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 16 Vec176: (XEN) Apic 0x00, Pin 16: vector=176, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 17 Vec168: (XEN) Apic 0x00, Pin 17: vector=168, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:8 (XEN) IRQ 18 Vec145: (XEN) Apic 0x00, Pin 18: vector=145, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 19 Vec169: (XEN) Apic 0x00, Pin 19: vector=169, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 21 Vec161: (XEN) Apic 0x00, Pin 21: vector=161, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 22 Vec147: (XEN) Apic 0x00, Pin 22: vector=147, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:16 (XEN) IRQ 23 Vec177: (XEN) Apic 0x00, Pin 23: vector=177, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 25 Vec162: (XEN) Apic 0x01, Pin 1: vector=162, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 26 Vec 74: (XEN) Apic 0x01, Pin 2: vector=74, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 27 Vec 65: (XEN) Apic 0x01, Pin 3: vector=65, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 29 Vec185: (XEN) Apic 0x01, Pin 5: vector=185, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 39 Vec153: (XEN) Apic 0x01, Pin 15: vector=153, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 43 Vec 51: (XEN) Apic 0x01, Pin 19: vector=51, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 44 Vec 67: (XEN) Apic 0x01, Pin 20: vector=67, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 45 Vec 83: (XEN) Apic 0x01, Pin 21: vector=83, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 46 Vec 99: (XEN) Apic 0x01, Pin 22: vector=99, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 [-- Attachment #3: dmesg_debugQ.txt --] [-- Type: text/plain, Size: 16384 bytes --] mapped, unbound (XEN) IRQ: 90 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d2 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 91 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:da type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 92 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:23 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 93 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:2b type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 94 affinity:00000000,00000000,00000000,00000008 vec:3b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:258(----), (XEN) IRQ: 95 affinity:00000000,00000000,00000000,00000008 vec:4b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:257(----), (XEN) IRQ: 96 affinity:00000000,00000000,00000000,00000008 vec:5b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:256(----), (XEN) IRQ: 97 affinity:00000000,00000000,00000000,00000008 vec:6b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:255(----), (XEN) IRQ: 98 affinity:00000000,00000000,00000000,00000008 vec:73 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:254(----), (XEN) IRQ: 99 affinity:00000000,00000000,00000000,00000008 vec:7b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:253(----), (XEN) IRQ: 100 affinity:00000000,00000000,00000000,00000008 vec:83 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:252(----), (XEN) IRQ: 101 affinity:00000000,00000000,00000000,00000008 vec:8b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:251(----), (XEN) IRQ: 102 affinity:00000000,00000000,00000000,00000001 vec:9b type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 103 affinity:00000000,00000000,00000000,00000001 vec:a3 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 104 affinity:00000000,00000000,00000000,00000004 vec:ab type=PCI-MSI status=00000010 in-flight=0 domain-list=3: 77(--M-), (XEN) IRQ: 105 affinity:00000000,00000000,00000000,00000004 vec:b3 type=PCI-MSI status=00000010 in-flight=0 domain-list=3: 76(--M-), (XEN) IRQ: 106 affinity:00000000,00000000,00000000,00000008 vec:bb type=PCI-MSI status=00000050 in-flight=0 domain-list=3: 75(--M-), (XEN) IO-APIC interrupt information: (XEN) IRQ 0 Vec240: (XEN) Apic 0x00, Pin 2: vector=240, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:255 (XEN) IRQ 1 Vec 48: (XEN) Apic 0x00, Pin 1: vector=48, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 3 Vec 56: (XEN) Apic 0x00, Pin 3: vector=56, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 4 Vec 64: (XEN) Apic 0x00, Pin 4: vector=64, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 5 Vec 72: (XEN) Apic 0x00, Pin 5: vector=72, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 6 Vec 80: (XEN) Apic 0x00, Pin 6: vector=80, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 7 Vec 88: (XEN) Apic 0x00, Pin 7: vector=88, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 8 Vec 96: (XEN) Apic 0x00, Pin 8: vector=96, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 9 Vec104: (XEN) Apic 0x00, Pin 9: vector=104, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 10 Vec112: (XEN) Apic 0x00, Pin 10: vector=112, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 11 Vec120: (XEN) Apic 0x00, Pin 11: vector=120, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 12 Vec136: (XEN) Apic 0x00, Pin 12: vector=136, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 13 Vec144: (XEN) Apic 0x00, Pin 13: vector=144, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 14 Vec152: (XEN) Apic 0x00, Pin 14: vector=152, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 15 Vec160: (XEN) Apic 0x00, Pin 15: vector=160, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 16 Vec176: (XEN) Apic 0x00, Pin 16: vector=176, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 17 Vec168: (XEN) Apic 0x00, Pin 17: vector=168, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:8 (XEN) IRQ 18 Vec145: (XEN) Apic 0x00, Pin 18: vector=145, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 19 Vec169: (XEN) Apic 0x00, Pin 19: vector=169, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 21 Vec161: (XEN) Apic 0x00, Pin 21: vector=161, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 22 Vec147: (XEN) Apic 0x00, Pin 22: vector=147, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:16 (XEN) IRQ 23 Vec177: (XEN) Apic 0x00, Pin 23: vector=177, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 25 Vec162: (XEN) Apic 0x01, Pin 1: vector=162, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 26 Vec 74: (XEN) Apic 0x01, Pin 2: vector=74, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 27 Vec 65: (XEN) Apic 0x01, Pin 3: vector=65, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 29 Vec185: (XEN) Apic 0x01, Pin 5: vector=185, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 39 Vec153: (XEN) Apic 0x01, Pin 15: vector=153, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 43 Vec 51: (XEN) Apic 0x01, Pin 19: vector=51, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 44 Vec 67: (XEN) Apic 0x01, Pin 20: vector=67, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 45 Vec 83: (XEN) Apic 0x01, Pin 21: vector=83, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 46 Vec 99: (XEN) Apic 0x01, Pin 22: vector=99, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) number of MP IRQ sources: 15. (XEN) number of IO-APIC #1 registers: 24. (XEN) number of IO-APIC #3 registers: 24. (XEN) testing the IO APIC....................... (XEN) IO APIC #1...... (XEN) .... register #00: 04000000 (XEN) ....... : physical APIC id: 04 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 001 01 0 0 0 0 0 1 1 30 (XEN) 02 0FF 0F 0 0 0 0 0 1 1 F0 (XEN) 03 0FF 0F 1 0 0 0 0 1 1 38 (XEN) 04 0FF 0F 1 0 0 0 0 1 1 40 (XEN) 05 001 01 0 0 0 0 0 1 1 48 (XEN) 06 001 01 0 0 0 0 0 1 1 50 (XEN) 07 001 01 0 0 0 0 0 1 1 58 (XEN) 08 001 01 0 0 0 0 0 1 1 60 (XEN) 09 001 01 0 1 0 0 0 1 1 68 (XEN) 0a 001 01 0 0 0 0 0 1 1 70 (XEN) 0b 001 01 0 0 0 0 0 1 1 78 (XEN) 0c 001 01 0 0 0 0 0 1 1 88 (XEN) 0d 0FF 0F 1 0 0 0 0 1 1 90 (XEN) 0e 001 01 0 0 0 0 0 1 1 98 (XEN) 0f 001 01 0 0 0 0 0 1 1 A0 (XEN) 10 001 01 0 1 0 1 0 1 1 B0 (XEN) 11 008 08 0 1 0 1 0 1 1 A8 (XEN) 12 001 01 0 1 0 1 0 1 1 91 (XEN) 13 001 01 0 1 0 1 0 1 1 A9 (XEN) 14 04B 0B 1 0 0 0 0 0 2 05 (XEN) 15 001 01 0 1 0 1 0 1 1 A1 (XEN) 16 010 00 0 1 0 1 0 1 1 93 (XEN) 17 001 01 0 1 0 1 0 1 1 B1 (XEN) IO APIC #3...... (XEN) .... register #00: 06000000 (XEN) ....... : physical APIC id: 06 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... register #02: 00000000 (XEN) ....... : arbitration: 00 (XEN) .... register #03: 00000001 (XEN) ....... : Boot DT : 1 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 0FF 0F 1 1 0 1 0 1 1 A2 (XEN) 02 0FF 0F 1 1 0 1 0 1 1 4A (XEN) 03 0FF 0F 1 1 0 1 0 1 1 41 (XEN) 04 000 00 1 0 0 0 0 0 0 00 (XEN) 05 0FF 0F 1 1 0 1 0 1 1 B9 (XEN) 06 000 00 1 0 0 0 0 0 0 00 (XEN) 07 000 00 1 0 0 0 0 0 0 00 (XEN) 08 000 00 1 0 0 0 0 0 0 00 (XEN) 09 000 00 1 0 0 0 0 0 0 00 (XEN) 0a 000 00 1 0 0 0 0 0 0 00 (XEN) 0b 000 00 1 0 0 0 0 0 0 00 (XEN) 0c 000 00 1 0 0 0 0 0 0 00 (XEN) 0d 000 00 1 0 0 0 0 0 0 00 (XEN) 0e 000 00 1 0 0 0 0 0 0 00 (XEN) 0f 001 01 0 1 0 1 0 1 1 99 (XEN) 10 000 00 1 0 0 0 0 0 0 00 (XEN) 11 000 00 1 0 0 0 0 0 0 00 (XEN) 12 000 00 1 0 0 0 0 0 0 00 (XEN) 13 0FF 0F 1 1 0 1 0 1 1 33 (XEN) 14 0FF 0F 1 1 0 1 0 1 1 43 (XEN) 15 0FF 0F 1 1 0 1 0 1 1 53 (XEN) 16 0FF 0F 1 1 0 1 0 1 1 63 (XEN) 17 000 00 1 0 0 0 0 0 0 00 (XEN) Using vector-based indexing (XEN) IRQ to pin mappings: (XEN) IRQ240 -> 0:2 (XEN) IRQ48 -> 0:1 (XEN) IRQ56 -> 0:3 (XEN) IRQ64 -> 0:4 (XEN) IRQ72 -> 0:5 (XEN) IRQ80 -> 0:6 (XEN) IRQ88 -> 0:7 (XEN) IRQ96 -> 0:8 (XEN) IRQ104 -> 0:9 (XEN) IRQ112 -> 0:10 (XEN) IRQ120 -> 0:11 (XEN) IRQ136 -> 0:12 (XEN) IRQ144 -> 0:13 (XEN) IRQ152 -> 0:14 (XEN) IRQ160 -> 0:15 (XEN) IRQ176 -> 0:16 (XEN) IRQ168 -> 0:17 (XEN) IRQ145 -> 0:18 (XEN) IRQ169 -> 0:19 (XEN) IRQ161 -> 0:21 (XEN) IRQ147 -> 0:22 (XEN) IRQ177 -> 0:23 (XEN) IRQ162 -> 1:1 (XEN) IRQ74 -> 1:2 (XEN) IRQ65 -> 1:3 (XEN) IRQ185 -> 1:5 (XEN) IRQ153 -> 1:15 (XEN) IRQ51 -> 1:19 (XEN) IRQ67 -> 1:20 (XEN) IRQ83 -> 1:21 (XEN) IRQ99 -> 1:22 (XEN) .................................... done. (XEN) ==== PCI devices ==== (XEN) ff:06.3 - dom 0 - MSIs < > (XEN) ff:06.2 - dom 0 - MSIs < > (XEN) ff:06.1 - dom 0 - MSIs < > (XEN) ff:06.0 - dom 0 - MSIs < > (XEN) ff:05.3 - dom 0 - MSIs < > (XEN) ff:05.2 - dom 0 - MSIs < > (XEN) ff:05.1 - dom 0 - MSIs < > (XEN) ff:05.0 - dom 0 - MSIs < > (XEN) ff:04.3 - dom 0 - MSIs < > (XEN) ff:04.2 - dom 0 - MSIs < > (XEN) ff:04.1 - dom 0 - MSIs < > (XEN) ff:04.0 - dom 0 - MSIs < > (XEN) ff:03.4 - dom 0 - MSIs < > (XEN) ff:03.2 - dom 0 - MSIs < > (XEN) ff:03.1 - dom 0 - MSIs < > (XEN) ff:03.0 - dom 0 - MSIs < > (XEN) ff:02.5 - dom 0 - MSIs < > (XEN) ff:02.4 - dom 0 - MSIs < > (XEN) ff:02.3 - dom 0 - MSIs < > (XEN) ff:02.2 - dom 0 - MSIs < > (XEN) ff:02.1 - dom 0 - MSIs < > (XEN) ff:02.0 - dom 0 - MSIs < > (XEN) ff:00.1 - dom 0 - MSIs < > (XEN) ff:00.0 - dom 0 - MSIs < > (XEN) fe:06.3 - dom 0 - MSIs < > (XEN) fe:06.2 - dom 0 - MSIs < > (XEN) fe:06.1 - dom 0 - MSIs < > (XEN) fe:06.0 - dom 0 - MSIs < > (XEN) fe:05.3 - dom 0 - MSIs < > (XEN) fe:05.2 - dom 0 - MSIs < > (XEN) fe:05.1 - dom 0 - MSIs < > (XEN) fe:05.0 - dom 0 - MSIs < > (XEN) fe:04.3 - dom 0 - MSIs < > (XEN) fe:04.2 - dom 0 - MSIs < > (XEN) fe:04.1 - dom 0 - MSIs < > (XEN) fe:04.0 - dom 0 - MSIs < > (XEN) fe:03.4 - dom 0 - MSIs < > (XEN) fe:03.2 - dom 0 - MSIs < > (XEN) fe:03.1 - dom 0 - MSIs < > (XEN) fe:03.0 - dom 0 - MSIs < > (XEN) fe:02.5 - dom 0 - MSIs < > (XEN) fe:02.4 - dom 0 - MSIs < > (XEN) fe:02.3 - dom 0 - MSIs < > (XEN) fe:02.2 - dom 0 - MSIs < > (XEN) fe:02.1 - dom 0 - MSIs < > (XEN) fe:02.0 - dom 0 - MSIs < > (XEN) fe:00.1 - dom 0 - MSIs < > (XEN) fe:00.0 - dom 0 - MSIs < > (XEN) 0c:00.1 - dom 0 - MSIs < 67 68 69 70 71 72 73 74 75 > (XEN) 0c:00.0 - dom 0 - MSIs < 58 59 60 61 62 63 64 65 66 > (XEN) 0a:00.1 - dom 0 - MSIs < 85 86 87 88 89 90 91 92 93 > (XEN) 0a:00.0 - dom 0 - MSIs < 76 77 78 79 80 81 82 83 84 > (XEN) 09:04.0 - dom 0 - MSIs < 57 > (XEN) 09:02.0 - dom 0 - MSIs < 56 > (XEN) 08:00.0 - dom 0 - MSIs < > (XEN) 07:0e.0 - dom 0 - MSIs < > (XEN) 05:00.2 - dom 0 - MSIs < > (XEN) 05:00.0 - dom 0 - MSIs < > (XEN) 03:00.0 - dom 3 - MSIs < 103 > (XEN) 02:00.0 - dom 3 - MSIs < 102 104 105 106 > (XEN) 01:05.7 - dom 0 - MSIs < > (XEN) 01:05.6 - dom 0 - MSIs < > (XEN) 01:05.5 - dom 0 - MSIs < > (XEN) 01:05.4 - dom 0 - MSIs < > (XEN) 01:05.3 - dom 0 - MSIs < > (XEN) 01:05.2 - dom 0 - MSIs < > (XEN) 01:05.1 - dom 0 - MSIs < > (XEN) 01:05.0 - dom 0 - MSIs < > (XEN) 01:00.7 - dom 0 - MSIs < > (XEN) 01:00.6 - dom 0 - MSIs < > (XEN) 01:00.5 - dom 0 - MSIs < > (XEN) 01:00.4 - dom 0 - MSIs < > (XEN) 01:00.3 - dom 0 - MSIs < > (XEN) 01:00.2 - dom 0 - MSIs < > (XEN) 01:00.1 - dom 0 - MSIs < > (XEN) 01:00.0 - dom 0 - MSIs < > (XEN) 00:1f.5 - dom 0 - MSIs < > (XEN) 00:1f.3 - dom 0 - MSIs < > (XEN) 00:1f.2 - dom 0 - MSIs < > (XEN) 00:1f.0 - dom 0 - MSIs < > (XEN) 00:1e.0 - dom 0 - MSIs < > (XEN) 00:1d.7 - dom 0 - MSIs < > (XEN) 00:1d.2 - dom 0 - MSIs < > (XEN) 00:1d.1 - dom 0 - MSIs < > (XEN) 00:1d.0 - dom 0 - MSIs < > (XEN) 00:1c.5 - dom 0 - MSIs < 55 > (XEN) 00:1c.4 - dom 0 - MSIs < 54 > (XEN) 00:1c.0 - dom 0 - MSIs < 53 > (XEN) 00:1b.0 - dom 0 - MSIs < > (XEN) 00:1a.7 - dom 0 - MSIs < > (XEN) 00:1a.2 - dom 0 - MSIs < > (XEN) 00:1a.1 - dom 0 - MSIs < > (XEN) 00:1a.0 - dom 0 - MSIs < > (XEN) 00:16.7 - dom 0 - MSIs < 101 > (XEN) 00:16.6 - dom 0 - MSIs < 100 > (XEN) 00:16.5 - dom 0 - MSIs < 99 > (XEN) 00:16.4 - dom 0 - MSIs < 98 > (XEN) 00:16.3 - dom 0 - MSIs < 97 > (XEN) 00:16.2 - dom 0 - MSIs < 96 > (XEN) 00:16.1 - dom 0 - MSIs < 95 > (XEN) 00:16.0 - dom 0 - MSIs < 94 > (XEN) 00:14.3 - dom 0 - MSIs < > (XEN) 00:14.2 - dom 0 - MSIs < > (XEN) 00:14.1 - dom 0 - MSIs < > (XEN) 00:14.0 - dom 0 - MSIs < > (XEN) 00:13.0 - dom 0 - MSIs < > (XEN) 00:07.0 - dom 0 - MSIs < 52 > (XEN) 00:05.0 - dom 0 - MSIs < 51 > (XEN) 00:03.0 - dom 0 - MSIs < 50 > (XEN) 00:01.0 - dom 0 - MSIs < 49 > (XEN) 00:00.0 - dom 0 - MSIs < > [-- Attachment #4: dmesg_debugz.txt --] [-- Type: text/plain, Size: 16384 bytes --] 58 affinity:00000000,00000000,00000000,00000002 vec:49 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:294(----), (XEN) IRQ: 59 affinity:00000000,00000000,00000000,00000002 vec:51 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:293(----), (XEN) IRQ: 60 affinity:00000000,00000000,00000000,00000002 vec:59 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:292(----), (XEN) IRQ: 61 affinity:00000000,00000000,00000000,00000002 vec:61 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:291(----), (XEN) IRQ: 62 affinity:00000000,00000000,00000000,00000002 vec:69 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:290(----), (XEN) IRQ: 63 affinity:00000000,00000000,00000000,00000002 vec:71 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:289(----), (XEN) IRQ: 64 affinity:00000000,00000000,00000000,00000002 vec:79 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:288(----), (XEN) IRQ: 65 affinity:00000000,00000000,00000000,00000002 vec:81 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:287(----), (XEN) IRQ: 66 affinity:00000000,00000000,00000000,00000002 vec:89 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:286(----), (XEN) IRQ: 67 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c1 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 68 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c9 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 69 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d1 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 70 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d9 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 71 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:22 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 72 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:2a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 73 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:32 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 74 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:3a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 75 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:42 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 76 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:52 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 77 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:5a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 78 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:62 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 79 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:6a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 80 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:72 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 81 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:7a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 82 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:8a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 83 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:92 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 84 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:9a type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 85 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:aa type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 86 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:b2 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 87 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:ba type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 88 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c2 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 89 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:ca type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 90 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d2 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 91 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:da type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 92 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:23 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 93 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:2b type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 94 affinity:00000000,00000000,00000000,00000008 vec:3b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:258(----), (XEN) IRQ: 95 affinity:00000000,00000000,00000000,00000008 vec:4b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:257(----), (XEN) IRQ: 96 affinity:00000000,00000000,00000000,00000008 vec:5b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:256(----), (XEN) IRQ: 97 affinity:00000000,00000000,00000000,00000008 vec:6b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:255(----), (XEN) IRQ: 98 affinity:00000000,00000000,00000000,00000008 vec:73 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:254(----), (XEN) IRQ: 99 affinity:00000000,00000000,00000000,00000008 vec:7b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:253(----), (XEN) IRQ: 100 affinity:00000000,00000000,00000000,00000008 vec:83 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:252(----), (XEN) IRQ: 101 affinity:00000000,00000000,00000000,00000008 vec:8b type=PCI-MSI status=00000010 in-flight=0 domain-list=0:251(----), (XEN) IRQ: 102 affinity:00000000,00000000,00000000,00000001 vec:9b type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 103 affinity:00000000,00000000,00000000,00000001 vec:a3 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 104 affinity:00000000,00000000,00000000,00000004 vec:ab type=PCI-MSI status=00000010 in-flight=0 domain-list=3: 77(--M-), (XEN) IRQ: 105 affinity:00000000,00000000,00000000,00000004 vec:b3 type=PCI-MSI status=00000010 in-flight=0 domain-list=3: 76(--M-), (XEN) IRQ: 106 affinity:00000000,00000000,00000000,00000008 vec:bb type=PCI-MSI status=00000050 in-flight=0 domain-list=3: 75(--M-), (XEN) IO-APIC interrupt information: (XEN) IRQ 0 Vec240: (XEN) Apic 0x00, Pin 2: vector=240, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:255 (XEN) IRQ 1 Vec 48: (XEN) Apic 0x00, Pin 1: vector=48, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 3 Vec 56: (XEN) Apic 0x00, Pin 3: vector=56, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 4 Vec 64: (XEN) Apic 0x00, Pin 4: vector=64, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 5 Vec 72: (XEN) Apic 0x00, Pin 5: vector=72, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 6 Vec 80: (XEN) Apic 0x00, Pin 6: vector=80, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 7 Vec 88: (XEN) Apic 0x00, Pin 7: vector=88, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 8 Vec 96: (XEN) Apic 0x00, Pin 8: vector=96, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 9 Vec104: (XEN) Apic 0x00, Pin 9: vector=104, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 10 Vec112: (XEN) Apic 0x00, Pin 10: vector=112, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 11 Vec120: (XEN) Apic 0x00, Pin 11: vector=120, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 12 Vec136: (XEN) Apic 0x00, Pin 12: vector=136, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 13 Vec144: (XEN) Apic 0x00, Pin 13: vector=144, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:255 (XEN) IRQ 14 Vec152: (XEN) Apic 0x00, Pin 14: vector=152, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 15 Vec160: (XEN) Apic 0x00, Pin 15: vector=160, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 16 Vec176: (XEN) Apic 0x00, Pin 16: vector=176, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 17 Vec168: (XEN) Apic 0x00, Pin 17: vector=168, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:8 (XEN) IRQ 18 Vec145: (XEN) Apic 0x00, Pin 18: vector=145, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 19 Vec169: (XEN) Apic 0x00, Pin 19: vector=169, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 21 Vec161: (XEN) Apic 0x00, Pin 21: vector=161, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 22 Vec147: (XEN) Apic 0x00, Pin 22: vector=147, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:16 (XEN) IRQ 23 Vec177: (XEN) Apic 0x00, Pin 23: vector=177, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 25 Vec162: (XEN) Apic 0x01, Pin 1: vector=162, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 26 Vec 74: (XEN) Apic 0x01, Pin 2: vector=74, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 27 Vec 65: (XEN) Apic 0x01, Pin 3: vector=65, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 29 Vec185: (XEN) Apic 0x01, Pin 5: vector=185, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 39 Vec153: (XEN) Apic 0x01, Pin 15: vector=153, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 43 Vec 51: (XEN) Apic 0x01, Pin 19: vector=51, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 44 Vec 67: (XEN) Apic 0x01, Pin 20: vector=67, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 45 Vec 83: (XEN) Apic 0x01, Pin 21: vector=83, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) IRQ 46 Vec 99: (XEN) Apic 0x01, Pin 22: vector=99, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:255 (XEN) number of MP IRQ sources: 15. (XEN) number of IO-APIC #1 registers: 24. (XEN) number of IO-APIC #3 registers: 24. (XEN) testing the IO APIC....................... (XEN) IO APIC #1...... (XEN) .... register #00: 04000000 (XEN) ....... : physical APIC id: 04 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 001 01 0 0 0 0 0 1 1 30 (XEN) 02 0FF 0F 0 0 0 0 0 1 1 F0 (XEN) 03 0FF 0F 1 0 0 0 0 1 1 38 (XEN) 04 0FF 0F 1 0 0 0 0 1 1 40 (XEN) 05 001 01 0 0 0 0 0 1 1 48 (XEN) 06 001 01 0 0 0 0 0 1 1 50 (XEN) 07 001 01 0 0 0 0 0 1 1 58 (XEN) 08 001 01 0 0 0 0 0 1 1 60 (XEN) 09 001 01 0 1 0 0 0 1 1 68 (XEN) 0a 001 01 0 0 0 0 0 1 1 70 (XEN) 0b 001 01 0 0 0 0 0 1 1 78 (XEN) 0c 001 01 0 0 0 0 0 1 1 88 (XEN) 0d 0FF 0F 1 0 0 0 0 1 1 90 (XEN) 0e 001 01 0 0 0 0 0 1 1 98 (XEN) 0f 001 01 0 0 0 0 0 1 1 A0 (XEN) 10 001 01 0 1 0 1 0 1 1 B0 (XEN) 11 008 08 0 1 0 1 0 1 1 A8 (XEN) 12 001 01 0 1 0 1 0 1 1 91 (XEN) 13 001 01 0 1 0 1 0 1 1 A9 (XEN) 14 04B 0B 1 0 0 0 0 0 2 05 (XEN) 15 001 01 0 1 0 1 0 1 1 A1 (XEN) 16 010 00 0 1 0 1 0 1 1 93 (XEN) 17 001 01 0 1 0 1 0 1 1 B1 (XEN) IO APIC #3...... (XEN) .... register #00: 06000000 (XEN) ....... : physical APIC id: 06 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... register #02: 00000000 (XEN) ....... : arbitration: 00 (XEN) .... register #03: 00000001 (XEN) ....... : Boot DT : 1 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 0FF 0F 1 1 0 1 0 1 1 A2 (XEN) 02 0FF 0F 1 1 0 1 0 1 1 4A (XEN) 03 0FF 0F 1 1 0 1 0 1 1 41 (XEN) 04 000 00 1 0 0 0 0 0 0 00 (XEN) 05 0FF 0F 1 1 0 1 0 1 1 B9 (XEN) 06 000 00 1 0 0 0 0 0 0 00 (XEN) 07 000 00 1 0 0 0 0 0 0 00 (XEN) 08 000 00 1 0 0 0 0 0 0 00 (XEN) 09 000 00 1 0 0 0 0 0 0 00 (XEN) 0a 000 00 1 0 0 0 0 0 0 00 (XEN) 0b 000 00 1 0 0 0 0 0 0 00 (XEN) 0c 000 00 1 0 0 0 0 0 0 00 (XEN) 0d 000 00 1 0 0 0 0 0 0 00 (XEN) 0e 000 00 1 0 0 0 0 0 0 00 (XEN) 0f 001 01 0 1 0 1 0 1 1 99 (XEN) 10 000 00 1 0 0 0 0 0 0 00 (XEN) 11 000 00 1 0 0 0 0 0 0 00 (XEN) 12 000 00 1 0 0 0 0 0 0 00 (XEN) 13 0FF 0F 1 1 0 1 0 1 1 33 (XEN) 14 0FF 0F 1 1 0 1 0 1 1 43 (XEN) 15 0FF 0F 1 1 0 1 0 1 1 53 (XEN) 16 0FF 0F 1 1 0 1 0 1 1 63 (XEN) 17 000 00 1 0 0 0 0 0 0 00 (XEN) Using vector-based indexing (XEN) IRQ to pin mappings: (XEN) IRQ240 -> 0:2 (XEN) IRQ48 -> 0:1 (XEN) IRQ56 -> 0:3 (XEN) IRQ64 -> 0:4 (XEN) IRQ72 -> 0:5 (XEN) IRQ80 -> 0:6 (XEN) IRQ88 -> 0:7 (XEN) IRQ96 -> 0:8 (XEN) IRQ104 -> 0:9 (XEN) IRQ112 -> 0:10 (XEN) IRQ120 -> 0:11 (XEN) IRQ136 -> 0:12 (XEN) IRQ144 -> 0:13 (XEN) IRQ152 -> 0:14 (XEN) IRQ160 -> 0:15 (XEN) IRQ176 -> 0:16 (XEN) IRQ168 -> 0:17 (XEN) IRQ145 -> 0:18 (XEN) IRQ169 -> 0:19 (XEN) IRQ161 -> 0:21 (XEN) IRQ147 -> 0:22 (XEN) IRQ177 -> 0:23 (XEN) IRQ162 -> 1:1 (XEN) IRQ74 -> 1:2 (XEN) IRQ65 -> 1:3 (XEN) IRQ185 -> 1:5 (XEN) IRQ153 -> 1:15 (XEN) IRQ51 -> 1:19 (XEN) IRQ67 -> 1:20 (XEN) IRQ83 -> 1:21 (XEN) IRQ99 -> 1:22 (XEN) .................................... done. [-- Attachment #5: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-11 19:22 ` [Xen-users] " Mark Adams @ 2010-11-11 19:42 ` Mark Adams 0 siblings, 0 replies; 48+ messages in thread From: Mark Adams @ 2010-11-11 19:42 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich [-- Attachment #1: Type: text/plain, Size: 1063 bytes --] Apols - Also see plain dmesg attached. This one from updated machine (4.0.1) still showing the msi issues. On Thu, Nov 11, 2010 at 07:22:56PM +0000, Mark Adams wrote: > See attached > > On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > > I've just noticed this at the end of xm dmesg > > > > > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > Something else trying to use the device being exported? (the nics are > > > 02:00.0 and 03:00.0) > > > > Hmm, looks like it, but it should not have happend. Can you attach > > the output of 'xm dmesg' and also do the 'xm debug-keys ..' that I > > asked for in the previous e-mail? > > > > Jan, the fixes for the MSI you did, they weren't for 4.0.1 right? Just > > for unstable? > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > [-- Attachment #2: dmesg.txt --] [-- Type: text/plain, Size: 5388 bytes --] (XEN) Xen version 4.0.1 (Debian 4.0.1-1) (waldi@debian.org) (gcc version 4.4.5 20100824 (prerelease) (Debian 4.4.4-11) ) Fri Sep 3 15:38:12 UTC 2010 (XEN) Bootloader: GRUB 1.98+20100804-7 (XEN) Command line: dom0_mem=2048M (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 2 MBR signatures (XEN) Found 2 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009e800 (usable) (XEN) 000000000009e800 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000bf770000 (usable) (XEN) 00000000bf770000 - 00000000bf77e000 (ACPI data) (XEN) 00000000bf77e000 - 00000000bf7d0000 (ACPI NVS) (XEN) 00000000bf7d0000 - 00000000bf7e0000 (reserved) (XEN) 00000000bf7ec000 - 00000000c0000000 (reserved) (XEN) 00000000e0000000 - 00000000f0000000 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000ffa00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000000640000000 (usable) (XEN) ACPI: RSDP 000FA7E0, 0024 (r2 ACPIAM) (XEN) ACPI: XSDT BF770100, 006C (r1 022410 XSDT1354 20100224 MSFT 97) (XEN) ACPI: FACP BF770290, 00F4 (r4 022410 FACP1354 20100224 MSFT 97) (XEN) ACPI: DSDT BF770540, 627D (r2 7010_ 7010_001 1 INTL 20051117) (XEN) ACPI: FACS BF77E000, 0040 (XEN) ACPI: APIC BF770390, 011E (r2 022410 APIC1354 20100224 MSFT 97) (XEN) ACPI: MCFG BF7704B0, 003C (r1 022410 OEMMCFG 20100224 MSFT 97) (XEN) ACPI: SPMI BF7704F0, 0041 (r5 022410 OEMSPMI 20100224 MSFT 97) (XEN) ACPI: OEMB BF77E040, 0082 (r1 022410 OEMB1354 20100224 MSFT 97) (XEN) ACPI: SRAT BF77A540, 0150 (r2 022410 OEMSRAT 1 INTL 1) (XEN) ACPI: HPET BF77A690, 0038 (r1 022410 OEMHPET 20100224 MSFT 97) (XEN) ACPI: DMAR BF77E0D0, 0128 (r1 AMI OEMDMAR 1 MSFT 97) (XEN) ACPI: SSDT BF780410, 0363 (r1 DpgPmm CpuPm 12 INTL 20051117) (XEN) System RAM: 24567MB (25156664kB) (XEN) Domain heap initialised DMA width 31 bits (XEN) Processor #0 6:12 APIC version 21 (XEN) Processor #2 6:12 APIC version 21 (XEN) Processor #18 6:12 APIC version 21 (XEN) Processor #20 6:12 APIC version 21 (XEN) Processor #32 6:12 APIC version 21 (XEN) Processor #34 6:12 APIC version 21 (XEN) Processor #50 6:12 APIC version 21 (XEN) Processor #52 6:12 APIC version 21 (XEN) IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23 (XEN) IOAPIC[1]: apic_id 3, version 32, address 0xfec8a000, GSI 24-47 (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 2400.140 MHz processor. (XEN) Initing memory sharing. (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) - Unrestricted Guest (XEN) EPT supports 2MB super page. (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging detected. (XEN) Intel VT-d Snoop Control not supported. (XEN) Intel VT-d DMA Passthrough not supported. (XEN) Intel VT-d Queued Invalidation supported. (XEN) Intel VT-d Interrupt Remapping supported. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Enabled directed EOI with ioapic_ack_old on! (XEN) Total of 8 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using old ACK method (XEN) TSC is reliable, synchronization unnecessary (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 16 KiB. (XEN) Brought up 8 CPUs (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x16b5000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000238000000->000000023c000000 (507904 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff816b5000 (XEN) Init. ramdisk: ffffffff816b5000->ffffffff83211200 (XEN) Phys-Mach map: ffffffff83212000->ffffffff83612000 (XEN) Start info: ffffffff83612000->ffffffff836124b4 (XEN) Page tables: ffffffff83613000->ffffffff83632000 (XEN) Boot stack: ffffffff83632000->ffffffff83633000 (XEN) TOTAL: ffffffff80000000->ffffffff83800000 (XEN) ENTRY ADDRESS: ffffffff81506200 (XEN) Dom0 has maximum 8 VCPUs (XEN) Scrubbing Free RAM: .............................................................................................................................................................................................................................done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 176kB init memory. (XEN) save.c:72:d0 Domain 1 expects freq 2400MHz but host's freq is 2400MHz: trap and emulate rdtsc (XEN) save.c:72:d0 Domain 2 expects freq 2400MHz but host's freq is 2400MHz: trap and emulate rdtsc (XEN) msi.c:715: MSI is already in use on device 02:00.0 (XEN) msi.c:715: MSI is already in use on device 02:00.0 (XEN) msi.c:715: MSI is already in use on device 02:00.0 [-- Attachment #3: Type: text/plain, Size: 137 bytes --] _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-11 19:06 ` Konrad Rzeszutek Wilk 2010-11-11 19:22 ` [Xen-users] " Mark Adams @ 2010-11-12 17:10 ` Mark Adams 2010-11-12 22:22 ` Konrad Rzeszutek Wilk 1 sibling, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-12 17:10 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > I've just noticed this at the end of xm dmesg > > > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > Something else trying to use the device being exported? (the nics are > > 02:00.0 and 03:00.0) > > Hmm, looks like it, but it should not have happend. Can you attach > the output of 'xm dmesg' and also do the 'xm debug-keys ..' that I > asked for in the previous e-mail? > > Jan, the fixes for the MSI you did, they weren't for 4.0.1 right? Just > for unstable? > Any further idea's on this? Is it a xen bug if the hidden device is being accessed in dom0? or is there an overlap somewhere? (not sure how this would work).. Regards, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-12 17:10 ` [Xen-users] " Mark Adams @ 2010-11-12 22:22 ` Konrad Rzeszutek Wilk 2010-11-14 17:15 ` Re: [Xen-devel] " Mark Adams 0 siblings, 1 reply; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-12 22:22 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users, JBeulich On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: > On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > > I've just noticed this at the end of xm dmesg > > > > > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > Something else trying to use the device being exported? (the nics are > > > 02:00.0 and 03:00.0) > > > > Hmm, looks like it, but it should not have happend. Can you attach > > the output of 'xm dmesg' and also do the 'xm debug-keys ..' that I > > asked for in the previous e-mail? > > > > Jan, the fixes for the MSI you did, they weren't for 4.0.1 right? Just > > for unstable? > > > > Any further idea's on this? Is it a xen bug if the hidden device is being > accessed in dom0? or is there an overlap somewhere? (not sure how this > would work).. I was going to look in the source today to get an idea but never got to it... You might, as I mentioned in earlier emails, try to setup a serial console or netconsole and log the Linux kernel output when it hangs/fails. ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-12 22:22 ` Konrad Rzeszutek Wilk @ 2010-11-14 17:15 ` Mark Adams 2010-11-15 17:11 ` [Xen-users] " Mark Adams 2010-11-15 17:15 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 0 siblings, 2 replies; 48+ messages in thread From: Mark Adams @ 2010-11-14 17:15 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich On 12 Nov 2010, at 22:22, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: >> On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: >>>> I've just noticed this at the end of xm dmesg >>>> >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 >>>> >>>> Something else trying to use the device being exported? (the nics are >>>> 02:00.0 and 03:00.0) >>> >>> Hmm, looks like it, but it should not have happend. Can you attach >>> the output of 'xm dmesg' and also do the 'xm debug-keys ..' that I >>> asked for in the previous e-mail? >>> >>> Jan, the fixes for the MSI you did, they weren't for 4.0.1 right? Just >>> for unstable? >>> >> >> Any further idea's on this? Is it a xen bug if the hidden device is being >> accessed in dom0? or is there an overlap somewhere? (not sure how this >> would work).. > > I was going to look in the source today to get an idea but never got to it... > > You might, as I mentioned in earlier emails, try to setup a serial console > or netconsole and log the Linux kernel output when it hangs/fails. > can't do this unfortunately, the server Is in use so not able to just let it hang again... The passthrough is not in use now until I think there is some possible solution to get rid of the MSI conflict (when it won't hang anymore!) > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-14 17:15 ` Re: [Xen-devel] " Mark Adams @ 2010-11-15 17:11 ` Mark Adams 2010-11-15 17:15 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 1 sibling, 0 replies; 48+ messages in thread From: Mark Adams @ 2010-11-15 17:11 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich On Sun, Nov 14, 2010 at 05:15:02PM +0000, Mark Adams wrote: > > On 12 Nov 2010, at 22:22, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > > > On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: > >> On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > >>>> I've just noticed this at the end of xm dmesg > >>>> > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > >>>> > >>>> Something else trying to use the device being exported? (the nics are > >>>> 02:00.0 and 03:00.0) > >>> > >>> Hmm, looks like it, but it should not have happend. Can you attach > >>> the output of 'xm dmesg' and also do the 'xm debug-keys ..' that I > >>> asked for in the previous e-mail? > >>> > >>> Jan, the fixes for the MSI you did, they weren't for 4.0.1 right? Just > >>> for unstable? > >>> > >> > >> Any further idea's on this? Is it a xen bug if the hidden device is being > >> accessed in dom0? or is there an overlap somewhere? (not sure how this > >> would work).. > > > > I was going to look in the source today to get an idea but never got to it... > > > > You might, as I mentioned in earlier emails, try to setup a serial console > > or netconsole and log the Linux kernel output when it hangs/fails. > > > > can't do this unfortunately, the server > Is in use so not able to just let it hang again... The passthrough is > not in use now until I think there is some possible solution to get > rid of the MSI conflict (when it won't hang anymore!) Is there anything else I can do to help with resolution on this issue? I see another user is also having a similar problem with (quiet possibly) PCI passthrough causing their RAID array to go offline. Did my logs show anything useful? You mentioned some fix for MSI earlier has this been corrected in a newer version of Xen? Regards, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-14 17:15 ` Re: [Xen-devel] " Mark Adams 2010-11-15 17:11 ` [Xen-users] " Mark Adams @ 2010-11-15 17:15 ` Konrad Rzeszutek Wilk 2010-11-15 17:23 ` [Xen-users] " Mark Adams 1 sibling, 1 reply; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-15 17:15 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users, JBeulich On Sun, Nov 14, 2010 at 05:15:02PM +0000, Mark Adams wrote: > > On 12 Nov 2010, at 22:22, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > > > On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: > >> On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > >>>> I've just noticed this at the end of xm dmesg > >>>> > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 Looking briefly at the code it means that somebody enabled the MSI already on the device and did not disable them. But I wonder how you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) or pciback.hide (for older kernels) to "hide" the devices away from the Linux Dom0 kernel? > > I was going to look in the source today to get an idea but never got to it... > > > > You might, as I mentioned in earlier emails, try to setup a serial console > > or netconsole and log the Linux kernel output when it hangs/fails. > > > > can't do this unfortunately, the server > Is in use so not able to just let it hang again... The passthrough is not in use now until I think there is some possible solution to get rid of the MSI conflict (when it won't hang anymore!) Didn't you say that you had two servers and saw this problem on another box too? Without more details on the Xen hypervisor line or the kernel line when the failure occurs I sadly can't help you. ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-15 17:15 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk @ 2010-11-15 17:23 ` Mark Adams 2010-11-15 17:44 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 0 siblings, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-15 17:23 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich On Mon, Nov 15, 2010 at 12:15:44PM -0500, Konrad Rzeszutek Wilk wrote: > On Sun, Nov 14, 2010 at 05:15:02PM +0000, Mark Adams wrote: > > > > On 12 Nov 2010, at 22:22, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > > > > > On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: > > >> On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > >>>> I've just noticed this at the end of xm dmesg > > >>>> > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > Looking briefly at the code it means that somebody enabled the MSI > already on the device and did not disable them. But I wonder how > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > or pciback.hide (for older kernels) to "hide" the devices away from the > Linux Dom0 kernel? using xen-pciback.hide as its a pvops kernel (debian squeeze 2.6.32-5-27) > > > > > I was going to look in the source today to get an idea but never got to it... > > > > > > You might, as I mentioned in earlier emails, try to setup a serial console > > > or netconsole and log the Linux kernel output when it hangs/fails. > > > > > > > can't do this unfortunately, the server > > Is in use so not able to just let it hang again... The passthrough is not in use now until I think there is some possible solution to get rid of the MSI conflict (when it won't hang anymore!) > > Didn't you say that you had two servers and saw this problem on another > box too? > > Without more details on the Xen hypervisor line or the kernel line when > the failure occurs I sadly can't help you. Yes this occurs on both servers that I've tried it on. Doesn't the MSI log above indicate that there is a conflict - which is what ends up causing the device to go offline? Is there no other way to identify the conflict? Regards, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-15 17:23 ` [Xen-users] " Mark Adams @ 2010-11-15 17:44 ` Konrad Rzeszutek Wilk 2010-11-15 17:56 ` [Xen-users] " Mark Adams ` (2 more replies) 0 siblings, 3 replies; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-15 17:44 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users, JBeulich On Mon, Nov 15, 2010 at 05:23:09PM +0000, Mark Adams wrote: > On Mon, Nov 15, 2010 at 12:15:44PM -0500, Konrad Rzeszutek Wilk wrote: > > On Sun, Nov 14, 2010 at 05:15:02PM +0000, Mark Adams wrote: > > > > > > On 12 Nov 2010, at 22:22, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > > > > > > > On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: > > > >> On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > > >>>> I've just noticed this at the end of xm dmesg > > > >>>> > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > Looking briefly at the code it means that somebody enabled the MSI > > already on the device and did not disable them. But I wonder how > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > > or pciback.hide (for older kernels) to "hide" the devices away from the > > Linux Dom0 kernel? > > using xen-pciback.hide as its a pvops kernel (debian squeeze > 2.6.32-5-27) Ok. Then it might be worth looking in when this happens. I think there is an argument on the Xen hyperisor line to include the time-stamp, but I don't remember it :-( > > Didn't you say that you had two servers and saw this problem on another > > box too? > > > > Without more details on the Xen hypervisor line or the kernel line when > > the failure occurs I sadly can't help you. > > Yes this occurs on both servers that I've tried it on. Doesn't the MSI > log above indicate that there is a conflict - which is what ends up > causing the device to go offline? Is there no other way to identify the Could be, but it is unclear - it depends on when the message pops out. But that does not help with finding out why your RAID controller goes offline. ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-15 17:44 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk @ 2010-11-15 17:56 ` Mark Adams 2010-11-24 17:59 ` Mark Adams 2010-11-15 19:26 ` [Xen-users] Re: pci-passthrough in pvops causing offline raid Pasi Kärkkäinen 2010-11-16 10:37 ` Re: [Xen-devel] " Mark Adams 2 siblings, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-15 17:56 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich On Mon, Nov 15, 2010 at 12:44:13PM -0500, Konrad Rzeszutek Wilk wrote: > On Mon, Nov 15, 2010 at 05:23:09PM +0000, Mark Adams wrote: > > On Mon, Nov 15, 2010 at 12:15:44PM -0500, Konrad Rzeszutek Wilk wrote: > > > On Sun, Nov 14, 2010 at 05:15:02PM +0000, Mark Adams wrote: > > > > > > > > On 12 Nov 2010, at 22:22, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > > > > > > > > > On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: > > > > >> On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > > > >>>> I've just noticed this at the end of xm dmesg > > > > >>>> > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > Looking briefly at the code it means that somebody enabled the MSI > > > already on the device and did not disable them. But I wonder how > > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > > > or pciback.hide (for older kernels) to "hide" the devices away from the > > > Linux Dom0 kernel? > > > > using xen-pciback.hide as its a pvops kernel (debian squeeze > > 2.6.32-5-27) > > Ok. Then it might be worth looking in when this happens. I think > there is an argument on the Xen hyperisor line to include the time-stamp, but > I don't remember it :-( > > > > Didn't you say that you had two servers and saw this problem on another > > > box too? > > > > > > Without more details on the Xen hypervisor line or the kernel line when > > > the failure occurs I sadly can't help you. > > > > Yes this occurs on both servers that I've tried it on. Doesn't the MSI > > log above indicate that there is a conflict - which is what ends up > > causing the device to go offline? Is there no other way to identify the > > Could be, but it is unclear - it depends on when the message pops out. The message appears immediately on boot. > > But that does not help with finding out why your RAID controller goes offline. Maybe the other user having a similar issue can help with logs if it is still happening to him. I'll ask on that thread... Regards, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-15 17:56 ` [Xen-users] " Mark Adams @ 2010-11-24 17:59 ` Mark Adams 2010-11-24 20:28 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 0 siblings, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-24 17:59 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich > > > > > >>>> > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > > > Looking briefly at the code it means that somebody enabled the MSI > > > > already on the device and did not disable them. But I wonder how > > > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > > > > or pciback.hide (for older kernels) to "hide" the devices away from the > > > > Linux Dom0 kernel? > > > > > > using xen-pciback.hide as its a pvops kernel (debian squeeze > > > 2.6.32-5-27) > > > > Ok. Then it might be worth looking in when this happens. I think > > there is an argument on the Xen hyperisor line to include the time-stamp, but > > I don't remember it :-( > > I've got a test setup in place now, and am trying to reproduce this. I've not connected up serial as yet, but can see the following logs in the qemu-dm log file when I get the "MSI is already in use" errors above. Note also that this error -always- shows for the first specified device in the pci= field, and not the 2nd. pt_msixctrl_reg_write: guest enabling MSI-X, disable MSI-INTx translation pci_intx: intx=1 pt_msix_update_one: Update msix entry 0 with pirq 4d gvec 59 pt_msix_update_one: Update msix entry 1 with pirq 4c gvec 61 pt_msix_update_one: Update msix entry 2 with pirq 4b gvec 69 pt_msixctrl_reg_write: guest enabling MSI-X, disable MSI-INTx translation pci_intx: intx=2 I have also seen the following log just once, not sure if it's related: (XEN) domctl.c:811:d0 XEN_DOMCTL_test_assign_device: 2:0.0 already assigned, or non-existent Does this help at all with debugging my issues? Regards, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-24 17:59 ` Mark Adams @ 2010-11-24 20:28 ` Konrad Rzeszutek Wilk 2010-11-26 11:15 ` Mark Adams 0 siblings, 1 reply; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-24 20:28 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users, JBeulich On Wed, Nov 24, 2010 at 05:59:26PM +0000, Mark Adams wrote: > > > > > > >>>> > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > > > > > Looking briefly at the code it means that somebody enabled the MSI > > > > > already on the device and did not disable them. But I wonder how > > > > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > > > > > or pciback.hide (for older kernels) to "hide" the devices away from the > > > > > Linux Dom0 kernel? > > > > > > > > using xen-pciback.hide as its a pvops kernel (debian squeeze > > > > 2.6.32-5-27) > > > > > > Ok. Then it might be worth looking in when this happens. I think > > > there is an argument on the Xen hyperisor line to include the time-stamp, but > > > I don't remember it :-( > > > > > I've got a test setup in place now, and am trying to reproduce this. > I've not connected up serial as yet, but can see the following logs in > the qemu-dm log file when I get the "MSI is already in use" errors > above. Note also that this error -always- shows for the first specified > device in the pci= field, and not the 2nd. > > pt_msixctrl_reg_write: guest enabling MSI-X, disable MSI-INTx translation > pci_intx: intx=1 > pt_msix_update_one: Update msix entry 0 with pirq 4d gvec 59 > pt_msix_update_one: Update msix entry 1 with pirq 4c gvec 61 > pt_msix_update_one: Update msix entry 2 with pirq 4b gvec 69 > pt_msixctrl_reg_write: guest enabling MSI-X, disable MSI-INTx translation > pci_intx: intx=2 > > I have also seen the following log just once, not sure if it's related: > > (XEN) domctl.c:811:d0 XEN_DOMCTL_test_assign_device: 2:0.0 already assigned, or non-existent > > Does this help at all with debugging my issues? Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your machine is toast. I mentioned in the previous email the key sequences - look on Google on how to pass in SysRQ if you are using a serial concentrator. ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-24 20:28 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk @ 2010-11-26 11:15 ` Mark Adams 2010-11-26 15:25 ` [Xen-users] " Mark Adams 2010-11-29 16:36 ` HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails Konrad Rzeszutek Wilk 0 siblings, 2 replies; 48+ messages in thread From: Mark Adams @ 2010-11-26 11:15 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich On Wed, Nov 24, 2010 at 03:28:43PM -0500, Konrad Rzeszutek Wilk wrote: > On Wed, Nov 24, 2010 at 05:59:26PM +0000, Mark Adams wrote: > > > > > > > >>>> > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > > > > > > > Looking briefly at the code it means that somebody enabled the MSI > > > > > > already on the device and did not disable them. But I wonder how > > > > > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > > > > > > or pciback.hide (for older kernels) to "hide" the devices away from the > > > > > > Linux Dom0 kernel? > > > > I've got a test setup in place now, and am trying to reproduce this. > > I've not connected up serial as yet, but can see the following logs in > > the qemu-dm log file when I get the "MSI is already in use" errors > > above. Note also that this error -always- shows for the first specified > > device in the pci= field, and not the 2nd. > > In my new test setup, I have seen some strange behaviour. 1 of the HVM's (with identical config in dom0 and domU) suddenly would not allow the igb driver to be loaded in domU, even though the device was visible in lspci. Shutting the machine down, removing the power cord, waiting 5 seconds then plugging it in again corrected that issue - Is this possibly a motherboard bug? I have also disabled the SR-IOV functionality in the BIOS incase this is causing any issues. In addition, to try to correct the MSI issue noted above, I have changed my pci= line to the following: pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] This has stopped the "already in use on device" log, and the devices appear to show correctly in the domU. Is it safe to disable msitranslate? as I understand it, its for allowing multifunction devices to be seen as such in domU. Is that correct? I haven't been able to reproduce the dropped raid issue yet, but I am awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause this due to their very high interrupt usage (2000 per second). In the mean time, I can see the following in the qemu-dm logs now with the msitranslate=0 enabled. Is it anything to worry about? pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your > machine is toast. I mentioned in the previous email the key sequences - look on Google > on how to pass in SysRQ if you are using a serial concentrator. I will do this when I can get the machine to crash. Best Regards, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-26 11:15 ` Mark Adams @ 2010-11-26 15:25 ` Mark Adams 2010-11-29 16:36 ` HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails Konrad Rzeszutek Wilk 1 sibling, 0 replies; 48+ messages in thread From: Mark Adams @ 2010-11-26 15:25 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich On Fri, Nov 26, 2010 at 11:15:20AM +0000, Mark Adams wrote: > On Wed, Nov 24, 2010 at 03:28:43PM -0500, Konrad Rzeszutek Wilk wrote: > > On Wed, Nov 24, 2010 at 05:59:26PM +0000, Mark Adams wrote: > > > > > > > > >>>> > > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > > > > > > > > > Looking briefly at the code it means that somebody enabled the MSI > > > > > > > already on the device and did not disable them. But I wonder how > > > > > > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > > > > > > > or pciback.hide (for older kernels) to "hide" the devices away from the > > > > > > > Linux Dom0 kernel? > > > > > > I've got a test setup in place now, and am trying to reproduce this. > > > I've not connected up serial as yet, but can see the following logs in > > > the qemu-dm log file when I get the "MSI is already in use" errors > > > above. Note also that this error -always- shows for the first specified > > > device in the pci= field, and not the 2nd. > > > > > In my new test setup, I have seen some strange behaviour. 1 of the HVM's > (with identical config in dom0 and domU) suddenly would not allow the > igb driver to be loaded in domU, even though the device was visible in > lspci. Shutting the machine down, removing the power cord, waiting 5 > seconds then plugging it in again corrected that issue - Is this > possibly a motherboard bug? I have also disabled the SR-IOV > functionality in the BIOS incase this is causing any issues. > > In addition, to try to correct the MSI issue noted above, I have changed > my pci= line to the following: > > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] Apologies for replying to my own post, but I'm having an issue with this setup in that I can not get a link on the 2nd interface that I'm passing through. I've tried with msitranslate both on and off, with no success. The device shows in the domU as an interface correctly, but even with the link up (link led's show on the interface) domU always shows the eth2 interface as not ready. [ 7.001784] ADDRCONF(NETDEV_UP): eth1: link is not ready [ 7.047903] ADDRCONF(NETDEV_UP): eth2: link is not ready [ 10.108995] e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None [ 10.109653] ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready [ 16.468102] eth0: no IPv6 routers present [ 20.404092] eth1: no IPv6 routers present I've tried using the ET dual port card (igb) aswell as the onboard interfaces (e1000e) with exactly the same result. The eth0 interface is a xen bridge device, and if I remove this, both passthrough interfaces work correctly. Can you not have a bridge and pci-passthrough operational? is there a limit of 2 NIC's in a HVM domU? (this doesn't sound right...) Regards, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-11-26 11:15 ` Mark Adams 2010-11-26 15:25 ` [Xen-users] " Mark Adams @ 2010-11-29 16:36 ` Konrad Rzeszutek Wilk 2010-12-08 12:58 ` Mark Adams 1 sibling, 1 reply; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-29 16:36 UTC (permalink / raw) To: Mark Adams, Stefano Stabellini, anthony.perard, yunhong.jiang, yuan.b.liu Cc: xen-devel, xen-users, JBeulich > > In my new test setup, I have seen some strange behaviour. 1 of the HVM's > (with identical config in dom0 and domU) suddenly would not allow the > igb driver to be loaded in domU, even though the device was visible in Let's create a new thread for this other issue. > lspci. Shutting the machine down, removing the power cord, waiting 5 > seconds then plugging it in again corrected that issue - Is this > possibly a motherboard bug? I have also disabled the SR-IOV > functionality in the BIOS incase this is causing any issues. > > In addition, to try to correct the MSI issue noted above, I have changed > my pci= line to the following: > > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] With the msi_translate=1 turned on the DomU HVM guests did work, right? > > This has stopped the "already in use on device" log, and the devices > appear to show correctly in the domU. Is it safe to disable > msitranslate? as I understand it, its for allowing multifunction devices > to be seen as such in domU. Is that correct? > > I haven't been able to reproduce the dropped raid issue yet, but I am > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause > this due to their very high interrupt usage (2000 per second). OK. > > In the mean time, I can see the following in the qemu-dm logs now with > the msitranslate=0 enabled. Is it anything to worry about? Well, the "Error" ones are pretty bad, thought I am having a hard time understanding what it means. Lets copy some of the QEMU folks on this. > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > > > > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your > > machine is toast. I mentioned in the previous email the key sequences - look on Google > > on how to pass in SysRQ if you are using a serial concentrator. > > I will do this when I can get the machine to crash. > > Best Regards, > Mark > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-11-29 16:36 ` HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails Konrad Rzeszutek Wilk @ 2010-12-08 12:58 ` Mark Adams 2010-12-08 13:37 ` Sander Eikelenboom 2010-12-08 13:48 ` Dietmar Hahn 0 siblings, 2 replies; 48+ messages in thread From: Mark Adams @ 2010-12-08 12:58 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: xen-devel, Stefano Stabellini, yunhong.jiang, JBeulich, yuan.b.liu, anthony.perard, xen-users Hi - Apologies to top post this, but after alot of testing, I believe there must be an issue with IRQ's going missing between domU and dom0. Unfortunately I have no data to prove this! With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel grub config, all 3 NIC's appear OK in the domU however I still had issues with the red-fone ISDN box. The interrupts were showing correctly (2000/s) in the domU but communication to the device via the NIC was still being interrupted (as shown in the asterisk console)Note that to get the igb driver to allow this many interrupts, the InterruptThrottleRate was set to 0. The same config (red-fone box, asterisk etc) works fine with a physical server. There is also the additional issue that I could not get the passthrough NIC's to show correctly when I also had a bridge setup. Throughout my testing however, I could not get the machine to crash. Not sure where to go with this one. For now we are keeping our VoIP servers physical when ISDN connections are required. Regards, Mark On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote: > > > > In my new test setup, I have seen some strange behaviour. 1 of the HVM's > > (with identical config in dom0 and domU) suddenly would not allow the > > igb driver to be loaded in domU, even though the device was visible in > > Let's create a new thread for this other issue. > > > lspci. Shutting the machine down, removing the power cord, waiting 5 > > seconds then plugging it in again corrected that issue - Is this > > possibly a motherboard bug? I have also disabled the SR-IOV > > functionality in the BIOS incase this is causing any issues. > > > > In addition, to try to correct the MSI issue noted above, I have changed > > my pci= line to the following: > > > > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] > > With the msi_translate=1 turned on the DomU HVM guests did work, right? > > > > > This has stopped the "already in use on device" log, and the devices > > appear to show correctly in the domU. Is it safe to disable > > msitranslate? as I understand it, its for allowing multifunction devices > > to be seen as such in domU. Is that correct? > > > > I haven't been able to reproduce the dropped raid issue yet, but I am > > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause > > this due to their very high interrupt usage (2000 per second). > > OK. > > > > In the mean time, I can see the following in the qemu-dm logs now with > > the msitranslate=0 enabled. Is it anything to worry about? > > Well, the "Error" ones are pretty bad, thought I am having a hard time > understanding what it means. Lets copy some of the QEMU folks on this. > > > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] > > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 > > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 > > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] > > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 > > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 > > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 > > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 > > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 > > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 > > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > > > > > > > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your > > > machine is toast. I mentioned in the previous email the key sequences - look on Google > > > on how to pass in SysRQ if you are using a serial concentrator. > > > > I will do this when I can get the machine to crash. > > > > Best Regards, > > Mark > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 12:58 ` Mark Adams @ 2010-12-08 13:37 ` Sander Eikelenboom 2010-12-08 13:48 ` Mark Adams 2010-12-08 13:48 ` Dietmar Hahn 1 sibling, 1 reply; 48+ messages in thread From: Sander Eikelenboom @ 2010-12-08 13:37 UTC (permalink / raw) To: Mark Adams Cc: xen-devel, xen-users, Stefano Stabellini, Konrad Rzeszutek Wilk Hello Mark, Just a recap: you pass through: - 3 physical nics/IGB - 1 ISDN pci ISDN box - all using msi/msi-x interrupts ? Have you tried using a PV domU instead of a HVM domU ? Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? -- Sander Wednesday, December 8, 2010, 1:58:55 PM, you wrote: > Hi - Apologies to top post this, but after alot of testing, I believe > there must be an issue with IRQ's going missing between domU and dom0. > Unfortunately I have no data to prove this! > With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel > grub config, all 3 NIC's appear OK in the domU however I still had > issues with the red-fone ISDN box. The interrupts were showing correctly > (2000/s) in the domU but communication to the device via the NIC was > still being interrupted (as shown in the asterisk console)Note that to > get the igb driver to allow this many interrupts, the > InterruptThrottleRate was set to 0. The same config (red-fone box, > asterisk etc) works fine with a physical server. > There is also the additional issue that I could not get the passthrough > NIC's to show correctly when I also had a bridge setup. > Throughout my testing however, I could not get the machine to crash. > Not sure where to go with this one. For now we are keeping our VoIP > servers physical when ISDN connections are required. > Regards, > Mark > On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote: >> > >> > In my new test setup, I have seen some strange behaviour. 1 of the HVM's >> > (with identical config in dom0 and domU) suddenly would not allow the >> > igb driver to be loaded in domU, even though the device was visible in >> >> Let's create a new thread for this other issue. >> >> > lspci. Shutting the machine down, removing the power cord, waiting 5 >> > seconds then plugging it in again corrected that issue - Is this >> > possibly a motherboard bug? I have also disabled the SR-IOV >> > functionality in the BIOS incase this is causing any issues. >> > >> > In addition, to try to correct the MSI issue noted above, I have changed >> > my pci= line to the following: >> > >> > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] >> >> With the msi_translate=1 turned on the DomU HVM guests did work, right? >> >> > >> > This has stopped the "already in use on device" log, and the devices >> > appear to show correctly in the domU. Is it safe to disable >> > msitranslate? as I understand it, its for allowing multifunction devices >> > to be seen as such in domU. Is that correct? >> > >> > I haven't been able to reproduce the dropped raid issue yet, but I am >> > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause >> > this due to their very high interrupt usage (2000 per second). >> >> OK. >> > >> > In the mean time, I can see the following in the qemu-dm logs now with >> > the msitranslate=0 enabled. Is it anything to worry about? >> >> Well, the "Error" ones are pretty bad, thought I am having a hard time >> understanding what it means. Lets copy some of the QEMU folks on this. >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] >> > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 >> > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] >> > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 >> > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 >> > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 >> > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 >> > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 >> > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 >> > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> > >> > > >> > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your >> > > machine is toast. I mentioned in the previous email the key sequences - look on Google >> > > on how to pass in SysRQ if you are using a serial concentrator. >> > >> > I will do this when I can get the machine to crash. >> > >> > Best Regards, >> > Mark >> > >> > _______________________________________________ >> > Xen-devel mailing list >> > Xen-devel@lists.xensource.com >> > http://lists.xensource.com/xen-devel -- Best regards, Sander mailto:linux@eikelenboom.it ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 13:37 ` Sander Eikelenboom @ 2010-12-08 13:48 ` Mark Adams 2010-12-08 14:05 ` Sander Eikelenboom 2010-12-08 17:01 ` [Xen-devel] " Konrad Rzeszutek Wilk 0 siblings, 2 replies; 48+ messages in thread From: Mark Adams @ 2010-12-08 13:48 UTC (permalink / raw) To: Sander Eikelenboom Cc: xen-devel, xen-users, Stefano Stabellini, Konrad Rzeszutek Wilk On Wed, Dec 08, 2010 at 02:37:15PM +0100, Sander Eikelenboom wrote: > Hello Mark, Hi > > Just a recap: > you pass through: > - 3 physical nics/IGB > - 1 ISDN pci ISDN box The redfone box runs on 1 of the nics - its not seperate. It converts ISDN to TDMoE see here.. http://www.red-fone.com/ > - all using msi/msi-x interrupts ? I tried using msi/msi-x interrupts, but it caused the raid card to drop off (after some use) and provided seemingly even worse performance than pegging everything back to legacy. > > Have you tried using a PV domU instead of a HVM domU ? I initially tried PV but had issues with the igb NIC's. There was another thread somewhere about my issues with that. > Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? > > > -- > Sander > > > > Wednesday, December 8, 2010, 1:58:55 PM, you wrote: > > > Hi - Apologies to top post this, but after alot of testing, I believe > > there must be an issue with IRQ's going missing between domU and dom0. > > Unfortunately I have no data to prove this! > > > With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel > > grub config, all 3 NIC's appear OK in the domU however I still had > > issues with the red-fone ISDN box. The interrupts were showing correctly > > (2000/s) in the domU but communication to the device via the NIC was > > still being interrupted (as shown in the asterisk console)Note that to > > get the igb driver to allow this many interrupts, the > > InterruptThrottleRate was set to 0. The same config (red-fone box, > > asterisk etc) works fine with a physical server. > > > There is also the additional issue that I could not get the passthrough > > NIC's to show correctly when I also had a bridge setup. > > > Throughout my testing however, I could not get the machine to crash. > > > Not sure where to go with this one. For now we are keeping our VoIP > > servers physical when ISDN connections are required. > > > Regards, > > Mark > > > On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote: > >> > > >> > In my new test setup, I have seen some strange behaviour. 1 of the HVM's > >> > (with identical config in dom0 and domU) suddenly would not allow the > >> > igb driver to be loaded in domU, even though the device was visible in > >> > >> Let's create a new thread for this other issue. > >> > >> > lspci. Shutting the machine down, removing the power cord, waiting 5 > >> > seconds then plugging it in again corrected that issue - Is this > >> > possibly a motherboard bug? I have also disabled the SR-IOV > >> > functionality in the BIOS incase this is causing any issues. > >> > > >> > In addition, to try to correct the MSI issue noted above, I have changed > >> > my pci= line to the following: > >> > > >> > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] > >> > >> With the msi_translate=1 turned on the DomU HVM guests did work, right? > >> > >> > > >> > This has stopped the "already in use on device" log, and the devices > >> > appear to show correctly in the domU. Is it safe to disable > >> > msitranslate? as I understand it, its for allowing multifunction devices > >> > to be seen as such in domU. Is that correct? > >> > > >> > I haven't been able to reproduce the dropped raid issue yet, but I am > >> > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause > >> > this due to their very high interrupt usage (2000 per second). > >> > >> OK. > >> > > >> > In the mean time, I can see the following in the qemu-dm logs now with > >> > the msitranslate=0 enabled. Is it anything to worry about? > >> > >> Well, the "Error" ones are pretty bad, thought I am having a hard time > >> understanding what it means. Lets copy some of the QEMU folks on this. > >> > >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] > >> > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 > >> > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 > >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] > >> > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 > >> > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 > >> > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 > >> > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 > >> > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 > >> > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 > >> > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 > >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> > > >> > > > >> > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your > >> > > machine is toast. I mentioned in the previous email the key sequences - look on Google > >> > > on how to pass in SysRQ if you are using a serial concentrator. > >> > > >> > I will do this when I can get the machine to crash. > >> > > >> > Best Regards, > >> > Mark > >> > > >> > _______________________________________________ > >> > Xen-devel mailing list > >> > Xen-devel@lists.xensource.com > >> > http://lists.xensource.com/xen-devel > > > > > > -- > Best regards, > Sander mailto:linux@eikelenboom.it > ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 13:48 ` Mark Adams @ 2010-12-08 14:05 ` Sander Eikelenboom 2010-12-08 15:48 ` [Xen-users] " Mark Adams 2010-12-08 17:01 ` [Xen-devel] " Konrad Rzeszutek Wilk 1 sibling, 1 reply; 48+ messages in thread From: Sander Eikelenboom @ 2010-12-08 14:05 UTC (permalink / raw) To: Mark Adams Cc: xen-devel, xen-users, Stefano Stabellini, Konrad Rzeszutek Wilk Wednesday, December 8, 2010, 2:48:48 PM, you wrote: > On Wed, Dec 08, 2010 at 02:37:15PM +0100, Sander Eikelenboom wrote: >> Hello Mark, > Hi >> >> Just a recap: >> you pass through: >> - 3 physical nics/IGB >> - 1 ISDN pci ISDN box > The redfone box runs on 1 of the nics - its not seperate. It converts > ISDN to TDMoE see here.. http://www.red-fone.com/ So the problem is probably with the igb's. Searching showed http://forums.virtualbox.org/viewtopic.php?f=7&t=32171 , perhaps worth a try ? Have you tried with just 1 IGB, and/or another simple 1gb NIC (non intel) to see if it's due to any of the special offload features ? >> - all using msi/msi-x interrupts ? > I tried using msi/msi-x interrupts, but it caused the raid card to drop > off (after some use) and provided seemingly even worse performance than > pegging everything back to legacy. >> >> Have you tried using a PV domU instead of a HVM domU ? > I initially tried PV but had issues with the igb NIC's. There was > another thread somewhere about my issues with that. >> Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? >> >> >> -- >> Sander >> >> >> >> Wednesday, December 8, 2010, 1:58:55 PM, you wrote: >> >> > Hi - Apologies to top post this, but after alot of testing, I believe >> > there must be an issue with IRQ's going missing between domU and dom0. >> > Unfortunately I have no data to prove this! >> >> > With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel >> > grub config, all 3 NIC's appear OK in the domU however I still had >> > issues with the red-fone ISDN box. The interrupts were showing correctly >> > (2000/s) in the domU but communication to the device via the NIC was >> > still being interrupted (as shown in the asterisk console)Note that to >> > get the igb driver to allow this many interrupts, the >> > InterruptThrottleRate was set to 0. The same config (red-fone box, >> > asterisk etc) works fine with a physical server. >> >> > There is also the additional issue that I could not get the passthrough >> > NIC's to show correctly when I also had a bridge setup. >> >> > Throughout my testing however, I could not get the machine to crash. >> >> > Not sure where to go with this one. For now we are keeping our VoIP >> > servers physical when ISDN connections are required. >> >> > Regards, >> > Mark >> >> > On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote: >> >> > >> >> > In my new test setup, I have seen some strange behaviour. 1 of the HVM's >> >> > (with identical config in dom0 and domU) suddenly would not allow the >> >> > igb driver to be loaded in domU, even though the device was visible in >> >> >> >> Let's create a new thread for this other issue. >> >> >> >> > lspci. Shutting the machine down, removing the power cord, waiting 5 >> >> > seconds then plugging it in again corrected that issue - Is this >> >> > possibly a motherboard bug? I have also disabled the SR-IOV >> >> > functionality in the BIOS incase this is causing any issues. >> >> > >> >> > In addition, to try to correct the MSI issue noted above, I have changed >> >> > my pci= line to the following: >> >> > >> >> > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] >> >> >> >> With the msi_translate=1 turned on the DomU HVM guests did work, right? >> >> >> >> > >> >> > This has stopped the "already in use on device" log, and the devices >> >> > appear to show correctly in the domU. Is it safe to disable >> >> > msitranslate? as I understand it, its for allowing multifunction devices >> >> > to be seen as such in domU. Is that correct? >> >> > >> >> > I haven't been able to reproduce the dropped raid issue yet, but I am >> >> > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause >> >> > this due to their very high interrupt usage (2000 per second). >> >> >> >> OK. >> >> > >> >> > In the mean time, I can see the following in the qemu-dm logs now with >> >> > the msitranslate=0 enabled. Is it anything to worry about? >> >> >> >> Well, the "Error" ones are pretty bad, thought I am having a hard time >> >> understanding what it means. Lets copy some of the QEMU folks on this. >> >> >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] >> >> > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 >> >> > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] >> >> > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 >> >> > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 >> >> > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 >> >> > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 >> >> > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 >> >> > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 >> >> > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> >> > >> >> > > >> >> > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your >> >> > > machine is toast. I mentioned in the previous email the key sequences - look on Google >> >> > > on how to pass in SysRQ if you are using a serial concentrator. >> >> > >> >> > I will do this when I can get the machine to crash. >> >> > >> >> > Best Regards, >> >> > Mark >> >> > >> >> > _______________________________________________ >> >> > Xen-devel mailing list >> >> > Xen-devel@lists.xensource.com >> >> > http://lists.xensource.com/xen-devel >> >> >> >> >> >> -- >> Best regards, >> Sander mailto:linux@eikelenboom.it >> -- Best regards, Sander mailto:linux@eikelenboom.it ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 14:05 ` Sander Eikelenboom @ 2010-12-08 15:48 ` Mark Adams 2010-12-08 16:44 ` Sander Eikelenboom 0 siblings, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-12-08 15:48 UTC (permalink / raw) To: Sander Eikelenboom Cc: xen-devel, xen-users, Konrad Rzeszutek Wilk, Stefano Stabellini On Wed, Dec 08, 2010 at 03:05:50PM +0100, Sander Eikelenboom wrote: > > Wednesday, December 8, 2010, 2:48:48 PM, you wrote: > > > On Wed, Dec 08, 2010 at 02:37:15PM +0100, Sander Eikelenboom wrote: > >> Hello Mark, > > > Hi > > >> > >> Just a recap: > >> you pass through: > >> - 3 physical nics/IGB > >> - 1 ISDN pci ISDN box > > > The redfone box runs on 1 of the nics - its not seperate. It converts > > ISDN to TDMoE see here.. http://www.red-fone.com/ > > So the problem is probably with the igb's. > Searching showed http://forums.virtualbox.org/viewtopic.php?f=7&t=32171 , perhaps worth a try ? Tried this - doesn't help. > > Have you tried with just 1 IGB, and/or another simple 1gb NIC (non intel) to see if it's due to any of the special offload features ? Haven't got any other NIC's to try unfortunately. Even if it did work with 1, it would be no use to me as I need 3. > > > >> - all using msi/msi-x interrupts ? > > > I tried using msi/msi-x interrupts, but it caused the raid card to drop > > off (after some use) and provided seemingly even worse performance than > > pegging everything back to legacy. > > >> > >> Have you tried using a PV domU instead of a HVM domU ? > > > I initially tried PV but had issues with the igb NIC's. There was > > another thread somewhere about my issues with that. > > > >> Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? > >> > >> > >> -- > >> Sander > >> > >> > >> > >> Wednesday, December 8, 2010, 1:58:55 PM, you wrote: > >> > >> > Hi - Apologies to top post this, but after alot of testing, I believe > >> > there must be an issue with IRQ's going missing between domU and dom0. > >> > Unfortunately I have no data to prove this! > >> > >> > With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel > >> > grub config, all 3 NIC's appear OK in the domU however I still had > >> > issues with the red-fone ISDN box. The interrupts were showing correctly > >> > (2000/s) in the domU but communication to the device via the NIC was > >> > still being interrupted (as shown in the asterisk console)Note that to > >> > get the igb driver to allow this many interrupts, the > >> > InterruptThrottleRate was set to 0. The same config (red-fone box, > >> > asterisk etc) works fine with a physical server. > >> > >> > There is also the additional issue that I could not get the passthrough > >> > NIC's to show correctly when I also had a bridge setup. > >> > >> > Throughout my testing however, I could not get the machine to crash. > >> > >> > Not sure where to go with this one. For now we are keeping our VoIP > >> > servers physical when ISDN connections are required. > >> > >> > Regards, > >> > Mark > >> > >> > On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote: > >> >> > > >> >> > In my new test setup, I have seen some strange behaviour. 1 of the HVM's > >> >> > (with identical config in dom0 and domU) suddenly would not allow the > >> >> > igb driver to be loaded in domU, even though the device was visible in > >> >> > >> >> Let's create a new thread for this other issue. > >> >> > >> >> > lspci. Shutting the machine down, removing the power cord, waiting 5 > >> >> > seconds then plugging it in again corrected that issue - Is this > >> >> > possibly a motherboard bug? I have also disabled the SR-IOV > >> >> > functionality in the BIOS incase this is causing any issues. > >> >> > > >> >> > In addition, to try to correct the MSI issue noted above, I have changed > >> >> > my pci= line to the following: > >> >> > > >> >> > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] > >> >> > >> >> With the msi_translate=1 turned on the DomU HVM guests did work, right? > >> >> > >> >> > > >> >> > This has stopped the "already in use on device" log, and the devices > >> >> > appear to show correctly in the domU. Is it safe to disable > >> >> > msitranslate? as I understand it, its for allowing multifunction devices > >> >> > to be seen as such in domU. Is that correct? > >> >> > > >> >> > I haven't been able to reproduce the dropped raid issue yet, but I am > >> >> > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause > >> >> > this due to their very high interrupt usage (2000 per second). > >> >> > >> >> OK. > >> >> > > >> >> > In the mean time, I can see the following in the qemu-dm logs now with > >> >> > the msitranslate=0 enabled. Is it anything to worry about? > >> >> > >> >> Well, the "Error" ones are pretty bad, thought I am having a hard time > >> >> understanding what it means. Lets copy some of the QEMU folks on this. > >> >> > >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] > >> >> > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 > >> >> > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 > >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] > >> >> > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 > >> >> > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 > >> >> > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 > >> >> > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 > >> >> > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 > >> >> > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 > >> >> > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 > >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> >> > > >> >> > > > >> >> > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your > >> >> > > machine is toast. I mentioned in the previous email the key sequences - look on Google > >> >> > > on how to pass in SysRQ if you are using a serial concentrator. > >> >> > > >> >> > I will do this when I can get the machine to crash. > >> >> > > >> >> > Best Regards, > >> >> > Mark > >> >> > > >> >> > _______________________________________________ > >> >> > Xen-devel mailing list > >> >> > Xen-devel@lists.xensource.com > >> >> > http://lists.xensource.com/xen-devel > >> > >> > >> > >> > >> > >> -- > >> Best regards, > >> Sander mailto:linux@eikelenboom.it > >> > > > > -- > Best regards, > Sander mailto:linux@eikelenboom.it > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 15:48 ` [Xen-users] " Mark Adams @ 2010-12-08 16:44 ` Sander Eikelenboom 2010-12-09 10:39 ` Mark Adams 0 siblings, 1 reply; 48+ messages in thread From: Sander Eikelenboom @ 2010-12-08 16:44 UTC (permalink / raw) To: Mark Adams Cc: xen-devel, xen-users, Konrad Rzeszutek Wilk, Stefano Stabellini Wednesday, December 8, 2010, 4:48:57 PM, you wrote: > On Wed, Dec 08, 2010 at 03:05:50PM +0100, Sander Eikelenboom wrote: >> >> Wednesday, December 8, 2010, 2:48:48 PM, you wrote: >> >> > On Wed, Dec 08, 2010 at 02:37:15PM +0100, Sander Eikelenboom wrote: >> >> Hello Mark, >> >> > Hi >> >> >> >> >> Just a recap: >> >> you pass through: >> >> - 3 physical nics/IGB >> >> - 1 ISDN pci ISDN box >> >> > The redfone box runs on 1 of the nics - its not seperate. It converts >> > ISDN to TDMoE see here.. http://www.red-fone.com/ >> >> So the problem is probably with the igb's. >> Searching showed http://forums.virtualbox.org/viewtopic.php?f=7&t=32171 , perhaps worth a try ? > Tried this - doesn't help. >> >> Have you tried with just 1 IGB, and/or another simple 1gb NIC (non intel) to see if it's due to any of the special offload features ? > Haven't got any other NIC's to try unfortunately. Even if it did work > with 1, it would be no use to me as I need 3. I understand, but simplifying the setup and trying to isolate the problem, could clarify things. I also read you previous thread, and i saw you hide the 02:00.0 and 03:00.0 with xen-pciback (e1000e driver) there, but now you seem to be passing through 08:00.0 and 08:00.1 (igb) ? So i assume you have already tried 2 different NIC's http://download.intel.com/design/network/specupdt/82574.pdf though shows some errata regarding msi-x interrupts and timing issues and workarounds on the 82574 (02:00.0 and 03:00.0) nics. -- Sander >> >> >> >> - all using msi/msi-x interrupts ? >> >> > I tried using msi/msi-x interrupts, but it caused the raid card to drop >> > off (after some use) and provided seemingly even worse performance than >> > pegging everything back to legacy. >> >> >> >> >> Have you tried using a PV domU instead of a HVM domU ? >> >> > I initially tried PV but had issues with the igb NIC's. There was >> > another thread somewhere about my issues with that. >> >> >> >> Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? >> >> >> >> >> >> -- >> >> Sander >> >> >> >> >> >> >> >> Wednesday, December 8, 2010, 1:58:55 PM, you wrote: >> >> >> >> > Hi - Apologies to top post this, but after alot of testing, I believe >> >> > there must be an issue with IRQ's going missing between domU and dom0. >> >> > Unfortunately I have no data to prove this! >> >> >> >> > With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel >> >> > grub config, all 3 NIC's appear OK in the domU however I still had >> >> > issues with the red-fone ISDN box. The interrupts were showing correctly >> >> > (2000/s) in the domU but communication to the device via the NIC was >> >> > still being interrupted (as shown in the asterisk console)Note that to >> >> > get the igb driver to allow this many interrupts, the >> >> > InterruptThrottleRate was set to 0. The same config (red-fone box, >> >> > asterisk etc) works fine with a physical server. >> >> >> >> > There is also the additional issue that I could not get the passthrough >> >> > NIC's to show correctly when I also had a bridge setup. >> >> >> >> > Throughout my testing however, I could not get the machine to crash. >> >> >> >> > Not sure where to go with this one. For now we are keeping our VoIP >> >> > servers physical when ISDN connections are required. >> >> >> >> > Regards, >> >> > Mark >> >> >> >> > On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote: >> >> >> > >> >> >> > In my new test setup, I have seen some strange behaviour. 1 of the HVM's >> >> >> > (with identical config in dom0 and domU) suddenly would not allow the >> >> >> > igb driver to be loaded in domU, even though the device was visible in >> >> >> >> >> >> Let's create a new thread for this other issue. >> >> >> >> >> >> > lspci. Shutting the machine down, removing the power cord, waiting 5 >> >> >> > seconds then plugging it in again corrected that issue - Is this >> >> >> > possibly a motherboard bug? I have also disabled the SR-IOV >> >> >> > functionality in the BIOS incase this is causing any issues. >> >> >> > >> >> >> > In addition, to try to correct the MSI issue noted above, I have changed >> >> >> > my pci= line to the following: >> >> >> > >> >> >> > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] >> >> >> >> >> >> With the msi_translate=1 turned on the DomU HVM guests did work, right? >> >> >> >> >> >> > >> >> >> > This has stopped the "already in use on device" log, and the devices >> >> >> > appear to show correctly in the domU. Is it safe to disable >> >> >> > msitranslate? as I understand it, its for allowing multifunction devices >> >> >> > to be seen as such in domU. Is that correct? >> >> >> > >> >> >> > I haven't been able to reproduce the dropped raid issue yet, but I am >> >> >> > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause >> >> >> > this due to their very high interrupt usage (2000 per second). >> >> >> >> >> >> OK. >> >> >> > >> >> >> > In the mean time, I can see the following in the qemu-dm logs now with >> >> >> > the msitranslate=0 enabled. Is it anything to worry about? >> >> >> >> >> >> Well, the "Error" ones are pretty bad, thought I am having a hard time >> >> >> understanding what it means. Lets copy some of the QEMU folks on this. >> >> >> >> >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] >> >> >> > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 >> >> >> > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 >> >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] >> >> >> > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 >> >> >> > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 >> >> >> > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 >> >> >> > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 >> >> >> > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 >> >> >> > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 >> >> >> > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 >> >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. >> >> >> > >> >> >> > > >> >> >> > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your >> >> >> > > machine is toast. I mentioned in the previous email the key sequences - look on Google >> >> >> > > on how to pass in SysRQ if you are using a serial concentrator. >> >> >> > >> >> >> > I will do this when I can get the machine to crash. >> >> >> > >> >> >> > Best Regards, >> >> >> > Mark >> >> >> > >> >> >> > _______________________________________________ >> >> >> > Xen-devel mailing list >> >> >> > Xen-devel@lists.xensource.com >> >> >> > http://lists.xensource.com/xen-devel >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> Best regards, >> >> Sander mailto:linux@eikelenboom.it >> >> >> >> >> >> -- >> Best regards, >> Sander mailto:linux@eikelenboom.it >> >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xensource.com >> http://lists.xensource.com/xen-users -- Best regards, Sander mailto:linux@eikelenboom.it ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 16:44 ` Sander Eikelenboom @ 2010-12-09 10:39 ` Mark Adams 0 siblings, 0 replies; 48+ messages in thread From: Mark Adams @ 2010-12-09 10:39 UTC (permalink / raw) To: Sander Eikelenboom Cc: xen-devel, xen-users, Stefano Stabellini, Konrad Rzeszutek Wilk On Wed, Dec 08, 2010 at 05:44:39PM +0100, Sander Eikelenboom wrote: > Wednesday, December 8, 2010, 4:48:57 PM, you wrote: > > > On Wed, Dec 08, 2010 at 03:05:50PM +0100, Sander Eikelenboom wrote: > >> > >> Wednesday, December 8, 2010, 2:48:48 PM, you wrote: > >> > >> > On Wed, Dec 08, 2010 at 02:37:15PM +0100, Sander Eikelenboom wrote: > >> >> Hello Mark, > >> > >> > Hi > >> > >> >> > >> >> Just a recap: > >> >> you pass through: > >> >> - 3 physical nics/IGB > >> >> - 1 ISDN pci ISDN box > >> > >> > The redfone box runs on 1 of the nics - its not seperate. It converts > >> > ISDN to TDMoE see here.. http://www.red-fone.com/ > >> > >> So the problem is probably with the igb's. > >> Searching showed http://forums.virtualbox.org/viewtopic.php?f=7&t=32171 , perhaps worth a try ? > > > Tried this - doesn't help. > > >> > >> Have you tried with just 1 IGB, and/or another simple 1gb NIC (non intel) to see if it's due to any of the special offload features ? > > > Haven't got any other NIC's to try unfortunately. Even if it did work > > with 1, it would be no use to me as I need 3. > > I understand, but simplifying the setup and trying to isolate the problem, could clarify things. > > I also read you previous thread, and i saw you hide the 02:00.0 and 03:00.0 with xen-pciback (e1000e driver) there, but now you seem to be passing through 08:00.0 and 08:00.1 (igb) ? > So i assume you have already tried 2 different NIC's > > http://download.intel.com/design/network/specupdt/82574.pdf though shows some errata regarding msi-x interrupts and timing issues and workarounds on the 82574 (02:00.0 and 03:00.0) nics. > I was initially using the onboard NICs (e1000e) when I had the crashing problem. To try to get around this, I disabled all the msi based stuff I could find - which seemed to correct the crashing issue. In order to do this I needed 3 NIC's because bridging would not work at the same time as passthrough (would not show all devices being passed through?) hence starting to use the igb based NIC card thats also in the machine. Unfortunately the servers I've been testing on need to go in to production now, so can't test any further (hence sticking the voip stuff on to a physical box). Xen works really well for me when I don't use pci-passthrough! Regards, Mark > -- > Sander > > > >> > >> > >> >> - all using msi/msi-x interrupts ? > >> > >> > I tried using msi/msi-x interrupts, but it caused the raid card to drop > >> > off (after some use) and provided seemingly even worse performance than > >> > pegging everything back to legacy. > >> > >> >> > >> >> Have you tried using a PV domU instead of a HVM domU ? > >> > >> > I initially tried PV but had issues with the igb NIC's. There was > >> > another thread somewhere about my issues with that. > >> > >> > >> >> Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? > >> >> > >> >> > >> >> -- > >> >> Sander > >> >> > >> >> > >> >> > >> >> Wednesday, December 8, 2010, 1:58:55 PM, you wrote: > >> >> > >> >> > Hi - Apologies to top post this, but after alot of testing, I believe > >> >> > there must be an issue with IRQ's going missing between domU and dom0. > >> >> > Unfortunately I have no data to prove this! > >> >> > >> >> > With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel > >> >> > grub config, all 3 NIC's appear OK in the domU however I still had > >> >> > issues with the red-fone ISDN box. The interrupts were showing correctly > >> >> > (2000/s) in the domU but communication to the device via the NIC was > >> >> > still being interrupted (as shown in the asterisk console)Note that to > >> >> > get the igb driver to allow this many interrupts, the > >> >> > InterruptThrottleRate was set to 0. The same config (red-fone box, > >> >> > asterisk etc) works fine with a physical server. > >> >> > >> >> > There is also the additional issue that I could not get the passthrough > >> >> > NIC's to show correctly when I also had a bridge setup. > >> >> > >> >> > Throughout my testing however, I could not get the machine to crash. > >> >> > >> >> > Not sure where to go with this one. For now we are keeping our VoIP > >> >> > servers physical when ISDN connections are required. > >> >> > >> >> > Regards, > >> >> > Mark > >> >> > >> >> > On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote: > >> >> >> > > >> >> >> > In my new test setup, I have seen some strange behaviour. 1 of the HVM's > >> >> >> > (with identical config in dom0 and domU) suddenly would not allow the > >> >> >> > igb driver to be loaded in domU, even though the device was visible in > >> >> >> > >> >> >> Let's create a new thread for this other issue. > >> >> >> > >> >> >> > lspci. Shutting the machine down, removing the power cord, waiting 5 > >> >> >> > seconds then plugging it in again corrected that issue - Is this > >> >> >> > possibly a motherboard bug? I have also disabled the SR-IOV > >> >> >> > functionality in the BIOS incase this is causing any issues. > >> >> >> > > >> >> >> > In addition, to try to correct the MSI issue noted above, I have changed > >> >> >> > my pci= line to the following: > >> >> >> > > >> >> >> > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] > >> >> >> > >> >> >> With the msi_translate=1 turned on the DomU HVM guests did work, right? > >> >> >> > >> >> >> > > >> >> >> > This has stopped the "already in use on device" log, and the devices > >> >> >> > appear to show correctly in the domU. Is it safe to disable > >> >> >> > msitranslate? as I understand it, its for allowing multifunction devices > >> >> >> > to be seen as such in domU. Is that correct? > >> >> >> > > >> >> >> > I haven't been able to reproduce the dropped raid issue yet, but I am > >> >> >> > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause > >> >> >> > this due to their very high interrupt usage (2000 per second). > >> >> >> > >> >> >> OK. > >> >> >> > > >> >> >> > In the mean time, I can see the following in the qemu-dm logs now with > >> >> >> > the msitranslate=0 enabled. Is it anything to worry about? > >> >> >> > >> >> >> Well, the "Error" ones are pretty bad, thought I am having a hard time > >> >> >> understanding what it means. Lets copy some of the QEMU folks on this. > >> >> >> > >> >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] > >> >> >> > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 > >> >> >> > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 > >> >> >> > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] > >> >> >> > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 > >> >> >> > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 > >> >> >> > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 > >> >> >> > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 > >> >> >> > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 > >> >> >> > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 > >> >> >> > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 > >> >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> >> >> > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > >> >> >> > > >> >> >> > > > >> >> >> > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your > >> >> >> > > machine is toast. I mentioned in the previous email the key sequences - look on Google > >> >> >> > > on how to pass in SysRQ if you are using a serial concentrator. > >> >> >> > > >> >> >> > I will do this when I can get the machine to crash. > >> >> >> > > >> >> >> > Best Regards, > >> >> >> > Mark > >> >> >> > > >> >> >> > _______________________________________________ > >> >> >> > Xen-devel mailing list > >> >> >> > Xen-devel@lists.xensource.com > >> >> >> > http://lists.xensource.com/xen-devel > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> -- > >> >> Best regards, > >> >> Sander mailto:linux@eikelenboom.it > >> >> > >> > >> > >> > >> -- > >> Best regards, > >> Sander mailto:linux@eikelenboom.it > >> > >> > >> _______________________________________________ > >> Xen-users mailing list > >> Xen-users@lists.xensource.com > >> http://lists.xensource.com/xen-users > > > > > -- > Best regards, > Sander mailto:linux@eikelenboom.it > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-devel] Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 13:48 ` Mark Adams 2010-12-08 14:05 ` Sander Eikelenboom @ 2010-12-08 17:01 ` Konrad Rzeszutek Wilk 2010-12-08 17:15 ` Sander Eikelenboom ` (2 more replies) 1 sibling, 3 replies; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-12-08 17:01 UTC (permalink / raw) To: Mark Adams; +Cc: Sander Eikelenboom, xen-devel, xen-users, Stefano Stabellini > > Just a recap: > > you pass through: > > - 3 physical nics/IGB > > - 1 ISDN pci ISDN box > > The redfone box runs on 1 of the nics - its not seperate. It converts > ISDN to TDMoE see here.. http://www.red-fone.com/ > > > - all using msi/msi-x interrupts ? > > I tried using msi/msi-x interrupts, but it caused the raid card to drop > off (after some use) and provided seemingly even worse performance than > pegging everything back to legacy. Were you able to get a serial log and hit all of the differetn debug options when the the RAID card died? > > > > > Have you tried using a PV domU instead of a HVM domU ? > > I initially tried PV but had issues with the igb NIC's. There was > another thread somewhere about my issues with that. Hmm, the only other thread I see from you is about the RAID. > > > > Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? Mr. Sander idea of isolating one piece by piece is the right way. There is a bunch of warnings in the QEMU output - some of them quite .. troubling. You seem to have issues with gntdev (as in, not found), but if you are using OpenSuSE then that would work - I think. When you tested xen-unstable did you use the OpenSUSE kernel or the PV-OPS one? ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 17:01 ` [Xen-devel] " Konrad Rzeszutek Wilk @ 2010-12-08 17:15 ` Sander Eikelenboom 2010-12-08 19:51 ` Jeremy Fitzhardinge 2010-12-08 17:18 ` [Xen-devel] " Sander Eikelenboom 2010-12-09 10:49 ` Mark Adams 2 siblings, 1 reply; 48+ messages in thread From: Sander Eikelenboom @ 2010-12-08 17:15 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: Jeremy Fitzhardinge, xen-devel, xen-users, Stefano Stabellini, Mark Adams It seems Jeremy has had some troubles with the Intel 82754L as well. Don't know if he has got those resolved, or has to use workarounds ? http://www.gossamer-threads.com/lists/linux/kernel/1166308 What kernel are you using in your domU ? Also the 2.6.32 debian ? Perhaps it's also worth a try to test a newer intel driver/kernel ? -- Sander Wednesday, December 8, 2010, 6:01:25 PM, you wrote: >> > Just a recap: >> > you pass through: >> > - 3 physical nics/IGB >> > - 1 ISDN pci ISDN box >> >> The redfone box runs on 1 of the nics - its not seperate. It converts >> ISDN to TDMoE see here.. http://www.red-fone.com/ >> >> > - all using msi/msi-x interrupts ? >> >> I tried using msi/msi-x interrupts, but it caused the raid card to drop >> off (after some use) and provided seemingly even worse performance than >> pegging everything back to legacy. > Were you able to get a serial log and hit all of the differetn debug options > when the the RAID card died? >> >> > >> > Have you tried using a PV domU instead of a HVM domU ? >> >> I initially tried PV but had issues with the igb NIC's. There was >> another thread somewhere about my issues with that. > Hmm, the only other thread I see from you is about the RAID. >> >> >> > Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? > Mr. Sander idea of isolating one piece by piece is the right way. > There is a bunch of warnings in the QEMU output - some of them quite .. troubling. > You seem to have issues with gntdev (as in, not found), but if you are using OpenSuSE > then that would work - I think. When you tested xen-unstable did you use the > OpenSUSE kernel or the PV-OPS one? -- Best regards, Sander mailto:linux@eikelenboom.it ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 17:15 ` Sander Eikelenboom @ 2010-12-08 19:51 ` Jeremy Fitzhardinge 0 siblings, 0 replies; 48+ messages in thread From: Jeremy Fitzhardinge @ 2010-12-08 19:51 UTC (permalink / raw) To: Sander Eikelenboom Cc: xen-users, Mark Adams, xen-devel, Stefano Stabellini, Konrad Rzeszutek Wilk On 12/08/2010 09:15 AM, Sander Eikelenboom wrote: > It seems Jeremy has had some troubles with the Intel 82754L as well. > > Don't know if he has got those resolved, or has to use workarounds ? Its basically a bug in that particular chip, I think. The workaround is to disable ASPM for it; I'm not sure if the upstream driver does that yet, but the workaround used to be to completely disable ASPM (pcie_aspm=off on the kernel command line, and in the BIOS). J ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-devel] Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 17:01 ` [Xen-devel] " Konrad Rzeszutek Wilk 2010-12-08 17:15 ` Sander Eikelenboom @ 2010-12-08 17:18 ` Sander Eikelenboom 2010-12-08 17:43 ` Konrad Rzeszutek Wilk 2010-12-09 10:49 ` Mark Adams 2 siblings, 1 reply; 48+ messages in thread From: Sander Eikelenboom @ 2010-12-08 17:18 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: xen-devel, xen-users, Stefano Stabellini, Mark Adams And another thread with problems and strange irq reports with regard to the 82754 http://www.linuxquestions.org/questions/linux-hardware-18/intel-82574l-gigabit-network-card-issues-and-resolution-831364/ (please Mr. Konrad, no "Mr." that makes me feel really really old ;-) ) Wednesday, December 8, 2010, 6:01:25 PM, you wrote: >> > Just a recap: >> > you pass through: >> > - 3 physical nics/IGB >> > - 1 ISDN pci ISDN box >> >> The redfone box runs on 1 of the nics - its not seperate. It converts >> ISDN to TDMoE see here.. http://www.red-fone.com/ >> >> > - all using msi/msi-x interrupts ? >> >> I tried using msi/msi-x interrupts, but it caused the raid card to drop >> off (after some use) and provided seemingly even worse performance than >> pegging everything back to legacy. > Were you able to get a serial log and hit all of the differetn debug options > when the the RAID card died? >> >> > >> > Have you tried using a PV domU instead of a HVM domU ? >> >> I initially tried PV but had issues with the igb NIC's. There was >> another thread somewhere about my issues with that. > Hmm, the only other thread I see from you is about the RAID. >> >> >> > Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? > Mr. Sander idea of isolating one piece by piece is the right way. > There is a bunch of warnings in the QEMU output - some of them quite .. troubling. > You seem to have issues with gntdev (as in, not found), but if you are using OpenSuSE > then that would work - I think. When you tested xen-unstable did you use the > OpenSUSE kernel or the PV-OPS one? -- Best regards, Sander mailto:linux@eikelenboom.it ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-devel] Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 17:18 ` [Xen-devel] " Sander Eikelenboom @ 2010-12-08 17:43 ` Konrad Rzeszutek Wilk 0 siblings, 0 replies; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-12-08 17:43 UTC (permalink / raw) To: Sander Eikelenboom; +Cc: xen-devel, xen-users, Stefano Stabellini, Mark Adams On Wed, Dec 08, 2010 at 06:18:06PM +0100, Sander Eikelenboom wrote: > And another thread with problems and strange irq reports with regard to the 82754 > http://www.linuxquestions.org/questions/linux-hardware-18/intel-82574l-gigabit-network-card-issues-and-resolution-831364/ Ugh, that device sure looks to have some faults. So I think I've confused myself. There was another person who tried a similar pass-through to an HVM guest of a sound card. While it worked it did not seem to work that well and spitted out lots of warnings. But those are quite different from what Mark had. Mark, right now we are all busy trying to get patches ready for 2.6.38 so hence the reason for not being so fast at responding to you or trying to reproduce this on our machines. The RAID is troubling, but the neat thing about it is that it hangs your machine so if you hit all of those debug options via the serial console, it can help us narrow down on where the problem is. The other issues you have - well, there are many posibilities (and it might be very well the same issue you are hitting with the RAID card) and narrowing it down to the exact cause (say - it might be what Sander suggested - one of the NICs is just funky or perhaps needs a firmware update) can help here. The latest kernel is 2.6.32.26 (I think?) and the latest xen-unstable.hg has some fixes to the MSI ownership and some IRQ migration issues fixed. > > (please Mr. Konrad, no "Mr." that makes me feel really really old ;-) ) Sure thing :-) ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-devel] Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 17:01 ` [Xen-devel] " Konrad Rzeszutek Wilk 2010-12-08 17:15 ` Sander Eikelenboom 2010-12-08 17:18 ` [Xen-devel] " Sander Eikelenboom @ 2010-12-09 10:49 ` Mark Adams 2 siblings, 0 replies; 48+ messages in thread From: Mark Adams @ 2010-12-09 10:49 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: Sander Eikelenboom, xen-devel, xen-users, Stefano Stabellini On Wed, Dec 08, 2010 at 12:01:25PM -0500, Konrad Rzeszutek Wilk wrote: > > > Just a recap: > > > you pass through: > > > - 3 physical nics/IGB > > > - 1 ISDN pci ISDN box > > > > The redfone box runs on 1 of the nics - its not seperate. It converts > > ISDN to TDMoE see here.. http://www.red-fone.com/ > > > > > - all using msi/msi-x interrupts ? > > > > I tried using msi/msi-x interrupts, but it caused the raid card to drop > > off (after some use) and provided seemingly even worse performance than > > pegging everything back to legacy. > > Were you able to get a serial log and hit all of the differetn debug options > when the the RAID card died? > > I didn't get it to crash - I have spent my limited time with the hardware trying to get the setup to work effectively on the current debian xen packages (hence disabling all the MSI stuff which seemed to be the problem). I understand that this isn't helpful in terms of getting whatever bugs may be there fixed for the future however! > > > > > > Have you tried using a PV domU instead of a HVM domU ? > > > > I initially tried PV but had issues with the igb NIC's. There was > > another thread somewhere about my issues with that. > > Hmm, the only other thread I see from you is about the RAID. > > > > > > > Have you tried passing through only the ISDN box, and let the network run with the xen backend/frontend to rule out the IGB/network stuff ? > > Mr. Sander idea of isolating one piece by piece is the right way. > > There is a bunch of warnings in the QEMU output - some of them quite .. troubling. > > You seem to have issues with gntdev (as in, not found), but if you are using OpenSuSE > then that would work - I think. When you tested xen-unstable did you use the > OpenSUSE kernel or the PV-OPS one? I'm running the Debian xen packages in squeeze (4.0.1-1 and 2.6.32-28) - haven't stepped out of the packages at all. Regards, Mark ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails. 2010-12-08 12:58 ` Mark Adams 2010-12-08 13:37 ` Sander Eikelenboom @ 2010-12-08 13:48 ` Dietmar Hahn 1 sibling, 0 replies; 48+ messages in thread From: Dietmar Hahn @ 2010-12-08 13:48 UTC (permalink / raw) To: xen-users Cc: xen-devel, Konrad Rzeszutek Wilk, yunhong.jiang, JBeulich, yuan.b.liu, anthony.perard, Stefano Stabellini, Mark Adams [-- Attachment #1.1: Type: text/plain, Size: 6279 bytes --] Hi, Am 08.12.2010 schrieb "Mark Adams <mark@campbell-lange.net>": > Hi - Apologies to top post this, but after alot of testing, I believe > there must be an issue with IRQ's going missing between domU and dom0. > Unfortunately I have no data to prove this! > > With msitranslate=0 as detailed below, and pci=nomsi in the guest kernel > grub config, all 3 NIC's appear OK in the domU however I still had > issues with the red-fone ISDN box. The interrupts were showing correctly > (2000/s) in the domU but communication to the device via the NIC was > still being interrupted (as shown in the asterisk console)Note that to > get the igb driver to allow this many interrupts, the > InterruptThrottleRate was set to 0. The same config (red-fone box, > asterisk etc) works fine with a physical server. > > There is also the additional issue that I could not get the passthrough > NIC's to show correctly when I also had a bridge setup. > > Throughout my testing however, I could not get the machine to crash. > > Not sure where to go with this one. For now we are keeping our VoIP > servers physical when ISDN connections are required. Today I did some tests with xen-unstable and found these problems too. I tried to passthrough 2 pci cards and got some error messages on the xen xonsole and in the qemu logs. With msitranslate=0 and pci=nomsi I got the soundcard working in a domU linux but it doesn't help on windows. I attached the logs from the xen serial console and the qemu logs. Thanks! Dietmar. > > Regards, > Mark > > On Mon, Nov 29, 2010 at 11:36:35AM -0500, Konrad Rzeszutek Wilk wrote: > > > > > > In my new test setup, I have seen some strange behaviour. 1 of the HVM's > > > (with identical config in dom0 and domU) suddenly would not allow the > > > igb driver to be loaded in domU, even though the device was visible in > > > > Let's create a new thread for this other issue. > > > > > lspci. Shutting the machine down, removing the power cord, waiting 5 > > > seconds then plugging it in again corrected that issue - Is this > > > possibly a motherboard bug? I have also disabled the SR-IOV > > > functionality in the BIOS incase this is causing any issues. > > > > > > In addition, to try to correct the MSI issue noted above, I have changed > > > my pci= line to the following: > > > > > > pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ] > > > > With the msi_translate=1 turned on the DomU HVM guests did work, right? > > > > > > > > This has stopped the "already in use on device" log, and the devices > > > appear to show correctly in the domU. Is it safe to disable > > > msitranslate? as I understand it, its for allowing multifunction devices > > > to be seen as such in domU. Is that correct? > > > > > > I haven't been able to reproduce the dropped raid issue yet, but I am > > > awaiting delivery of the Red-Fone boxes (ISDN VoIP) which seem to cause > > > this due to their very high interrupt usage (2000 per second). > > > > OK. > > > > > > In the mean time, I can see the following in the qemu-dm logs now with > > > the msitranslate=0 enabled. Is it anything to worry about? > > > > Well, the "Error" ones are pretty bad, thought I am having a hard time > > understanding what it means. Lets copy some of the QEMU folks on this. > > > > > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:14h][Length:4] > > > pt_ioport_map: e_phys=ffff pio_base=e880 len=32 index=2 first_map=0 > > > pt_ioport_map: e_phys=c220 pio_base=e880 len=32 index=2 first_map=0 > > > pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:06.0][Offset:14h][Length:4] > > > pt_ioport_map: e_phys=ffff pio_base=ec00 len=32 index=2 first_map=0 > > > pt_ioport_map: e_phys=c240 pio_base=ec00 len=32 index=2 first_map=0 > > > pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 > > > pt_msix_update_one: Update msix entry 1 with pirq 4e gvec 61 > > > pt_msix_update_one: Update msix entry 2 with pirq 4d gvec 69 > > > pt_msix_update_one: Update msix entry 3 with pirq 4c gvec 71 > > > pt_msix_update_one: Update msix entry 4 with pirq 4b gvec 79 > > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 0 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 1 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 2 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 3 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > > > pci_msix_writel: Error: Can't update msix entry 4 since MSI-X is already function. > > > > > > > > > > > Not yet. Need to serial log of the Linux kernel and the Xen hypervisor when your > > > > machine is toast. I mentioned in the previous email the key sequences - look on Google > > > > on how to pass in SysRQ if you are using a serial concentrator. > > > > > > I will do this when I can get the machine to crash. > > > > > > Best Regards, > > > Mark > > > > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.xensource.com > > > http://lists.xensource.com/xen-devel > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > -- Company details: http://ts.fujitsu.com/imprint.html [-- Attachment #1.2: Type: text/html, Size: 25276 bytes --] [-- Attachment #2: xen-serial-log.txt --] [-- Type: text/plain, Size: 7666 bytes --] (XEN) tmem: all pools thawed for all domains (XEN) tmem: all pools frozen for all domains (XEN) tmem: all pools thawed for all domains (XEN) memory.c:133:d0 Could not allocate order=9 extent: id=18 memflags=0 (3 of 4) (XEN) memory.c:133:d0 Could not allocate order=9 extent: id=18 memflags=0 (0 of 3) (XEN) [VT-D]iommu.c:1514: d0:PCIe: unmap bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1387: d18:PCIe: map bdf = 0:1b.0 (XEN) [VT-D]io.c:303: d18: bind: m_gsi=0 g_gsi=32 device=4 intx=0 (XEN) [VT-D]iommu.c:1514: d0:PCIe: unmap bdf = 10:0.0 (XEN) HAHN-me_wifi_quirk: id 422c8086 (XEN) [VT-D]iommu.c:1387: d18:PCIe: map bdf = 10:0.0 (XEN) HAHN-me_wifi_quirk: id 422c8086 (XEN) irq.c:1511: dom18: pirq 0 or irq 40 already mapped (XEN) [VT-D]io.c:303: d18: bind: m_gsi=0 g_gsi=36 device=5 intx=0 (XEN) HVM18: HVM Loader (XEN) HVM18: Detected Xen v4.1-unstable (XEN) HVM18: CPU speed is 2394 MHz (XEN) HVM18: Xenbus rings @0xfeffc000, event channel 2 (XEN) irq.c:258: Dom18 PCI link 0 changed 0 -> 5 (XEN) HVM18: PCI-ISA link 0 routed to IRQ5 (XEN) irq.c:258: Dom18 PCI link 1 changed 0 -> 10 (XEN) HVM18: PCI-ISA link 1 routed to IRQ10 (XEN) irq.c:258: Dom18 PCI link 2 changed 0 -> 11 (XEN) HVM18: PCI-ISA link 2 routed to IRQ11 (XEN) irq.c:258: Dom18 PCI link 3 changed 0 -> 5 (XEN) HVM18: PCI-ISA link 3 routed to IRQ5 (XEN) HVM18: pci dev 01:2 INTD->IRQ5 (XEN) HVM18: pci dev 01:3 INTA->IRQ10 (XEN) HVM18: pci dev 03:0 INTA->IRQ5 (XEN) HVM18: pci dev 04:0 INTA->IRQ5 (XEN) HVM18: pci dev 05:0 INTA->IRQ10 (XEN) HVM18: pci dev 02:0 bar 10 size 02000000: f0000008 (XEN) HVM18: pci dev 03:0 bar 14 size 01000000: f2000008 (XEN) HVM18: pci dev 04:0 bar 10 size 00004000: f3000004 (XEN) domctl.c:982:d0 memory_map:add: gfn=f3000 mfn=f2720 nr_mfns=4 (XEN) HVM18: pci dev 05:0 bar 10 size 00002000: f3004004 (XEN) domctl.c:982:d0 memory_map:add: gfn=f3004 mfn=f2400 nr_mfns=2 (XEN) HVM18: pci dev 02:0 bar 14 size 00001000: f3006000 (XEN) HVM18: pci dev 03:0 bar 10 size 00000100: 0000c001 (XEN) HVM18: pci dev 01:2 bar 20 size 00000020: 0000c101 (XEN) HVM18: pci dev 01:1 bar 20 size 00000010: 0000c121 (XEN) HVM18: Multiprocessor initialisation: (XEN) HVM18: - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [2/8] ... done. (XEN) HVM18: Testing HVM environment: (XEN) HVM18: - REP INSB across page boundaries ... passed (XEN) HVM18: - GS base MSRs and SWAPGS ... passed (XEN) HVM18: Passed 2 of 2 tests (XEN) HVM18: Writing SMBIOS tables ... (XEN) HVM18: Loading ROMBIOS ... (XEN) HVM18: 9660 bytes of ROMBIOS high-memory extensions: (XEN) HVM18: Relocating to 0xfc000000-0xfc0025bc ... done (XEN) HVM18: Creating MP tables ... (XEN) HVM18: Loading Cirrus VGABIOS ... (XEN) HVM18: Loading ACPI ... (XEN) HVM18: - Lo data: 000ea020-000ea04f (XEN) HVM18: - Hi data: fc002800-fc01291f (XEN) HVM18: vm86 TSS at fc012c00 (XEN) HVM18: BIOS map: (XEN) HVM18: c0000-c8fff: VGA BIOS (XEN) HVM18: eb000-eb158: SMBIOS tables (XEN) HVM18: f0000-fffff: Main BIOS (XEN) HVM18: E820 table: (XEN) HVM18: [00]: 00000000:00000000 - 00000000:0009e000: RAM (XEN) HVM18: [01]: 00000000:0009e000 - 00000000:0009fc00: RESERVED (XEN) HVM18: [02]: 00000000:0009fc00 - 00000000:000a0000: RESERVED (XEN) HVM18: HOLE: 00000000:000a0000 - 00000000:000e0000 (XEN) HVM18: [03]: 00000000:000e0000 - 00000000:00100000: RESERVED (XEN) HVM18: [04]: 00000000:00100000 - 00000000:40000000: RAM (XEN) HVM18: HOLE: 00000000:40000000 - 00000000:fc000000 (XEN) HVM18: [05]: 00000000:fc000000 - 00000001:00000000: RESERVED (XEN) HVM18: Invoking ROMBIOS ... (XEN) HVM18: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ (XEN) stdvga.c:147:d18 entering stdvga and caching modes (XEN) HVM18: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $ (XEN) HVM18: Bochs BIOS - build: 06/23/99 (XEN) HVM18: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $ (XEN) HVM18: Options: apmbios pcibios eltorito PMM (XEN) HVM18: (XEN) HVM18: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 (XEN) HVM18: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (8192 MBytes) (XEN) HVM18: IDE time out (XEN) HVM18: (XEN) HVM18: (XEN) HVM18: (XEN) HVM18: Press F12 for boot menu. (XEN) HVM18: (XEN) HVM18: Booting from Hard Disk... (XEN) HVM18: Booting from 0000:7c00 (XEN) HVM18: int13_harddisk: function 41, unmapped device for ELDL=81 (XEN) HVM18: int13_harddisk: function 08, unmapped device for ELDL=81 (XEN) HVM18: *** int 15h function AX=00c0, BX=0000 not yet supported! (XEN) HVM18: *** int 15h function AX=ec00, BX=0002 not yet supported! (XEN) HVM18: KBD: unsupported int 16h function 03 (XEN) HVM18: *** int 15h function AX=e980, BX=0000 not yet supported! (XEN) HVM18: int13_harddisk: function 41, unmapped device for ELDL=81 (XEN) HVM18: int13_harddisk: function 02, unmapped device for ELDL=81 (XEN) HVM18: int13_harddisk: function 41, unmapped device for ELDL=82 (XEN) HVM18: int13_harddisk: function 02, unmapped device for ELDL=82 (XEN) HVM18: int13_harddisk: function 41, unmapped device for ELDL=83 (XEN) HVM18: int13_harddisk: function 02, unmapped device for ELDL=83 (XEN) HVM18: int13_harddisk: function 41, unmapped device for ELDL=84 (XEN) HVM18: int13_harddisk: function 02, unmapped device for ELDL=84 (XEN) HVM18: int13_harddisk: function 41, unmapped device for ELDL=85 (XEN) HVM18: int13_harddisk: function 02, unmapped device for ELDL=85 (XEN) HVM18: int13_harddisk: function 41, unmapped device for ELDL=86 (XEN) HVM18: int13_harddisk: function 02, unmapped device for ELDL=86 (XEN) HVM18: int13_harddisk: function 41, unmapped device for ELDL=87 (XEN) HVM18: int13_harddisk: function 02, unmapped device for ELDL=87 (XEN) HVM18: int13_harddisk: function 41, ELDL out of range 88 (XEN) HVM18: int13_harddisk: function 02, ELDL out of range 88 (XEN) HVM18: int13_harddisk: function 41, ELDL out of range 89 (XEN) HVM18: int13_harddisk: function 02, ELDL out of range 89 (XEN) HVM18: int13_harddisk: function 41, ELDL out of range 8a (XEN) HVM18: int13_harddisk: function 02, ELDL out of range 8a (XEN) HVM18: int13_harddisk: function 41, ELDL out of range 8b (XEN) HVM18: int13_harddisk: function 02, ELDL out of range 8b (XEN) HVM18: int13_harddisk: function 41, ELDL out of range 8c (XEN) HVM18: int13_harddisk: function 02, ELDL out of range 8c (XEN) HVM18: int13_harddisk: function 41, ELDL out of range 8d (XEN) HVM18: int13_harddisk: function 02, ELDL out of range 8d (XEN) HVM18: int13_harddisk: function 41, ELDL out of range 8e (XEN) HVM18: int13_harddisk: function 02, ELDL out of range 8e (XEN) HVM18: int13_harddisk: function 41, ELDL out of range 8f (XEN) HVM18: int13_harddisk: function 02, ELDL out of range 8f (XEN) stdvga.c:151:d18 leaving stdvga (XEN) vlapic.c:699:d18 Local APIC Write to read-only register 0x30 (XEN) vlapic.c:699:d18 Local APIC Write to read-only register 0x20 (XEN) vlapic.c:699:d18 Local APIC Write to read-only register 0x20 (XEN) irq.c:258: Dom18 PCI link 0 changed 5 -> 0 (XEN) irq.c:258: Dom18 PCI link 1 changed 10 -> 0 (XEN) irq.c:258: Dom18 PCI link 2 changed 11 -> 0 (XEN) irq.c:258: Dom18 PCI link 3 changed 5 -> 0 (XEN) [VT-D]io.c:327: d18: unbind: m_gsi=0 g_gsi=36 device=5 intx=0 (XEN) [VT-D]io.c:386: d18 unmap: m_irq=0 device=5 intx=0 (XEN) [VT-D]io.c:303: d18: bind: m_gsi=17 g_gsi=36 device=5 intx=0 (XEN) domctl.c:920:d0 pt_irq_create_bind failed! (XEN) irq.c:1590: dom18: forcing unbind of pirq 0 (XEN) [VT-D]io.c:327: d18: unbind: m_gsi=0 g_gsi=32 device=4 intx=0 (XEN) irq.c:1856: dom18: pirq 0 not mapped (XEN) [VT-D]io.c:386: d18 unmap: m_irq=0 device=4 intx=0 (XEN) [VT-D]io.c:303: d18: bind: m_gsi=16 g_gsi=32 device=4 intx=0 (XEN) domctl.c:920:d0 pt_irq_create_bind failed! (XEN) irq.c:1856: dom18: pirq 0 not mapped [-- Attachment #3: qemu-dm-OpenSuseHVM.log --] [-- Type: text/x-log, Size: 4425 bytes --] domid: 18 Strip off blktap sub-type prefix to /home/VMs/openSuse11.3.img (drv 'aio') Using file /home/VMs/openSuse11.3.img in read-write mode Watching /local/domain/0/device-model/18/logdirty/cmd Watching /local/domain/0/device-model/18/command Watching /local/domain/18/cpu char device redirected to /dev/pts/2 qemu_map_cache_init nr_buckets = 10000 size 4194304 shared page at pfn feffd buffered io page at pfn feffb Guest uuid = 53580b33-0ad5-8aa1-5736-93a294fd0a6c Time offset set 0 populating video RAM at ff000000 mapping video RAM from ff000000 Register xen platform. Done register platform. platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state. xs_read(/local/domain/0/device-model/18/xen_extended_power_mgmt): read error Log-dirty: no command yet. I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0 dm-command: hot insert pass-through pci dev register_real_device: Assigning real physical device 00:1b.0 ... register_real_device: Enable MSI translation via per device option register_real_device: Disable power management pt_iomul_init: Error: pt_iomul_init: No such device: setup io multiplexing failed! 0x0:0x1b.0x0 pt_register_regions: IO region registered (size=0x00004000 base_addr=0xf2720004) pt_msi_setup: pt_msi_setup requested pirq = 0 pt_msi_setup: msi mapped with pirq 0 pci_intx: intx=1 register_real_device: Real physical device 00:1b.0 registered successfuly! IRQ type = MSI-INTx vcpu-set: watch node error. xs_read(/local/domain/18/log-throttling): read error qemu: ignoring not-understood drive `/local/domain/18/log-throttling' medium change watch on `/local/domain/18/log-throttling' - unknown device, ignored dm-command: hot insert pass-through pci dev register_real_device: Assigning real physical device 10:00.0 ... register_real_device: Enable MSI translation via per device option register_real_device: Disable power management pt_iomul_init: Error: pt_iomul_init: No such device: setup io multiplexing failed! 0x10:0x0.0x0 pt_register_regions: IO region registered (size=0x00002000 base_addr=0xf2400004) pt_msi_setup: pt_msi_setup requested pirq = 0 pt_msi_setup: msi mapped with pirq 0 pci_intx: intx=1 register_real_device: Real physical device 10:00.0 registered successfuly! IRQ type = MSI-INTx xen be core: xen be core: can't open gnttab device can't open gnttab device xen be core: xen be core: can't open gnttab device can't open gnttab device xen be core: xen be core: can't open gnttab device can't open gnttab device xen be core: xen be core: can't open gnttab device can't open gnttab device xen be core: xen be core: can't open gnttab device can't open gnttab device xen be core: xen be core: can't open gnttab device can't open gnttab device xen be core: xen be core: can't open gnttab device can't open gnttab device xen be core: xen be core: can't open gnttab device can't open gnttab device xen be core: xen be core: can't open gnttab device can't open gnttab device cirrus vga map change while on lfb mode pt_iomem_map: e_phys=f3000000 maddr=f2720000 type=0 len=16384 index=0 first_map=1 pt_iomem_map: e_phys=f3004000 maddr=f2400000 type=0 len=8192 index=0 first_map=1 mapping vram to f0000000 - f0400000 platform_fixed_ioport: changed ro/rw state of ROM memory area. now is rw state. platform_fixed_ioport: changed ro/rw state of ROM memory area. now is ro state. pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:04.0][Offset:30h][Length:4] pt_pci_write_config: Warning: Guest attempt to set address to unused Base Address Register. [00:05.0][Offset:30h][Length:4] pt_msgctrl_reg_write: guest enabling MSI, disable MSI-INTx translation pci_intx: intx=1 pt_msi_update: Update msi with pirq 0 gvec 49 gflags 0 pt_msi_update: Error: Binding of MSI failed. pt_msgctrl_reg_write: Warning: Can not bind MSI for dev 28 pt_msgctrl_reg_write: setup msi for dev 28 pt_msi_setup: msi mapped with pirq 37 pt_msi_update: Update msi with pirq 37 gvec 49 gflags 0 pt_msgctrl_reg_write: guest enabling MSI, disable MSI-INTx translation pci_intx: intx=1 pt_msi_update: Update msi with pirq 0 gvec 59 gflags 0 pt_msi_update: Error: Binding of MSI failed. pt_msi_update: Error: Unmapping of MSI failed. pt_msgctrl_reg_write: Warning: Can not bind MSI for dev 20 pt_msgctrl_reg_write: setup msi for dev 20 pt_msi_setup: msi mapped with pirq 36 pt_msi_update: Update msi with pirq 36 gvec 59 gflags 0 [-- Attachment #4: Type: text/plain, Size: 137 bytes --] _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-15 17:44 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 2010-11-15 17:56 ` [Xen-users] " Mark Adams @ 2010-11-15 19:26 ` Pasi Kärkkäinen 2010-11-16 10:37 ` Re: [Xen-devel] " Mark Adams 2 siblings, 0 replies; 48+ messages in thread From: Pasi Kärkkäinen @ 2010-11-15 19:26 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich, Mark Adams On Mon, Nov 15, 2010 at 12:44:13PM -0500, Konrad Rzeszutek Wilk wrote: > On Mon, Nov 15, 2010 at 05:23:09PM +0000, Mark Adams wrote: > > On Mon, Nov 15, 2010 at 12:15:44PM -0500, Konrad Rzeszutek Wilk wrote: > > > On Sun, Nov 14, 2010 at 05:15:02PM +0000, Mark Adams wrote: > > > > > > > > On 12 Nov 2010, at 22:22, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > > > > > > > > > On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: > > > > >> On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > > > >>>> I've just noticed this at the end of xm dmesg > > > > >>>> > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > Looking briefly at the code it means that somebody enabled the MSI > > > already on the device and did not disable them. But I wonder how > > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > > > or pciback.hide (for older kernels) to "hide" the devices away from the > > > Linux Dom0 kernel? > > > > using xen-pciback.hide as its a pvops kernel (debian squeeze > > 2.6.32-5-27) > > Ok. Then it might be worth looking in when this happens. I think > there is an argument on the Xen hyperisor line to include the time-stamp, but > I don't remember it :-( > http://wiki.xen.org/xenwiki/XenHypervisorBootOptions So I think it's "console_timestamps" -- Pasi ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-15 17:44 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 2010-11-15 17:56 ` [Xen-users] " Mark Adams 2010-11-15 19:26 ` [Xen-users] Re: pci-passthrough in pvops causing offline raid Pasi Kärkkäinen @ 2010-11-16 10:37 ` Mark Adams 2010-11-16 16:04 ` [Xen-users] " Konrad Rzeszutek Wilk 2 siblings, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-16 10:37 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich On Mon, Nov 15, 2010 at 12:44:13PM -0500, Konrad Rzeszutek Wilk wrote: > On Mon, Nov 15, 2010 at 05:23:09PM +0000, Mark Adams wrote: > > On Mon, Nov 15, 2010 at 12:15:44PM -0500, Konrad Rzeszutek Wilk wrote: > > > On Sun, Nov 14, 2010 at 05:15:02PM +0000, Mark Adams wrote: > > > > > > > > On 12 Nov 2010, at 22:22, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > > > > > > > > > On Fri, Nov 12, 2010 at 05:10:58PM +0000, Mark Adams wrote: > > > > >> On Thu, Nov 11, 2010 at 02:06:58PM -0500, Konrad Rzeszutek Wilk wrote: > > > > >>>> I've just noticed this at the end of xm dmesg > > > > >>>> > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0 > > > > > > Looking briefly at the code it means that somebody enabled the MSI > > > already on the device and did not disable them. But I wonder how > > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels) > > > or pciback.hide (for older kernels) to "hide" the devices away from the > > > Linux Dom0 kernel? > > > > using xen-pciback.hide as its a pvops kernel (debian squeeze > > 2.6.32-5-27) > > Ok. Then it might be worth looking in when this happens. I think > there is an argument on the Xen hyperisor line to include the time-stamp, but > I don't remember it :-( > > > > Didn't you say that you had two servers and saw this problem on another > > > box too? > > > > > > Without more details on the Xen hypervisor line or the kernel line when > > > the failure occurs I sadly can't help you. > > > > Yes this occurs on both servers that I've tried it on. Doesn't the MSI > > log above indicate that there is a conflict - which is what ends up > > causing the device to go offline? Is there no other way to identify the > > Could be, but it is unclear - it depends on when the message pops out. > > But that does not help with finding out why your RAID controller goes offline. Stephan Austermuhle advises that nothing is logged via remote syslog when this hang occurs. I'll reply on that thread to see if he can add the additional logging > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-16 10:37 ` Re: [Xen-devel] " Mark Adams @ 2010-11-16 16:04 ` Konrad Rzeszutek Wilk 2010-11-16 16:47 ` Mark Adams 2010-11-16 21:19 ` [Xen-users] " Pasi Kärkkäinen 0 siblings, 2 replies; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-16 16:04 UTC (permalink / raw) To: Mark Adams, pasik; +Cc: xen-devel, xen-users, JBeulich > > Could be, but it is unclear - it depends on when the message pops out. > > > > But that does not help with finding out why your RAID controller goes offline. > > Stephan Austermuhle advises that nothing is logged via remote syslog > when this hang occurs. I'll reply on that thread to see if he can add He can also look at http://wiki.xensource.com/xenwiki/XenSerialConsole > the additional logging Pasi, is there a Wiki with this?: When the hang happens, he needs to do two things: 1) In the Linux kernel, hit SysRq-L, SysRQ-T 2). Then go in the hypervisor, hit Ctrl-A three times. He should see a prompt saying (XEN) ** Serial ... and hit '*' - that will collect all of the relevant information. 3). Send the full serial log from the start of the machine to us (or in this case, to you Mark - or you can just CC him on this thread). ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-16 16:04 ` [Xen-users] " Konrad Rzeszutek Wilk @ 2010-11-16 16:47 ` Mark Adams 2010-11-18 8:42 ` Re: [Xen-devel] " Stephan Austermühle 2010-11-16 21:19 ` [Xen-users] " Pasi Kärkkäinen 1 sibling, 1 reply; 48+ messages in thread From: Mark Adams @ 2010-11-16 16:47 UTC (permalink / raw) To: Konrad Rzeszutek Wilk, au; +Cc: xen-devel, JBeulich, xen-users Hi Stephan, please see debugging instructions from Konrad below. Regards, Mark On Tue, Nov 16, 2010 at 11:04:22AM -0500, Konrad Rzeszutek Wilk wrote: > > > Could be, but it is unclear - it depends on when the message pops out. > > > > > > But that does not help with finding out why your RAID controller goes offline. > > > > Stephan Austermuhle advises that nothing is logged via remote syslog > > when this hang occurs. I'll reply on that thread to see if he can add > > He can also look at http://wiki.xensource.com/xenwiki/XenSerialConsole > > > the additional logging > > Pasi, is there a Wiki with this?: > > When the hang happens, he needs to do two things: > > 1) In the Linux kernel, hit SysRq-L, SysRQ-T > > 2). Then go in the hypervisor, hit Ctrl-A three times. He should see a > prompt saying (XEN) ** Serial ... > and hit '*' - that will collect all of the relevant information. > > 3). Send the full serial log from the start of the machine to us (or in this > case, to you Mark - or you can just CC him on this thread). ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-16 16:47 ` Mark Adams @ 2010-11-18 8:42 ` Stephan Austermühle 2010-11-18 8:45 ` [Xen-users] " Pasi Kärkkäinen 0 siblings, 1 reply; 48+ messages in thread From: Stephan Austermühle @ 2010-11-18 8:42 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, JBeulich, xen-users, pasik, Konrad Rzeszutek Wilk [-- Attachment #1.1: Type: text/plain, Size: 1119 bytes --] Hello Mark! Am 16.11.2010 17:47, schrieb Mark Adams: >>> Stephan Austermuhle advises that nothing is logged via remote syslog >>> when this hang occurs. I'll reply on that thread to see if he can add >> >> He can also look at http://wiki.xensource.com/xenwiki/XenSerialConsole >> >>> the additional logging >> >> Pasi, is there a Wiki with this?: >> >> When the hang happens, he needs to do two things: >> >> 1) In the Linux kernel, hit SysRq-L, SysRQ-T >> >> 2). Then go in the hypervisor, hit Ctrl-A three times. He should see a >> prompt saying (XEN) ** Serial ... >> and hit '*' - that will collect all of the relevant information. >> >> 3). Send the full serial log from the start of the machine to us (or in this >> case, to you Mark - or you can just CC him on this thread). Thanks for your support. The server is far away from me (some hundred kilometers) with no chance to connect a serial console. The only thing that I have access to is a network console (kind of iLO). Is it sufficient to collect additional debug data? Best regards, Stephan [-- Attachment #1.2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 5179 bytes --] [-- Attachment #2: Type: text/plain, Size: 137 bytes --] _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-18 8:42 ` Re: [Xen-devel] " Stephan Austermühle @ 2010-11-18 8:45 ` Pasi Kärkkäinen 2010-11-18 8:48 ` Re: [Xen-devel] " Stephan Austermühle 0 siblings, 1 reply; 48+ messages in thread From: Pasi Kärkkäinen @ 2010-11-18 8:45 UTC (permalink / raw) To: Stephan Austermühle Cc: xen-users, Konrad Rzeszutek Wilk, xen-devel, JBeulich, Mark Adams On Thu, Nov 18, 2010 at 09:42:32AM +0100, Stephan Austermühle wrote: > Hello Mark! > > Am 16.11.2010 17:47, schrieb Mark Adams: > > >>> Stephan Austermuhle advises that nothing is logged via remote syslog > >>> when this hang occurs. I'll reply on that thread to see if he can add > >> > >> He can also look at http://wiki.xensource.com/xenwiki/XenSerialConsole > >> > >>> the additional logging > >> > >> Pasi, is there a Wiki with this?: > >> > >> When the hang happens, he needs to do two things: > >> > >> 1) In the Linux kernel, hit SysRq-L, SysRQ-T > >> > >> 2). Then go in the hypervisor, hit Ctrl-A three times. He should see a > >> prompt saying (XEN) ** Serial ... > >> and hit '*' - that will collect all of the relevant information. > >> > >> 3). Send the full serial log from the start of the machine to us (or in this > >> case, to you Mark - or you can just CC him on this thread). > > Thanks for your support. > > The server is far away from me (some hundred kilometers) with no chance > to connect a serial console. The only thing that I have access to is a > network console (kind of iLO). Is it sufficient to collect additional > debug data? > If it's SOL (Serial Over LAN), then yes, that's enough. See: http://wiki.xensource.com/xenwiki/XenSerialConsole -- Pasi ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-18 8:45 ` [Xen-users] " Pasi Kärkkäinen @ 2010-11-18 8:48 ` Stephan Austermühle 0 siblings, 0 replies; 48+ messages in thread From: Stephan Austermühle @ 2010-11-18 8:48 UTC (permalink / raw) To: Pasi Kärkkäinen Cc: xen-users, Konrad Rzeszutek Wilk, xen-devel, JBeulich, Mark Adams [-- Attachment #1.1: Type: text/plain, Size: 458 bytes --] Hi Pasi, Am 18.11.2010 09:45, schrieb Pasi Kärkkäinen: >> The server is far away from me (some hundred kilometers) with no chance >> to connect a serial console. The only thing that I have access to is a >> network console (kind of iLO). Is it sufficient to collect additional >> debug data? > > If it's SOL (Serial Over LAN), then yes, that's enough. > > See: http://wiki.xensource.com/xenwiki/XenSerialConsole I'll check. Stephan [-- Attachment #1.2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 5179 bytes --] [-- Attachment #2: Type: text/plain, Size: 137 bytes --] _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid 2010-11-16 16:04 ` [Xen-users] " Konrad Rzeszutek Wilk 2010-11-16 16:47 ` Mark Adams @ 2010-11-16 21:19 ` Pasi Kärkkäinen 1 sibling, 0 replies; 48+ messages in thread From: Pasi Kärkkäinen @ 2010-11-16 21:19 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel, xen-users, JBeulich, Mark Adams On Tue, Nov 16, 2010 at 11:04:22AM -0500, Konrad Rzeszutek Wilk wrote: > > > Could be, but it is unclear - it depends on when the message pops out. > > > > > > But that does not help with finding out why your RAID controller goes offline. > > > > Stephan Austermuhle advises that nothing is logged via remote syslog > > when this hang occurs. I'll reply on that thread to see if he can add > > He can also look at http://wiki.xensource.com/xenwiki/XenSerialConsole > > > the additional logging > > Pasi, is there a Wiki with this?: > I don't think we have this in the wiki.. at least I haven't seen one.. -- Pasi > When the hang happens, he needs to do two things: > > 1) In the Linux kernel, hit SysRq-L, SysRQ-T > > 2). Then go in the hypervisor, hit Ctrl-A three times. He should see a > prompt saying (XEN) ** Serial ... > and hit '*' - that will collect all of the relevant information. > > 3). Send the full serial log from the start of the machine to us (or in this > case, to you Mark - or you can just CC him on this thread). ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: pci-passthrough in pvops causing offline raid 2010-11-11 18:13 ` [Xen-devel] " Mark Adams 2010-11-11 18:47 ` Mark Adams @ 2010-11-11 18:57 ` Konrad Rzeszutek Wilk 1 sibling, 0 replies; 48+ messages in thread From: Konrad Rzeszutek Wilk @ 2010-11-11 18:57 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users > > > It is an Areca 1220. I can't see anything when the device goes offline > > > apart from > > > > > > [77324.264270] sd 0:0:0:1: rejecting I/O to offline device > > > [77334.005854] sd 0:0:0:0: rejecting I/O to offline device > > > > That is it? No other details from the driver? Did you poke at the driver (modinfo) > > to see if there are any options to increase its verbosity. > > I can't do anything once its happened, everything is offline so I have > no utils... An easy is to use netconsole. You can make all of the kernel log output got a different machine on your network. > > > > > > > > Unfortunately nothing get's logged because there is nothing to write to > > > anymore. I'm not sure how I can see the IRQs otherwise. There is no > > > > cat /proc/interrupts > > > > > pinning being done at all, and the machine was running for a few months > > > OK before the pciback was added. > > > > Ok, what about your NICs? Are they on-board? Are they sharing the IRQ > > with the card? You should be able to see this by looking at /proc/interrupts. > > Which NICs are they? lspci can you help you there. As of matter of fact, run > > lspci -vvv and send that. > > It is the onboard nics, they are Intel 82574L. I can see the arcmsr > line, but not anything for the NICS (because they are hidden?) Your lspci tells me it is on 16 and 17. You should see in /proc/interrupts on that line something about pciback? > > 39: 1126249 0 0 0 0 0 0 0 xen-pirq-ioapic-level arcmsr > > Nothing else is on 1126249 You mean IRQ 39. > > see lspci.txt attached. thanks. > > When this hang occurs, can you do 'xm debug-key Q', 'xm debug-key i', 'xm debug-key z'. > > Then run 'xm dmesg' and provide that to me? > > I can try this, but It probably won't work as the device is will not be > readable. Look on Google for 'Wiki PVOPS' and there is a section on how to connect a serial console. With the serial console we can send those commands to the hypervisor even if your box is hanged. http://wiki.xen.org/xenwiki/XenSerialConsole > > > > Is your boot disk on the same disk as the RAID? > > There are 2 raids, a Raid1 for the OS (/boot / /var /tmp /usr) and a > raid5 for VM's - They both dissapear at the same time so it appears the > card is dissapearing.. > I wonder if we have your IRQs confused. Can you provide the full cat /proc/interrupts and as well the serial bootup of the console? Or just the 'xm dmesg' and 'dmesg' output if you don't have the serial console hooked up yet. ^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: Re: [Xen-devel] pci-passthrough in pvops causing offline raid 2010-11-11 16:53 ` [Xen-devel] " Konrad Rzeszutek Wilk 2010-11-11 17:38 ` Mark Adams @ 2010-11-11 17:40 ` Richie 1 sibling, 0 replies; 48+ messages in thread From: Richie @ 2010-11-11 17:40 UTC (permalink / raw) To: Mark Adams; +Cc: xen-devel, xen-users On 11/11/2010 11:53 AM, Konrad Rzeszutek Wilk wrote: > On Thu, Nov 11, 2010 at 10:24:17AM +0000, Mark Adams wrote: > >> Hi All, >> >> Running xen 4.0.1-rc6, debian squeeze 2.6.32-21. >> >> In a voip setup, where I have forwarded the onboard NIC interfaces >> through to domU using the following grub config: >> >> module /vmlinuz-2.6.32-5-xen-amd64 placeholder root=UUID=25c3ac79-6850-498d-afcf-ea42970e94fd ro quiet xen-pciback.permissive xen-pciback.hide=(02:00.0)(03:00.0) pci=resource_alignment=02:00.0;03:00.0 >> >> I'm having a serious issue where the raid card goes offline after an >> indefinate period of time. Sometimes runs fine for a week, other times 1 >> day before I get "offline device" errors. Rebooting the machine fixes it >> straight away, and everything is back online. >> >> What in the Xen pciback is causing the raid card to go offline? The >> only devices hidden are the 2 onboard NIC's. >> > You need to give more details. Is the RAID card a 3Ware? An LSI? Do you > run with an IOMMU? When the RAID card goes offline, do you see a stop of > IRQs going to the device? Are the IRQs for the RAID card sent to all of your > CPUs or just a specific one? Are you pinning your guests to specific CPUs? > Does the issue disappear if you don't passthrough the NIC interfaces? If so have > you run this setup for "a week" to make sure? > >> I know that this issue is with Xen, as I had this running on a different >> server (same xen setup) and it had the same issues, which I initially >> thought were to do with the raid card. >> > So you never ran this setup on this kernel (2.6.32-5) without the Xen hypervisor? > > >> Is there known issues in this kernel and xen version with pciback? I'm >> > No. It all works perfectly :-) > > >> going to update to the current package versions this evening (4.0.1-1 >> and 2.6.32-27) however would appreciate if anyone has any other insight >> into this issue, or even just a note to say it is a bug that has been >> fixed in current versions! >> > Well, there were issues with the LSI cards having a hidden PCI device. But those > are pretty obvious as you can't even use it correctly. There is also > a problem with 3Ware 9506 IDE card - which on my box stops sending IRQs > on the IOAPIC it has been assigned (28) and instead uses another one (17). > Not sure if this is just the PCI card using the wrong PCI interrupt pin on the > card and it ends up poking the wrong IOAPIC. > > Note: I have no idea if this would be related to your issue or that my assessment is completely accurate. I had an issue that I feel the debian squeeze kernel running under domU played a part in. My dom0 is 2.6.34.7 Xenified w/Andrew lyon's patches and I running Xen 4.0.2-pre (xen testing). I passthrough a pci tuner card but have not considered that this could also contribute. Sometimes when I shutdown the domU and upon halt I started getting libata style DRIVE_NOT_READY errors in my dom0. Either one drive would drop from my mdadm raid (which houses my lvm filesystems including root for dom0 and domU) or perhaps they would drop and cause a panic. A reboot fixes everything though a rebuild would occur. I was not able to capture those errors in the few times it happened, but I have since changed to use a 2.6.31 pvops kernel from jeremy's stable branch in my domU and I have yet to reproduce the issue. I did note that it might take a number of days for the problem to manifest and so far I've tested a domU shutdown after 24 and 72 hours using the new kernel with no issues. My next test is @ 7 days. I wish I had more information myself, but I don't. Regardless of the accuracy of this claim, I recommend trying other kernels to see if the problem persists. ^ permalink raw reply [flat|nested] 48+ messages in thread
end of thread, other threads:[~2010-12-09 10:49 UTC | newest] Thread overview: 48+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-11-11 10:24 pci-passthrough in pvops causing offline raid Mark Adams 2010-11-11 11:13 ` Olivier Hanesse 2010-11-11 12:03 ` Re: [Xen-users] " Mark Adams 2010-11-11 16:53 ` [Xen-devel] " Konrad Rzeszutek Wilk 2010-11-11 17:38 ` Mark Adams 2010-11-11 17:58 ` Konrad Rzeszutek Wilk 2010-11-11 18:13 ` [Xen-devel] " Mark Adams 2010-11-11 18:47 ` Mark Adams 2010-11-11 19:06 ` Konrad Rzeszutek Wilk 2010-11-11 19:22 ` [Xen-users] " Mark Adams 2010-11-11 19:42 ` Re: [Xen-devel] " Mark Adams 2010-11-12 17:10 ` [Xen-users] " Mark Adams 2010-11-12 22:22 ` Konrad Rzeszutek Wilk 2010-11-14 17:15 ` Re: [Xen-devel] " Mark Adams 2010-11-15 17:11 ` [Xen-users] " Mark Adams 2010-11-15 17:15 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 2010-11-15 17:23 ` [Xen-users] " Mark Adams 2010-11-15 17:44 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 2010-11-15 17:56 ` [Xen-users] " Mark Adams 2010-11-24 17:59 ` Mark Adams 2010-11-24 20:28 ` Re: [Xen-devel] " Konrad Rzeszutek Wilk 2010-11-26 11:15 ` Mark Adams 2010-11-26 15:25 ` [Xen-users] " Mark Adams 2010-11-29 16:36 ` HVM DomU, msi_translate=0, MSI/MSI-X PCI passthrough fails Konrad Rzeszutek Wilk 2010-12-08 12:58 ` Mark Adams 2010-12-08 13:37 ` Sander Eikelenboom 2010-12-08 13:48 ` Mark Adams 2010-12-08 14:05 ` Sander Eikelenboom 2010-12-08 15:48 ` [Xen-users] " Mark Adams 2010-12-08 16:44 ` Sander Eikelenboom 2010-12-09 10:39 ` Mark Adams 2010-12-08 17:01 ` [Xen-devel] " Konrad Rzeszutek Wilk 2010-12-08 17:15 ` Sander Eikelenboom 2010-12-08 19:51 ` Jeremy Fitzhardinge 2010-12-08 17:18 ` [Xen-devel] " Sander Eikelenboom 2010-12-08 17:43 ` Konrad Rzeszutek Wilk 2010-12-09 10:49 ` Mark Adams 2010-12-08 13:48 ` Dietmar Hahn 2010-11-15 19:26 ` [Xen-users] Re: pci-passthrough in pvops causing offline raid Pasi Kärkkäinen 2010-11-16 10:37 ` Re: [Xen-devel] " Mark Adams 2010-11-16 16:04 ` [Xen-users] " Konrad Rzeszutek Wilk 2010-11-16 16:47 ` Mark Adams 2010-11-18 8:42 ` Re: [Xen-devel] " Stephan Austermühle 2010-11-18 8:45 ` [Xen-users] " Pasi Kärkkäinen 2010-11-18 8:48 ` Re: [Xen-devel] " Stephan Austermühle 2010-11-16 21:19 ` [Xen-users] " Pasi Kärkkäinen 2010-11-11 18:57 ` Konrad Rzeszutek Wilk 2010-11-11 17:40 ` Re: [Xen-devel] " Richie
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.