Re: pci-passthrough in pvops causing offline raid

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

> > > It is an Areca 1220. I can't see anything when the device goes offline
> > > apart from 
> > > 
> > >     [77324.264270] sd 0:0:0:1: rejecting I/O to offline device
> > >     [77334.005854] sd 0:0:0:0: rejecting I/O to offline device
> > 
> > That is it? No other details from the driver? Did you poke at the driver (modinfo)
> > to see if there are any options to increase its verbosity.
> 
> I can't do anything once its happened, everything is offline so I have
> no utils...

An easy is to use netconsole. You can make all of the kernel log output
got a different machine on your network.

> > 
> > > 
> > > Unfortunately nothing get's logged because there is nothing to write to
> > > anymore. I'm not sure how I can see the IRQs otherwise. There is no
> > 
> > cat /proc/interrupts
> > 
> > > pinning being done at all, and the machine was running for a few months
> > > OK before the pciback was added.
> > 
> > Ok, what about your NICs? Are they on-board? Are they sharing the IRQ
> > with the card? You should be able to see this by looking at /proc/interrupts.
> > Which NICs are they? lspci can you help you there. As of matter of fact, run
> > lspci -vvv and send that.
> 
> It is the onboard nics, they are Intel 82574L. I can see the arcmsr
> line, but not anything for the NICS (because they are hidden?)

Your lspci tells me it is on 16 and 17. You should see in /proc/interrupts
on that line something about pciback?
> 
> 39:    1126249          0          0          0          0          0         0          0  xen-pirq-ioapic-level  arcmsr
> 
> Nothing else is on 1126249

You mean IRQ 39.
> 
> see lspci.txt attached.

thanks.
> > When this hang occurs, can you do 'xm debug-key Q', 'xm debug-key i', 'xm debug-key z'.
> > Then run 'xm dmesg' and provide that to me?
> 
> I can try this, but It probably won't work as the device is will not be
> readable.

Look on Google for 'Wiki PVOPS' and there is a section on how to connect a serial console.
With the serial console we can send those commands to the hypervisor even if your box
is hanged.

http://wiki.xen.org/xenwiki/XenSerialConsole

> > 
> > Is your boot disk on the same disk as the RAID?
> 
> There are 2 raids, a Raid1 for the OS (/boot / /var /tmp /usr) and a
> raid5 for VM's - They both dissapear at the same time so it appears the
> card is dissapearing..
> 

I wonder if we have your IRQs confused. Can you provide the full cat /proc/interrupts
and as well the serial bootup of the console? Or just the 'xm dmesg' and 'dmesg' output
if you don't have the serial console hooked up yet.