Re: [Xen-users] Re: pci-passthrough in pvops causing offline raid

[Mark Adams <mark@campbell-lange.net>]

On Fri, Nov 26, 2010 at 11:15:20AM +0000, Mark Adams wrote:
> On Wed, Nov 24, 2010 at 03:28:43PM -0500, Konrad Rzeszutek Wilk wrote:
> > On Wed, Nov 24, 2010 at 05:59:26PM +0000, Mark Adams wrote:
> > > > > > > > >>>> 
> > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0
> > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0
> > > > > > > > >>>> (XEN) msi.c:715: MSI is already in use on device 02:00.0
> > > > > > > 
> > > > > > > Looking briefly at the code it means that somebody enabled the MSI
> > > > > > > already on the device and did not disable them. But I wonder how
> > > > > > > you got those in the first place. Did you use xen-pciback.hide (for PVOPS kernels)
> > > > > > > or pciback.hide (for older kernels) to "hide" the devices away from the
> > > > > > > Linux Dom0 kernel?
> > > 
> > > I've got a test setup in place now, and am trying to reproduce this.
> > > I've not connected up serial as yet, but can see the following logs in
> > > the qemu-dm log file when I get the "MSI is already in use" errors
> > > above. Note also that this error -always- shows for the first specified
> > > device in the pci= field, and not the 2nd.
> > > 
> 
> In my new test setup, I have seen some strange behaviour. 1 of the HVM's
> (with identical config in dom0 and domU) suddenly would not allow the
> igb driver to be loaded in domU, even though the device was visible in
> lspci. Shutting the machine down, removing the power cord, waiting 5
> seconds then plugging it in again corrected that issue - Is this
> possibly a motherboard bug? I have also disabled the SR-IOV
> functionality in the BIOS incase this is causing any issues.
> 
> In addition, to try to correct the MSI issue noted above, I have changed
> my pci= line to the following:
> 
> pci=[ '08:00.0,msitranslate=0', '08:00.1,msitranslate=0' ]

Apologies for replying to my own post, but I'm having an issue with this
setup in that I can not get a link on the 2nd interface that I'm passing
through. I've tried with msitranslate both on and off, with no success.

The device shows in the domU as an interface correctly, but even with
the link up (link led's show on the interface) domU always shows the
eth2 interface as not ready.

[    7.001784] ADDRCONF(NETDEV_UP): eth1: link is not ready
[    7.047903] ADDRCONF(NETDEV_UP): eth2: link is not ready
[   10.108995] e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
[   10.109653] ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[   16.468102] eth0: no IPv6 routers present
[   20.404092] eth1: no IPv6 routers present

I've tried using the ET dual port card (igb) aswell as the onboard
interfaces (e1000e) with exactly the same result. 

The eth0 interface is a xen bridge device, and if I remove this, both
passthrough interfaces work correctly.

Can you not have a bridge and pci-passthrough operational? is there a
limit of 2 NIC's in a HVM domU? (this doesn't sound right...)

Regards,
Mark