Xen 4.10.x and PCI passthrough

Xen 4.10.x and PCI passthrough

[Andreas Kinzler]

Hello Roger,

in August 2017, I reported a problem with PCI passthrough and MSI  
interrupts  
(https://lists.xenproject.org/archives/html/xen-devel/2017-08/msg01433.html).

That report lead to some patches for Xen and qemu.

Some weeks ago I tried a quite new version of Xen 4.10.2-pre  
(http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=a645331a9f4190e92ccf41a950bc4692f8904239)  
and the PCI card (LSI SAS HBA) using Windows 2012 R2 as a guest.  
Everything works but only to the point where Windows reboots -> then the  
card is no longer usable. If you destroy the domain and recreate the card  
again works.

Did I miss something simple or should we analyze the problem again using  
similar debug prints as before?

Regards Andreas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Xen 4.10.x and PCI passthrough

[Roger Pau Monné]

On Fri, Sep 07, 2018 at 07:23:43PM +0200, Andreas Kinzler wrote:
> Hello Roger,
> 
> in August 2017, I reported a problem with PCI passthrough and MSI interrupts
> (https://lists.xenproject.org/archives/html/xen-devel/2017-08/msg01433.html).
> 
> That report lead to some patches for Xen and qemu.
> 
> Some weeks ago I tried a quite new version of Xen 4.10.2-pre (http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=a645331a9f4190e92ccf41a950bc4692f8904239)
> and the PCI card (LSI SAS HBA) using Windows 2012 R2 as a guest. Everything
> works but only to the point where Windows reboots -> then the card is no
> longer usable. If you destroy the domain and recreate the card again works.
> 
> Did I miss something simple or should we analyze the problem again using
> similar debug prints as before?

Not sure, but it doesn't look to me like this issue is related to the
one fixed by the patches mentioned above, I think this is a different
issue, and by the looks of it it's a toolstack issue.

Can you paste the output of `xl -vvv create <config.cfg>` and the
contents of the log that you will find in
/var/log/xen/xl-<domain-name>.log after you have attempted a reboot?

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Xen 4.10.x and PCI passthrough

[Andreas Kinzler]

[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]

Hello Roger,

>> Some weeks ago I tried a quite new version of Xen 4.10.2-pre  
>> (http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=a645331a9f4190e92ccf41a950bc4692f8904239)
>> and the PCI card (LSI SAS HBA) using Windows 2012 R2 as a guest.  
>> Everything
>> works but only to the point where Windows reboots -> then the card is no
>> longer usable. If you destroy the domain and recreate the card again  
>> Did I miss something simple or should we analyze the problem again using
>> similar debug prints as before?
> Not sure, but it doesn't look to me like this issue is related to the
> one fixed by the patches mentioned above, I think this is a different
> issue, and by the looks of it it's a toolstack issue.
> Can you paste the output of `xl -vvv create <config.cfg>` and the
> contents of the log that you will find in
> /var/log/xen/xl-<domain-name>.log after you have attempted a reboot?

xl-domain.log is attached. From what I can see, the problem is that the  
card is deleted after domid 3 and not added back later. To confirm I ran  
"xl pci-list winsrv" during domid 3 and the card is shown. After Windows  
reboot (domid 4) "xl pci-list winsrv" gives an empty output.

Regards Andreas

[-- Attachment #2: xl-pcipt.zip --]
[-- Type: application/zip, Size: 7362 bytes --]

[-- Attachment #3: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Xen 4.10.x and PCI passthrough

[Roger Pau Monné]

On Mon, Sep 17, 2018 at 12:05:26PM +0200, Andreas Kinzler wrote:
> Hello Roger,
> 
> > > Some weeks ago I tried a quite new version of Xen 4.10.2-pre (http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=a645331a9f4190e92ccf41a950bc4692f8904239)
> > > and the PCI card (LSI SAS HBA) using Windows 2012 R2 as a guest.
> > > Everything
> > > works but only to the point where Windows reboots -> then the card is no
> > > longer usable. If you destroy the domain and recreate the card again
> > > Did I miss something simple or should we analyze the problem again
> > > using
> > > similar debug prints as before?
> > Not sure, but it doesn't look to me like this issue is related to the
> > one fixed by the patches mentioned above, I think this is a different
> > issue, and by the looks of it it's a toolstack issue.
> > Can you paste the output of `xl -vvv create <config.cfg>` and the
> > contents of the log that you will find in
> > /var/log/xen/xl-<domain-name>.log after you have attempted a reboot?
> 
> xl-domain.log is attached. From what I can see, the problem is that the card
> is deleted after domid 3 and not added back later. To confirm I ran "xl
> pci-list winsrv" during domid 3 and the card is shown. After Windows reboot
> (domid 4) "xl pci-list winsrv" gives an empty output.

Hello,

I'm not very familiar with the way xl/libxl keeps state between
reboots, I think Wei (now on Cc) was the last one to look into this.
If not I can try to take a look later this week.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Xen 4.10.x and PCI passthrough

[Wei Liu]

On Wed, Sep 19, 2018 at 04:48:58PM +0200, Roger Pau Monné wrote:
> On Mon, Sep 17, 2018 at 12:05:26PM +0200, Andreas Kinzler wrote:
> > Hello Roger,
> > 
> > > > Some weeks ago I tried a quite new version of Xen 4.10.2-pre (http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=a645331a9f4190e92ccf41a950bc4692f8904239)
> > > > and the PCI card (LSI SAS HBA) using Windows 2012 R2 as a guest.
> > > > Everything
> > > > works but only to the point where Windows reboots -> then the card is no
> > > > longer usable. If you destroy the domain and recreate the card again
> > > > Did I miss something simple or should we analyze the problem again
> > > > using
> > > > similar debug prints as before?
> > > Not sure, but it doesn't look to me like this issue is related to the
> > > one fixed by the patches mentioned above, I think this is a different
> > > issue, and by the looks of it it's a toolstack issue.
> > > Can you paste the output of `xl -vvv create <config.cfg>` and the
> > > contents of the log that you will find in
> > > /var/log/xen/xl-<domain-name>.log after you have attempted a reboot?
> > 
> > xl-domain.log is attached. From what I can see, the problem is that the card
> > is deleted after domid 3 and not added back later. To confirm I ran "xl
> > pci-list winsrv" during domid 3 and the card is shown. After Windows reboot
> > (domid 4) "xl pci-list winsrv" gives an empty output.
> 
> Hello,
> 
> I'm not very familiar with the way xl/libxl keeps state between
> reboots, I think Wei (now on Cc) was the last one to look into this.
> If not I can try to take a look later this week.

Rebooting the domain involves saving the domain state and recreating the
guest from that state.

PCI devices don't get saved when saving domain state. The framework is
generic, but there are some missing hooks for PCI type devices.

See libxl_domain.c:libxl_retrieve_domain_configuration and libxl_pci.c.

Wei.

> 
> Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel