Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Pasi Kärkkäinen]

Hello,

Now that Xen 4.0.0 is out and has support for primary graphics passthru
and many people have been using it already and asking for a status update
for secondary adapter passthru I thought of opening a new thread about this.

What are the actual issues that need to be resolved to get the 
secondary graphics passthru working? Probably related to VGA BIOS execution
in the guest? 

How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
How do we get that functionality integrated so people don't need additional patches? 

Was there also a problem that a working VGA BIOS cannot be copied from some cards 
and they need to use/load it from a file? 

Thanks!

-- Pasi

btw. I've created a wiki page where we can list all the tested/working graphics adapters:
http://wiki.xensource.com/xenwiki/XenVGAPassthroughTestedAdapters

Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> Hello,
> 
> Now that Xen 4.0.0 is out and has support for primary graphics passthru
> and many people have been using it already and asking for a status update
> for secondary adapter passthru I thought of opening a new thread about this.
> 
> What are the actual issues that need to be resolved to get the 
> secondary graphics passthru working? Probably related to VGA BIOS execution
> in the guest? 
> 
> How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> How do we get that functionality integrated so people don't need additional patches? 
> 
> Was there also a problem that a working VGA BIOS cannot be copied from some cards 
> and they need to use/load it from a file? 
> 
> Thanks!
> 

I think is worth letting people know that the only devices supported by
Nvidia in the pci passthrough scenario are Quadro FX 3800, 4800 and
5800 at the moment.

This doesn't mean we have to stop trying making the other devices work
too ;)

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Pasi Kärkkäinen]

On Mon, Apr 12, 2010 at 10:34:14AM +0100, Stefano Stabellini wrote:
> On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> > Hello,
> > 
> > Now that Xen 4.0.0 is out and has support for primary graphics passthru
> > and many people have been using it already and asking for a status update
> > for secondary adapter passthru I thought of opening a new thread about this.
> > 
> > What are the actual issues that need to be resolved to get the 
> > secondary graphics passthru working? Probably related to VGA BIOS execution
> > in the guest? 
> > 
> > How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> > How do we get that functionality integrated so people don't need additional patches? 
> > 
> > Was there also a problem that a working VGA BIOS cannot be copied from some cards 
> > and they need to use/load it from a file? 
> > 
> > Thanks!
> > 
> 
> I think is worth letting people know that the only devices supported by
> Nvidia in the pci passthrough scenario are Quadro FX 3800, 4800 and
> 5800 at the moment.
> 

Yeah.. I guess you mean 'official support' here? What does that mean practically? 
FLR support? 

> This doesn't mean we have to stop trying making the other devices work
> too ;)
>

01:00.0 VGA compatible controller: nVidia Corporation G92 [GeForce 8800 GTS 512] (rev a2)

That card was reported to work.. :)

-- Pasi

Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> On Mon, Apr 12, 2010 at 10:34:14AM +0100, Stefano Stabellini wrote:
> > On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> > > Hello,
> > > 
> > > Now that Xen 4.0.0 is out and has support for primary graphics passthru
> > > and many people have been using it already and asking for a status update
> > > for secondary adapter passthru I thought of opening a new thread about this.
> > > 
> > > What are the actual issues that need to be resolved to get the 
> > > secondary graphics passthru working? Probably related to VGA BIOS execution
> > > in the guest? 
> > > 
> > > How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> > > How do we get that functionality integrated so people don't need additional patches? 
> > > 
> > > Was there also a problem that a working VGA BIOS cannot be copied from some cards 
> > > and they need to use/load it from a file? 
> > > 
> > > Thanks!
> > > 
> > 
> > I think is worth letting people know that the only devices supported by
> > Nvidia in the pci passthrough scenario are Quadro FX 3800, 4800 and
> > 5800 at the moment.
> > 
> 
> Yeah.. I guess you mean 'official support' here? What does that mean practically? 
> FLR support? 
> 

It means that the retail Nvidia drivers have been tested in this use
case and probably contain some changes to make sure it works properly.

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Pasi Kärkkäinen]

On Mon, Apr 12, 2010 at 12:03:26PM +0100, Stefano Stabellini wrote:
> On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> > On Mon, Apr 12, 2010 at 10:34:14AM +0100, Stefano Stabellini wrote:
> > > On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> > > > Hello,
> > > > 
> > > > Now that Xen 4.0.0 is out and has support for primary graphics passthru
> > > > and many people have been using it already and asking for a status update
> > > > for secondary adapter passthru I thought of opening a new thread about this.
> > > > 
> > > > What are the actual issues that need to be resolved to get the 
> > > > secondary graphics passthru working? Probably related to VGA BIOS execution
> > > > in the guest? 
> > > > 
> > > > How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> > > > How do we get that functionality integrated so people don't need additional patches? 
> > > > 
> > > > Was there also a problem that a working VGA BIOS cannot be copied from some cards 
> > > > and they need to use/load it from a file? 
> > > > 
> > > > Thanks!
> > > > 
> > > 
> > > I think is worth letting people know that the only devices supported by
> > > Nvidia in the pci passthrough scenario are Quadro FX 3800, 4800 and
> > > 5800 at the moment.
> > > 
> > 
> > Yeah.. I guess you mean 'official support' here? What does that mean practically? 
> > FLR support? 
> > 
> 
> It means that the retail Nvidia drivers have been tested in this use
> case and probably contain some changes to make sure it works properly.
>

Ok. I'll add a note about this to the wiki page.

-- Pasi

Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 113 bytes --]

On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> 
> Ok. I'll add a note about this to the wiki page.
 
Thanks :)

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Christian Tramnitz]

12.04.2010 09:21, Pasi Kärkkäinen wrote:
> Hello,
>
> Now that Xen 4.0.0 is out and has support for primary graphics passthru
> and many people have been using it already and asking for a status update
> for secondary adapter passthru I thought of opening a new thread about this.
>
> What are the actual issues that need to be resolved to get the
> secondary graphics passthru working? Probably related to VGA BIOS execution
> in the guest?
>
> How about the vBAR=pBAR stuff required for some (Nvidia) cards?
> How do we get that functionality integrated so people don't need additional patches?
>
> Was there also a problem that a working VGA BIOS cannot be copied from some cards
> and they need to use/load it from a file?
>
> Thanks!
>
> -- Pasi
>
> btw. I've created a wiki page where we can list all the tested/working graphics adapters:
> http://wiki.xensource.com/xenwiki/XenVGAPassthroughTestedAdapters


I already added that suggestion to the 4.1 suggested features some days 
ago and planned to rebase the patches to 4.0 if they need any adjustments.
One major change I'd like to see is dynamic (and thus optional) 
configuration of the mem space to be mapped and the BIOS file to be used 
(instead of configuration at compile time).
Also afair the previous patch set also had a maximum size (correct me if 
I'm wrong) for the BIOS size that was to small for recent (HD4800+) ATI 
devices.


Best regards,
    Christian

Re: Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Weidong Han]

Christian Tramnitz wrote:
> 12.04.2010 09:21, Pasi Kärkkäinen wrote:
>   
>> Hello,
>>
>> Now that Xen 4.0.0 is out and has support for primary graphics passthru
>> and many people have been using it already and asking for a status update
>> for secondary adapter passthru I thought of opening a new thread about this.
>>
>> What are the actual issues that need to be resolved to get the
>> secondary graphics passthru working? Probably related to VGA BIOS execution
>> in the guest?
>>
>> How about the vBAR=pBAR stuff required for some (Nvidia) cards?
>> How do we get that functionality integrated so people don't need additional patches?
>>
>> Was there also a problem that a working VGA BIOS cannot be copied from some cards
>> and they need to use/load it from a file?
>>
>> Thanks!
>>
>> -- Pasi
>>
>> btw. I've created a wiki page where we can list all the tested/working graphics adapters:
>> http://wiki.xensource.com/xenwiki/XenVGAPassthroughTestedAdapters
>>     
>
>
> I already added that suggestion to the 4.1 suggested features some days 
> ago and planned to rebase the patches to 4.0 if they need any adjustments.
> One major change I'd like to see is dynamic (and thus optional) 
> configuration of the mem space to be mapped and the BIOS file to be used 
>   
Could you explain "dynamic configuration of the mem space to be mapped"? 
do you mean MMIO memories?
> (instead of configuration at compile time).
> Also afair the previous patch set also had a maximum size (correct me if 
> I'm wrong) for the BIOS size that was to small for recent (HD4800+) ATI 
> devices.
>   
What is the size of HD4800 bios? It can adjust memory map address 
definitions in hvmloader to solve it.

Regards,
Weidong
>
> Best regards,
>     Christian
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Christian Tramnitz]

12.04.2010 12:08, Weidong Han wrote:
>> I already added that suggestion to the 4.1 suggested features some
>> days ago and planned to rebase the patches to 4.0 if they need any
>> adjustments.
>> One major change I'd like to see is dynamic (and thus optional)
>> configuration of the mem space to be mapped and the BIOS file to be used
> Could you explain "dynamic configuration of the mem space to be mapped"?
> do you mean MMIO memories?
Yes, the MMIO BARs that currently need manual changes to dsdt.asl before 
compilation. If we could make that a runtime configuration option or 
even allow dynamic detection (via libpci?) this would help a lot. It 
would also make the changes less intrusive (and increase the chances of 
being accepted I hope).

>> Also afair the previous patch set also had a maximum size (correct me
>> if I'm wrong) for the BIOS size that was to small for recent (HD4800+)
>> ATI devices.
> What is the size of HD4800 bios? It can adjust memory map address
> definitions in hvmloader to solve it.
It's 128kB (current limitation is 64k), but that should be easy enough 
to fix, just wanted to mention it while we're discussing the topic ;-)


Are you still actively working on this topic? Haven't seen a reply to my 
previous mails to list if work is still being done. Just want to avoid, 
that we're both working on the same thing...


Best regards,
    Christian

Re: Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Konrad Rzeszutek Wilk]

On Mon, Apr 12, 2010 at 12:18:50PM +0200, Christian Tramnitz wrote:
> 12.04.2010 12:08, Weidong Han wrote:
>>> I already added that suggestion to the 4.1 suggested features some
>>> days ago and planned to rebase the patches to 4.0 if they need any
>>> adjustments.
>>> One major change I'd like to see is dynamic (and thus optional)
>>> configuration of the mem space to be mapped and the BIOS file to be used
>> Could you explain "dynamic configuration of the mem space to be mapped"?
>> do you mean MMIO memories?
> Yes, the MMIO BARs that currently need manual changes to dsdt.asl before  
> compilation. If we could make that a runtime configuration option or  
> even allow dynamic detection (via libpci?) this would help a lot. It  
> would also make the changes less intrusive (and increase the chances of  
> being accepted I hope).
>
>>> Also afair the previous patch set also had a maximum size (correct me
>>> if I'm wrong) for the BIOS size that was to small for recent (HD4800+)
>>> ATI devices.
>> What is the size of HD4800 bios? It can adjust memory map address
>> definitions in hvmloader to solve it.
> It's 128kB (current limitation is 64k), but that should be easy enough  
> to fix, just wanted to mention it while we're discussing the topic ;-)

Oh, that might also fix the QLogic BIOS issue that some folks reported.
>
>
> Are you still actively working on this topic? Haven't seen a reply to my  
> previous mails to list if work is still being done. Just want to avoid,  
> that we're both working on the same thing...

I wasn't sure on the status of this and wrote up a task item for this
exact work on the Google Summer Code Code:

http://wiki.xensource.com/xenwiki/XAPI_project_suggestions

It is great to see that there is interest in this. Christian, since you
are taking a lead in this, would you like me to nuke the entry from the
GSOC and start hacking at this yourself? Or would you like to be a mentor
for this project?

Re: Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

[Weidong Han]

Pasi Kärkkäinen wrote:
> Hello,
>
> Now that Xen 4.0.0 is out and has support for primary graphics passthru
> and many people have been using it already and asking for a status update
> for secondary adapter passthru I thought of opening a new thread about this.
>
> What are the actual issues that need to be resolved to get the 
> secondary graphics passthru working? Probably related to VGA BIOS execution
> in the guest? 
>   
I think of two things need to be solved for secondary gfx assigned:
 1) claim VGA cycle for it to display in VGA for guest, VGA is owned by 
primary gfx.
 2) load VGA bios from file, because you cannot get it from memory like 
primary gfx.
> How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> How do we get that functionality integrated so people don't need additional patches? 
>   
vBAR=pBAR patch is a bit tricky. It needs to cook a new clean vBAR=pBAR 
patch if integrate it in upstream.
> Was there also a problem that a working VGA BIOS cannot be copied from some cards 
> and they need to use/load it from a file? 
>   
Load it from a file should work for all cards, but it's not user 
friendly  (involving manual steps). It's better to implement a generic 
approach to get VGA bios for various gfx cards including primary and 
secondary gfx. The direct way is to load it from expansion rom, but it 
doesn't always work because expansion rom address may be invalid (the 
bridge may doesn't have enough memory for gfx card).

Regards,
Weidong
> Thanks!
>
> -- Pasi
>
> btw. I've created a wiki page where we can list all the tested/working graphics adapters:
> http://wiki.xensource.com/xenwiki/XenVGAPassthroughTestedAdapters
>
>
>