1 GPU multiple VMs

1 GPU multiple VMs

[Jacobs Jordi]

[-- Attachment #1.1: Type: text/plain, Size: 555 bytes --]

Hi,

I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.

I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich one would be the best solution for Xen.

API remoting (gallium3D) (ex. Xen3D project)
Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)

Can you guys give me your idea on the matter?
Please also mention any existing projects you know that are related to this problem.

Thanks in advance,

Jordi

[-- Attachment #1.2: Type: text/html, Size: 1095 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 1 GPU multiple VMs

[Stefano Stabellini]

On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> Hi,
> 
> I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> 
> I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> one would be the best solution for Xen.
> 
> API remoting (gallium3D) (ex. Xen3D project)
> Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> 
> Can you guys give me your idea on the matter?
> Please also mention any existing projects you know that are related to this problem.

My personal opinion is that the simplest thing to do is OpenGL
remoting. Gallium remoting could also be OK but considering that many
cards don't have Gallium drivers we would probably end up doing two
conversions instead of one (DirectX->Gallium in the guest,
Gallium->OpenGL in dom0).
Mediated passthrough is very card specific so I am afraid you would end
up writing virtualization specific drivers for all the cards you want to
support. Not to mention that you might need to access some videocard
interfaces that on Nvidia are not discosed.

I believe that virtualbox already supports OpenGL remoting in a decent
way, so I would start from there, port what they have to Xen, and
improve it.

Re: 1 GPU multiple VMs

[Pasi Kärkkäinen]

On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > Hi,
> > 
> > I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> > 
> > I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> > one would be the best solution for Xen.
> > 
> > API remoting (gallium3D) (ex. Xen3D project)
> > Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> > 
> > Can you guys give me your idea on the matter?
> > Please also mention any existing projects you know that are related to this problem.
> 
> My personal opinion is that the simplest thing to do is OpenGL
> remoting. Gallium remoting could also be OK but considering that many
> cards don't have Gallium drivers we would probably end up doing two
> conversions instead of one (DirectX->Gallium in the guest,
> Gallium->OpenGL in dom0).
> Mediated passthrough is very card specific so I am afraid you would end
> up writing virtualization specific drivers for all the cards you want to
> support. Not to mention that you might need to access some videocard
> interfaces that on Nvidia are not discosed.
> 
> I believe that virtualbox already supports OpenGL remoting in a decent
> way, so I would start from there, port what they have to Xen, and
> improve it.
> 

I wonder if SPICE already supports OpenGL stuff..
I think it's at least supposed to be able to support OpenGL.

(http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.html)

-- Pasi

Re: 1 GPU multiple VMs

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1919 bytes --]

On Wed, 8 Feb 2012, Pasi Kärkkäinen wrote:
> On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > Hi,
> > > 
> > > I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> > > 
> > > I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> > > one would be the best solution for Xen.
> > > 
> > > API remoting (gallium3D) (ex. Xen3D project)
> > > Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> > > 
> > > Can you guys give me your idea on the matter?
> > > Please also mention any existing projects you know that are related to this problem.
> > 
> > My personal opinion is that the simplest thing to do is OpenGL
> > remoting. Gallium remoting could also be OK but considering that many
> > cards don't have Gallium drivers we would probably end up doing two
> > conversions instead of one (DirectX->Gallium in the guest,
> > Gallium->OpenGL in dom0).
> > Mediated passthrough is very card specific so I am afraid you would end
> > up writing virtualization specific drivers for all the cards you want to
> > support. Not to mention that you might need to access some videocard
> > interfaces that on Nvidia are not discosed.
> > 
> > I believe that virtualbox already supports OpenGL remoting in a decent
> > way, so I would start from there, port what they have to Xen, and
> > improve it.
> > 
> 
> I wonder if SPICE already supports OpenGL stuff..
> I think it's at least supposed to be able to support OpenGL.
> 
> (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.html)
 
Not yet, but I think that they are working on it. Still very early days
though.
Also it would only work with HVM guests, while having a proper xen
frontend/backend OpenGL remoting driver pair would work for both.

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 1 GPU multiple VMs

[Pasi Kärkkäinen]

On Wed, Feb 08, 2012 at 06:11:56PM +0000, Stefano Stabellini wrote:
> On Wed, 8 Feb 2012, Pasi K�rkk�inen wrote:
> > On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > > Hi,
> > > > 
> > > > I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> > > > 
> > > > I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> > > > one would be the best solution for Xen.
> > > > 
> > > > API remoting (gallium3D) (ex. Xen3D project)
> > > > Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> > > > 
> > > > Can you guys give me your idea on the matter?
> > > > Please also mention any existing projects you know that are related to this problem.
> > > 
> > > My personal opinion is that the simplest thing to do is OpenGL
> > > remoting. Gallium remoting could also be OK but considering that many
> > > cards don't have Gallium drivers we would probably end up doing two
> > > conversions instead of one (DirectX->Gallium in the guest,
> > > Gallium->OpenGL in dom0).
> > > Mediated passthrough is very card specific so I am afraid you would end
> > > up writing virtualization specific drivers for all the cards you want to
> > > support. Not to mention that you might need to access some videocard
> > > interfaces that on Nvidia are not discosed.
> > > 
> > > I believe that virtualbox already supports OpenGL remoting in a decent
> > > way, so I would start from there, port what they have to Xen, and
> > > improve it.
> > > 
> > 
> > I wonder if SPICE already supports OpenGL stuff..
> > I think it's at least supposed to be able to support OpenGL.
> > 
> > (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.html)
>  
> Not yet, but I think that they are working on it. Still very early days
> though.
> Also it would only work with HVM guests, while having a proper xen
> frontend/backend OpenGL remoting driver pair would work for both.

Yep.

There's also: http://sourceforge.net/projects/vmgl/

"OpenGL apps running inside a VM use VMGL to obtain graphics hardware acceleration. VMGL supports VMware, Xen PV and HVM, qemu, and KVM VMs; X11-based OS such as Linux, FreeBSD and OpenSolaris; and ATI, Nvidia and Intel GPUs. "

-- Pasi

Re: 1 GPU multiple VMs

[Pasi Kärkkäinen]

On Wed, Feb 08, 2012 at 08:18:47PM +0200, Pasi Kärkkäinen wrote:
> On Wed, Feb 08, 2012 at 06:11:56PM +0000, Stefano Stabellini wrote:
> > On Wed, 8 Feb 2012, Pasi K�rkk�inen wrote:
> > > On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > > > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > > > Hi,
> > > > > 
> > > > > I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> > > > > 
> > > > > I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> > > > > one would be the best solution for Xen.
> > > > > 
> > > > > API remoting (gallium3D) (ex. Xen3D project)
> > > > > Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> > > > > 
> > > > > Can you guys give me your idea on the matter?
> > > > > Please also mention any existing projects you know that are related to this problem.
> > > > 
> > > > My personal opinion is that the simplest thing to do is OpenGL
> > > > remoting. Gallium remoting could also be OK but considering that many
> > > > cards don't have Gallium drivers we would probably end up doing two
> > > > conversions instead of one (DirectX->Gallium in the guest,
> > > > Gallium->OpenGL in dom0).
> > > > Mediated passthrough is very card specific so I am afraid you would end
> > > > up writing virtualization specific drivers for all the cards you want to
> > > > support. Not to mention that you might need to access some videocard
> > > > interfaces that on Nvidia are not discosed.
> > > > 
> > > > I believe that virtualbox already supports OpenGL remoting in a decent
> > > > way, so I would start from there, port what they have to Xen, and
> > > > improve it.
> > > > 
> > > 
> > > I wonder if SPICE already supports OpenGL stuff..
> > > I think it's at least supposed to be able to support OpenGL.
> > > 
> > > (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.html)
> >  
> > Not yet, but I think that they are working on it. Still very early days
> > though.
> > Also it would only work with HVM guests, while having a proper xen
> > frontend/backend OpenGL remoting driver pair would work for both.
> 
> Yep.
> 
> There's also: http://sourceforge.net/projects/vmgl/
> 
> "OpenGL apps running inside a VM use VMGL to obtain graphics hardware acceleration. VMGL supports VMware, Xen PV and HVM, qemu, and KVM VMs; X11-based OS such as Linux, FreeBSD and OpenSolaris; and ATI, Nvidia and Intel GPUs. "
> 

And Chromium might have something related aswell:
http://chromium.sourceforge.net/doc/index.html


-- Pasi

Re: 1 GPU multiple VMs

[Pasi Kärkkäinen]

On Wed, Feb 08, 2012 at 08:22:33PM +0200, Pasi Kärkkäinen wrote:
> On Wed, Feb 08, 2012 at 08:18:47PM +0200, Pasi Kärkkäinen wrote:
> > On Wed, Feb 08, 2012 at 06:11:56PM +0000, Stefano Stabellini wrote:
> > > On Wed, 8 Feb 2012, Pasi K�rkk�inen wrote:
> > > > On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > > > > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> > > > > > 
> > > > > > I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> > > > > > one would be the best solution for Xen.
> > > > > > 
> > > > > > API remoting (gallium3D) (ex. Xen3D project)
> > > > > > Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> > > > > > 
> > > > > > Can you guys give me your idea on the matter?
> > > > > > Please also mention any existing projects you know that are related to this problem.
> > > > > 
> > > > > My personal opinion is that the simplest thing to do is OpenGL
> > > > > remoting. Gallium remoting could also be OK but considering that many
> > > > > cards don't have Gallium drivers we would probably end up doing two
> > > > > conversions instead of one (DirectX->Gallium in the guest,
> > > > > Gallium->OpenGL in dom0).
> > > > > Mediated passthrough is very card specific so I am afraid you would end
> > > > > up writing virtualization specific drivers for all the cards you want to
> > > > > support. Not to mention that you might need to access some videocard
> > > > > interfaces that on Nvidia are not discosed.
> > > > > 
> > > > > I believe that virtualbox already supports OpenGL remoting in a decent
> > > > > way, so I would start from there, port what they have to Xen, and
> > > > > improve it.
> > > > > 
> > > > 
> > > > I wonder if SPICE already supports OpenGL stuff..
> > > > I think it's at least supposed to be able to support OpenGL.
> > > > 
> > > > (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.html)
> > >  
> > > Not yet, but I think that they are working on it. Still very early days
> > > though.
> > > Also it would only work with HVM guests, while having a proper xen
> > > frontend/backend OpenGL remoting driver pair would work for both.
> > 
> > Yep.
> > 
> > There's also: http://sourceforge.net/projects/vmgl/
> > 
> > "OpenGL apps running inside a VM use VMGL to obtain graphics hardware acceleration. VMGL supports VMware, Xen PV and HVM, qemu, and KVM VMs; X11-based OS such as Linux, FreeBSD and OpenSolaris; and ATI, Nvidia and Intel GPUs. "
> > 
> 
> And Chromium might have something related aswell:
> http://chromium.sourceforge.net/doc/index.html
> 

And one more related interesting link:

"VMware's Virtual GPU Driver Is Running Fast":
http://www.phoronix.com/scan.php?page=article&item=vmware_vmwgfx_g3d&num=1

So the Gallium3D virtual GPU/OpenGL stuff with VMware VMs seems to work pretty nicely these days!
That might be worth a look.. and to get it running with Xen.

I think SPICE is going to use Gallium3D aswell.

-- Pasi

Re: 1 GPU multiple VMs

[Pasi Kärkkäinen]

On Mon, Feb 13, 2012 at 04:27:38PM +0200, Pasi Kärkkäinen wrote:
> On Wed, Feb 08, 2012 at 08:22:33PM +0200, Pasi Kärkkäinen wrote:
> > On Wed, Feb 08, 2012 at 08:18:47PM +0200, Pasi Kärkkäinen wrote:
> > > On Wed, Feb 08, 2012 at 06:11:56PM +0000, Stefano Stabellini wrote:
> > > > On Wed, 8 Feb 2012, Pasi K�rkk�inen wrote:
> > > > > On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > > > > > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> > > > > > > 
> > > > > > > I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> > > > > > > one would be the best solution for Xen.
> > > > > > > 
> > > > > > > API remoting (gallium3D) (ex. Xen3D project)
> > > > > > > Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> > > > > > > 
> > > > > > > Can you guys give me your idea on the matter?
> > > > > > > Please also mention any existing projects you know that are related to this problem.
> > > > > > 
> > > > > > My personal opinion is that the simplest thing to do is OpenGL
> > > > > > remoting. Gallium remoting could also be OK but considering that many
> > > > > > cards don't have Gallium drivers we would probably end up doing two
> > > > > > conversions instead of one (DirectX->Gallium in the guest,
> > > > > > Gallium->OpenGL in dom0).
> > > > > > Mediated passthrough is very card specific so I am afraid you would end
> > > > > > up writing virtualization specific drivers for all the cards you want to
> > > > > > support. Not to mention that you might need to access some videocard
> > > > > > interfaces that on Nvidia are not discosed.
> > > > > > 
> > > > > > I believe that virtualbox already supports OpenGL remoting in a decent
> > > > > > way, so I would start from there, port what they have to Xen, and
> > > > > > improve it.
> > > > > > 
> > > > > 
> > > > > I wonder if SPICE already supports OpenGL stuff..
> > > > > I think it's at least supposed to be able to support OpenGL.
> > > > > 
> > > > > (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.html)
> > > >  
> > > > Not yet, but I think that they are working on it. Still very early days
> > > > though.
> > > > Also it would only work with HVM guests, while having a proper xen
> > > > frontend/backend OpenGL remoting driver pair would work for both.
> > > 
> > > Yep.
> > > 
> > > There's also: http://sourceforge.net/projects/vmgl/
> > > 
> > > "OpenGL apps running inside a VM use VMGL to obtain graphics hardware acceleration. VMGL supports VMware, Xen PV and HVM, qemu, and KVM VMs; X11-based OS such as Linux, FreeBSD and OpenSolaris; and ATI, Nvidia and Intel GPUs. "
> > > 
> > 
> > And Chromium might have something related aswell:
> > http://chromium.sourceforge.net/doc/index.html
> > 
> 
> And one more related interesting link:
> 
> "VMware's Virtual GPU Driver Is Running Fast":
> http://www.phoronix.com/scan.php?page=article&item=vmware_vmwgfx_g3d&num=1
> 
> So the Gallium3D virtual GPU/OpenGL stuff with VMware VMs seems to work pretty nicely these days!
> That might be worth a look.. and to get it running with Xen.
> 
> I think SPICE is going to use Gallium3D aswell.
> 

I thought i'd already stop spamming but here's more relevant links:

"SPICE On KVM/QEMU Works Towards Gallium3D":
http://www.phoronix.com/scan.php?page=news_item&px=MTA1NTQ

And even more interesting!:

"The Gallium3D Driver That Few Know About / Xen vGallium":
http://www.phoronix.com/scan.php?page=news_item&px=ODE2NQ


-- Pasi

Re: 1 GPU multiple VMs

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 4444 bytes --]

On Mon, 13 Feb 2012, Pasi Kärkkäinen wrote:
> On Mon, Feb 13, 2012 at 04:27:38PM +0200, Pasi Kärkkäinen wrote:
> > On Wed, Feb 08, 2012 at 08:22:33PM +0200, Pasi Kärkkäinen wrote:
> > > On Wed, Feb 08, 2012 at 08:18:47PM +0200, Pasi Kärkkäinen wrote:
> > > > On Wed, Feb 08, 2012 at 06:11:56PM +0000, Stefano Stabellini wrote:
> > > > > On Wed, 8 Feb 2012, Pasi K�rkk�inen wrote:
> > > > > > On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > > > > > > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > > > > > > Hi,
> > > > > > > > 
> > > > > > > > I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> > > > > > > > 
> > > > > > > > I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> > > > > > > > one would be the best solution for Xen.
> > > > > > > > 
> > > > > > > > API remoting (gallium3D) (ex. Xen3D project)
> > > > > > > > Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> > > > > > > > 
> > > > > > > > Can you guys give me your idea on the matter?
> > > > > > > > Please also mention any existing projects you know that are related to this problem.
> > > > > > > 
> > > > > > > My personal opinion is that the simplest thing to do is OpenGL
> > > > > > > remoting. Gallium remoting could also be OK but considering that many
> > > > > > > cards don't have Gallium drivers we would probably end up doing two
> > > > > > > conversions instead of one (DirectX->Gallium in the guest,
> > > > > > > Gallium->OpenGL in dom0).
> > > > > > > Mediated passthrough is very card specific so I am afraid you would end
> > > > > > > up writing virtualization specific drivers for all the cards you want to
> > > > > > > support. Not to mention that you might need to access some videocard
> > > > > > > interfaces that on Nvidia are not discosed.
> > > > > > > 
> > > > > > > I believe that virtualbox already supports OpenGL remoting in a decent
> > > > > > > way, so I would start from there, port what they have to Xen, and
> > > > > > > improve it.
> > > > > > > 
> > > > > > 
> > > > > > I wonder if SPICE already supports OpenGL stuff..
> > > > > > I think it's at least supposed to be able to support OpenGL.
> > > > > > 
> > > > > > (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.html)
> > > > >  
> > > > > Not yet, but I think that they are working on it. Still very early days
> > > > > though.
> > > > > Also it would only work with HVM guests, while having a proper xen
> > > > > frontend/backend OpenGL remoting driver pair would work for both.
> > > > 
> > > > Yep.
> > > > 
> > > > There's also: http://sourceforge.net/projects/vmgl/
> > > > 
> > > > "OpenGL apps running inside a VM use VMGL to obtain graphics hardware acceleration. VMGL supports VMware, Xen PV and HVM, qemu, and KVM VMs; X11-based OS such as Linux, FreeBSD and OpenSolaris; and ATI, Nvidia and Intel GPUs. "
> > > > 
> > > 
> > > And Chromium might have something related aswell:
> > > http://chromium.sourceforge.net/doc/index.html
> > > 
> > 
> > And one more related interesting link:
> > 
> > "VMware's Virtual GPU Driver Is Running Fast":
> > http://www.phoronix.com/scan.php?page=article&item=vmware_vmwgfx_g3d&num=1
> > 
> > So the Gallium3D virtual GPU/OpenGL stuff with VMware VMs seems to work pretty nicely these days!
> > That might be worth a look.. and to get it running with Xen.
> > 
> > I think SPICE is going to use Gallium3D aswell.
> > 
> 
> I thought i'd already stop spamming but here's more relevant links:
> 
> "SPICE On KVM/QEMU Works Towards Gallium3D":
> http://www.phoronix.com/scan.php?page=news_item&px=MTA1NTQ
> 
> And even more interesting!:
> 
> "The Gallium3D Driver That Few Know About / Xen vGallium":
> http://www.phoronix.com/scan.php?page=news_item&px=ODE2NQ
 
That's right: Xen had a Gallium3D based virtual GPU implementation
before anyone else, however it never went out of the prototype phase. It
would be great if a well motivated developer could get the sources
(https://github.com/smowton/vgallium,
http://www.cl.cam.ac.uk/~cs448/git/trunk/doc/Build-instructions) and
continue the work!
Ideally we would have a PV frontend/backend drivers pair that would work
for both PV and HVM guests. On the guest side the work would need to be
upstreamed to Mesa/DRM, on the host side we would need support in
QEMU/Linux.

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 1 GPU multiple VMs

[Pasi Kärkkäinen]

On Mon, Feb 13, 2012 at 03:03:23PM +0000, Stefano Stabellini wrote:
> On Mon, 13 Feb 2012, Pasi Kärkkäinen wrote:
> > On Mon, Feb 13, 2012 at 04:27:38PM +0200, Pasi Kärkkäinen wrote:
> > > On Wed, Feb 08, 2012 at 08:22:33PM +0200, Pasi Kärkkäinen wrote:
> > > > On Wed, Feb 08, 2012 at 08:18:47PM +0200, Pasi Kärkkäinen wrote:
> > > > > On Wed, Feb 08, 2012 at 06:11:56PM +0000, Stefano Stabellini wrote:
> > > > > > On Wed, 8 Feb 2012, Pasi KÃ??Â?rkkÃ??Â?inen wrote:
> > > > > > > On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > > > > > > > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > > > > > > > Hi,
> > > > > > > > > 
> > > > > > > > > I am wondering how GPU sharing could/should be implemented for the Xen Hypervisor.
> > > > > > > > > 
> > > > > > > > > I have come across several papers that explain many possibilities on GPU sharing for multiple VMs but I'm not sure wich
> > > > > > > > > one would be the best solution for Xen.
> > > > > > > > > 
> > > > > > > > > API remoting (gallium3D) (ex. Xen3D project)
> > > > > > > > > Mediated passthrough (Multiplexing the GPU while maintaining the different contexts.)
> > > > > > > > > 
> > > > > > > > > Can you guys give me your idea on the matter?
> > > > > > > > > Please also mention any existing projects you know that are related to this problem.
> > > > > > > > 
> > > > > > > > My personal opinion is that the simplest thing to do is OpenGL
> > > > > > > > remoting. Gallium remoting could also be OK but considering that many
> > > > > > > > cards don't have Gallium drivers we would probably end up doing two
> > > > > > > > conversions instead of one (DirectX->Gallium in the guest,
> > > > > > > > Gallium->OpenGL in dom0).
> > > > > > > > Mediated passthrough is very card specific so I am afraid you would end
> > > > > > > > up writing virtualization specific drivers for all the cards you want to
> > > > > > > > support. Not to mention that you might need to access some videocard
> > > > > > > > interfaces that on Nvidia are not discosed.
> > > > > > > > 
> > > > > > > > I believe that virtualbox already supports OpenGL remoting in a decent
> > > > > > > > way, so I would start from there, port what they have to Xen, and
> > > > > > > > improve it.
> > > > > > > > 
> > > > > > > 
> > > > > > > I wonder if SPICE already supports OpenGL stuff..
> > > > > > > I think it's at least supposed to be able to support OpenGL.
> > > > > > > 
> > > > > > > (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.html)
> > > > > >  
> > > > > > Not yet, but I think that they are working on it. Still very early days
> > > > > > though.
> > > > > > Also it would only work with HVM guests, while having a proper xen
> > > > > > frontend/backend OpenGL remoting driver pair would work for both.
> > > > > 
> > > > > Yep.
> > > > > 
> > > > > There's also: http://sourceforge.net/projects/vmgl/
> > > > > 
> > > > > "OpenGL apps running inside a VM use VMGL to obtain graphics hardware acceleration. VMGL supports VMware, Xen PV and HVM, qemu, and KVM VMs; X11-based OS such as Linux, FreeBSD and OpenSolaris; and ATI, Nvidia and Intel GPUs. "
> > > > > 
> > > > 
> > > > And Chromium might have something related aswell:
> > > > http://chromium.sourceforge.net/doc/index.html
> > > > 
> > > 
> > > And one more related interesting link:
> > > 
> > > "VMware's Virtual GPU Driver Is Running Fast":
> > > http://www.phoronix.com/scan.php?page=article&item=vmware_vmwgfx_g3d&num=1
> > > 
> > > So the Gallium3D virtual GPU/OpenGL stuff with VMware VMs seems to work pretty nicely these days!
> > > That might be worth a look.. and to get it running with Xen.
> > > 
> > > I think SPICE is going to use Gallium3D aswell.
> > > 
> > 
> > I thought i'd already stop spamming but here's more relevant links:
> > 
> > "SPICE On KVM/QEMU Works Towards Gallium3D":
> > http://www.phoronix.com/scan.php?page=news_item&px=MTA1NTQ
> > 
> > And even more interesting!:
> > 
> > "The Gallium3D Driver That Few Know About / Xen vGallium":
> > http://www.phoronix.com/scan.php?page=news_item&px=ODE2NQ
>  
> That's right: Xen had a Gallium3D based virtual GPU implementation
> before anyone else, however it never went out of the prototype phase. It
> would be great if a well motivated developer could get the sources
> (https://github.com/smowton/vgallium,
> http://www.cl.cam.ac.uk/~cs448/git/trunk/doc/Build-instructions) and
> continue the work!
> Ideally we would have a PV frontend/backend drivers pair that would work
> for both PV and HVM guests. On the guest side the work would need to be
> upstreamed to Mesa/DRM, on the host side we would need support in
> QEMU/Linux.
>

And some more related info for the mailinglist archives:

OpenSuse has some rpms with the xen3d/vgallium patches:
https://build.opensuse.org/package/files?package=xen-vgallium&project=security%3AOpenTC

Research paper from XenSummit 2009:
"Flexible and Secure Hardware 3D Rendering on Xen":
http://www.xen.org/files/xensummit_oracle09/xensummit_chris.pdf


-- Pasi

Re: 1 GPU multiple VMs

[Lv, Zhiyuan]

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Pasi K?rkk?inen
> Sent: Thursday, February 09, 2012 2:23 AM
> To: Stefano Stabellini
> Cc: Xen-devel@lists.xensource.com; Jacobs Jordi
> Subject: Re: [Xen-devel] 1 GPU multiple VMs
> 
> On Wed, Feb 08, 2012 at 08:18:47PM +0200, Pasi Kärkkäinen wrote:
> > On Wed, Feb 08, 2012 at 06:11:56PM +0000, Stefano Stabellini wrote:
> > > On Wed, 8 Feb 2012, Pasi K�rkk�inen wrote:
> > > > On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > > > > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I am wondering how GPU sharing could/should be implemented for the
> Xen Hypervisor.
> > > > > >
> > > > > > I have come across several papers that explain many possibilities on
> GPU sharing for multiple VMs but I'm not sure wich
> > > > > > one would be the best solution for Xen.
> > > > > >
> > > > > > API remoting (gallium3D) (ex. Xen3D project)
> > > > > > Mediated passthrough (Multiplexing the GPU while maintaining the
> different contexts.)
> > > > > >
> > > > > > Can you guys give me your idea on the matter?
> > > > > > Please also mention any existing projects you know that are related to
> this problem.
> > > > >
> > > > > My personal opinion is that the simplest thing to do is OpenGL
> > > > > remoting. Gallium remoting could also be OK but considering that many
> > > > > cards don't have Gallium drivers we would probably end up doing two
> > > > > conversions instead of one (DirectX->Gallium in the guest,
> > > > > Gallium->OpenGL in dom0).
> > > > > Mediated passthrough is very card specific so I am afraid you would end
> > > > > up writing virtualization specific drivers for all the cards you want to
> > > > > support. Not to mention that you might need to access some videocard
> > > > > interfaces that on Nvidia are not discosed.
> > > > >
> > > > > I believe that virtualbox already supports OpenGL remoting in a decent
> > > > > way, so I would start from there, port what they have to Xen, and
> > > > > improve it.
> > > > >
> > > >
> > > > I wonder if SPICE already supports OpenGL stuff..
> > > > I think it's at least supposed to be able to support OpenGL.
> > > >
> > > >
> (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.ht
> ml)
> > >
> > > Not yet, but I think that they are working on it. Still very early days
> > > though.
> > > Also it would only work with HVM guests, while having a proper xen
> > > frontend/backend OpenGL remoting driver pair would work for both.
> >
> > Yep.
> >
> > There's also: http://sourceforge.net/projects/vmgl/
> >
> > "OpenGL apps running inside a VM use VMGL to obtain graphics hardware
> acceleration. VMGL supports VMware, Xen PV and HVM, qemu, and KVM VMs;
> X11-based OS such as Linux, FreeBSD and OpenSolaris; and ATI, Nvidia and Intel
> GPUs. "
> >
> 
> And Chromium might have something related aswell:
> http://chromium.sourceforge.net/doc/index.html
> 
Hello,

Another related work is as below:
http://lists.gnu.org/archive/html/qemu-devel/2012-01/msg01027.html

Similar to VMGL, it is an API remoting approach but not based on chromium. Another difference is that it does not require X server extensions. Just for your reference. Thanks!

> 
> -- Pasi
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: 1 GPU multiple VMs

[Stefano Stabellini]

On Tue, 14 Feb 2012, Lv, Zhiyuan wrote:
> Hello,
> 
> Another related work is as below:
> http://lists.gnu.org/archive/html/qemu-devel/2012-01/msg01027.html
> 
> Similar to VMGL, it is an API remoting approach but not based on chromium. Another difference is that it does not require X server extensions. Just for your reference. Thanks!

Thanks for the very useful link.
Ideally instead of virtio it would be based on a Xen frontend/backend
pair of paravirtualized devices, so that it would work with PV guests
too (virtio only works with Xen HVM guests).

Re: 1 GPU multiple VMs

[Jacobs Jordi]

http://xen.org/products/xci_source.html
XCI (Xen Client Initiative)
XenClient from Citrix is also capable to share a GPU.
However the software doesn't support many GPUs, so that might be a downside.
(I have not yet analysed the repositories.)
________________________________________
From: Lv, Zhiyuan [zhiyuan.lv@intel.com]
Sent: Wednesday, February 15, 2012 12:47 AM
To: Pasi Kärkkäinen; Stefano Stabellini
Cc: Xen-devel@lists.xensource.com; Jacobs Jordi; Zaborowski, Andrew
Subject: RE: [Xen-devel] 1 GPU multiple VMs

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Pasi K?rkk?inen
> Sent: Thursday, February 09, 2012 2:23 AM
> To: Stefano Stabellini
> Cc: Xen-devel@lists.xensource.com; Jacobs Jordi
> Subject: Re: [Xen-devel] 1 GPU multiple VMs
>
> On Wed, Feb 08, 2012 at 08:18:47PM +0200, Pasi Kärkkäinen wrote:
> > On Wed, Feb 08, 2012 at 06:11:56PM +0000, Stefano Stabellini wrote:
> > > On Wed, 8 Feb 2012, Pasi K�rkk�inen wrote:
> > > > On Wed, Feb 08, 2012 at 05:20:31PM +0000, Stefano Stabellini wrote:
> > > > > On Fri, 3 Feb 2012, Jacobs Jordi wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I am wondering how GPU sharing could/should be implemented for the
> Xen Hypervisor.
> > > > > >
> > > > > > I have come across several papers that explain many possibilities on
> GPU sharing for multiple VMs but I'm not sure wich
> > > > > > one would be the best solution for Xen.
> > > > > >
> > > > > > API remoting (gallium3D) (ex. Xen3D project)
> > > > > > Mediated passthrough (Multiplexing the GPU while maintaining the
> different contexts.)
> > > > > >
> > > > > > Can you guys give me your idea on the matter?
> > > > > > Please also mention any existing projects you know that are related to
> this problem.
> > > > >
> > > > > My personal opinion is that the simplest thing to do is OpenGL
> > > > > remoting. Gallium remoting could also be OK but considering that many
> > > > > cards don't have Gallium drivers we would probably end up doing two
> > > > > conversions instead of one (DirectX->Gallium in the guest,
> > > > > Gallium->OpenGL in dom0).
> > > > > Mediated passthrough is very card specific so I am afraid you would end
> > > > > up writing virtualization specific drivers for all the cards you want to
> > > > > support. Not to mention that you might need to access some videocard
> > > > > interfaces that on Nvidia are not discosed.
> > > > >
> > > > > I believe that virtualbox already supports OpenGL remoting in a decent
> > > > > way, so I would start from there, port what they have to Xen, and
> > > > > improve it.
> > > > >
> > > >
> > > > I wonder if SPICE already supports OpenGL stuff..
> > > > I think it's at least supposed to be able to support OpenGL.
> > > >
> > > >
> (http://lists.freedesktop.org/archives/spice-devel/2011-November/006018.ht
> ml)
> > >
> > > Not yet, but I think that they are working on it. Still very early days
> > > though.
> > > Also it would only work with HVM guests, while having a proper xen
> > > frontend/backend OpenGL remoting driver pair would work for both.
> >
> > Yep.
> >
> > There's also: http://sourceforge.net/projects/vmgl/
> >
> > "OpenGL apps running inside a VM use VMGL to obtain graphics hardware
> acceleration. VMGL supports VMware, Xen PV and HVM, qemu, and KVM VMs;
> X11-based OS such as Linux, FreeBSD and OpenSolaris; and ATI, Nvidia and Intel
> GPUs. "
> >
>
> And Chromium might have something related aswell:
> http://chromium.sourceforge.net/doc/index.html
>
Hello,

Another related work is as below:
http://lists.gnu.org/archive/html/qemu-devel/2012-01/msg01027.html

Similar to VMGL, it is an API remoting approach but not based on chromium. Another difference is that it does not require X server extensions. Just for your reference. Thanks!

>
> -- Pasi
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

1 GPU multiple VMs

[Jacobs Jordi]

[-- Attachment #1.1: Type: text/plain, Size: 165 bytes --]

Hi,
Something recent that might also fit:
http://www.phoronix.com/scan.php?page=news_item&px=MTA3OTI
https://github.com/shinpei0208/Gdev

Kind regards,
Jordi

[-- Attachment #1.2: Type: text/html, Size: 684 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel