GPU passthrough hot-swapping driver development?

GPU passthrough hot-swapping driver development?

[nucleare2]

Hello, I'd like to discuss the possibility of developing a method for hot-swapping video cards between a KVM host and the guest machine when using passthrough.

It seems that currently passthrough is pretty restrictive as far as assignment to the host vs guest goes.  This means for applications like accelerating macOS fully a video card needs to be dedicated to passthrough for macOS so that native drivers grab a hold of it at macOS guest startup.

I've been thinking about the possibility of creating some kind of a dummy passthrough driver that may, at some minimal level, virtualize the GPU access so as to allow the KVM host and guest (macOS, Windows, Linux, whatever would need the full GPU access) to "hot-swap" access to the GPU.

As I said above, right now it appears that it's necessary to have multiple GPUs for the host vs guests that need/want passthrough, but if an appropriate driver is developed that somehow captures some core functionality, couldn't a kernel level key combination capture be implemented that would flip passthrough between host and guest(s)?

Basically if a macOS or Windows (or Linux) guest BELIEVES it still has control of the GPU, then the guest kernel should not panic and should still remain operational while the host get's access.  I'm not talking about any kind of capturing here, just simply making the host or guest think it's still getting GPU access and happily spinning away while the user is flipping between host/guest instances.

Has anyone considered this or put any work into this so far?

I've seen someone else mention this before somewhere, but it seemed to not get any attention. This is a pretty critical function that would benefit KVM users enormously.

-nuc

Re: GPU passthrough hot-swapping driver development?

[nucleare2]

Given that there's been no response yet, I wanted to hopefully clarify what I'm talking about by providing a couple diagrams and maybe some clarifying terminology. I'm not sure if these diagrams will look right for everyone, so I'll also include a link to an image for each one above it.

First, please correct me if I'm wrong, but I believe the following diagram represents a very basic visualization of KVM + QEMU:

Image: https://i.ibb.co/smh9Rv4/KVM-and-QEMU.png

┌───────────────┐   ┌──────────────┐   ┌──────────────┐
│    Native     │   │    Guest     │   │    Guest     │
│   Software    │   │      VM      │   │      VM      │
│               │   │              │   │              │
│               │   │              │   │              │
│               │   │              │   │              │
│               │   │              │   │              │
└───────┬───────┘   │              │   │              │
        │           │              │   │              │
        │           │              │   │              │
        │           │              │   │              │
        │           │      ║       │   │              │
        │           ├──────╬───────┴───┴──────────────┤
        │           │      ║       QEMU               │
        │           │      ║                          │
        │           └──────╬────────────────┬─────────┘
        │                  ║  GPU           │
        │                  ║Passthru        │
        │                  ╚════════╗       │
┌───────▼───────────────────────────╬───────┼─────────┐
│                                ┌──╬───────▼──────┐  │
│Linux (kernel)                  │  ║    KVM       │  │
│                                └──╬──────────────┘  │
└───────────────────────────┬───────╬─────────────────┘
┌───────────────────────────▼───────╬─────────────────┐
│                                   ║                 │
│                 ┌─────────┐┌──────▼──┐   ┌─────────┐│
│HARDWARE         │  GPU1   ││         │   │         ││
│                 │  (host  ││  GPU2   │...│  GPUn   ││
│                 │reserved)││         │   │         ││
│                 └─────────┘└─────────┘   └─────────┘│
└─────────────────────────────────────────────────────┘


What I'm considering or asking about is the development of something I'll call a "soft video switch" within Linux that perhaps lives beside KVM or somehow works with KVM to facilitate quick switching between direct passthrough of graphics for the HOST or GUEST(s):

Image: https://i.ibb.co/gSLybZy/KVM-and-QEMU-with-soft-video-switch.png
                                                         ││
┌───────────────┐   ┌──────────────┐   ┌──────────────┐      ╔═════════════════════════════════════════╗
│    Native     │   │    Guest     │   │    Guest     │  ││  ║  EXPANDED VIEW of "soft video switch"   ║
│   Software    │   │      VM      │   │      VM      │      ║                                         ║
│               │   │              │   │              │  ││  ║───────direct guest video──┐             ║
│               │   │              │   │              │      ║         driver access     │             ║
│               │   │              │   │              │  ││  ║────────────────┐          │             ║
│               │   │              │   │              │      ║       direct host video   │             ║
└───────┬───────┘   │              │   │              │  ││  ║         driver access     │             ║
        │           │              │   │              │      ║         ┌──────▼──────────▼──────┐      ║
        │           │              │   │              │  ││  ║         │   soft video switch    │      ║
        │           │      ║       │   │              │      ║         │                        │      ║
        │           │      ║       │   │              │  ││  ║         │           Λ            │      ║
        │           ├──────╬───────┴───┴──────────────┤      ║         │          ╱ ╲           │      ║
        │           │      ║       QEMU               │  ││  ║         │         ╱   ╲          │      ║
        │           │      ║                          │      ║         │        ╱     ╲         │      ║
        │           └──────╬────────────────┬─────────┘  ││  ║         │    ┌──▕       ▏───┐    │      ║
        │                  ║   GPU          │                ║         │    │   ╲     ╱    │    │      ║
        │                  ║ Passthru       │            ││  ║         │    │    ╲   ╱     │    │      ║
        │                  ╚══════════╗     │                ║         │    │     ╲ ╱      │    │      ║
┌───────▼─────────────────────────────╬─────┼─────────┐  ││  ║         │  Active   V   Inactive │      ║
│LINUX (kernel)   ┌─────────────┐┌────╬─────▼──────┐  │      ║         │   Input         Input  │      ║
│                 │system video ││    ║  KVM       │  │  ││  ║         │    │              │    │      ║
│                 │   driver    ││    ║            │  │      ║         └────┼──────────────┼────┘      ║
│                 └──────┬──────┘└────╬────────────┘  │  ││  ║              │              │           ║
│                        │            ║               │      ║         ┌────▼─────┐  ┌─────▼────┐      ║
│                        └──┐         ║               │  ││ ▷║         │          │  │          │      ║
│                           │         ║             see    ╱ ║         │   GPU    │  │/dev/null │      ║
│                         ┌─▼─────────▼─┐  ╱─────expanded ╱  ║         │          │  │          │      ║
│                         │ soft video  │ ╱        view      ║         └──────────┘  └──────────┘      ║
│                         │   switch    │▷        (right)││  ║  ┌───────────────────────────────────┐  ║
│                         └──────┬──────┘             │      ╚══╣    NOTE: User presses some key    ╠══╝
└─────────────┬──────────────────┼────────────────────┘  ││     │       combination (example:       │
              │                  │                              │ control-alt-shift-ESC) to toggle  │
┌─────────────▼──────────────────┼────────────────────┐  ││     │  between which "OS" get's ACTIVE  │
│HARDWARE                        │                    │         │    direct GPU access while the    │
│                           ┌────▼────┐               │  ││     │    INACTIVE "OS" is discarded.    │
│                           │         │               │         └───────────────────────────────────┘
│                           │   GPU   │               │  ││
│                           │         │               │
│                           └─────────┘               │  ││
└─────────────────────────────────────────────────────┘
                                                         ││

If I'm missing any diagrammatic details, I would very much appreciate some correction.  Further, I'd like to discuss what could stop the development of such a "soft video switch" as explained above.


Thanks for any input here!

-nuc


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, June 1, 2019 6:07 PM, nucleare2 <nucleare2@protonmail.com> wrote:

> Hello, I'd like to discuss the possibility of developing a method for hot-swapping video cards between a KVM host and the guest machine when using passthrough.
>
> It seems that currently passthrough is pretty restrictive as far as assignment to the host vs guest goes. This means for applications like accelerating macOS fully a video card needs to be dedicated to passthrough for macOS so that native drivers grab a hold of it at macOS guest startup.
>
> I've been thinking about the possibility of creating some kind of a dummy passthrough driver that may, at some minimal level, virtualize the GPU access so as to allow the KVM host and guest (macOS, Windows, Linux, whatever would need the full GPU access) to "hot-swap" access to the GPU.
>
> As I said above, right now it appears that it's necessary to have multiple GPUs for the host vs guests that need/want passthrough, but if an appropriate driver is developed that somehow captures some core functionality, couldn't a kernel level key combination capture be implemented that would flip passthrough between host and guest(s)?
>
> Basically if a macOS or Windows (or Linux) guest BELIEVES it still has control of the GPU, then the guest kernel should not panic and should still remain operational while the host get's access. I'm not talking about any kind of capturing here, just simply making the host or guest think it's still getting GPU access and happily spinning away while the user is flipping between host/guest instances.
>
> Has anyone considered this or put any work into this so far?
>
> I've seen someone else mention this before somewhere, but it seemed to not get any attention. This is a pretty critical function that would benefit KVM users enormously.
>
> -nuc

Re: GPU passthrough hot-swapping driver development?

[nucleare2]

(I believe my last message may have been garbled from the way it's looking on MARC so I'm going to resend that message WITHOUT the inline diagrams (just links to images)...)

Given that there's been no response yet, I wanted to hopefully clarify what I'm talking about by providing a couple diagrams and maybe some clarifying terminology. I'm not sure if these diagrams will look right for everyone, so I'll also include a link to an image for each one above it.

First, please correct me if I'm wrong, but I believe the following diagram represents a very basic visualization of KVM + QEMU:

IMAGE: https://i.ibb.co/smh9Rv4/KVM-and-QEMU.png

What I'm considering or asking about is the development of something I'll call a "soft video switch" within Linux that perhaps lives beside KVM or somehow works with KVM to facilitate quick switching between direct passthrough of graphics for the HOST or GUEST(s):

IMAGE: https://i.ibb.co/gSLybZy/KVM-and-QEMU-with-soft-video-switch.png

If I'm missing any diagrammatic details, I would very much appreciate some correction.  Further, I'd like to discuss what could stop the development of such a "soft video switch" as explained above.


Thanks for any input here!

-nuc


> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Saturday, June 1, 2019 6:07 PM, nucleare2 nucleare2@protonmail.com wrote:
>
> > Hello, I'd like to discuss the possibility of developing a method for hot-swapping video cards between a KVM host and the guest machine when using passthrough.
> > It seems that currently passthrough is pretty restrictive as far as assignment to the host vs guest goes. This means for applications like accelerating macOS fully a video card needs to be dedicated to passthrough for macOS so that native drivers grab a hold of it at macOS guest startup.
> > I've been thinking about the possibility of creating some kind of a dummy passthrough driver that may, at some minimal level, virtualize the GPU access so as to allow the KVM host and guest (macOS, Windows, Linux, whatever would need the full GPU access) to "hot-swap" access to the GPU.
> > As I said above, right now it appears that it's necessary to have multiple GPUs for the host vs guests that need/want passthrough, but if an appropriate driver is developed that somehow captures some core functionality, couldn't a kernel level key combination capture be implemented that would flip passthrough between host and guest(s)?
> > Basically if a macOS or Windows (or Linux) guest BELIEVES it still has control of the GPU, then the guest kernel should not panic and should still remain operational while the host get's access. I'm not talking about any kind of capturing here, just simply making the host or guest think it's still getting GPU access and happily spinning away while the user is flipping between host/guest instances.
> > Has anyone considered this or put any work into this so far?
> > I've seen someone else mention this before somewhere, but it seemed to not get any attention. This is a pretty critical function that would benefit KVM users enormously.
> > -nuc