Re: vga passthrough // questions about pci passthrough

[Alex Williamson <alex.williamson@redhat.com>]

On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote:
> soundcard logs added
> 
> On 18.07.2012 14:08, Martin Wolf wrote:
> > On 18.07.2012 11:26, Jan Kiszka wrote:
> >> On 2012-07-18 07:45, Martin Wolf wrote:
> >>> Hello,
> >>>
> >>> i was able to passthrough an AMD 7870 videocard to my win7 guest 
> >>> machine.
> >> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
> > sure, i will prepare something
> >>
> >>> my host is ubuntu 12.04 with stock kernel.
> >>> my system contains:
> >>> dq67sw q67 mainboard
> >>> i5-2400s cpu
> >>> sapphire 7870 amd videocard
> >>> xonar d2x (problems to passthrough)
> >>>
> >>> for full functionality i just needed two options
> >>>
> >>> - kernel : iommu=on
> >>> - kvm module: ignore_msrs=1
> >>> (if i would not set it the guest os would crash with a bluescreen)
> >> Can you report (=> kernel log) which MSRs are unknown to KVM?
> > Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >
> > i hope thats the info you need, i booted it with ignore_msrs=1 since 
> > if i dont do that i get less output.
> > (do you need it without the "option"?)
> >
> >>
> >>> the unigine benchmark ran flawlessly
> >>> also the benchmark included in windows gave my videocard
> >>> similar values (7.7) comparable with my native win7 (7.9)
> >>>
> >>>
> >>> now to my questions...
> >>> 1.     is it possible to reset the videocard properly to be able to
> >>>      reboot the vm?
> >> Which versions of kernel and qemu-kvm are involved via your distro? Can
> >> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
> >> something got fixed meanwhile.
> >>
> >> In general, there are many adapters that require special procedures to
> >> perform resets. This one may fall into that category as well.
> > i will do a test today.
> >>> 2.    the xonar d2x is a very nice audio card, it would be very handy
> >>>      to be able to use it in the vm. in my oppinion the card is a
> >>>      d2 with a pci-e to pci bridge.
> >>>      i tried to passthrough the card alone and with the pci-bridge
> >>>      that was shown though lspci, but i had no success.
> >>>      maybe you guys here have an idea on that topic?
> >> Any further details about the error? Does the adapter work with a Linux
> >> guest or provide more information that way?
> >>
> >> Jan
> 
> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI 
> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
>          Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- 
> ParErr- Stepping- SERR- FastB2B- DisINTx-
>          Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- 
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>          Latency: 0, Cache Line Size: 64 bytes
>          Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
>          I/O behind bridge: 0000d000-0000dfff
>          Memory behind bridge: fff00000-000fffff
>          Prefetchable memory behind bridge: fff00000-000fffff
>          Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium 
>  >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>          BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
>                  PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>          Capabilities: <access denied>
>          Kernel driver in use: pci-stub
>          Kernel modules: shpchp
> 
> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 
> [Oxygen HD Audio]
>          Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
>          Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- 
> ParErr- Stepping- SERR- FastB2B- DisINTx-
>          Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium 
>  >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>          Interrupt: pin A routed to IRQ 11
>          Region 0: I/O ports at d000 [disabled] [size=256]
>          Capabilities: <access denied>
>          Kernel driver in use: pci-stub
>          Kernel modules: snd-virtuoso
> 
> 
>   /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda 
> /var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device 
> pci-assign,host=03:04.0
> Device assignment only supports endpoint assignment, device type 7
> qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign' 
> could not be initialized
> 
> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c 
> -hda /var/lib/libvirt/images/Win7.img -device 
> pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
> Perhaps you are assigning a device that shares an IRQ with another device?
> qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign' 
> could not be initialized

Either your distro isn't shipping a version of kvm that supports
interrupt sharing on PCI 2.3 complaint devices (likely) or your device
doesn't support PCI 2.3 INTx disable (possible).  You either need to
unbind any other devices that may be sharing IRQ 11 with this device
from their driver or upgrade your kernel and qemu-kvm.  Thanks,

Alex