Re: vga passthrough // questions about pci passthrough

[Martin Wolf <mwolf@adiumentum.com>]

thank you for the information.

i will try what you mentioned...
do you have some additional information about rebooting a VM with a 
passed through videocard?
(amd / ati 7870)

thanks in advance

Am 27.09.2012 19:56, schrieb Alex Williamson:
> On Thu, 2012-09-27 at 13:19 +0200, Martin Wolf wrote:
>> i waited 2 months to get a more stable ubuntu 12.10
>> now that i dont get crashes every 5 minutes im going to test again.
>>
>> as you remember my two only problems are
>> first the reboot issue of the gues os
>> and second the xonar 2 d2x audio card.
>>
>> but first i would like to work on the reboot issue since
>> i guess this is far more important.
>>
>> when i read the faq i found that one problem could be the "VBIOS ROM access"
>> i tried -device pci-assign,...,romfile=... but have not found any output
>> or change.
>> is there some documentation of the romfile command to generate usable
>> output?
> If you can read the rom from the host, this probably isn't adding
> anything.  You can test this by finding the rom in /sys (find /sys -name
> rom), match it up to the PCI device you're assigning.  Then do:
>
> # echo 1 > rom
> # xxd rom | head
> #echo 0 > rom
>
> If you get something out that starts with 55aa then qemu will be able to
> read the rom directly.  If you're replacing the rom with the romfile=
> option, you can boot a linux guest, do the same as above to read the rom
> from the guest, save it to a file and compare md5sum of what the guest
> sees to the file provided.  If qemu is reading the rom on the device
> correctly, it's no surprise that providing one doesn't make any
> difference.
>
>> for testing i downloaded the bios of my videocard from techpowerup.
>> they have a huge database and my card and biosrevision is listed.
>> was that ok or do i have to retrieve the bios from inside the vm?
> Thanks for mentioning that.  From other reports I've seen, it looks like
> nvidia cards are the troublesome ones for being able to read the BIOS
> from the host.  Thanks,
>
> Alex
>
>
>> thank you in advance
>>
>> Am 19.07.2012 18:16, schrieb Alex Williamson:
>>> On Thu, 2012-07-19 at 16:54 +0200, Martin Wolf wrote:
>>>> i tried now the alpha of ubuntu 12.10 that includes qemu-kvm 1.1
>>>> and a 3.5 kernel version.
>>>>
>>>> the "msr" problem is gone now, i was able to select "sandybridge" as
>>>> cpu topology, this removed the MSR error messages.
>>>> thats the positive part...
>>>>
>>>> unfortunately i had no success with the other topics (rebootability and
>>>> the soundcard)
>>>>
>>>> rebootability:
>>>> the guest vm still crashes with a page fault bluescreen
>>>>
>>>> soundcard:
>>>> it would be really nice if someone would be able to give me
>>>> some advise how to debug this problem.
>>> If kernel/qemu-kvm update didn't help, then the device probably doesn't
>>> support the interrupt disable mechanism that allows shared interrupts.
>>> Therefore the device needs to be on an exclusive interrupt on the host.
>>> Look in /proc/interrupts and see what else is on the same interrupt
>>> line.  Your output below indicates the device is on IRQ 11.  Sometimes
>>> moving the device to a different physical slot will change the IRQ and
>>> give you an exclusive interrupt, otherwise you need to find the devices
>>> sharing that interrupt line and disable them by attaching the device to
>>> pci-stub.  Thanks,
>>>
>>> Alex
>>>
>>>> Am 18.07.2012 16:23, schrieb Alex Williamson:
>>>>> On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote:
>>>>>> soundcard logs added
>>>>>>
>>>>>> On 18.07.2012 14:08, Martin Wolf wrote:
>>>>>>> On 18.07.2012 11:26, Jan Kiszka wrote:
>>>>>>>> On 2012-07-18 07:45, Martin Wolf wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> i was able to passthrough an AMD 7870 videocard to my win7 guest
>>>>>>>>> machine.
>>>>>>>> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
>>>>>>> sure, i will prepare something
>>>>>>>>> my host is ubuntu 12.04 with stock kernel.
>>>>>>>>> my system contains:
>>>>>>>>> dq67sw q67 mainboard
>>>>>>>>> i5-2400s cpu
>>>>>>>>> sapphire 7870 amd videocard
>>>>>>>>> xonar d2x (problems to passthrough)
>>>>>>>>>
>>>>>>>>> for full functionality i just needed two options
>>>>>>>>>
>>>>>>>>> - kernel : iommu=on
>>>>>>>>> - kvm module: ignore_msrs=1
>>>>>>>>> (if i would not set it the guest os would crash with a bluescreen)
>>>>>>>> Can you report (=> kernel log) which MSRs are unknown to KVM?
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>>
>>>>>>> i hope thats the info you need, i booted it with ignore_msrs=1 since
>>>>>>> if i dont do that i get less output.
>>>>>>> (do you need it without the "option"?)
>>>>>>>
>>>>>>>>> the unigine benchmark ran flawlessly
>>>>>>>>> also the benchmark included in windows gave my videocard
>>>>>>>>> similar values (7.7) comparable with my native win7 (7.9)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> now to my questions...
>>>>>>>>> 1.     is it possible to reset the videocard properly to be able to
>>>>>>>>>         reboot the vm?
>>>>>>>> Which versions of kernel and qemu-kvm are involved via your distro? Can
>>>>>>>> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
>>>>>>>> something got fixed meanwhile.
>>>>>>>>
>>>>>>>> In general, there are many adapters that require special procedures to
>>>>>>>> perform resets. This one may fall into that category as well.
>>>>>>> i will do a test today.
>>>>>>>>> 2.    the xonar d2x is a very nice audio card, it would be very handy
>>>>>>>>>         to be able to use it in the vm. in my oppinion the card is a
>>>>>>>>>         d2 with a pci-e to pci bridge.
>>>>>>>>>         i tried to passthrough the card alone and with the pci-bridge
>>>>>>>>>         that was shown though lspci, but i had no success.
>>>>>>>>>         maybe you guys here have an idea on that topic?
>>>>>>>> Any further details about the error? Does the adapter work with a Linux
>>>>>>>> guest or provide more information that way?
>>>>>>>>
>>>>>>>> Jan
>>>>>> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
>>>>>> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
>>>>>>             Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>>>>>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>>>>>             Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
>>>>>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>>>>>             Latency: 0, Cache Line Size: 64 bytes
>>>>>>             Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
>>>>>>             I/O behind bridge: 0000d000-0000dfff
>>>>>>             Memory behind bridge: fff00000-000fffff
>>>>>>             Prefetchable memory behind bridge: fff00000-000fffff
>>>>>>             Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium
>>>>>>     >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>>>>>>             BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
>>>>>>                     PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>>>>>>             Capabilities: <access denied>
>>>>>>             Kernel driver in use: pci-stub
>>>>>>             Kernel modules: shpchp
>>>>>>
>>>>>> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788
>>>>>> [Oxygen HD Audio]
>>>>>>             Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
>>>>>>             Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-
>>>>>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>>>>>             Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium
>>>>>>     >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>>>>>             Interrupt: pin A routed to IRQ 11
>>>>>>             Region 0: I/O ports at d000 [disabled] [size=256]
>>>>>>             Capabilities: <access denied>
>>>>>>             Kernel driver in use: pci-stub
>>>>>>             Kernel modules: snd-virtuoso
>>>>>>
>>>>>>
>>>>>>      /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda
>>>>>> /var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device
>>>>>> pci-assign,host=03:04.0
>>>>>> Device assignment only supports endpoint assignment, device type 7
>>>>>> qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign'
>>>>>> could not be initialized
>>>>>>
>>>>>> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c
>>>>>> -hda /var/lib/libvirt/images/Win7.img -device
>>>>>> pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
>>>>>> Perhaps you are assigning a device that shares an IRQ with another device?
>>>>>> qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign'
>>>>>> could not be initialized
>>>>> Either your distro isn't shipping a version of kvm that supports
>>>>> interrupt sharing on PCI 2.3 complaint devices (likely) or your device
>>>>> doesn't support PCI 2.3 INTx disable (possible).  You either need to
>>>>> unbind any other devices that may be sharing IRQ 11 with this device
>>>>> from their driver or upgrade your kernel and qemu-kvm.  Thanks,
>>>>>
>>>>> Alex
>>>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>


-- 
Adiumentum GmbH
Gf. Martin Wolf
Banderbacherstraße 76
90513 Zirndorf

0911 / 9601470
mwolf@adiumentum.com