From mboxrd@z Thu Jan 1 00:00:00 1970 From: Martin Wolf Subject: Re: vga passthrough // questions about pci passthrough Date: Thu, 27 Sep 2012 20:43:40 +0200 Message-ID: <50649E5C.6020808@adiumentum.com> References: <50064D87.2090809@adiumentum.com> <50068134.4060306@siemens.com> <5006A72D.3080408@adiumentum.com> <5006ADF3.9030304@adiumentum.com> <1342621405.2229.194.camel@bling.home> <50081FBC.7060208@adiumentum.com> <1342714590.3142.4.camel@ul30vt> <5064364E.7050508@adiumentum.com> <1348768586.2320.204.camel@ul30vt.home> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: kvm@vger.kernel.org To: Alex Williamson Return-path: Received: from mail.andre-duewel.de ([178.63.55.43]:54429 "EHLO mail.andre-duewel.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752578Ab2I0Sns (ORCPT ); Thu, 27 Sep 2012 14:43:48 -0400 In-Reply-To: <1348768586.2320.204.camel@ul30vt.home> Sender: kvm-owner@vger.kernel.org List-ID: thank you for the information. i will try what you mentioned... do you have some additional information about rebooting a VM with a=20 passed through videocard? (amd / ati 7870) thanks in advance Am 27.09.2012 19:56, schrieb Alex Williamson: > On Thu, 2012-09-27 at 13:19 +0200, Martin Wolf wrote: >> i waited 2 months to get a more stable ubuntu 12.10 >> now that i dont get crashes every 5 minutes im going to test again. >> >> as you remember my two only problems are >> first the reboot issue of the gues os >> and second the xonar 2 d2x audio card. >> >> but first i would like to work on the reboot issue since >> i guess this is far more important. >> >> when i read the faq i found that one problem could be the "VBIOS ROM= access" >> i tried -device pci-assign,...,romfile=3D... but have not found any = output >> or change. >> is there some documentation of the romfile command to generate usabl= e >> output? > If you can read the rom from the host, this probably isn't adding > anything. You can test this by finding the rom in /sys (find /sys -n= ame > rom), match it up to the PCI device you're assigning. Then do: > > # echo 1 > rom > # xxd rom | head > #echo 0 > rom > > If you get something out that starts with 55aa then qemu will be able= to > read the rom directly. If you're replacing the rom with the romfile=3D > option, you can boot a linux guest, do the same as above to read the = rom > from the guest, save it to a file and compare md5sum of what the gues= t > sees to the file provided. If qemu is reading the rom on the device > correctly, it's no surprise that providing one doesn't make any > difference. > >> for testing i downloaded the bios of my videocard from techpowerup. >> they have a huge database and my card and biosrevision is listed. >> was that ok or do i have to retrieve the bios from inside the vm? > Thanks for mentioning that. From other reports I've seen, it looks l= ike > nvidia cards are the troublesome ones for being able to read the BIOS > from the host. Thanks, > > Alex > > >> thank you in advance >> >> Am 19.07.2012 18:16, schrieb Alex Williamson: >>> On Thu, 2012-07-19 at 16:54 +0200, Martin Wolf wrote: >>>> i tried now the alpha of ubuntu 12.10 that includes qemu-kvm 1.1 >>>> and a 3.5 kernel version. >>>> >>>> the "msr" problem is gone now, i was able to select "sandybridge" = as >>>> cpu topology, this removed the MSR error messages. >>>> thats the positive part... >>>> >>>> unfortunately i had no success with the other topics (rebootabilit= y and >>>> the soundcard) >>>> >>>> rebootability: >>>> the guest vm still crashes with a page fault bluescreen >>>> >>>> soundcard: >>>> it would be really nice if someone would be able to give me >>>> some advise how to debug this problem. >>> If kernel/qemu-kvm update didn't help, then the device probably doe= sn't >>> support the interrupt disable mechanism that allows shared interrup= ts. >>> Therefore the device needs to be on an exclusive interrupt on the h= ost. >>> Look in /proc/interrupts and see what else is on the same interrupt >>> line. Your output below indicates the device is on IRQ 11. Someti= mes >>> moving the device to a different physical slot will change the IRQ = and >>> give you an exclusive interrupt, otherwise you need to find the dev= ices >>> sharing that interrupt line and disable them by attaching the devic= e to >>> pci-stub. Thanks, >>> >>> Alex >>> >>>> Am 18.07.2012 16:23, schrieb Alex Williamson: >>>>> On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote: >>>>>> soundcard logs added >>>>>> >>>>>> On 18.07.2012 14:08, Martin Wolf wrote: >>>>>>> On 18.07.2012 11:26, Jan Kiszka wrote: >>>>>>>> On 2012-07-18 07:45, Martin Wolf wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> i was able to passthrough an AMD 7870 videocard to my win7 gu= est >>>>>>>>> machine. >>>>>>>> Would you add it to http://www.linux-kvm.org/page/VGA_device_a= ssignment? >>>>>>> sure, i will prepare something >>>>>>>>> my host is ubuntu 12.04 with stock kernel. >>>>>>>>> my system contains: >>>>>>>>> dq67sw q67 mainboard >>>>>>>>> i5-2400s cpu >>>>>>>>> sapphire 7870 amd videocard >>>>>>>>> xonar d2x (problems to passthrough) >>>>>>>>> >>>>>>>>> for full functionality i just needed two options >>>>>>>>> >>>>>>>>> - kernel : iommu=3Don >>>>>>>>> - kvm module: ignore_msrs=3D1 >>>>>>>>> (if i would not set it the guest os would crash with a bluesc= reen) >>>>>>>> Can you report (=3D> kernel log) which MSRs are unknown to KVM= ? >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.309931] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522724] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522733] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x1c9 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522736] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x60 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522752] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x1c9 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522755] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x60 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522821] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x1c9 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522823] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x60 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522834] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522840] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x1c9 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522842] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x60 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522865] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x1c9 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522867] kvm: 3347: cpu1 = ignored >>>>>>> rdmsr: 0x60 >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.522921] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.523005] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.523081] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.523175] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.523248] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.523333] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> Jul 18 14:03:33 kvm-xen kernel: [ 437.523430] kvm: 3347: cpu1 >>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop >>>>>>> >>>>>>> i hope thats the info you need, i booted it with ignore_msrs=3D= 1 since >>>>>>> if i dont do that i get less output. >>>>>>> (do you need it without the "option"?) >>>>>>> >>>>>>>>> the unigine benchmark ran flawlessly >>>>>>>>> also the benchmark included in windows gave my videocard >>>>>>>>> similar values (7.7) comparable with my native win7 (7.9) >>>>>>>>> >>>>>>>>> >>>>>>>>> now to my questions... >>>>>>>>> 1. is it possible to reset the videocard properly to be a= ble to >>>>>>>>> reboot the vm? >>>>>>>> Which versions of kernel and qemu-kvm are involved via your di= stro? Can >>>>>>>> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Mayb= e >>>>>>>> something got fixed meanwhile. >>>>>>>> >>>>>>>> In general, there are many adapters that require special proce= dures to >>>>>>>> perform resets. This one may fall into that category as well. >>>>>>> i will do a test today. >>>>>>>>> 2. the xonar d2x is a very nice audio card, it would be ve= ry handy >>>>>>>>> to be able to use it in the vm. in my oppinion the ca= rd is a >>>>>>>>> d2 with a pci-e to pci bridge. >>>>>>>>> i tried to passthrough the card alone and with the pc= i-bridge >>>>>>>>> that was shown though lspci, but i had no success. >>>>>>>>> maybe you guys here have an idea on that topic? >>>>>>>> Any further details about the error? Does the adapter work wit= h a Linux >>>>>>>> guest or provide more information that way? >>>>>>>> >>>>>>>> Jan >>>>>> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI >>>>>> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode]) >>>>>> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VG= ASnoop- >>>>>> ParErr- Stepping- SERR- FastB2B- DisINTx- >>>>>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Df= ast >TAbort- >>>>>> SERR- >>>>> Latency: 0, Cache Line Size: 64 bytes >>>>>> Bus: primary=3D02, secondary=3D03, subordinate=3D03,= sec-latency=3D32 >>>>>> I/O behind bridge: 0000d000-0000dfff >>>>>> Memory behind bridge: fff00000-000fffff >>>>>> Prefetchable memory behind bridge: fff00000-000fffff >>>>>> Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=3Dm= edium >>>>>> >TAbort- >>>>> BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset-= FastB2B- >>>>>> PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmr= SERREn- >>>>>> Capabilities: >>>>>> Kernel driver in use: pci-stub >>>>>> Kernel modules: shpchp >>>>>> >>>>>> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI= 8788 >>>>>> [Oxygen HD Audio] >>>>>> Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar= D2X) >>>>>> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VG= ASnoop- >>>>>> ParErr- Stepping- SERR- FastB2B- DisINTx- >>>>>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dm= edium >>>>>> >TAbort- SERR- >>>>> Interrupt: pin A routed to IRQ 11 >>>>>> Region 0: I/O ports at d000 [disabled] [size=3D256] >>>>>> Capabilities: >>>>>> Kernel driver in use: pci-stub >>>>>> Kernel modules: snd-virtuoso >>>>>> >>>>>> >>>>>> /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda >>>>>> /var/lib/libvirt/images/Win7.img -device pci-assign,host=3D02:00= =2E0 -device >>>>>> pci-assign,host=3D03:04.0 >>>>>> Device assignment only supports endpoint assignment, device type= 7 >>>>>> qemu-system-x86_64: -device pci-assign,host=3D02:00.0: Device 'p= ci-assign' >>>>>> could not be initialized >>>>>> >>>>>> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -b= oot c >>>>>> -hda /var/lib/libvirt/images/Win7.img -device >>>>>> pci-assign,host=3D03:04.0Failed to assign irq for "(null)": Inpu= t/output error >>>>>> Perhaps you are assigning a device that shares an IRQ with anoth= er device? >>>>>> qemu-system-x86_64: -device pci-assign,host=3D03:04.0: Device 'p= ci-assign' >>>>>> could not be initialized >>>>> Either your distro isn't shipping a version of kvm that supports >>>>> interrupt sharing on PCI 2.3 complaint devices (likely) or your d= evice >>>>> doesn't support PCI 2.3 INTx disable (possible). You either need= to >>>>> unbind any other devices that may be sharing IRQ 11 with this dev= ice >>>>> from their driver or upgrade your kernel and qemu-kvm. Thanks, >>>>> >>>>> Alex >>>>> >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe kvm" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > --=20 Adiumentum GmbH Gf. Martin Wolf Banderbacherstra=C3=9Fe 76 90513 Zirndorf 0911 / 9601470 mwolf@adiumentum.com