vga passthrough // questions about pci passthrough

vga passthrough // questions about pci passthrough

[Martin Wolf]

Hello,

i was able to passthrough an AMD 7870 videocard to my win7 guest machine.
my host is ubuntu 12.04 with stock kernel.
my system contains:
dq67sw q67 mainboard
i5-2400s cpu
sapphire 7870 amd videocard
xonar d2x (problems to passthrough)

for full functionality i just needed two options

- kernel : iommu=on
- kvm module: ignore_msrs=1
(if i would not set it the guest os would crash with a bluescreen)

the unigine benchmark ran flawlessly
also the benchmark included in windows gave my videocard
similar values (7.7) comparable with my native win7 (7.9)


now to my questions...
1. 	is it possible to reset the videocard properly to be able to
	reboot the vm?

2.	the xonar d2x is a very nice audio card, it would be very handy
	to be able to use it in the vm. in my oppinion the card is a
	d2 with a pci-e to pci bridge.
	i tried to passthrough the card alone and with the pci-bridge
	that was shown though lspci, but i had no success.
	maybe you guys here have an idea on that topic?

thank you for you great work ;)

Re: vga passthrough // questions about pci passthrough

[Jan Kiszka]

On 2012-07-18 07:45, Martin Wolf wrote:
> Hello,
> 
> i was able to passthrough an AMD 7870 videocard to my win7 guest machine.

Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?

> my host is ubuntu 12.04 with stock kernel.
> my system contains:
> dq67sw q67 mainboard
> i5-2400s cpu
> sapphire 7870 amd videocard
> xonar d2x (problems to passthrough)
> 
> for full functionality i just needed two options
> 
> - kernel : iommu=on
> - kvm module: ignore_msrs=1
> (if i would not set it the guest os would crash with a bluescreen)

Can you report (=> kernel log) which MSRs are unknown to KVM?

> 
> the unigine benchmark ran flawlessly
> also the benchmark included in windows gave my videocard
> similar values (7.7) comparable with my native win7 (7.9)
> 
> 
> now to my questions...
> 1.     is it possible to reset the videocard properly to be able to
>     reboot the vm?

Which versions of kernel and qemu-kvm are involved via your distro? Can
you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
something got fixed meanwhile.

In general, there are many adapters that require special procedures to
perform resets. This one may fall into that category as well.

> 
> 2.    the xonar d2x is a very nice audio card, it would be very handy
>     to be able to use it in the vm. in my oppinion the card is a
>     d2 with a pci-e to pci bridge.
>     i tried to passthrough the card alone and with the pci-bridge
>     that was shown though lspci, but i had no success.
>     maybe you guys here have an idea on that topic?

Any further details about the error? Does the adapter work with a Linux
guest or provide more information that way?

Jan


-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

On 18.07.2012 11:26, Jan Kiszka wrote:
> On 2012-07-18 07:45, Martin Wolf wrote:
>> Hello,
>>
>> i was able to passthrough an AMD 7870 videocard to my win7 guest machine.
> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
sure, i will prepare something
>
>> my host is ubuntu 12.04 with stock kernel.
>> my system contains:
>> dq67sw q67 mainboard
>> i5-2400s cpu
>> sapphire 7870 amd videocard
>> xonar d2x (problems to passthrough)
>>
>> for full functionality i just needed two options
>>
>> - kernel : iommu=on
>> - kvm module: ignore_msrs=1
>> (if i would not set it the guest os would crash with a bluescreen)
> Can you report (=> kernel log) which MSRs are unknown to KVM?
Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored 
rdmsr: 0x1c9
Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored 
rdmsr: 0x60
Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored 
rdmsr: 0x1c9
Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored 
rdmsr: 0x60
Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored 
rdmsr: 0x1c9
Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored 
rdmsr: 0x60
Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored 
rdmsr: 0x1c9
Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored 
rdmsr: 0x60
Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored 
rdmsr: 0x1c9
Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored 
rdmsr: 0x60
Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1 
kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop

i hope thats the info you need, i booted it with ignore_msrs=1 since if 
i dont do that i get less output.
(do you need it without the "option"?)

>
>> the unigine benchmark ran flawlessly
>> also the benchmark included in windows gave my videocard
>> similar values (7.7) comparable with my native win7 (7.9)
>>
>>
>> now to my questions...
>> 1.     is it possible to reset the videocard properly to be able to
>>      reboot the vm?
> Which versions of kernel and qemu-kvm are involved via your distro? Can
> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
> something got fixed meanwhile.
>
> In general, there are many adapters that require special procedures to
> perform resets. This one may fall into that category as well.
i will do a test today.
>> 2.    the xonar d2x is a very nice audio card, it would be very handy
>>      to be able to use it in the vm. in my oppinion the card is a
>>      d2 with a pci-e to pci bridge.
>>      i tried to passthrough the card alone and with the pci-bridge
>>      that was shown though lspci, but i had no success.
>>      maybe you guys here have an idea on that topic?
> Any further details about the error? Does the adapter work with a Linux
> guest or provide more information that way?
>
> Jan
i will also add info here later
>

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

soundcard logs added

On 18.07.2012 14:08, Martin Wolf wrote:
> On 18.07.2012 11:26, Jan Kiszka wrote:
>> On 2012-07-18 07:45, Martin Wolf wrote:
>>> Hello,
>>>
>>> i was able to passthrough an AMD 7870 videocard to my win7 guest 
>>> machine.
>> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
> sure, i will prepare something
>>
>>> my host is ubuntu 12.04 with stock kernel.
>>> my system contains:
>>> dq67sw q67 mainboard
>>> i5-2400s cpu
>>> sapphire 7870 amd videocard
>>> xonar d2x (problems to passthrough)
>>>
>>> for full functionality i just needed two options
>>>
>>> - kernel : iommu=on
>>> - kvm module: ignore_msrs=1
>>> (if i would not set it the guest os would crash with a bluescreen)
>> Can you report (=> kernel log) which MSRs are unknown to KVM?
> Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored 
> rdmsr: 0x1c9
> Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored 
> rdmsr: 0x60
> Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored 
> rdmsr: 0x1c9
> Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored 
> rdmsr: 0x60
> Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored 
> rdmsr: 0x1c9
> Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored 
> rdmsr: 0x60
> Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored 
> rdmsr: 0x1c9
> Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored 
> rdmsr: 0x60
> Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored 
> rdmsr: 0x1c9
> Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored 
> rdmsr: 0x60
> Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1 
> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>
> i hope thats the info you need, i booted it with ignore_msrs=1 since 
> if i dont do that i get less output.
> (do you need it without the "option"?)
>
>>
>>> the unigine benchmark ran flawlessly
>>> also the benchmark included in windows gave my videocard
>>> similar values (7.7) comparable with my native win7 (7.9)
>>>
>>>
>>> now to my questions...
>>> 1.     is it possible to reset the videocard properly to be able to
>>>      reboot the vm?
>> Which versions of kernel and qemu-kvm are involved via your distro? Can
>> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
>> something got fixed meanwhile.
>>
>> In general, there are many adapters that require special procedures to
>> perform resets. This one may fall into that category as well.
> i will do a test today.
>>> 2.    the xonar d2x is a very nice audio card, it would be very handy
>>>      to be able to use it in the vm. in my oppinion the card is a
>>>      d2 with a pci-e to pci bridge.
>>>      i tried to passthrough the card alone and with the pci-bridge
>>>      that was shown though lspci, but i had no success.
>>>      maybe you guys here have an idea on that topic?
>> Any further details about the error? Does the adapter work with a Linux
>> guest or provide more information that way?
>>
>> Jan

02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI 
Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- 
ParErr- Stepping- SERR- FastB2B- DisINTx-
         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- 
<TAbort- <MAbort- >SERR- <PERR- INTx-
         Latency: 0, Cache Line Size: 64 bytes
         Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
         I/O behind bridge: 0000d000-0000dfff
         Memory behind bridge: fff00000-000fffff
         Prefetchable memory behind bridge: fff00000-000fffff
         Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium 
 >TAbort- <TAbort- <MAbort- <SERR- <PERR-
         BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
         Capabilities: <access denied>
         Kernel driver in use: pci-stub
         Kernel modules: shpchp

03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 
[Oxygen HD Audio]
         Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- 
ParErr- Stepping- SERR- FastB2B- DisINTx-
         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium 
 >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
         Interrupt: pin A routed to IRQ 11
         Region 0: I/O ports at d000 [disabled] [size=256]
         Capabilities: <access denied>
         Kernel driver in use: pci-stub
         Kernel modules: snd-virtuoso


  /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda 
/var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device 
pci-assign,host=03:04.0
Device assignment only supports endpoint assignment, device type 7
qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign' 
could not be initialized

root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c 
-hda /var/lib/libvirt/images/Win7.img -device 
pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
Perhaps you are assigning a device that shares an IRQ with another device?
qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign' 
could not be initialized

these kernel log lines showed up after the the second qemu-system line
Jul 18 14:32:08 kvm-xen kernel: [ 2149.180353] assign device 0:3:4.0
Jul 18 14:32:08 kvm-xen kernel: [ 2149.180575] deassign device 0:3:4.0
Jul 18 14:32:08 kvm-xen kernel: [ 2149.209408] pci-stub 0000:03:04.0: 
restoring config space at offset 0x1 (was 0x2100000, writing 0x2100001)
Jul 18 14:32:08 kvm-xen kernel: [ 2149.209551] pci-stub 0000:03:04.0: 
PCI INT A disabled



>>
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: vga passthrough // questions about pci passthrough

[Alex Williamson]

On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote:
> soundcard logs added
> 
> On 18.07.2012 14:08, Martin Wolf wrote:
> > On 18.07.2012 11:26, Jan Kiszka wrote:
> >> On 2012-07-18 07:45, Martin Wolf wrote:
> >>> Hello,
> >>>
> >>> i was able to passthrough an AMD 7870 videocard to my win7 guest 
> >>> machine.
> >> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
> > sure, i will prepare something
> >>
> >>> my host is ubuntu 12.04 with stock kernel.
> >>> my system contains:
> >>> dq67sw q67 mainboard
> >>> i5-2400s cpu
> >>> sapphire 7870 amd videocard
> >>> xonar d2x (problems to passthrough)
> >>>
> >>> for full functionality i just needed two options
> >>>
> >>> - kernel : iommu=on
> >>> - kvm module: ignore_msrs=1
> >>> (if i would not set it the guest os would crash with a bluescreen)
> >> Can you report (=> kernel log) which MSRs are unknown to KVM?
> > Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x1c9
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored 
> > rdmsr: 0x60
> > Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> > Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1 
> > kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >
> > i hope thats the info you need, i booted it with ignore_msrs=1 since 
> > if i dont do that i get less output.
> > (do you need it without the "option"?)
> >
> >>
> >>> the unigine benchmark ran flawlessly
> >>> also the benchmark included in windows gave my videocard
> >>> similar values (7.7) comparable with my native win7 (7.9)
> >>>
> >>>
> >>> now to my questions...
> >>> 1.     is it possible to reset the videocard properly to be able to
> >>>      reboot the vm?
> >> Which versions of kernel and qemu-kvm are involved via your distro? Can
> >> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
> >> something got fixed meanwhile.
> >>
> >> In general, there are many adapters that require special procedures to
> >> perform resets. This one may fall into that category as well.
> > i will do a test today.
> >>> 2.    the xonar d2x is a very nice audio card, it would be very handy
> >>>      to be able to use it in the vm. in my oppinion the card is a
> >>>      d2 with a pci-e to pci bridge.
> >>>      i tried to passthrough the card alone and with the pci-bridge
> >>>      that was shown though lspci, but i had no success.
> >>>      maybe you guys here have an idea on that topic?
> >> Any further details about the error? Does the adapter work with a Linux
> >> guest or provide more information that way?
> >>
> >> Jan
> 
> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI 
> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
>          Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- 
> ParErr- Stepping- SERR- FastB2B- DisINTx-
>          Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- 
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>          Latency: 0, Cache Line Size: 64 bytes
>          Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
>          I/O behind bridge: 0000d000-0000dfff
>          Memory behind bridge: fff00000-000fffff
>          Prefetchable memory behind bridge: fff00000-000fffff
>          Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium 
>  >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>          BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
>                  PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>          Capabilities: <access denied>
>          Kernel driver in use: pci-stub
>          Kernel modules: shpchp
> 
> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 
> [Oxygen HD Audio]
>          Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
>          Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- 
> ParErr- Stepping- SERR- FastB2B- DisINTx-
>          Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium 
>  >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>          Interrupt: pin A routed to IRQ 11
>          Region 0: I/O ports at d000 [disabled] [size=256]
>          Capabilities: <access denied>
>          Kernel driver in use: pci-stub
>          Kernel modules: snd-virtuoso
> 
> 
>   /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda 
> /var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device 
> pci-assign,host=03:04.0
> Device assignment only supports endpoint assignment, device type 7
> qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign' 
> could not be initialized
> 
> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c 
> -hda /var/lib/libvirt/images/Win7.img -device 
> pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
> Perhaps you are assigning a device that shares an IRQ with another device?
> qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign' 
> could not be initialized

Either your distro isn't shipping a version of kvm that supports
interrupt sharing on PCI 2.3 complaint devices (likely) or your device
doesn't support PCI 2.3 INTx disable (possible).  You either need to
unbind any other devices that may be sharing IRQ 11 with this device
from their driver or upgrade your kernel and qemu-kvm.  Thanks,

Alex

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

i tried now the alpha of ubuntu 12.10 that includes qemu-kvm 1.1
and a 3.5 kernel version.

the "msr" problem is gone now, i was able to select "sandybridge" as
cpu topology, this removed the MSR error messages.
thats the positive part...

unfortunately i had no success with the other topics (rebootability and
the soundcard)

rebootability:
the guest vm still crashes with a page fault bluescreen

soundcard:
it would be really nice if someone would be able to give me
some advise how to debug this problem.

thank you for your help


Am 18.07.2012 16:23, schrieb Alex Williamson:
> On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote:
>> soundcard logs added
>>
>> On 18.07.2012 14:08, Martin Wolf wrote:
>>> On 18.07.2012 11:26, Jan Kiszka wrote:
>>>> On 2012-07-18 07:45, Martin Wolf wrote:
>>>>> Hello,
>>>>>
>>>>> i was able to passthrough an AMD 7870 videocard to my win7 guest
>>>>> machine.
>>>> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
>>> sure, i will prepare something
>>>>> my host is ubuntu 12.04 with stock kernel.
>>>>> my system contains:
>>>>> dq67sw q67 mainboard
>>>>> i5-2400s cpu
>>>>> sapphire 7870 amd videocard
>>>>> xonar d2x (problems to passthrough)
>>>>>
>>>>> for full functionality i just needed two options
>>>>>
>>>>> - kernel : iommu=on
>>>>> - kvm module: ignore_msrs=1
>>>>> (if i would not set it the guest os would crash with a bluescreen)
>>>> Can you report (=> kernel log) which MSRs are unknown to KVM?
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x1c9
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x60
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x1c9
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x60
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x1c9
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x60
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x1c9
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x60
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x1c9
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored
>>> rdmsr: 0x60
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1
>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>
>>> i hope thats the info you need, i booted it with ignore_msrs=1 since
>>> if i dont do that i get less output.
>>> (do you need it without the "option"?)
>>>
>>>>> the unigine benchmark ran flawlessly
>>>>> also the benchmark included in windows gave my videocard
>>>>> similar values (7.7) comparable with my native win7 (7.9)
>>>>>
>>>>>
>>>>> now to my questions...
>>>>> 1.     is it possible to reset the videocard properly to be able to
>>>>>       reboot the vm?
>>>> Which versions of kernel and qemu-kvm are involved via your distro? Can
>>>> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
>>>> something got fixed meanwhile.
>>>>
>>>> In general, there are many adapters that require special procedures to
>>>> perform resets. This one may fall into that category as well.
>>> i will do a test today.
>>>>> 2.    the xonar d2x is a very nice audio card, it would be very handy
>>>>>       to be able to use it in the vm. in my oppinion the card is a
>>>>>       d2 with a pci-e to pci bridge.
>>>>>       i tried to passthrough the card alone and with the pci-bridge
>>>>>       that was shown though lspci, but i had no success.
>>>>>       maybe you guys here have an idea on that topic?
>>>> Any further details about the error? Does the adapter work with a Linux
>>>> guest or provide more information that way?
>>>>
>>>> Jan
>> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
>> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
>>           Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>           Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>           Latency: 0, Cache Line Size: 64 bytes
>>           Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
>>           I/O behind bridge: 0000d000-0000dfff
>>           Memory behind bridge: fff00000-000fffff
>>           Prefetchable memory behind bridge: fff00000-000fffff
>>           Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium
>>   >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>>           BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
>>                   PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>>           Capabilities: <access denied>
>>           Kernel driver in use: pci-stub
>>           Kernel modules: shpchp
>>
>> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788
>> [Oxygen HD Audio]
>>           Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
>>           Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-
>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>           Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium
>>   >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>           Interrupt: pin A routed to IRQ 11
>>           Region 0: I/O ports at d000 [disabled] [size=256]
>>           Capabilities: <access denied>
>>           Kernel driver in use: pci-stub
>>           Kernel modules: snd-virtuoso
>>
>>
>>    /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda
>> /var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device
>> pci-assign,host=03:04.0
>> Device assignment only supports endpoint assignment, device type 7
>> qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign'
>> could not be initialized
>>
>> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c
>> -hda /var/lib/libvirt/images/Win7.img -device
>> pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
>> Perhaps you are assigning a device that shares an IRQ with another device?
>> qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign'
>> could not be initialized
> Either your distro isn't shipping a version of kvm that supports
> interrupt sharing on PCI 2.3 complaint devices (likely) or your device
> doesn't support PCI 2.3 INTx disable (possible).  You either need to
> unbind any other devices that may be sharing IRQ 11 with this device
> from their driver or upgrade your kernel and qemu-kvm.  Thanks,
>
> Alex
>


-- 
Adiumentum GmbH
Gf. Martin Wolf
Banderbacherstraße 76
90513 Zirndorf

0911 / 9601470
mwolf@adiumentum.com

Re: vga passthrough // questions about pci passthrough

[Jan Kiszka]

On 2012-07-19 16:54, Martin Wolf wrote:
> i tried now the alpha of ubuntu 12.10 that includes qemu-kvm 1.1
> and a 3.5 kernel version.
> 
> the "msr" problem is gone now, i was able to select "sandybridge" as
> cpu topology, this removed the MSR error messages.
> thats the positive part...
> 
> unfortunately i had no success with the other topics (rebootability and
> the soundcard)
> 
> rebootability:
> the guest vm still crashes with a page fault bluescreen

So this card likely needs special steps to trigger a reset. Then only
the vendor can help.

> 
> soundcard:
> it would be really nice if someone would be able to give me
> some advise how to debug this problem.

Check /proc/interrupts for other users of IRQ 11. Disable them as a
first try.

What would also be interesting is the output of qemu-kvm 1.1 regarding
why the card still doesn't work. Did it change?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: vga passthrough // questions about pci passthrough

[Alex Williamson]

On Thu, 2012-07-19 at 16:54 +0200, Martin Wolf wrote:
> i tried now the alpha of ubuntu 12.10 that includes qemu-kvm 1.1
> and a 3.5 kernel version.
> 
> the "msr" problem is gone now, i was able to select "sandybridge" as
> cpu topology, this removed the MSR error messages.
> thats the positive part...
> 
> unfortunately i had no success with the other topics (rebootability and
> the soundcard)
> 
> rebootability:
> the guest vm still crashes with a page fault bluescreen
> 
> soundcard:
> it would be really nice if someone would be able to give me
> some advise how to debug this problem.

If kernel/qemu-kvm update didn't help, then the device probably doesn't
support the interrupt disable mechanism that allows shared interrupts.
Therefore the device needs to be on an exclusive interrupt on the host.
Look in /proc/interrupts and see what else is on the same interrupt
line.  Your output below indicates the device is on IRQ 11.  Sometimes
moving the device to a different physical slot will change the IRQ and
give you an exclusive interrupt, otherwise you need to find the devices
sharing that interrupt line and disable them by attaching the device to
pci-stub.  Thanks,

Alex

> Am 18.07.2012 16:23, schrieb Alex Williamson:
> > On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote:
> >> soundcard logs added
> >>
> >> On 18.07.2012 14:08, Martin Wolf wrote:
> >>> On 18.07.2012 11:26, Jan Kiszka wrote:
> >>>> On 2012-07-18 07:45, Martin Wolf wrote:
> >>>>> Hello,
> >>>>>
> >>>>> i was able to passthrough an AMD 7870 videocard to my win7 guest
> >>>>> machine.
> >>>> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
> >>> sure, i will prepare something
> >>>>> my host is ubuntu 12.04 with stock kernel.
> >>>>> my system contains:
> >>>>> dq67sw q67 mainboard
> >>>>> i5-2400s cpu
> >>>>> sapphire 7870 amd videocard
> >>>>> xonar d2x (problems to passthrough)
> >>>>>
> >>>>> for full functionality i just needed two options
> >>>>>
> >>>>> - kernel : iommu=on
> >>>>> - kvm module: ignore_msrs=1
> >>>>> (if i would not set it the guest os would crash with a bluescreen)
> >>>> Can you report (=> kernel log) which MSRs are unknown to KVM?
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x1c9
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x60
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x1c9
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x60
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x1c9
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x60
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x1c9
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x60
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x1c9
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored
> >>> rdmsr: 0x60
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>> Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1
> >>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>
> >>> i hope thats the info you need, i booted it with ignore_msrs=1 since
> >>> if i dont do that i get less output.
> >>> (do you need it without the "option"?)
> >>>
> >>>>> the unigine benchmark ran flawlessly
> >>>>> also the benchmark included in windows gave my videocard
> >>>>> similar values (7.7) comparable with my native win7 (7.9)
> >>>>>
> >>>>>
> >>>>> now to my questions...
> >>>>> 1.     is it possible to reset the videocard properly to be able to
> >>>>>       reboot the vm?
> >>>> Which versions of kernel and qemu-kvm are involved via your distro? Can
> >>>> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
> >>>> something got fixed meanwhile.
> >>>>
> >>>> In general, there are many adapters that require special procedures to
> >>>> perform resets. This one may fall into that category as well.
> >>> i will do a test today.
> >>>>> 2.    the xonar d2x is a very nice audio card, it would be very handy
> >>>>>       to be able to use it in the vm. in my oppinion the card is a
> >>>>>       d2 with a pci-e to pci bridge.
> >>>>>       i tried to passthrough the card alone and with the pci-bridge
> >>>>>       that was shown though lspci, but i had no success.
> >>>>>       maybe you guys here have an idea on that topic?
> >>>> Any further details about the error? Does the adapter work with a Linux
> >>>> guest or provide more information that way?
> >>>>
> >>>> Jan
> >> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
> >> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
> >>           Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> >> ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>           Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> >> <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>           Latency: 0, Cache Line Size: 64 bytes
> >>           Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
> >>           I/O behind bridge: 0000d000-0000dfff
> >>           Memory behind bridge: fff00000-000fffff
> >>           Prefetchable memory behind bridge: fff00000-000fffff
> >>           Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium
> >>   >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> >>           BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
> >>                   PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> >>           Capabilities: <access denied>
> >>           Kernel driver in use: pci-stub
> >>           Kernel modules: shpchp
> >>
> >> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788
> >> [Oxygen HD Audio]
> >>           Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
> >>           Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-
> >> ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>           Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium
> >>   >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>           Interrupt: pin A routed to IRQ 11
> >>           Region 0: I/O ports at d000 [disabled] [size=256]
> >>           Capabilities: <access denied>
> >>           Kernel driver in use: pci-stub
> >>           Kernel modules: snd-virtuoso
> >>
> >>
> >>    /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda
> >> /var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device
> >> pci-assign,host=03:04.0
> >> Device assignment only supports endpoint assignment, device type 7
> >> qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign'
> >> could not be initialized
> >>
> >> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c
> >> -hda /var/lib/libvirt/images/Win7.img -device
> >> pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
> >> Perhaps you are assigning a device that shares an IRQ with another device?
> >> qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign'
> >> could not be initialized
> > Either your distro isn't shipping a version of kvm that supports
> > interrupt sharing on PCI 2.3 complaint devices (likely) or your device
> > doesn't support PCI 2.3 INTx disable (possible).  You either need to
> > unbind any other devices that may be sharing IRQ 11 with this device
> > from their driver or upgrade your kernel and qemu-kvm.  Thanks,
> >
> > Alex
> >
> 
> 

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

i waited 2 months to get a more stable ubuntu 12.10
now that i dont get crashes every 5 minutes im going to test again.

as you remember my two only problems are
first the reboot issue of the gues os
and second the xonar 2 d2x audio card.

but first i would like to work on the reboot issue since
i guess this is far more important.

when i read the faq i found that one problem could be the "VBIOS ROM access"
i tried -device pci-assign,...,romfile=... but have not found any output 
or change.
is there some documentation of the romfile command to generate usable
output? for testing i downloaded the bios of my videocard from techpowerup.
they have a huge database and my card and biosrevision is listed.
was that ok or do i have to retrieve the bios from inside the vm?

thank you in advance

Am 19.07.2012 18:16, schrieb Alex Williamson:
> On Thu, 2012-07-19 at 16:54 +0200, Martin Wolf wrote:
>> i tried now the alpha of ubuntu 12.10 that includes qemu-kvm 1.1
>> and a 3.5 kernel version.
>>
>> the "msr" problem is gone now, i was able to select "sandybridge" as
>> cpu topology, this removed the MSR error messages.
>> thats the positive part...
>>
>> unfortunately i had no success with the other topics (rebootability and
>> the soundcard)
>>
>> rebootability:
>> the guest vm still crashes with a page fault bluescreen
>>
>> soundcard:
>> it would be really nice if someone would be able to give me
>> some advise how to debug this problem.
> If kernel/qemu-kvm update didn't help, then the device probably doesn't
> support the interrupt disable mechanism that allows shared interrupts.
> Therefore the device needs to be on an exclusive interrupt on the host.
> Look in /proc/interrupts and see what else is on the same interrupt
> line.  Your output below indicates the device is on IRQ 11.  Sometimes
> moving the device to a different physical slot will change the IRQ and
> give you an exclusive interrupt, otherwise you need to find the devices
> sharing that interrupt line and disable them by attaching the device to
> pci-stub.  Thanks,
>
> Alex
>
>> Am 18.07.2012 16:23, schrieb Alex Williamson:
>>> On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote:
>>>> soundcard logs added
>>>>
>>>> On 18.07.2012 14:08, Martin Wolf wrote:
>>>>> On 18.07.2012 11:26, Jan Kiszka wrote:
>>>>>> On 2012-07-18 07:45, Martin Wolf wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> i was able to passthrough an AMD 7870 videocard to my win7 guest
>>>>>>> machine.
>>>>>> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
>>>>> sure, i will prepare something
>>>>>>> my host is ubuntu 12.04 with stock kernel.
>>>>>>> my system contains:
>>>>>>> dq67sw q67 mainboard
>>>>>>> i5-2400s cpu
>>>>>>> sapphire 7870 amd videocard
>>>>>>> xonar d2x (problems to passthrough)
>>>>>>>
>>>>>>> for full functionality i just needed two options
>>>>>>>
>>>>>>> - kernel : iommu=on
>>>>>>> - kvm module: ignore_msrs=1
>>>>>>> (if i would not set it the guest os would crash with a bluescreen)
>>>>>> Can you report (=> kernel log) which MSRs are unknown to KVM?
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x1c9
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x60
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x1c9
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x60
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x1c9
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x60
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x1c9
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x60
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x1c9
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored
>>>>> rdmsr: 0x60
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1
>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>
>>>>> i hope thats the info you need, i booted it with ignore_msrs=1 since
>>>>> if i dont do that i get less output.
>>>>> (do you need it without the "option"?)
>>>>>
>>>>>>> the unigine benchmark ran flawlessly
>>>>>>> also the benchmark included in windows gave my videocard
>>>>>>> similar values (7.7) comparable with my native win7 (7.9)
>>>>>>>
>>>>>>>
>>>>>>> now to my questions...
>>>>>>> 1.     is it possible to reset the videocard properly to be able to
>>>>>>>        reboot the vm?
>>>>>> Which versions of kernel and qemu-kvm are involved via your distro? Can
>>>>>> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
>>>>>> something got fixed meanwhile.
>>>>>>
>>>>>> In general, there are many adapters that require special procedures to
>>>>>> perform resets. This one may fall into that category as well.
>>>>> i will do a test today.
>>>>>>> 2.    the xonar d2x is a very nice audio card, it would be very handy
>>>>>>>        to be able to use it in the vm. in my oppinion the card is a
>>>>>>>        d2 with a pci-e to pci bridge.
>>>>>>>        i tried to passthrough the card alone and with the pci-bridge
>>>>>>>        that was shown though lspci, but i had no success.
>>>>>>>        maybe you guys here have an idea on that topic?
>>>>>> Any further details about the error? Does the adapter work with a Linux
>>>>>> guest or provide more information that way?
>>>>>>
>>>>>> Jan
>>>> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
>>>> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
>>>>            Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>>>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>>>            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
>>>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>>>            Latency: 0, Cache Line Size: 64 bytes
>>>>            Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
>>>>            I/O behind bridge: 0000d000-0000dfff
>>>>            Memory behind bridge: fff00000-000fffff
>>>>            Prefetchable memory behind bridge: fff00000-000fffff
>>>>            Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium
>>>>    >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>>>>            BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
>>>>                    PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>>>>            Capabilities: <access denied>
>>>>            Kernel driver in use: pci-stub
>>>>            Kernel modules: shpchp
>>>>
>>>> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788
>>>> [Oxygen HD Audio]
>>>>            Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
>>>>            Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-
>>>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>>>            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium
>>>>    >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>>>            Interrupt: pin A routed to IRQ 11
>>>>            Region 0: I/O ports at d000 [disabled] [size=256]
>>>>            Capabilities: <access denied>
>>>>            Kernel driver in use: pci-stub
>>>>            Kernel modules: snd-virtuoso
>>>>
>>>>
>>>>     /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda
>>>> /var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device
>>>> pci-assign,host=03:04.0
>>>> Device assignment only supports endpoint assignment, device type 7
>>>> qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign'
>>>> could not be initialized
>>>>
>>>> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c
>>>> -hda /var/lib/libvirt/images/Win7.img -device
>>>> pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
>>>> Perhaps you are assigning a device that shares an IRQ with another device?
>>>> qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign'
>>>> could not be initialized
>>> Either your distro isn't shipping a version of kvm that supports
>>> interrupt sharing on PCI 2.3 complaint devices (likely) or your device
>>> doesn't support PCI 2.3 INTx disable (possible).  You either need to
>>> unbind any other devices that may be sharing IRQ 11 with this device
>>> from their driver or upgrade your kernel and qemu-kvm.  Thanks,
>>>
>>> Alex
>>>
>>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Adiumentum GmbH
Gf. Martin Wolf
Banderbacherstraße 76
90513 Zirndorf

0911 / 9601470
mwolf@adiumentum.com

Re: vga passthrough // questions about pci passthrough

[Alex Williamson]

On Thu, 2012-09-27 at 13:19 +0200, Martin Wolf wrote:
> i waited 2 months to get a more stable ubuntu 12.10
> now that i dont get crashes every 5 minutes im going to test again.
> 
> as you remember my two only problems are
> first the reboot issue of the gues os
> and second the xonar 2 d2x audio card.
> 
> but first i would like to work on the reboot issue since
> i guess this is far more important.
> 
> when i read the faq i found that one problem could be the "VBIOS ROM access"
> i tried -device pci-assign,...,romfile=... but have not found any output 
> or change.
> is there some documentation of the romfile command to generate usable
> output?

If you can read the rom from the host, this probably isn't adding
anything.  You can test this by finding the rom in /sys (find /sys -name
rom), match it up to the PCI device you're assigning.  Then do:

# echo 1 > rom
# xxd rom | head
#echo 0 > rom

If you get something out that starts with 55aa then qemu will be able to
read the rom directly.  If you're replacing the rom with the romfile=
option, you can boot a linux guest, do the same as above to read the rom
from the guest, save it to a file and compare md5sum of what the guest
sees to the file provided.  If qemu is reading the rom on the device
correctly, it's no surprise that providing one doesn't make any
difference.

> for testing i downloaded the bios of my videocard from techpowerup.
> they have a huge database and my card and biosrevision is listed.
> was that ok or do i have to retrieve the bios from inside the vm?

Thanks for mentioning that.  From other reports I've seen, it looks like
nvidia cards are the troublesome ones for being able to read the BIOS
from the host.  Thanks,

Alex


> thank you in advance
> 
> Am 19.07.2012 18:16, schrieb Alex Williamson:
> > On Thu, 2012-07-19 at 16:54 +0200, Martin Wolf wrote:
> >> i tried now the alpha of ubuntu 12.10 that includes qemu-kvm 1.1
> >> and a 3.5 kernel version.
> >>
> >> the "msr" problem is gone now, i was able to select "sandybridge" as
> >> cpu topology, this removed the MSR error messages.
> >> thats the positive part...
> >>
> >> unfortunately i had no success with the other topics (rebootability and
> >> the soundcard)
> >>
> >> rebootability:
> >> the guest vm still crashes with a page fault bluescreen
> >>
> >> soundcard:
> >> it would be really nice if someone would be able to give me
> >> some advise how to debug this problem.
> > If kernel/qemu-kvm update didn't help, then the device probably doesn't
> > support the interrupt disable mechanism that allows shared interrupts.
> > Therefore the device needs to be on an exclusive interrupt on the host.
> > Look in /proc/interrupts and see what else is on the same interrupt
> > line.  Your output below indicates the device is on IRQ 11.  Sometimes
> > moving the device to a different physical slot will change the IRQ and
> > give you an exclusive interrupt, otherwise you need to find the devices
> > sharing that interrupt line and disable them by attaching the device to
> > pci-stub.  Thanks,
> >
> > Alex
> >
> >> Am 18.07.2012 16:23, schrieb Alex Williamson:
> >>> On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote:
> >>>> soundcard logs added
> >>>>
> >>>> On 18.07.2012 14:08, Martin Wolf wrote:
> >>>>> On 18.07.2012 11:26, Jan Kiszka wrote:
> >>>>>> On 2012-07-18 07:45, Martin Wolf wrote:
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> i was able to passthrough an AMD 7870 videocard to my win7 guest
> >>>>>>> machine.
> >>>>>> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
> >>>>> sure, i will prepare something
> >>>>>>> my host is ubuntu 12.04 with stock kernel.
> >>>>>>> my system contains:
> >>>>>>> dq67sw q67 mainboard
> >>>>>>> i5-2400s cpu
> >>>>>>> sapphire 7870 amd videocard
> >>>>>>> xonar d2x (problems to passthrough)
> >>>>>>>
> >>>>>>> for full functionality i just needed two options
> >>>>>>>
> >>>>>>> - kernel : iommu=on
> >>>>>>> - kvm module: ignore_msrs=1
> >>>>>>> (if i would not set it the guest os would crash with a bluescreen)
> >>>>>> Can you report (=> kernel log) which MSRs are unknown to KVM?
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x1c9
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x60
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x1c9
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x60
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x1c9
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x60
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x1c9
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x60
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x1c9
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored
> >>>>> rdmsr: 0x60
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1
> >>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> >>>>>
> >>>>> i hope thats the info you need, i booted it with ignore_msrs=1 since
> >>>>> if i dont do that i get less output.
> >>>>> (do you need it without the "option"?)
> >>>>>
> >>>>>>> the unigine benchmark ran flawlessly
> >>>>>>> also the benchmark included in windows gave my videocard
> >>>>>>> similar values (7.7) comparable with my native win7 (7.9)
> >>>>>>>
> >>>>>>>
> >>>>>>> now to my questions...
> >>>>>>> 1.     is it possible to reset the videocard properly to be able to
> >>>>>>>        reboot the vm?
> >>>>>> Which versions of kernel and qemu-kvm are involved via your distro? Can
> >>>>>> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
> >>>>>> something got fixed meanwhile.
> >>>>>>
> >>>>>> In general, there are many adapters that require special procedures to
> >>>>>> perform resets. This one may fall into that category as well.
> >>>>> i will do a test today.
> >>>>>>> 2.    the xonar d2x is a very nice audio card, it would be very handy
> >>>>>>>        to be able to use it in the vm. in my oppinion the card is a
> >>>>>>>        d2 with a pci-e to pci bridge.
> >>>>>>>        i tried to passthrough the card alone and with the pci-bridge
> >>>>>>>        that was shown though lspci, but i had no success.
> >>>>>>>        maybe you guys here have an idea on that topic?
> >>>>>> Any further details about the error? Does the adapter work with a Linux
> >>>>>> guest or provide more information that way?
> >>>>>>
> >>>>>> Jan
> >>>> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
> >>>> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
> >>>>            Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> >>>> ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>>>            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> >>>> <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>>>            Latency: 0, Cache Line Size: 64 bytes
> >>>>            Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
> >>>>            I/O behind bridge: 0000d000-0000dfff
> >>>>            Memory behind bridge: fff00000-000fffff
> >>>>            Prefetchable memory behind bridge: fff00000-000fffff
> >>>>            Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium
> >>>>    >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> >>>>            BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
> >>>>                    PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> >>>>            Capabilities: <access denied>
> >>>>            Kernel driver in use: pci-stub
> >>>>            Kernel modules: shpchp
> >>>>
> >>>> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788
> >>>> [Oxygen HD Audio]
> >>>>            Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
> >>>>            Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-
> >>>> ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>>>            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium
> >>>>    >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>>>            Interrupt: pin A routed to IRQ 11
> >>>>            Region 0: I/O ports at d000 [disabled] [size=256]
> >>>>            Capabilities: <access denied>
> >>>>            Kernel driver in use: pci-stub
> >>>>            Kernel modules: snd-virtuoso
> >>>>
> >>>>
> >>>>     /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda
> >>>> /var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device
> >>>> pci-assign,host=03:04.0
> >>>> Device assignment only supports endpoint assignment, device type 7
> >>>> qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign'
> >>>> could not be initialized
> >>>>
> >>>> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c
> >>>> -hda /var/lib/libvirt/images/Win7.img -device
> >>>> pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
> >>>> Perhaps you are assigning a device that shares an IRQ with another device?
> >>>> qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign'
> >>>> could not be initialized
> >>> Either your distro isn't shipping a version of kvm that supports
> >>> interrupt sharing on PCI 2.3 complaint devices (likely) or your device
> >>> doesn't support PCI 2.3 INTx disable (possible).  You either need to
> >>> unbind any other devices that may be sharing IRQ 11 with this device
> >>> from their driver or upgrade your kernel and qemu-kvm.  Thanks,
> >>>
> >>> Alex
> >>>
> >>
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

thank you for the information.

i will try what you mentioned...
do you have some additional information about rebooting a VM with a 
passed through videocard?
(amd / ati 7870)

thanks in advance

Am 27.09.2012 19:56, schrieb Alex Williamson:
> On Thu, 2012-09-27 at 13:19 +0200, Martin Wolf wrote:
>> i waited 2 months to get a more stable ubuntu 12.10
>> now that i dont get crashes every 5 minutes im going to test again.
>>
>> as you remember my two only problems are
>> first the reboot issue of the gues os
>> and second the xonar 2 d2x audio card.
>>
>> but first i would like to work on the reboot issue since
>> i guess this is far more important.
>>
>> when i read the faq i found that one problem could be the "VBIOS ROM access"
>> i tried -device pci-assign,...,romfile=... but have not found any output
>> or change.
>> is there some documentation of the romfile command to generate usable
>> output?
> If you can read the rom from the host, this probably isn't adding
> anything.  You can test this by finding the rom in /sys (find /sys -name
> rom), match it up to the PCI device you're assigning.  Then do:
>
> # echo 1 > rom
> # xxd rom | head
> #echo 0 > rom
>
> If you get something out that starts with 55aa then qemu will be able to
> read the rom directly.  If you're replacing the rom with the romfile=
> option, you can boot a linux guest, do the same as above to read the rom
> from the guest, save it to a file and compare md5sum of what the guest
> sees to the file provided.  If qemu is reading the rom on the device
> correctly, it's no surprise that providing one doesn't make any
> difference.
>
>> for testing i downloaded the bios of my videocard from techpowerup.
>> they have a huge database and my card and biosrevision is listed.
>> was that ok or do i have to retrieve the bios from inside the vm?
> Thanks for mentioning that.  From other reports I've seen, it looks like
> nvidia cards are the troublesome ones for being able to read the BIOS
> from the host.  Thanks,
>
> Alex
>
>
>> thank you in advance
>>
>> Am 19.07.2012 18:16, schrieb Alex Williamson:
>>> On Thu, 2012-07-19 at 16:54 +0200, Martin Wolf wrote:
>>>> i tried now the alpha of ubuntu 12.10 that includes qemu-kvm 1.1
>>>> and a 3.5 kernel version.
>>>>
>>>> the "msr" problem is gone now, i was able to select "sandybridge" as
>>>> cpu topology, this removed the MSR error messages.
>>>> thats the positive part...
>>>>
>>>> unfortunately i had no success with the other topics (rebootability and
>>>> the soundcard)
>>>>
>>>> rebootability:
>>>> the guest vm still crashes with a page fault bluescreen
>>>>
>>>> soundcard:
>>>> it would be really nice if someone would be able to give me
>>>> some advise how to debug this problem.
>>> If kernel/qemu-kvm update didn't help, then the device probably doesn't
>>> support the interrupt disable mechanism that allows shared interrupts.
>>> Therefore the device needs to be on an exclusive interrupt on the host.
>>> Look in /proc/interrupts and see what else is on the same interrupt
>>> line.  Your output below indicates the device is on IRQ 11.  Sometimes
>>> moving the device to a different physical slot will change the IRQ and
>>> give you an exclusive interrupt, otherwise you need to find the devices
>>> sharing that interrupt line and disable them by attaching the device to
>>> pci-stub.  Thanks,
>>>
>>> Alex
>>>
>>>> Am 18.07.2012 16:23, schrieb Alex Williamson:
>>>>> On Wed, 2012-07-18 at 14:37 +0200, Martin Wolf wrote:
>>>>>> soundcard logs added
>>>>>>
>>>>>> On 18.07.2012 14:08, Martin Wolf wrote:
>>>>>>> On 18.07.2012 11:26, Jan Kiszka wrote:
>>>>>>>> On 2012-07-18 07:45, Martin Wolf wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> i was able to passthrough an AMD 7870 videocard to my win7 guest
>>>>>>>>> machine.
>>>>>>>> Would you add it to http://www.linux-kvm.org/page/VGA_device_assignment?
>>>>>>> sure, i will prepare something
>>>>>>>>> my host is ubuntu 12.04 with stock kernel.
>>>>>>>>> my system contains:
>>>>>>>>> dq67sw q67 mainboard
>>>>>>>>> i5-2400s cpu
>>>>>>>>> sapphire 7870 amd videocard
>>>>>>>>> xonar d2x (problems to passthrough)
>>>>>>>>>
>>>>>>>>> for full functionality i just needed two options
>>>>>>>>>
>>>>>>>>> - kernel : iommu=on
>>>>>>>>> - kvm module: ignore_msrs=1
>>>>>>>>> (if i would not set it the guest os would crash with a bluescreen)
>>>>>>>> Can you report (=> kernel log) which MSRs are unknown to KVM?
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.309931] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522724] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522733] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522736] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522752] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522755] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522821] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522823] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522834] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522840] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522842] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522865] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x1c9
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522867] kvm: 3347: cpu1 ignored
>>>>>>> rdmsr: 0x60
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.522921] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523005] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523081] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523175] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523248] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523333] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>> Jul 18 14:03:33 kvm-xen kernel: [  437.523430] kvm: 3347: cpu1
>>>>>>> kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>>>>>>>
>>>>>>> i hope thats the info you need, i booted it with ignore_msrs=1 since
>>>>>>> if i dont do that i get less output.
>>>>>>> (do you need it without the "option"?)
>>>>>>>
>>>>>>>>> the unigine benchmark ran flawlessly
>>>>>>>>> also the benchmark included in windows gave my videocard
>>>>>>>>> similar values (7.7) comparable with my native win7 (7.9)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> now to my questions...
>>>>>>>>> 1.     is it possible to reset the videocard properly to be able to
>>>>>>>>>         reboot the vm?
>>>>>>>> Which versions of kernel and qemu-kvm are involved via your distro? Can
>>>>>>>> you retry with latest Linux (3.5-rcX) / lastest qemu-kvm? Maybe
>>>>>>>> something got fixed meanwhile.
>>>>>>>>
>>>>>>>> In general, there are many adapters that require special procedures to
>>>>>>>> perform resets. This one may fall into that category as well.
>>>>>>> i will do a test today.
>>>>>>>>> 2.    the xonar d2x is a very nice audio card, it would be very handy
>>>>>>>>>         to be able to use it in the vm. in my oppinion the card is a
>>>>>>>>>         d2 with a pci-e to pci bridge.
>>>>>>>>>         i tried to passthrough the card alone and with the pci-bridge
>>>>>>>>>         that was shown though lspci, but i had no success.
>>>>>>>>>         maybe you guys here have an idea on that topic?
>>>>>>>> Any further details about the error? Does the adapter work with a Linux
>>>>>>>> guest or provide more information that way?
>>>>>>>>
>>>>>>>> Jan
>>>>>> 02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
>>>>>> Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
>>>>>>             Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>>>>>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>>>>>             Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
>>>>>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>>>>>             Latency: 0, Cache Line Size: 64 bytes
>>>>>>             Bus: primary=02, secondary=03, subordinate=03, sec-latency=32
>>>>>>             I/O behind bridge: 0000d000-0000dfff
>>>>>>             Memory behind bridge: fff00000-000fffff
>>>>>>             Prefetchable memory behind bridge: fff00000-000fffff
>>>>>>             Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium
>>>>>>     >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>>>>>>             BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
>>>>>>                     PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>>>>>>             Capabilities: <access denied>
>>>>>>             Kernel driver in use: pci-stub
>>>>>>             Kernel modules: shpchp
>>>>>>
>>>>>> 03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788
>>>>>> [Oxygen HD Audio]
>>>>>>             Subsystem: ASUSTeK Computer Inc. Virtuoso 200 (Xonar D2X)
>>>>>>             Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop-
>>>>>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>>>>>             Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium
>>>>>>     >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>>>>>             Interrupt: pin A routed to IRQ 11
>>>>>>             Region 0: I/O ports at d000 [disabled] [size=256]
>>>>>>             Capabilities: <access denied>
>>>>>>             Kernel driver in use: pci-stub
>>>>>>             Kernel modules: snd-virtuoso
>>>>>>
>>>>>>
>>>>>>      /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c -hda
>>>>>> /var/lib/libvirt/images/Win7.img -device pci-assign,host=02:00.0 -device
>>>>>> pci-assign,host=03:04.0
>>>>>> Device assignment only supports endpoint assignment, device type 7
>>>>>> qemu-system-x86_64: -device pci-assign,host=02:00.0: Device 'pci-assign'
>>>>>> could not be initialized
>>>>>>
>>>>>> root@kvm-xen:~# /usr/bin/qemu-system-x86_64 -cpu host -m 8192 -boot c
>>>>>> -hda /var/lib/libvirt/images/Win7.img -device
>>>>>> pci-assign,host=03:04.0Failed to assign irq for "(null)": Input/output error
>>>>>> Perhaps you are assigning a device that shares an IRQ with another device?
>>>>>> qemu-system-x86_64: -device pci-assign,host=03:04.0: Device 'pci-assign'
>>>>>> could not be initialized
>>>>> Either your distro isn't shipping a version of kvm that supports
>>>>> interrupt sharing on PCI 2.3 complaint devices (likely) or your device
>>>>> doesn't support PCI 2.3 INTx disable (possible).  You either need to
>>>>> unbind any other devices that may be sharing IRQ 11 with this device
>>>>> from their driver or upgrade your kernel and qemu-kvm.  Thanks,
>>>>>
>>>>> Alex
>>>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>


-- 
Adiumentum GmbH
Gf. Martin Wolf
Banderbacherstraße 76
90513 Zirndorf

0911 / 9601470
mwolf@adiumentum.com

Re: vga passthrough // questions about pci passthrough

[Alex Williamson]

On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
> thank you for the information.
> 
> i will try what you mentioned...
> do you have some additional information about rebooting a VM with a 
> passed through videocard?
> (amd / ati 7870)

I don't.  Is the bsod on reboot only or does it also happen on shutdown?
There's a slim chance it could be traced by enabling debug in the
pci-assign driver and analyzing what the guest driver is trying to do.
I'm hoping that q35 chipset support might resolve some issues with vga
assignment as it exposes a topology that looks a bit more like one that
a driver would expect on physical hardware.  Thanks,

Alex

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

when will that support be implemented?
how do i enable debug support for the "pci-assign" driver?


Am 27.09.2012 21:18, schrieb Alex Williamson:
> On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
>> thank you for the information.
>>
>> i will try what you mentioned...
>> do you have some additional information about rebooting a VM with a
>> passed through videocard?
>> (amd / ati 7870)
> I don't.  Is the bsod on reboot only or does it also happen on shutdown?
> There's a slim chance it could be traced by enabling debug in the
> pci-assign driver and analyzing what the guest driver is trying to do.
> I'm hoping that q35 chipset support might resolve some issues with vga
> assignment as it exposes a topology that looks a bit more like one that
> a driver would expect on physical hardware.  Thanks,
>
> Alex
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Adiumentum GmbH
Gf. Martin Wolf
Banderbacherstraße 76
90513 Zirndorf

0911 / 9601470
mwolf@adiumentum.com

Re: vga passthrough // questions about pci passthrough

[Alex Williamson]

On Thu, 2012-09-27 at 21:37 +0200, Martin Wolf wrote:
> when will that support be implemented?
> how do i enable debug support for the "pci-assign" driver?

I believe the target is qemu 1.3 for q35.  Debug is enabled by
uncommenting the DEVICE_ASSIGNMENT_DEBUG define in hw/kvm/pci-assign.c
(or hw/device-assignment.c on qemu-kvm in an older tree) and rebuilding.
Thanks,

Alex

> Am 27.09.2012 21:18, schrieb Alex Williamson:
> > On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
> >> thank you for the information.
> >>
> >> i will try what you mentioned...
> >> do you have some additional information about rebooting a VM with a
> >> passed through videocard?
> >> (amd / ati 7870)
> > I don't.  Is the bsod on reboot only or does it also happen on shutdown?
> > There's a slim chance it could be traced by enabling debug in the
> > pci-assign driver and analyzing what the guest driver is trying to do.
> > I'm hoping that q35 chipset support might resolve some issues with vga
> > assignment as it exposes a topology that looks a bit more like one that
> > a driver would expect on physical hardware.  Thanks,
> >
> > Alex
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

about your question if the bsod also happens on shutdown, i have
to deny it. it just happens after an msi message in kernel log on 
rebooting the vm.
(but the msi message is also there on initial boot where the system 
works perfectly)
in my oppinion at the moment when the video driver is loaded.

Am 27.09.2012 22:46, schrieb Alex Williamson:
> On Thu, 2012-09-27 at 21:37 +0200, Martin Wolf wrote:
>> when will that support be implemented?
>> how do i enable debug support for the "pci-assign" driver?
> I believe the target is qemu 1.3 for q35.  Debug is enabled by
> uncommenting the DEVICE_ASSIGNMENT_DEBUG define in hw/kvm/pci-assign.c
> (or hw/device-assignment.c on qemu-kvm in an older tree) and rebuilding.
> Thanks,
>
> Alex
>
>> Am 27.09.2012 21:18, schrieb Alex Williamson:
>>> On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
>>>> thank you for the information.
>>>>
>>>> i will try what you mentioned...
>>>> do you have some additional information about rebooting a VM with a
>>>> passed through videocard?
>>>> (amd / ati 7870)
>>> I don't.  Is the bsod on reboot only or does it also happen on shutdown?
>>> There's a slim chance it could be traced by enabling debug in the
>>> pci-assign driver and analyzing what the guest driver is trying to do.
>>> I'm hoping that q35 chipset support might resolve some issues with vga
>>> assignment as it exposes a topology that looks a bit more like one that
>>> a driver would expect on physical hardware.  Thanks,
>>>
>>> Alex
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>


-- 
Adiumentum GmbH
Gf. Martin Wolf
Banderbacherstraße 76
90513 Zirndorf

0911 / 9601470
mwolf@adiumentum.com

Re: vga passthrough // questions about pci passthrough

[Jan Kiszka]

On 2012-09-27 21:18, Alex Williamson wrote:
> On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
>> thank you for the information.
>>
>> i will try what you mentioned...
>> do you have some additional information about rebooting a VM with a 
>> passed through videocard?
>> (amd / ati 7870)
> 
> I don't.  Is the bsod on reboot only or does it also happen on shutdown?
> There's a slim chance it could be traced by enabling debug in the
> pci-assign driver and analyzing what the guest driver is trying to do.
> I'm hoping that q35 chipset support might resolve some issues with vga
> assignment as it exposes a topology that looks a bit more like one that
> a driver would expect on physical hardware.  Thanks,

>From our attempts to get more working than what NVIDIA Quadro cards
support officially, my own experiments with q35 in this context and our
discussions with NVIDIA, I'm pretty skeptical that this chipset will
make a difference here. Most problems are due to those non-standard side
channels to configure the hardware, memory mappings etc. And getting
this working requires either cooperation of the vendor or *a lot* of
reverse engineering.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

well my first tests with the vga rom were useless because of apparmor 
rules i guess
now i placed the vga.rom in /usr/share/qemu ... well the error is gone 
now but no changes ;)
so i added the "bar" parameter but it also made no difference :(

are you interested in the windows memory dump from the bsod?

another thing, after i ran some benchmarks after a fresh reboot on win7 
i wanted to measure some
values of the 7870 so i started gpu-z ( http://www.techpowerup.com/gpuz/ )
then almost immediately the vm froze
i found one log entry in one of the libvirt log files:

kvm: /build/buildd/qemu-kvm-1.2.0+noroms/exec.c:2255: register_subpage: 
Assertion `existing->mr->subpage || existing->mr == &io_mem_unassigned' 
failed.
maybe you know what this is about.

thanks again for your patience and help ;)

Am 28.09.2012 10:12, schrieb Jan Kiszka:
> On 2012-09-27 21:18, Alex Williamson wrote:
>> On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
>>> thank you for the information.
>>>
>>> i will try what you mentioned...
>>> do you have some additional information about rebooting a VM with a
>>> passed through videocard?
>>> (amd / ati 7870)
>> I don't.  Is the bsod on reboot only or does it also happen on shutdown?
>> There's a slim chance it could be traced by enabling debug in the
>> pci-assign driver and analyzing what the guest driver is trying to do.
>> I'm hoping that q35 chipset support might resolve some issues with vga
>> assignment as it exposes a topology that looks a bit more like one that
>> a driver would expect on physical hardware.  Thanks,
>  From our attempts to get more working than what NVIDIA Quadro cards
> support officially, my own experiments with q35 in this context and our
> discussions with NVIDIA, I'm pretty skeptical that this chipset will
> make a difference here. Most problems are due to those non-standard side
> channels to configure the hardware, memory mappings etc. And getting
> this working requires either cooperation of the vendor or *a lot* of
> reverse engineering.
>
> Jan
>


-- 
Adiumentum GmbH
Gf. Martin Wolf
Banderbacherstraße 76
90513 Zirndorf

0911 / 9601470
mwolf@adiumentum.com

Re: vga passthrough // questions about pci passthrough

[Alex Williamson]

On Fri, 2012-09-28 at 10:12 +0200, Jan Kiszka wrote:
> On 2012-09-27 21:18, Alex Williamson wrote:
> > On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
> >> thank you for the information.
> >>
> >> i will try what you mentioned...
> >> do you have some additional information about rebooting a VM with a 
> >> passed through videocard?
> >> (amd / ati 7870)
> > 
> > I don't.  Is the bsod on reboot only or does it also happen on shutdown?
> > There's a slim chance it could be traced by enabling debug in the
> > pci-assign driver and analyzing what the guest driver is trying to do.
> > I'm hoping that q35 chipset support might resolve some issues with vga
> > assignment as it exposes a topology that looks a bit more like one that
> > a driver would expect on physical hardware.  Thanks,
> 
> From our attempts to get more working than what NVIDIA Quadro cards
> support officially, my own experiments with q35 in this context and our
> discussions with NVIDIA, I'm pretty skeptical that this chipset will
> make a difference here. Most problems are due to those non-standard side
> channels to configure the hardware, memory mappings etc. And getting
> this working requires either cooperation of the vendor or *a lot* of
> reverse engineering.

I heard from an nvidia guy that the driver behaves differently depending
on whether it finds an upstream express port, so we're probably causing
ourselves more problems if it's trying to run in AGP mode.  There was
also a lot of FUD in Xen (maybe justified) around how the BIOS
determines the memory ranges and whether it bypasses the PCI BARs and
gets them directly.  That means some cards may require identity mapping
to work.  It seems like the very high-end cards are possibly fixing
this, but they're far more expensive than I can justify.  Thanks,

Alex

Re: vga passthrough // questions about pci passthrough

[Jan Kiszka]

On 2012-09-28 17:50, Alex Williamson wrote:
> On Fri, 2012-09-28 at 10:12 +0200, Jan Kiszka wrote:
>> On 2012-09-27 21:18, Alex Williamson wrote:
>>> On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
>>>> thank you for the information.
>>>>
>>>> i will try what you mentioned...
>>>> do you have some additional information about rebooting a VM with a 
>>>> passed through videocard?
>>>> (amd / ati 7870)
>>>
>>> I don't.  Is the bsod on reboot only or does it also happen on shutdown?
>>> There's a slim chance it could be traced by enabling debug in the
>>> pci-assign driver and analyzing what the guest driver is trying to do.
>>> I'm hoping that q35 chipset support might resolve some issues with vga
>>> assignment as it exposes a topology that looks a bit more like one that
>>> a driver would expect on physical hardware.  Thanks,
>>
>> From our attempts to get more working than what NVIDIA Quadro cards
>> support officially, my own experiments with q35 in this context and our
>> discussions with NVIDIA, I'm pretty skeptical that this chipset will
>> make a difference here. Most problems are due to those non-standard side
>> channels to configure the hardware, memory mappings etc. And getting
>> this working requires either cooperation of the vendor or *a lot* of
>> reverse engineering.
> 
> I heard from an nvidia guy that the driver behaves differently depending
> on whether it finds an upstream express port, so we're probably causing
> ourselves more problems if it's trying to run in AGP mode.

May be a point for the low- to mid-range cards. It does not apply to the
"virtualization-ready" Quadro series according to our information back then.

>  There was
> also a lot of FUD in Xen (maybe justified) around how the BIOS
> determines the memory ranges and whether it bypasses the PCI BARs and
> gets them directly.  That means some cards may require identity mapping
> to work.  It seems like the very high-end cards are possibly fixing
> this, but they're far more expensive than I can justify.  Thanks,

Yes, that is what makes them virtualization ready. But they also come
with limitations. So far, you can't pass-through a primary card or use
it for early boot messages of the guest as the BIOS is not ready for
that - without identity mapping or even more.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: vga passthrough // questions about pci passthrough

[Martin Wolf]

Hello,

would a Memory Dump from inside the VM help solving the BSOD,
or is that pointless?

Thank you for your patience and help ;)


Am 28.09.2012 18:29, schrieb Jan Kiszka:
> On 2012-09-28 17:50, Alex Williamson wrote:
>> On Fri, 2012-09-28 at 10:12 +0200, Jan Kiszka wrote:
>>> On 2012-09-27 21:18, Alex Williamson wrote:
>>>> On Thu, 2012-09-27 at 20:43 +0200, Martin Wolf wrote:
>>>>> thank you for the information.
>>>>>
>>>>> i will try what you mentioned...
>>>>> do you have some additional information about rebooting a VM with a
>>>>> passed through videocard?
>>>>> (amd / ati 7870)
>>>> I don't.  Is the bsod on reboot only or does it also happen on shutdown?
>>>> There's a slim chance it could be traced by enabling debug in the
>>>> pci-assign driver and analyzing what the guest driver is trying to do.
>>>> I'm hoping that q35 chipset support might resolve some issues with vga
>>>> assignment as it exposes a topology that looks a bit more like one that
>>>> a driver would expect on physical hardware.  Thanks,
>>>  From our attempts to get more working than what NVIDIA Quadro cards
>>> support officially, my own experiments with q35 in this context and our
>>> discussions with NVIDIA, I'm pretty skeptical that this chipset will
>>> make a difference here. Most problems are due to those non-standard side
>>> channels to configure the hardware, memory mappings etc. And getting
>>> this working requires either cooperation of the vendor or *a lot* of
>>> reverse engineering.
>> I heard from an nvidia guy that the driver behaves differently depending
>> on whether it finds an upstream express port, so we're probably causing
>> ourselves more problems if it's trying to run in AGP mode.
> May be a point for the low- to mid-range cards. It does not apply to the
> "virtualization-ready" Quadro series according to our information back then.
>
>>   There was
>> also a lot of FUD in Xen (maybe justified) around how the BIOS
>> determines the memory ranges and whether it bypasses the PCI BARs and
>> gets them directly.  That means some cards may require identity mapping
>> to work.  It seems like the very high-end cards are possibly fixing
>> this, but they're far more expensive than I can justify.  Thanks,
> Yes, that is what makes them virtualization ready. But they also come
> with limitations. So far, you can't pass-through a primary card or use
> it for early boot messages of the guest as the BIOS is not ready for
> that - without identity mapping or even more.
>
> Jan
>


-- 
Adiumentum GmbH
Gf. Martin Wolf
Banderbacherstraße 76
90513 Zirndorf

0911 / 9601470
mwolf@adiumentum.com

Re: vga passthrough // questions about pci passthrough

[Erik Hardesty]

I'm not sure if this will help the KVM camp at all but I have a very similar 
situation. I have a Radeon 7970 and a Xonar DX which uses the same chipset and
pci-pcie bridge.

The best I could get passthrough to work in KVM was exactly the same as the 
original poster's experience. I eventually got the 7970 working in the guest but 
it would not reset properly on guest restart or shutdown. After getting 
frustrated with my experience I decided to try Xen. To my surprise both the 7970 
and the Xonar DX worked perfectly on the first try and performance was 
excellent.

Re: vga passthrough // questions about pci passthrough

[YunQiang Su]

It seems that winxp 32 bit has no BSOD, while the xp 64bit and
Win7 (both 32 bit and 64bit) will death of blue.

-- 
YunQiang Su