* PCI passthrough resource remapping
@ 2010-01-09 2:45 Ryan C. Underwood
2010-01-09 3:22 ` Alexander Graf
0 siblings, 1 reply; 39+ messages in thread
From: Ryan C. Underwood @ 2010-01-09 2:45 UTC (permalink / raw)
To: kvm
I have a multifunction PCI device that I'd like to pass through to KVM.
In order to do that, I'm reading that the PCI memory region must be 4K-page
aligned and the PCI memory resources itself must also be exact multiples
of 4K pages.
I have added the following on my kernel command line:
reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
But I don't know if it has any effect. The resources are still not
sized in 4K pages. Also, this seems to screw up the last device.
Am I doing the right thing. I am on amd64 and using the latest qemu-kvm
release 0.12.1.2 with kernel 2.6.31. It seems very hard to find
documentation on the kernel parameters related to pci-passthrough.
08:09.0 0c00: 1180:0832 (rev 05) (prog-if 10)
Subsystem: 103c:30cd
Flags: bus master, medium devsel, latency 32, IRQ 16
Memory at f4400000 (32-bit, non-prefetchable) [size=2K]
Capabilities: <access denied>
Kernel driver in use: ohci1394
Kernel modules: firewire-ohci, ohci1394
08:09.1 0805: 1180:0822 (rev 22)
Subsystem: 103c:30cd
Flags: bus master, medium devsel, latency 32, IRQ 18
Memory at f4400800 (32-bit, non-prefetchable) [size=256]
Capabilities: <access denied>
Kernel driver in use: sdhci-pci
Kernel modules: sdhci-pci
08:09.2 0880: 1180:0843 (rev 12)
Subsystem: 103c:30cd
Flags: bus master, medium devsel, latency 32, IRQ 10
Memory at f4400c00 (32-bit, non-prefetchable) [size=256]
Capabilities: <access denied>
Kernel driver in use: ricoh-mmc
Kernel modules: ricoh_mmc
08:09.3 0880: 1180:0592 (rev 12)
Subsystem: 103c:30cd
Flags: bus master, medium devsel, latency 32, IRQ 10
Memory at f4401000 (32-bit, non-prefetchable) [size=256]
Capabilities: <access denied>
08:09.4 0880: 1180:0852 (rev ff) (prog-if ff)
!!! Unknown header type 7f
--
Ryan C. Underwood, <nemesis@icequake.net>
^ permalink raw reply [flat|nested] 39+ messages in thread* Re: PCI passthrough resource remapping 2010-01-09 2:45 PCI passthrough resource remapping Ryan C. Underwood @ 2010-01-09 3:22 ` Alexander Graf 2010-01-10 21:53 ` Ryan C. Underwood ` (2 more replies) 0 siblings, 3 replies; 39+ messages in thread From: Alexander Graf @ 2010-01-09 3:22 UTC (permalink / raw) To: nemesis; +Cc: kvm On 09.01.2010, at 03:45, Ryan C. Underwood wrote: > > I have a multifunction PCI device that I'd like to pass through to KVM. > In order to do that, I'm reading that the PCI memory region must be 4K-page > aligned and the PCI memory resources itself must also be exact multiples > of 4K pages. > > I have added the following on my kernel command line: > reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 > > But I don't know if it has any effect. The resources are still not > sized in 4K pages. Also, this seems to screw up the last device. I submitted a patch to qemu-kvm recently that got rid of that limitation. Please try out if the current git head works for you. Alex ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-09 3:22 ` Alexander Graf @ 2010-01-10 21:53 ` Ryan C. Underwood 2010-01-10 22:15 ` Ryan C. Underwood 2010-03-26 2:37 ` Kenni Lund 2 siblings, 0 replies; 39+ messages in thread From: Ryan C. Underwood @ 2010-01-10 21:53 UTC (permalink / raw) To: Alexander Graf; +Cc: kvm > > I have a multifunction PCI device that I'd like to pass through to KVM. > > In order to do that, I'm reading that the PCI memory region must be 4K-page > > aligned and the PCI memory resources itself must also be exact multiples > > of 4K pages. > > > > I have added the following on my kernel command line: > > reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 > > > > But I don't know if it has any effect. The resources are still not > > sized in 4K pages. Also, this seems to screw up the last device. > > I submitted a patch to qemu-kvm recently that got rid of that limitation. Please try out if the current git head works for you. This works around that particular limitation, but now it looks like I have to have VT-D for this to work at all. (It seems so hard to find documentation about PCI passthrough, probably because things are changing all the time.) device: 08:09.0: driver="pci-assign" host="08:09.0" device: 08:09.1: driver="pci-assign" host="08:09.1" device: 08:09.2: driver="pci-assign" host="08:09.2" device: 08:09.3: driver="pci-assign" host="08:09.3" device: 08:09.4: driver="pci-assign" host="08:09.4" PCI region 0 at address 0xf4400000 has size 0x800, which is not a multiple of 4K. You might experience some performance hit due to that. No IOMMU found. Unable to assign device "08:09.0" Failed to deassign device "08:09.0" : Invalid argument Error initializing device pci-assign -- Ryan C. Underwood, <nemesis@icequake.net> ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-09 3:22 ` Alexander Graf 2010-01-10 21:53 ` Ryan C. Underwood @ 2010-01-10 22:15 ` Ryan C. Underwood 2010-01-14 13:59 ` Avi Kivity 2010-03-26 2:37 ` Kenni Lund 2 siblings, 1 reply; 39+ messages in thread From: Ryan C. Underwood @ 2010-01-10 22:15 UTC (permalink / raw) To: Alexander Graf; +Cc: kvm I guess I'll run the things I've found by the list to see if I'm off track or not. There is this patch: http://marc.info/?l=xen-devel&m=124748015304566&w=4 which would seem to be related to what I'm doing, trying to pass through a multifunction device (a Ricoh firewire controller which has digital media card reader subfunctions). Then there is this patch: http://patchwork.kernel.org/patch/20195/ Which is related to PCI passthrough without VT-d, but I have no idea if this patch is necessary or sufficient. Also, just for further complication, the Ricoh chip does not support MSI and shares an IRQ on the system board with the USB host controller. I have rebound the USB host controller to pci-stub, but I'm not sure if that totally takes care of the IRQ-sharing-without-MSI issue. -- Ryan C. Underwood, <nemesis@icequake.net> ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-10 22:15 ` Ryan C. Underwood @ 2010-01-14 13:59 ` Avi Kivity 2010-01-14 15:26 ` Ryan C. Underwood 0 siblings, 1 reply; 39+ messages in thread From: Avi Kivity @ 2010-01-14 13:59 UTC (permalink / raw) To: nemesis; +Cc: Alexander Graf, kvm On 01/11/2010 12:15 AM, Ryan C. Underwood wrote: > I guess I'll run the things I've found by the list to see if I'm off > track or not. > > There is this patch: > http://marc.info/?l=xen-devel&m=124748015304566&w=4 > > which would seem to be related to what I'm doing, trying to pass through > a multifunction device (a Ricoh firewire controller which has digital > media card reader subfunctions). > > Then there is this patch: > http://patchwork.kernel.org/patch/20195/ > > Which is related to PCI passthrough without VT-d, but I have no idea if > this patch is necessary or sufficient. > That effort has been discontinues. > Also, just for further complication, the Ricoh chip does not support > MSI and shares an IRQ on the system board with the USB host controller. > I have rebound the USB host controller to pci-stub, but I'm not sure if > that totally takes care of the IRQ-sharing-without-MSI issue. > Can you post lspci -vv output for that card? If it is pci 2.3 compliant we might be able to handle the sharing. -- error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 13:59 ` Avi Kivity @ 2010-01-14 15:26 ` Ryan C. Underwood 2010-01-14 15:34 ` Avi Kivity 0 siblings, 1 reply; 39+ messages in thread From: Ryan C. Underwood @ 2010-01-14 15:26 UTC (permalink / raw) To: Avi Kivity; +Cc: Alexander Graf, kvm [-- Attachment #1: Type: text/plain, Size: 713 bytes --] On Thu, Jan 14, 2010 at 03:59:53PM +0200, Avi Kivity wrote: > > >Also, just for further complication, the Ricoh chip does not support > >MSI and shares an IRQ on the system board with the USB host controller. > >I have rebound the USB host controller to pci-stub, but I'm not sure if > >that totally takes care of the IRQ-sharing-without-MSI issue. > > Can you post lspci -vv output for that card? If it is pci 2.3 > compliant we might be able to handle the sharing. It claims to be PCI 3.0 compliant. But I don't see any mention MSI anywhere in the specification: http://www.aeneas.com.cn/PDF/Ricoh/2005/R5C832E1%5B1%5D.00.pdf Attached is lspci -vv, thanks. -- Ryan C. Underwood, <nemesis@icequake.net> [-- Attachment #2: lspci.txt --] [-- Type: text/plain, Size: 14993 bytes --] 00:00.0 Host bridge: Intel Corporation Mobile PM965/GM965/GL960 Memory Controller Hub (rev 0c) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx- Latency: 0 Capabilities: <access denied> Kernel driver in use: agpgart-intel Kernel modules: intel-agp 00:02.0 VGA compatible controller: Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller (rev 0c) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 30 Region 0: Memory at f4000000 (64-bit, non-prefetchable) [size=1M] Region 2: Memory at d0000000 (64-bit, prefetchable) [size=256M] Region 4: I/O ports at 1800 [size=8] Capabilities: <access denied> Kernel driver in use: i915 Kernel modules: i915 00:02.1 Display controller: Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller (rev 0c) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Region 0: Memory at f4100000 (64-bit, non-prefetchable) [size=1M] Capabilities: <access denied> 00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #4 (rev 03) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 16 Region 4: I/O ports at 1840 [size=32] Kernel driver in use: uhci_hcd 00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #5 (rev 03) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 21 Region 4: I/O ports at 1860 [size=32] Kernel driver in use: uhci_hcd 00:1a.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #2 (rev 03) (prog-if 20) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin C routed to IRQ 18 Region 0: Memory at f4705c00 (32-bit, non-prefetchable) [size=1K] Capabilities: <access denied> Kernel driver in use: ehci_hcd 00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 03) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 22 Region 0: Memory at f4500000 (64-bit, non-prefetchable) [size=16K] Capabilities: <access denied> Kernel driver in use: HDA Intel Kernel modules: snd-hda-intel 00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 (rev 03) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=02, subordinate=03, sec-latency=0 I/O behind bridge: 00002000-00002fff Memory behind bridge: f2000000-f3ffffff Prefetchable memory behind bridge: 00000000f0000000-00000000f1ffffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: <access denied> Kernel driver in use: pcieport-driver Kernel modules: shpchp 00:1c.1 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 2 (rev 03) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=05, subordinate=05, sec-latency=0 I/O behind bridge: 00003000-00003fff Memory behind bridge: f4200000-f42fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: <access denied> Kernel driver in use: pcieport-driver Kernel modules: shpchp 00:1c.2 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 3 (rev 03) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=06, subordinate=06, sec-latency=0 Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: <access denied> Kernel driver in use: pcieport-driver Kernel modules: shpchp 00:1c.3 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 4 (rev 03) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=00, secondary=07, subordinate=07, sec-latency=0 Memory behind bridge: f4300000-f43fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: <access denied> Kernel driver in use: pcieport-driver Kernel modules: shpchp 00:1d.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #1 (rev 03) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 23 Region 4: I/O ports at 1880 [size=32] Kernel driver in use: uhci_hcd 00:1d.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #2 (rev 03) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 19 Region 4: I/O ports at 18a0 [size=32] Kernel driver in use: uhci_hcd 00:1d.2 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #3 (rev 03) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin C routed to IRQ 18 Region 4: I/O ports at 18c0 [size=32] Kernel driver in use: uhci_hcd 00:1d.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #1 (rev 03) (prog-if 20) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 23 Region 0: Memory at f4706000 (32-bit, non-prefetchable) [size=1K] Capabilities: <access denied> Kernel driver in use: ehci_hcd 00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev f3) (prog-if 01) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Bus: primary=00, secondary=08, subordinate=08, sec-latency=32 Memory behind bridge: f4400000-f44fffff Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: <access denied> 00:1f.0 ISA bridge: Intel Corporation 82801HEM (ICH8M) LPC Interface Controller (rev 03) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Capabilities: <access denied> Kernel modules: iTCO_wdt 00:1f.1 IDE interface: Intel Corporation 82801HBM/HEM (ICH8M/ICH8M-E) IDE Controller (rev 03) (prog-if 8a [Master SecP PriP]) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 19 Region 0: I/O ports at 01f0 [size=8] Region 1: I/O ports at 03f4 [size=1] Region 2: I/O ports at 0170 [size=8] Region 3: I/O ports at 0374 [size=1] Region 4: I/O ports at 1830 [size=16] Kernel driver in use: ata_piix 00:1f.2 SATA controller: Intel Corporation 82801HBM/HEM (ICH8M/ICH8M-E) SATA AHCI Controller (rev 03) (prog-if 01) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin B routed to IRQ 28 Region 0: I/O ports at 18f8 [size=8] Region 1: I/O ports at 18ec [size=4] Region 2: I/O ports at 18f0 [size=8] Region 3: I/O ports at 18e8 [size=4] Region 4: I/O ports at 1c00 [size=32] Region 5: Memory at f4705000 (32-bit, non-prefetchable) [size=2K] Capabilities: <access denied> Kernel driver in use: ahci 00:1f.3 SMBus: Intel Corporation 82801H (ICH8 Family) SMBus Controller (rev 03) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin C routed to IRQ 11 Region 0: Memory at c0000000 (32-bit, non-prefetchable) [size=256] Region 4: I/O ports at 1c20 [size=32] Kernel modules: i2c-i801 07:00.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN [Kedron] Network Connection (rev 61) Subsystem: Intel Corporation Device 1100 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 29 Region 0: Memory at f4300000 (64-bit, non-prefetchable) [size=8K] Capabilities: <access denied> Kernel driver in use: iwlagn Kernel modules: iwlagn 08:09.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 05) (prog-if 10) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 64 (500ns min, 1000ns max) Interrupt: pin A routed to IRQ 16 Region 0: [virtual] Memory at f4400000 (32-bit, non-prefetchable) [size=2K] Capabilities: <access denied> Kernel driver in use: pci-stub Kernel modules: firewire-ohci, ohci1394 08:09.1 SD Host controller: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev 22) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin B routed to IRQ 18 Region 0: [virtual] Memory at f4400800 (32-bit, non-prefetchable) [disabled] [size=256] Capabilities: <access denied> Kernel driver in use: pci-stub Kernel modules: sdhci-pci 08:09.2 System peripheral: Ricoh Co Ltd R5C843 MMC Host Controller (rev 12) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin B routed to IRQ 10 Region 0: [virtual] Memory at f4400c00 (32-bit, non-prefetchable) [disabled] [size=256] Capabilities: <access denied> Kernel driver in use: pci-stub Kernel modules: ricoh_mmc 08:09.3 System peripheral: Ricoh Co Ltd R5C592 Memory Stick Bus Host Adapter (rev 12) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin B routed to IRQ 10 Region 0: [virtual] Memory at f4401000 (32-bit, non-prefetchable) [disabled] [size=256] Capabilities: <access denied> Kernel driver in use: pci-stub 08:09.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev 12) Subsystem: Hewlett-Packard Company Device 30cd Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 248, Cache Line Size: 1020 bytes Interrupt: pin B routed to IRQ 10 Region 0: Memory at f4401400 (32-bit, non-prefetchable) [size=256] Capabilities: <access denied> Kernel driver in use: pci-stub ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 15:26 ` Ryan C. Underwood @ 2010-01-14 15:34 ` Avi Kivity 2010-01-14 15:47 ` Michael S. Tsirkin 0 siblings, 1 reply; 39+ messages in thread From: Avi Kivity @ 2010-01-14 15:34 UTC (permalink / raw) To: nemesis; +Cc: Alexander Graf, kvm, Michael S. Tsirkin On 01/14/2010 05:26 PM, Ryan C. Underwood wrote: > On Thu, Jan 14, 2010 at 03:59:53PM +0200, Avi Kivity wrote: > >> > >> >>> > >Also, just for further complication, the Ricoh chip does not support >>> > >MSI and shares an IRQ on the system board with the USB host controller. >>> > >I have rebound the USB host controller to pci-stub, but I'm not sure if >>> > >that totally takes care of the IRQ-sharing-without-MSI issue. >>> >> > >> > Can you post lspci -vv output for that card? If it is pci 2.3 >> > compliant we might be able to handle the sharing. >> > It claims to be PCI 3.0 compliant. But I don't see any mention MSI > anywhere in the specification: > http://www.aeneas.com.cn/PDF/Ricoh/2005/R5C832E1%5B1%5D.00.pdf > > Attached is lspci -vv, thanks. > > > > 08:09.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 05) (prog-if 10) > Subsystem: Hewlett-Packard Company Device 30cd > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx- > Latency: 64 (500ns min, 1000ns max) > Interrupt: pin A routed to IRQ 16 > Region 0: [virtual] Memory at f4400000 (32-bit, non-prefetchable) [size=2K] > Capabilities:<access denied> > Kernel driver in use: pci-stub > Kernel modules: firewire-ohci, ohci1394 > > Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant? > 08:09.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev 12) > Subsystem: Hewlett-Packard Company Device 30cd > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx- > Latency: 248, Cache Line Size: 1020 bytes > Interrupt: pin B routed to IRQ 10 > Region 0: Memory at f4401400 (32-bit, non-prefetchable) [size=256] > Capabilities:<access denied> > Kernel driver in use: pci-stub > But another function has this feature? -- error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 15:34 ` Avi Kivity @ 2010-01-14 15:47 ` Michael S. Tsirkin 2010-01-14 15:54 ` Avi Kivity 0 siblings, 1 reply; 39+ messages in thread From: Michael S. Tsirkin @ 2010-01-14 15:47 UTC (permalink / raw) To: Avi Kivity; +Cc: nemesis, Alexander Graf, kvm On Thu, Jan 14, 2010 at 05:34:42PM +0200, Avi Kivity wrote: > On 01/14/2010 05:26 PM, Ryan C. Underwood wrote: >> On Thu, Jan 14, 2010 at 03:59:53PM +0200, Avi Kivity wrote: >> >>> > >>>> > >Also, just for further complication, the Ricoh chip does not support >>>> > >MSI and shares an IRQ on the system board with the USB host controller. >>>> > >I have rebound the USB host controller to pci-stub, but I'm not sure if >>>> > >that totally takes care of the IRQ-sharing-without-MSI issue. >>>> >>> > > Can you post lspci -vv output for that card? If it is pci 2.3 >>> > compliant we might be able to handle the sharing. >>> >> It claims to be PCI 3.0 compliant. But I don't see any mention MSI >> anywhere in the specification: >> http://www.aeneas.com.cn/PDF/Ricoh/2005/R5C832E1%5B1%5D.00.pdf >> >> Attached is lspci -vv, thanks. >> >> >> >> 08:09.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 05) (prog-if 10) >> Subsystem: Hewlett-Packard Company Device 30cd >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx- >> Latency: 64 (500ns min, 1000ns max) >> Interrupt: pin A routed to IRQ 16 >> Region 0: [virtual] Memory at f4400000 (32-bit, non-prefetchable) [size=2K] >> Capabilities:<access denied> >> Kernel driver in use: pci-stub >> Kernel modules: firewire-ohci, ohci1394 >> >> > > Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant? No it doesn't, just that interrupt disable bit is not set. >> 08:09.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev 12) >> Subsystem: Hewlett-Packard Company Device 30cd >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx- >> Latency: 248, Cache Line Size: 1020 bytes >> Interrupt: pin B routed to IRQ 10 >> Region 0: Memory at f4401400 (32-bit, non-prefetchable) [size=256] >> Capabilities:<access denied> >> Kernel driver in use: pci-stub >> > > But another function has this feature? Another function has this bit set. > -- > error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 15:47 ` Michael S. Tsirkin @ 2010-01-14 15:54 ` Avi Kivity 2010-01-14 18:31 ` Ryan C. Underwood 0 siblings, 1 reply; 39+ messages in thread From: Avi Kivity @ 2010-01-14 15:54 UTC (permalink / raw) To: Michael S. Tsirkin; +Cc: nemesis, Alexander Graf, kvm On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote: > >> Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant? >> > No it doesn't, just that interrupt disable bit is not set. > Thanks. Ryan, while kvm doesn't support assigning a device with shared interrupts now, in the future it will likely be possible to share it. You'll still need an iommu. -- error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 15:54 ` Avi Kivity @ 2010-01-14 18:31 ` Ryan C. Underwood 2010-01-14 19:09 ` Avi Kivity 2010-01-15 13:11 ` Pasi Kärkkäinen 0 siblings, 2 replies; 39+ messages in thread From: Ryan C. Underwood @ 2010-01-14 18:31 UTC (permalink / raw) To: Avi Kivity; +Cc: Michael S. Tsirkin, Alexander Graf, kvm On Thu, Jan 14, 2010 at 05:54:51PM +0200, Avi Kivity wrote: > On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote: > > > >>Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant? > >No it doesn't, just that interrupt disable bit is not set. > > Thanks. Ryan, while kvm doesn't support assigning a device with > shared interrupts now, in the future it will likely be possible to > share it. You'll still need an iommu. No IOMMU on this machine and this is all integrated hardware. This IOMMU requirement seems so strange. I used to pass through PCI devices ages ago when using the DOSEMU emulator. It emulated PCI BIOS functions and mapped the PCI config space and memory regions into the emulator process. The device interrupt was grabbed and handled in the emulator's kernel support, waking up the emulator when an interrupt came in. I don't really know anything about kvm internals, but I'd like to understand more about the particulars of the IOMMU requirement if you don't mind. -- Ryan C. Underwood, <nemesis@icequake.net> ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 18:31 ` Ryan C. Underwood @ 2010-01-14 19:09 ` Avi Kivity 2010-01-14 19:34 ` Ryan C. Underwood 2010-01-15 13:11 ` Pasi Kärkkäinen 1 sibling, 1 reply; 39+ messages in thread From: Avi Kivity @ 2010-01-14 19:09 UTC (permalink / raw) To: nemesis; +Cc: Michael S. Tsirkin, Alexander Graf, kvm On 01/14/2010 08:31 PM, Ryan C. Underwood wrote: > > On Thu, Jan 14, 2010 at 05:54:51PM +0200, Avi Kivity wrote: > >> On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote: >> >>> >>>> Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant? >>>> >>> No it doesn't, just that interrupt disable bit is not set. >>> >> Thanks. Ryan, while kvm doesn't support assigning a device with >> shared interrupts now, in the future it will likely be possible to >> share it. You'll still need an iommu. >> > No IOMMU on this machine and this is all integrated hardware. > > This IOMMU requirement seems so strange. I used to pass through PCI > devices ages ago when using the DOSEMU emulator. It emulated PCI BIOS > functions and mapped the PCI config space and memory regions into the > emulator process. The device interrupt was grabbed and handled in the > emulator's kernel support, waking up the emulator when an interrupt came > in. > > I don't really know anything about kvm internals, but I'd like to > understand more about the particulars of the IOMMU requirement if you > don't mind. > PCI cards can access system memory directly. If you assign a card to a guest, the guest will program the card to transfer data to system memory using guest addresses; since guest addresses don't correspond to host addresses, memory corruption will ensue. Not sure about your experience with DOSEMU; maybe these devices were not dma capable. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 19:09 ` Avi Kivity @ 2010-01-14 19:34 ` Ryan C. Underwood 2010-01-16 9:23 ` Avi Kivity 0 siblings, 1 reply; 39+ messages in thread From: Ryan C. Underwood @ 2010-01-14 19:34 UTC (permalink / raw) To: Avi Kivity; +Cc: Michael S. Tsirkin, Alexander Graf, kvm On Thu, Jan 14, 2010 at 09:09:32PM +0200, Avi Kivity wrote: > > PCI cards can access system memory directly. If you assign a card > to a guest, the guest will program the card to transfer data to > system memory using guest addresses; since guest addresses don't > correspond to host addresses, memory corruption will ensue. I see, so the only way to fix this would be either with a special guest driver for the device that does not perform DMA, or if that is impossible (due to no docs), to trap and rewrite any command writes to the device's MMIO region that reference a DMA write target buffer. Forgive my ignorance, but is it possible that the latter is already possible with qemu-kvm (somewhat like hardware memory breakpoints in Soft-ICE)? If qemu-kvm can be made to break and log on PCI memory accesses, I would then hack around the safety limitations, assuming that's all they are, and analyze the PCI writes one by one to find the cases where a physical address is passed to the card. Then I would perform the IOMMU translation myself in software whenever a physmem address shows up in the command stream. (Somewhat like the security validation of a 3D graphics card command stream in the DRM.) > Not sure about your experience with DOSEMU; maybe these devices were > not dma capable. Well, more likely the MS-DOS drivers were simply not using PCI DMA transfers, since there is very little point to add such complexity to the driver in a non-multitasking OS. Of course that makes perfect sense now! -- Ryan C. Underwood, <nemesis@icequake.net> ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 19:34 ` Ryan C. Underwood @ 2010-01-16 9:23 ` Avi Kivity 0 siblings, 0 replies; 39+ messages in thread From: Avi Kivity @ 2010-01-16 9:23 UTC (permalink / raw) To: nemesis; +Cc: Michael S. Tsirkin, Alexander Graf, kvm On 01/14/2010 09:34 PM, Ryan C. Underwood wrote: > On Thu, Jan 14, 2010 at 09:09:32PM +0200, Avi Kivity wrote: > >> PCI cards can access system memory directly. If you assign a card >> to a guest, the guest will program the card to transfer data to >> system memory using guest addresses; since guest addresses don't >> correspond to host addresses, memory corruption will ensue. >> > I see, so the only way to fix this would be either with a special guest > driver for the device that does not perform DMA, or if that is > impossible (due to no docs), to trap and rewrite any command writes to > the device's MMIO region that reference a DMA write target buffer. > > Forgive my ignorance, but is it possible that the latter is already > possible with qemu-kvm (somewhat like hardware memory breakpoints in > Soft-ICE)? > > Yes, you can easily trap mmio writes to a device, and in fact kvm does this in some non-default scenarios. > If qemu-kvm can be made to break and log on PCI memory accesses, I would > then hack around the safety limitations, assuming that's all they are, > and analyze the PCI writes one by one to find the cases where a physical > address is passed to the card. > > Then I would perform the IOMMU translation myself in software whenever a > physmem address shows up in the command stream. (Somewhat like the > security validation of a 3D graphics card command stream in the DRM.) > > That's definitely doable, but you would need to know exactly how the device does dma. You would also need to lock all pages into memory (mlockall()) and how pages are mapped (/proc/$pid/pagemap?) -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain. ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-14 18:31 ` Ryan C. Underwood 2010-01-14 19:09 ` Avi Kivity @ 2010-01-15 13:11 ` Pasi Kärkkäinen 2010-01-15 13:15 ` Alexander Graf 1 sibling, 1 reply; 39+ messages in thread From: Pasi Kärkkäinen @ 2010-01-15 13:11 UTC (permalink / raw) To: Ryan C. Underwood; +Cc: Avi Kivity, Michael S. Tsirkin, Alexander Graf, kvm On Thu, Jan 14, 2010 at 12:31:32PM -0600, Ryan C. Underwood wrote: > > > On Thu, Jan 14, 2010 at 05:54:51PM +0200, Avi Kivity wrote: > > On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote: > > > > > >>Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant? > > >No it doesn't, just that interrupt disable bit is not set. > > > > Thanks. Ryan, while kvm doesn't support assigning a device with > > shared interrupts now, in the future it will likely be possible to > > share it. You'll still need an iommu. > > No IOMMU on this machine and this is all integrated hardware. > > This IOMMU requirement seems so strange. I used to pass through PCI > devices ages ago when using the DOSEMU emulator. It emulated PCI BIOS > functions and mapped the PCI config space and memory regions into the > emulator process. The device interrupt was grabbed and handled in the > emulator's kernel support, waking up the emulator when an interrupt came > in. > > I don't really know anything about kvm internals, but I'd like to > understand more about the particulars of the IOMMU requirement if you > don't mind. > Xen supports PCI passthrough to PV guests without IOMMU. This can create security problems, since the guests get DMA access to physical hardware, but that's usually OK in the situations where you want to use PCI passthrough on your desktop or on your development box. -- Pasi ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-15 13:11 ` Pasi Kärkkäinen @ 2010-01-15 13:15 ` Alexander Graf 0 siblings, 0 replies; 39+ messages in thread From: Alexander Graf @ 2010-01-15 13:15 UTC (permalink / raw) To: Pasi Kärkkäinen Cc: Ryan C. Underwood, Avi Kivity, Michael S. Tsirkin, kvm On 15.01.2010, at 14:11, Pasi Kärkkäinen wrote: > On Thu, Jan 14, 2010 at 12:31:32PM -0600, Ryan C. Underwood wrote: >> >> >> On Thu, Jan 14, 2010 at 05:54:51PM +0200, Avi Kivity wrote: >>> On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote: >>>> >>>>> Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant? >>>> No it doesn't, just that interrupt disable bit is not set. >>> >>> Thanks. Ryan, while kvm doesn't support assigning a device with >>> shared interrupts now, in the future it will likely be possible to >>> share it. You'll still need an iommu. >> >> No IOMMU on this machine and this is all integrated hardware. >> >> This IOMMU requirement seems so strange. I used to pass through PCI >> devices ages ago when using the DOSEMU emulator. It emulated PCI BIOS >> functions and mapped the PCI config space and memory regions into the >> emulator process. The device interrupt was grabbed and handled in the >> emulator's kernel support, waking up the emulator when an interrupt came >> in. >> >> I don't really know anything about kvm internals, but I'd like to >> understand more about the particulars of the IOMMU requirement if you >> don't mind. >> > > Xen supports PCI passthrough to PV guests without IOMMU. This can create > security problems, since the guests get DMA access to physical hardware, > but that's usually OK in the situations where you want to use PCI > passthrough on your desktop or on your development box. That's why there way PV support for DMA in KVM too, but it turned out to be rather unmaintained and hard to detect if it actually works. Because if the guest then just didn't use the PV parts to remap its DMA regions, your PCI card ended up writing into random host memory regions. WIthout you knowing. Xen doesn't have that problem as badly as we do, because it can guarantee that a PV guest is PV aware. On KVM PV is an optional add-in. All guests start off being fully virtualized. So we voted for dropping PV DMA support in KVM and just went with the IOMMU only approach. In the long run that's a pretty straight-forward hardware requirement. And if you're using KVM you're used to hardware requirements already anyways ;-). Alex ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-01-09 3:22 ` Alexander Graf 2010-01-10 21:53 ` Ryan C. Underwood 2010-01-10 22:15 ` Ryan C. Underwood @ 2010-03-26 2:37 ` Kenni Lund 2010-03-26 3:00 ` Brian Jackson 2 siblings, 1 reply; 39+ messages in thread From: Kenni Lund @ 2010-03-26 2:37 UTC (permalink / raw) To: kvm 2010/1/9 Alexander Graf <agraf@suse.de>: > > On 09.01.2010, at 03:45, Ryan C. Underwood wrote: > >> >> I have a multifunction PCI device that I'd like to pass through to KVM. >> In order to do that, I'm reading that the PCI memory region must be 4K-page >> aligned and the PCI memory resources itself must also be exact multiples >> of 4K pages. >> >> I have added the following on my kernel command line: >> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 >> >> But I don't know if it has any effect. The resources are still not >> sized in 4K pages. Also, this seems to screw up the last device. > > I submitted a patch to qemu-kvm recently that got rid of that limitation. Please try out if the current git head works for you. > > Alex-- I just upgraded to kernel 2.6.32.10 with qemu-kvm 0.12.3 and I still get the following error when trying to pass through a dedicated PCI USB card: "Unable to assign device: PCI region 0 at address 0xe9403000 has size 0x100, which is not a multiple of 4K Error initializing device pci-assign" Didn't the above patch make it into qemu-kvm? I don't know why, but I was under the impression that this was fixed when I upgraded to qemu-kvm 0.12.3. Thanks Best Regards Kenni ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-26 2:37 ` Kenni Lund @ 2010-03-26 3:00 ` Brian Jackson 2010-03-29 17:23 ` Kenni Lund 0 siblings, 1 reply; 39+ messages in thread From: Brian Jackson @ 2010-03-26 3:00 UTC (permalink / raw) To: Kenni Lund; +Cc: kvm It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there is one Sent from my iPhone On Mar 25, 2010, at 9:37 PM, Kenni Lund <kenni@kelu.dk> wrote: > 2010/1/9 Alexander Graf <agraf@suse.de>: >> >> On 09.01.2010, at 03:45, Ryan C. Underwood wrote: >> >>> >>> I have a multifunction PCI device that I'd like to pass through to >>> KVM. >>> In order to do that, I'm reading that the PCI memory region must >>> be 4K-page >>> aligned and the PCI memory resources itself must also be exact >>> multiples >>> of 4K pages. >>> >>> I have added the following on my kernel command line: >>> reassign_resources >>> reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 >>> >>> But I don't know if it has any effect. The resources are still not >>> sized in 4K pages. Also, this seems to screw up the last device. >> >> I submitted a patch to qemu-kvm recently that got rid of that >> limitation. Please try out if the current git head works for you. >> >> Alex-- > > I just upgraded to kernel 2.6.32.10 with qemu-kvm 0.12.3 and I still > get the following error when trying to pass through a dedicated PCI > USB card: > > "Unable to assign device: PCI region 0 at address 0xe9403000 has size > 0x100, which is not a multiple of 4K > Error initializing device pci-assign" > > Didn't the above patch make it into qemu-kvm? I don't know why, but I > was under the impression that this was fixed when I upgraded to > qemu-kvm 0.12.3. > > Thanks > > Best Regards > Kenni > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-26 3:00 ` Brian Jackson @ 2010-03-29 17:23 ` Kenni Lund 2010-03-29 19:17 ` Alexander Graf 0 siblings, 1 reply; 39+ messages in thread From: Kenni Lund @ 2010-03-29 17:23 UTC (permalink / raw) To: kvm >> 2010/1/9 Alexander Graf <agraf@suse.de>: >>> >>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote: >>> >>>> >>>> I have a multifunction PCI device that I'd like to pass through to KVM. >>>> In order to do that, I'm reading that the PCI memory region must be >>>> 4K-page >>>> aligned and the PCI memory resources itself must also be exact multiples >>>> of 4K pages. >>>> >>>> I have added the following on my kernel command line: >>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 >>>> >>>> But I don't know if it has any effect. The resources are still not >>>> sized in 4K pages. Also, this seems to screw up the last device. >>> >>> I submitted a patch to qemu-kvm recently that got rid of that limitation. >>> Please try out if the current git head works for you. >>> >>> Alex-- >> >> I just upgraded to kernel 2.6.32.10 with qemu-kvm 0.12.3 and I still >> get the following error when trying to pass through a dedicated PCI >> USB card: >> >> "Unable to assign device: PCI region 0 at address 0xe9403000 has size >> 0x100, which is not a multiple of 4K >> Error initializing device pci-assign" >> >> Didn't the above patch make it into qemu-kvm? I don't know why, but I >> was under the impression that this was fixed when I upgraded to >> qemu-kvm 0.12.3. >> > It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there > is one That would be highly appriciated...with the current USB support in QEMU, PCI passthrough is the only way to get USB 2.0 support. I've bought two dedicated PCI USB cards for this, but none of them works due to the above limitation. Perhaps a developer can comment on this? Are there any plans on including this patch in the stable releases in the near future? Thanks :) Best Regards Kenni Lund ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-29 17:23 ` Kenni Lund @ 2010-03-29 19:17 ` Alexander Graf 2010-03-29 23:00 ` Kenni Lund 0 siblings, 1 reply; 39+ messages in thread From: Alexander Graf @ 2010-03-29 19:17 UTC (permalink / raw) To: Kenni Lund; +Cc: kvm On 29.03.2010, at 19:23, Kenni Lund wrote: >>> 2010/1/9 Alexander Graf <agraf@suse.de>: >>>> >>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote: >>>> >>>>> >>>>> I have a multifunction PCI device that I'd like to pass through to KVM. >>>>> In order to do that, I'm reading that the PCI memory region must be >>>>> 4K-page >>>>> aligned and the PCI memory resources itself must also be exact multiples >>>>> of 4K pages. >>>>> >>>>> I have added the following on my kernel command line: >>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 >>>>> >>>>> But I don't know if it has any effect. The resources are still not >>>>> sized in 4K pages. Also, this seems to screw up the last device. >>>> >>>> I submitted a patch to qemu-kvm recently that got rid of that limitation. >>>> Please try out if the current git head works for you. >>>> >>>> Alex-- >>> >>> I just upgraded to kernel 2.6.32.10 with qemu-kvm 0.12.3 and I still >>> get the following error when trying to pass through a dedicated PCI >>> USB card: >>> >>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size >>> 0x100, which is not a multiple of 4K >>> Error initializing device pci-assign" >>> >>> Didn't the above patch make it into qemu-kvm? I don't know why, but I >>> was under the impression that this was fixed when I upgraded to >>> qemu-kvm 0.12.3. >>> >> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there >> is one > > That would be highly appriciated...with the current USB support in > QEMU, PCI passthrough is the only way to get USB 2.0 support. I've > bought two dedicated PCI USB cards for this, but none of them works > due to the above limitation. > > Perhaps a developer can comment on this? Are there any plans on > including this patch in the stable releases in the near future? Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable. I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-). Alex ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-29 19:17 ` Alexander Graf @ 2010-03-29 23:00 ` Kenni Lund 2010-03-29 23:12 ` Alexander Graf 0 siblings, 1 reply; 39+ messages in thread From: Kenni Lund @ 2010-03-29 23:00 UTC (permalink / raw) To: Alexander Graf; +Cc: kvm 2010/3/29 Alexander Graf <agraf@suse.de>: > > On 29.03.2010, at 19:23, Kenni Lund wrote: > >>>> 2010/1/9 Alexander Graf <agraf@suse.de>: >>>>> >>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote: >>>>> >>>>>> >>>>>> I have a multifunction PCI device that I'd like to pass through to KVM. >>>>>> In order to do that, I'm reading that the PCI memory region must be >>>>>> 4K-page >>>>>> aligned and the PCI memory resources itself must also be exact multiples >>>>>> of 4K pages. >>>>>> >>>>>> I have added the following on my kernel command line: >>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 >>>>>> >>>>>> But I don't know if it has any effect. The resources are still not >>>>>> sized in 4K pages. Also, this seems to screw up the last device. >>>>> >>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation. >>>>> Please try out if the current git head works for you. >>>>> >>>>> Alex-- >>>> >>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm 0.12.3 and I still >>>> get the following error when trying to pass through a dedicated PCI >>>> USB card: >>>> >>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size >>>> 0x100, which is not a multiple of 4K >>>> Error initializing device pci-assign" >>>> >>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I >>>> was under the impression that this was fixed when I upgraded to >>>> qemu-kvm 0.12.3. >>>> >>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there >>> is one >> >> That would be highly appriciated...with the current USB support in >> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've >> bought two dedicated PCI USB cards for this, but none of them works >> due to the above limitation. >> >> Perhaps a developer can comment on this? Are there any plans on >> including this patch in the stable releases in the near future? > > Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable. > > I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-). Sure, I have compiled the current git snapshot and performed some tests...It's at least mostly working, so I'm a bit unsure if this is a bug related to this or to something else. Here's my test results on trying to passthrough a PCI USB card (I've copy-pasted the text below into http://pastebin.com/8RJE36wG in case formatting is lost below): -------- qemu-kvm complete command line: qemu-kvm -usbdevice tablet -net nic,macaddr=52:54:00:00:00:01,model=virtio -net tap,ifname=tap0 -vnc :1 -smp 2 -m 2048 -cdrom /data/server/Linux/mythbuntu-9.10-desktop-amd64.iso -drive file=/data/virtualization/01_Mythbuntu.img,if=virtio,boot=on -boot c -localtime -daemonize -pcidevice host=02:01.0 -pcidevice host=02:01.1 -pcidevice host=02:01.2 -pcidevice host=02:01.3 -monitor unix:/var/run/kvm/01.socket,server,nowait -k da -------- uname -a on host Linux mediaserver 2.6.32-ARCH #1 SMP PREEMPT Fri Mar 26 02:03:53 CET 2010 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux Exact kernel version is 2.6.32.10. -------- lspci -v on host, only for the USB PCI card: 02:01.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10 [OHCI]) Subsystem: ALi Corporation ASRock 939Dual-SATA2 Motherboard Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 18 Memory at e9400000 (32-bit, non-prefetchable) [size=4K] Capabilities: [60] Power Management version 2 Kernel driver in use: pci-stub Kernel modules: ohci-hcd 02:01.1 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10 [OHCI]) Subsystem: ALi Corporation ASRock 939Dual-SATA2 Motherboard Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 16 Memory at e9401000 (32-bit, non-prefetchable) [size=4K] Capabilities: [60] Power Management version 2 Kernel driver in use: pci-stub Kernel modules: ohci-hcd 02:01.2 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10 [OHCI]) Subsystem: ALi Corporation ASRock 939Dual-SATA2 Motherboard Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 20 Memory at e9402000 (32-bit, non-prefetchable) [size=4K] Capabilities: [60] Power Management version 2 Kernel driver in use: pci-stub Kernel modules: ohci-hcd 02:01.3 USB Controller: ALi Corporation USB 2.0 Controller (rev 01) (prog-if 20 [EHCI]) Subsystem: Device 2020:8888 Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 19 Memory at e9403000 (32-bit, non-prefetchable) [size=256] Capabilities: [50] Power Management version 2 Capabilities: [58] Debug port: BAR=1 offset=0090 Kernel driver in use: pci-stub Kernel modules: ehci-hcd -------- Commands on host used to prepare PCI USB card: echo "10b9 5237" > /sys/bus/pci/drivers/pci-stub/new_id echo "0000:02:01.0" > /sys/bus/pci/drivers/ohci_hcd/unbind echo "0000:02:01.1" > /sys/bus/pci/drivers/ohci_hcd/unbind echo "0000:02:01.2" > /sys/bus/pci/drivers/ohci_hcd/unbind echo "0000:02:01.0" > /sys/bus/pci/drivers/pci-stub/bind echo "0000:02:01.1" > /sys/bus/pci/drivers/pci-stub/bind echo "0000:02:01.2" > /sys/bus/pci/drivers/pci-stub/bind echo "10b9 5237" > /sys/bus/pci/drivers/pci-stub/remove_id echo "10b9 5239" > /sys/bus/pci/drivers/pci-stub/new_id echo "0000:02:01.3" > /sys/bus/pci/drivers/ehci_hcd/unbind echo "0000:02:01.3" > /sys/bus/pci/drivers/pci-stub/bind echo "10b9 5239" > /sys/bus/pci/drivers/pci-stub/remove_id -------- Starting qemu-kvm 0.12.3 (fails): device: 02:01.0: driver="pci-assign" host="02:01.0" device: 02:01.1: driver="pci-assign" host="02:01.1" device: 02:01.2: driver="pci-assign" host="02:01.2" device: 02:01.3: driver="pci-assign" host="02:01.3" Unable to assign device: PCI region 0 at address 0xe9403000 has size 0x100, which is not a multiple of 4K Error initializing device pci-assign -------- Starting qemu-kvm git (succeeds): device: 02:01.0: driver="pci-assign" host="02:01.0" device: 02:01.1: driver="pci-assign" host="02:01.1" device: 02:01.2: driver="pci-assign" host="02:01.2" device: 02:01.3: driver="pci-assign" host="02:01.3" PCI region 0 at address 0xe9403000 has size 0x100, which is not a multiple of 4K. You might experience some performance hit due to that. -------- lspci -v in guest with qemu-kvm git (seems fine?): 00:05.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10) Subsystem: ALi Corporation USB 1.1 Controller Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 10 Memory at f2041000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:06.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10) Subsystem: ALi Corporation USB 1.1 Controller Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 11 Memory at f2042000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:07.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10) Subsystem: ALi Corporation USB 1.1 Controller Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 10 Memory at f2043000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd 00:08.0 USB Controller: ALi Corporation USB 2.0 Controller (rev 01) (prog-if 20) Subsystem: Device 2020:8888 Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 11 Memory at f2044000 (32-bit, non-prefetchable) [size=256] Kernel driver in use: ehci_hcd -------- dmesg in guest when inserting a DVB-T USB tuner into the PCI USB card: [ 190.900076] usb 1-6: new high speed USB device using ehci_hcd and address 2 [ 191.223314] usb 1-6: configuration #1 chosen from 1 choice [ 191.505921] dvb-usb: found a 'Intel CE9500 reference design' in warm state. [ 191.523481] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer. [ 191.537924] DVB: registering new adapter (Intel CE9500 reference design) [ 191.647202] DVB: registering adapter 0 frontend 0 (Zarlink ZL10353 DVB-T)... [ 191.716542] MXL5005S: Attached at address 0xc6 [ 191.719948] dvb-usb: Intel CE9500 reference design successfully initialized and connected. [ 191.719989] usbcore: registered new interface driver dvb_usb_ce6230 So far, so good...I'm actually able to scan and find channels with this USB tuner, but once I try to actually watch a channel, it'll only give me garbage, eg. artifacts. Probably at the same time the following message is written to the host: irq 19: nobody cared (try booting with the "irqpoll" option) Pid: 0, comm: swapper Not tainted 2.6.32-ARCH #1 Call Trace: <IRQ> [<ffffffff810ab7ee>] ? __report_bad_irq+0x1e/0x90 [<ffffffff810ab9eb>] ? note_interrupt+0x18b/0x1d0 [<ffffffff810ac2fd>] ? handle_fasteoi_irq+0xcd/0xf0 [<ffffffff810153e5>] ? handle_irq+0x15/0x20 [<ffffffff81014902>] ? do_IRQ+0x62/0xe0 [<ffffffff81012a53>] ? ret_from_intr+0x0/0x11 <EOI> [<ffffffff81033d22>] ? native_safe_halt+0x2/0x10 [<ffffffffa01d2233>] ? acpi_idle_enter_c1+0x9b/0x112 [processor] [<ffffffff81285342>] ? menu_select+0x102/0x290 [<ffffffff8128422f>] ? cpuidle_idle_call+0x9f/0x170 [<ffffffff81011202>] ? cpu_idle+0xb2/0x110 [<ffffffff81516d33>] ? start_kernel+0x3c6/0x3d1 [<ffffffff81516430>] ? x86_64_start_kernel+0xfd/0x10c handlers: [<ffffffffa009a7f0>] (usb_hcd_irq+0x0/0x80 [usbcore]) Disabling IRQ #19 ...after which the PCI USB card stops working in the guest. Is this issue related to the issue the other person had? I'm more than willing to help out with testing, if you or another developer are interested in fixing this passthrough support. Thanks :) Best Regards Kenni Lund ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-29 23:00 ` Kenni Lund @ 2010-03-29 23:12 ` Alexander Graf 2010-03-29 23:47 ` Chris Wright 0 siblings, 1 reply; 39+ messages in thread From: Alexander Graf @ 2010-03-29 23:12 UTC (permalink / raw) To: Chris Wright; +Cc: kvm list, Kenni Lund On 30.03.2010, at 01:00, Kenni Lund wrote: > 2010/3/29 Alexander Graf <agraf@suse.de>: >> >> On 29.03.2010, at 19:23, Kenni Lund wrote: >> >>>>> 2010/1/9 Alexander Graf <agraf@suse.de>: >>>>>> >>>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote: >>>>>> >>>>>>> >>>>>>> I have a multifunction PCI device that I'd like to pass through to KVM. >>>>>>> In order to do that, I'm reading that the PCI memory region must be >>>>>>> 4K-page >>>>>>> aligned and the PCI memory resources itself must also be exact multiples >>>>>>> of 4K pages. >>>>>>> >>>>>>> I have added the following on my kernel command line: >>>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 >>>>>>> >>>>>>> But I don't know if it has any effect. The resources are still not >>>>>>> sized in 4K pages. Also, this seems to screw up the last device. >>>>>> >>>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation. >>>>>> Please try out if the current git head works for you. >>>>>> >>>>>> Alex-- >>>>> >>>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm 0.12.3 and I still >>>>> get the following error when trying to pass through a dedicated PCI >>>>> USB card: >>>>> >>>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size >>>>> 0x100, which is not a multiple of 4K >>>>> Error initializing device pci-assign" >>>>> >>>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I >>>>> was under the impression that this was fixed when I upgraded to >>>>> qemu-kvm 0.12.3. >>>>> >>>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there >>>> is one >>> >>> That would be highly appriciated...with the current USB support in >>> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've >>> bought two dedicated PCI USB cards for this, but none of them works >>> due to the above limitation. >>> >>> Perhaps a developer can comment on this? Are there any plans on >>> including this patch in the stable releases in the near future? >> >> Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable. >> >> I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-). > > Sure, I have compiled the current git snapshot and performed some > tests...It's at least mostly working, so I'm a bit unsure if this is a > bug related to this or to something else. Chris, any idea on this? Looks like something's going wrong with function assignment. Alex ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-29 23:12 ` Alexander Graf @ 2010-03-29 23:47 ` Chris Wright 2010-03-30 0:21 ` Kenni Lund 0 siblings, 1 reply; 39+ messages in thread From: Chris Wright @ 2010-03-29 23:47 UTC (permalink / raw) To: Alexander Graf; +Cc: Chris Wright, kvm list, Kenni Lund * Alexander Graf (agraf@suse.de) wrote: > On 30.03.2010, at 01:00, Kenni Lund wrote: > > > 2010/3/29 Alexander Graf <agraf@suse.de>: > >> > >> On 29.03.2010, at 19:23, Kenni Lund wrote: > >> > >>>>> 2010/1/9 Alexander Graf <agraf@suse.de>: > >>>>>> > >>>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote: > >>>>>> > >>>>>>> > >>>>>>> I have a multifunction PCI device that I'd like to pass through to KVM. > >>>>>>> In order to do that, I'm reading that the PCI memory region must be > >>>>>>> 4K-page > >>>>>>> aligned and the PCI memory resources itself must also be exact multiples > >>>>>>> of 4K pages. > >>>>>>> > >>>>>>> I have added the following on my kernel command line: > >>>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 > >>>>>>> > >>>>>>> But I don't know if it has any effect. The resources are still not > >>>>>>> sized in 4K pages. Also, this seems to screw up the last device. > >>>>>> > >>>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation. > >>>>>> Please try out if the current git head works for you. > >>>>>> > >>>>>> Alex-- > >>>>> > >>>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm 0.12.3 and I still > >>>>> get the following error when trying to pass through a dedicated PCI > >>>>> USB card: > >>>>> > >>>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size > >>>>> 0x100, which is not a multiple of 4K > >>>>> Error initializing device pci-assign" > >>>>> > >>>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I > >>>>> was under the impression that this was fixed when I upgraded to > >>>>> qemu-kvm 0.12.3. > >>>>> > >>>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there > >>>> is one > >>> > >>> That would be highly appriciated...with the current USB support in > >>> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've > >>> bought two dedicated PCI USB cards for this, but none of them works > >>> due to the above limitation. > >>> > >>> Perhaps a developer can comment on this? Are there any plans on > >>> including this patch in the stable releases in the near future? > >> > >> Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable. > >> > >> I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-). > > > > Sure, I have compiled the current git snapshot and performed some > > tests...It's at least mostly working, so I'm a bit unsure if this is a > > bug related to this or to something else. > > Chris, any idea on this? Looks like something's going wrong with function assignment. Hmm, one thing that sticks out to me is the debug port. Kenni, can you post full dmesg on both host and guest, nothing is obviously broken (and in fact the guest should never "see" the debug port). thanks, -chris ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-29 23:47 ` Chris Wright @ 2010-03-30 0:21 ` Kenni Lund 2010-03-30 2:08 ` Chris Wright 0 siblings, 1 reply; 39+ messages in thread From: Kenni Lund @ 2010-03-30 0:21 UTC (permalink / raw) To: Chris Wright; +Cc: Alexander Graf, kvm list 2010/3/30 Chris Wright <chrisw@redhat.com>: > * Alexander Graf (agraf@suse.de) wrote: >> On 30.03.2010, at 01:00, Kenni Lund wrote: >> >> > 2010/3/29 Alexander Graf <agraf@suse.de>: >> >> >> >> On 29.03.2010, at 19:23, Kenni Lund wrote: >> >> >> >>>>> 2010/1/9 Alexander Graf <agraf@suse.de>: >> >>>>>> >> >>>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote: >> >>>>>> >> >>>>>>> >> >>>>>>> I have a multifunction PCI device that I'd like to pass through to KVM. >> >>>>>>> In order to do that, I'm reading that the PCI memory region must be >> >>>>>>> 4K-page >> >>>>>>> aligned and the PCI memory resources itself must also be exact multiples >> >>>>>>> of 4K pages. >> >>>>>>> >> >>>>>>> I have added the following on my kernel command line: >> >>>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4 >> >>>>>>> >> >>>>>>> But I don't know if it has any effect. The resources are still not >> >>>>>>> sized in 4K pages. Also, this seems to screw up the last device. >> >>>>>> >> >>>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation. >> >>>>>> Please try out if the current git head works for you. >> >>>>>> >> >>>>>> Alex-- >> >>>>> >> >>>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm 0.12.3 and I still >> >>>>> get the following error when trying to pass through a dedicated PCI >> >>>>> USB card: >> >>>>> >> >>>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size >> >>>>> 0x100, which is not a multiple of 4K >> >>>>> Error initializing device pci-assign" >> >>>>> >> >>>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I >> >>>>> was under the impression that this was fixed when I upgraded to >> >>>>> qemu-kvm 0.12.3. >> >>>>> >> >>>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there >> >>>> is one >> >>> >> >>> That would be highly appriciated...with the current USB support in >> >>> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've >> >>> bought two dedicated PCI USB cards for this, but none of them works >> >>> due to the above limitation. >> >>> >> >>> Perhaps a developer can comment on this? Are there any plans on >> >>> including this patch in the stable releases in the near future? >> >> >> >> Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable. >> >> >> >> I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-). >> > >> > Sure, I have compiled the current git snapshot and performed some >> > tests...It's at least mostly working, so I'm a bit unsure if this is a >> > bug related to this or to something else. >> >> Chris, any idea on this? Looks like something's going wrong with function assignment. > > Hmm, one thing that sticks out to me is the debug port. Kenni, can you > post full dmesg on both host and guest, nothing is obviously broken (and > in fact the guest should never "see" the debug port). > Uploaded here: Client dmesg: http://pastebin.com/uNG4QK5j Host dmesg: http://pastebin.com/jZu3WKZW I just verified it and I do get the call trace in the host (which disables IRQ 19, used by the PCI USB card), exactly at the same second I ask the DVB-T tuner to view a channel in the guest. Thanks.. Best Regards Kenni Lund ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-30 0:21 ` Kenni Lund @ 2010-03-30 2:08 ` Chris Wright 2010-03-30 22:27 ` Kenni Lund 0 siblings, 1 reply; 39+ messages in thread From: Chris Wright @ 2010-03-30 2:08 UTC (permalink / raw) To: Kenni Lund; +Cc: Chris Wright, Alexander Graf, kvm list * Kenni Lund (kenni@kelu.dk) wrote: > Client dmesg: http://pastebin.com/uNG4QK5j > Host dmesg: http://pastebin.com/jZu3WKZW > > I just verified it and I do get the call trace in the host (which > disables IRQ 19, used by the PCI USB card), exactly at the same second It looks like IRQ 19 is shared between the ehci controller and the ivtv tuner. What do you see in /proc/interrupts on the host (before you unbind and after you bind to pci stub)? The host dmesg shows other possible devices sharing that interrupt: $ grep "IRQ 19" jZu3WKZW 482. ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19 578. ehci_hcd 0000:02:01.3: PCI INT A -> GSI 19 (level, low) -> IRQ 19 623. uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19 795. ivtv 0000:03:09.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19 814. IRQ 19/ivtv1: IRQF_DISABLED is not guaranteed on shared IRQs 955. pci-stub 0000:02:01.3: PCI INT A -> GSI 19 (level, low) -> IRQ 19 thanks, -chris ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-30 2:08 ` Chris Wright @ 2010-03-30 22:27 ` Kenni Lund 2010-03-30 22:29 ` Alexander Graf 2010-03-30 23:58 ` Chris Wright 0 siblings, 2 replies; 39+ messages in thread From: Kenni Lund @ 2010-03-30 22:27 UTC (permalink / raw) To: Chris Wright; +Cc: Alexander Graf, kvm list 2010/3/30 Chris Wright <chrisw@redhat.com>: > * Kenni Lund (kenni@kelu.dk) wrote: >> Client dmesg: http://pastebin.com/uNG4QK5j >> Host dmesg: http://pastebin.com/jZu3WKZW >> >> I just verified it and I do get the call trace in the host (which >> disables IRQ 19, used by the PCI USB card), exactly at the same second > > It looks like IRQ 19 is shared between the ehci controller and the > ivtv tuner. What do you see in /proc/interrupts on the host (before > you unbind and after you bind to pci stub)? Ahh, and even if the ivtv module is not loaded, I will still have a shared IRQ, right? I didn't see ivtv in /proc/interrupts before, as I unload the ivtv driver on boot in /etc/rc.local, before unbinding the ivtv tuner and binding it to pci stub. (the ivtv tuner is normally assigned to the same guest, but not now while testing the PCI USB card). If I don't unload (and unbind/bind) the ivtv driver/tuner on boot in /etc/local, I get the following in /proc/interrupts on a clean boot: http://pastebin.com/SFQj58LC If I now unbind and bind the PCI USB card to pci stub, I get no changes in /proc/interrupts. So I suppose I'll need to get rid of this shared IRQ before I can conclude anything on the patch in git. Hmm, is there some cleaver way of fixing this in Linux, or do I have to fix it by changing BIOS IRQ settings, disabling hardware and/or moving the hardware around in various PCI slots? Best Regards Kenni ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-30 22:27 ` Kenni Lund @ 2010-03-30 22:29 ` Alexander Graf 2010-03-30 23:52 ` Kenni Lund 2010-03-30 23:58 ` Chris Wright 1 sibling, 1 reply; 39+ messages in thread From: Alexander Graf @ 2010-03-30 22:29 UTC (permalink / raw) To: Kenni Lund; +Cc: Chris Wright, kvm list On 31.03.2010, at 00:27, Kenni Lund wrote: > 2010/3/30 Chris Wright <chrisw@redhat.com>: >> * Kenni Lund (kenni@kelu.dk) wrote: >>> Client dmesg: http://pastebin.com/uNG4QK5j >>> Host dmesg: http://pastebin.com/jZu3WKZW >>> >>> I just verified it and I do get the call trace in the host (which >>> disables IRQ 19, used by the PCI USB card), exactly at the same second >> >> It looks like IRQ 19 is shared between the ehci controller and the >> ivtv tuner. What do you see in /proc/interrupts on the host (before >> you unbind and after you bind to pci stub)? > > Ahh, and even if the ivtv module is not loaded, I will still have a > shared IRQ, right? I didn't see ivtv in /proc/interrupts before, as I > unload the ivtv driver on boot in /etc/rc.local, before unbinding the > ivtv tuner and binding it to pci stub. (the ivtv tuner is normally > assigned to the same guest, but not now while testing the PCI USB > card). > > If I don't unload (and unbind/bind) the ivtv driver/tuner on boot in > /etc/local, I get the following in /proc/interrupts on a clean boot: > http://pastebin.com/SFQj58LC > > If I now unbind and bind the PCI USB card to pci stub, I get no > changes in /proc/interrupts. > > So I suppose I'll need to get rid of this shared IRQ before I can > conclude anything on the patch in git. Hmm, is there some cleaver way > of fixing this in Linux, or do I have to fix it by changing BIOS IRQ > settings, disabling hardware and/or moving the hardware around in > various PCI slots? The easiest thing coming to mind is to unplug the ivtv card for now. It's really only to verify that the patch does something useful :-). Alex ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-30 22:29 ` Alexander Graf @ 2010-03-30 23:52 ` Kenni Lund 2010-03-31 0:59 ` Chris Wright 0 siblings, 1 reply; 39+ messages in thread From: Kenni Lund @ 2010-03-30 23:52 UTC (permalink / raw) To: Alexander Graf; +Cc: Chris Wright, kvm list 2010/3/31 Alexander Graf <agraf@suse.de>: > > On 31.03.2010, at 00:27, Kenni Lund wrote: > >> 2010/3/30 Chris Wright <chrisw@redhat.com>: >>> * Kenni Lund (kenni@kelu.dk) wrote: >>>> Client dmesg: http://pastebin.com/uNG4QK5j >>>> Host dmesg: http://pastebin.com/jZu3WKZW >>>> >>>> I just verified it and I do get the call trace in the host (which >>>> disables IRQ 19, used by the PCI USB card), exactly at the same second >>> >>> It looks like IRQ 19 is shared between the ehci controller and the >>> ivtv tuner. What do you see in /proc/interrupts on the host (before >>> you unbind and after you bind to pci stub)? >> >> Ahh, and even if the ivtv module is not loaded, I will still have a >> shared IRQ, right? I didn't see ivtv in /proc/interrupts before, as I >> unload the ivtv driver on boot in /etc/rc.local, before unbinding the >> ivtv tuner and binding it to pci stub. (the ivtv tuner is normally >> assigned to the same guest, but not now while testing the PCI USB >> card). >> >> If I don't unload (and unbind/bind) the ivtv driver/tuner on boot in >> /etc/local, I get the following in /proc/interrupts on a clean boot: >> http://pastebin.com/SFQj58LC >> >> If I now unbind and bind the PCI USB card to pci stub, I get no >> changes in /proc/interrupts. >> >> So I suppose I'll need to get rid of this shared IRQ before I can >> conclude anything on the patch in git. Hmm, is there some cleaver way >> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ >> settings, disabling hardware and/or moving the hardware around in >> various PCI slots? > > The easiest thing coming to mind is to unplug the ivtv card for now. It's really only to verify that the patch does something useful :-). This was not sufficient. Same issue with the ivtv card unplugged...if I interpret the content of /proc/interrupt correctly, the USB PCI cards uses 4 IRQs and out of these three IRQs are still shared with other components. The card uses IRQ 16, 18, 19, 20: pci-stub 0000:02:01.0: PCI INT B -> GSI 18 (level, low) -> IRQ 18 pci-stub 0000:02:01.1: PCI INT C -> GSI 16 (level, low) -> IRQ 16 pci-stub 0000:02:01.2: PCI INT D -> GSI 20 (level, low) -> IRQ 20 pci-stub 0000:02:01.3: PCI INT A -> GSI 19 (level, low) -> IRQ 19 Only IRQ 20 gets "kvm_assigned_intx_device" after unbinding+binding+qemu-kvm launch: interrupts before: http://pastebin.com/DdJEv29z interrupts after: http://pastebin.com/zasWEZsL It seems like i still have shared IRQs with the onboard USB controller as well as the PATA controller. In the BIOS i have the option of setting the IRQ of "PCI Card 1" and "PCI Card 2". I tried changing these into IRQ 10 and 11 (which are free according to /proc/interrupts), but it changed absolutely nothing in /proc/interrupts after booting(?) :-( Any kind of hint which could lead me in hte right direction, would be highly appreciated...thanks. Best Regards Kenni ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-30 23:52 ` Kenni Lund @ 2010-03-31 0:59 ` Chris Wright 0 siblings, 0 replies; 39+ messages in thread From: Chris Wright @ 2010-03-31 0:59 UTC (permalink / raw) To: Kenni Lund; +Cc: Alexander Graf, Chris Wright, kvm list * Kenni Lund (kenni@kelu.dk) wrote: > 2010/3/31 Alexander Graf <agraf@suse.de>: > > The easiest thing coming to mind is to unplug the ivtv card for now. It's really only to verify that the patch does something useful :-). > > This was not sufficient. Same issue with the ivtv card unplugged...if > I interpret the content of /proc/interrupt correctly, the USB PCI > cards uses 4 IRQs and out of these three IRQs are still shared with > other components. > > The card uses IRQ 16, 18, 19, 20: > pci-stub 0000:02:01.0: PCI INT B -> GSI 18 (level, low) -> IRQ 18 > pci-stub 0000:02:01.1: PCI INT C -> GSI 16 (level, low) -> IRQ 16 > pci-stub 0000:02:01.2: PCI INT D -> GSI 20 (level, low) -> IRQ 20 > pci-stub 0000:02:01.3: PCI INT A -> GSI 19 (level, low) -> IRQ 19 > > Only IRQ 20 gets "kvm_assigned_intx_device" after > unbinding+binding+qemu-kvm launch: What is your qemu command line? It's acting as if function .2 (one of the ochi controllers) is generating the interrupt rather than .3 (the ehci controller). And indeed, there is no handler to ack that interrupt (just the likely on-board uhci one). before: 19: 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb3, uhci_hcd:usb9 20: 0 0 0 0 IO-APIC-fasteoi ohci_hcd:usb12 after: 19: 75010 73438 76479 75074 IO-APIC-fasteoi uhci_hcd:usb9 20: 0 0 0 0 IO-APIC-fasteoi kvm_assigned_intx_device Can you unbind uhci_hcd:usb9 and then give both .2 and .3 to the guest? thanks, -chris ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-30 22:27 ` Kenni Lund 2010-03-30 22:29 ` Alexander Graf @ 2010-03-30 23:58 ` Chris Wright 2010-03-31 0:47 ` Kenni Lund 1 sibling, 1 reply; 39+ messages in thread From: Chris Wright @ 2010-03-30 23:58 UTC (permalink / raw) To: Kenni Lund; +Cc: Chris Wright, Alexander Graf, kvm list * Kenni Lund (kenni@kelu.dk) wrote: > 2010/3/30 Chris Wright <chrisw@redhat.com>: > > * Kenni Lund (kenni@kelu.dk) wrote: > >> Client dmesg: http://pastebin.com/uNG4QK5j > >> Host dmesg: http://pastebin.com/jZu3WKZW > >> > >> I just verified it and I do get the call trace in the host (which > >> disables IRQ 19, used by the PCI USB card), exactly at the same second > > > > It looks like IRQ 19 is shared between the ehci controller and the > > ivtv tuner. What do you see in /proc/interrupts on the host (before > > you unbind and after you bind to pci stub)? > > Ahh, and even if the ivtv module is not loaded, I will still have a > shared IRQ, right? I didn't see ivtv in /proc/interrupts before, as I > unload the ivtv driver on boot in /etc/rc.local, before unbinding the > ivtv tuner and binding it to pci stub. (the ivtv tuner is normally > assigned to the same guest, but not now while testing the PCI USB > card). > > If I don't unload (and unbind/bind) the ivtv driver/tuner on boot in > /etc/local, I get the following in /proc/interrupts on a clean boot: > http://pastebin.com/SFQj58LC > > If I now unbind and bind the PCI USB card to pci stub, I get no > changes in /proc/interrupts. Sorry, I meant bind to pci_stub and launch guest. IOW, you should see kvm_assigned_{msi,msix,intx}_device (from your lspci, I'd expect intx). What's odd is a device is asserting an interrupt to a line w/ no handler acking. The IRQ 19, should have kvm handling the interrupt, in which case it'd always return IRQ_HANDLED. And for the case of shared interrupts, we won't let you start the guest with an assigned device that's sharing an interrupt. IOW, we do request_irq() w/out specifying IRQF_SHARED (meaning we want an exclusive irq). > So I suppose I'll need to get rid of this shared IRQ before I can > conclude anything on the patch in git. Hmm, is there some cleaver way > of fixing this in Linux, or do I have to fix it by changing BIOS IRQ > settings, disabling hardware and/or moving the hardware around in > various PCI slots? The way I typically work around this is simply unbinding the driver from the device in the host (and thus freeing the irq). thanks, -chris ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-30 23:58 ` Chris Wright @ 2010-03-31 0:47 ` Kenni Lund 2010-03-31 1:32 ` Chris Wright 2010-03-31 11:43 ` Kenni Lund 0 siblings, 2 replies; 39+ messages in thread From: Kenni Lund @ 2010-03-31 0:47 UTC (permalink / raw) To: Chris Wright; +Cc: Alexander Graf, kvm list 2010/3/31 Chris Wright <chrisw@redhat.com>: >> So I suppose I'll need to get rid of this shared IRQ before I can >> conclude anything on the patch in git. Hmm, is there some cleaver way >> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ >> settings, disabling hardware and/or moving the hardware around in >> various PCI slots? > > The way I typically work around this is simply unbinding the driver from > the device in the host (and thus freeing the irq). Doh...anyway, I went all the way, found a USB->PS2 adaptor, disabled the onboard USB controller and PATA controller in BIOS, and now got "kvm_assigned_intx_device" for all 4 IRQs :) Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks a lot for your help...I have one more question, though: If I have two devices (like the ivtv tuner and the USB card) and they share an IRQ, if I then assign BOTH of them to the same guest, will it then work? Alexander, the patch works, I hope to see it in a stable release in the near future ;) Best Regards Kenni ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 0:47 ` Kenni Lund @ 2010-03-31 1:32 ` Chris Wright 2010-03-31 10:07 ` Kenni Lund 2010-03-31 11:43 ` Kenni Lund 1 sibling, 1 reply; 39+ messages in thread From: Chris Wright @ 2010-03-31 1:32 UTC (permalink / raw) To: Kenni Lund; +Cc: Chris Wright, Alexander Graf, kvm list * Kenni Lund (kenni@kelu.dk) wrote: > 2010/3/31 Chris Wright <chrisw@redhat.com>: > >> So I suppose I'll need to get rid of this shared IRQ before I can > >> conclude anything on the patch in git. Hmm, is there some cleaver way > >> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ > >> settings, disabling hardware and/or moving the hardware around in > >> various PCI slots? > > > > The way I typically work around this is simply unbinding the driver from > > the device in the host (and thus freeing the irq). > > Doh...anyway, I went all the way, found a USB->PS2 adaptor, disabled > the onboard USB controller and PATA controller in BIOS, and now got > "kvm_assigned_intx_device" for all 4 IRQs :) A little extreme...but hey, that works ;-) I'm still curious what's going on w/ your PCI card and the irq routing. Something is suspect. > Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks > a lot for your help...I have one more question, though: If I have two > devices (like the ivtv tuner and the USB card) and they share an IRQ, > if I then assign BOTH of them to the same guest, will it then work? No, it won't. The first one will request an exclusive interrupt, and the second one will fail its request_irq. There's two options, one is to use a different driver for kvm (if the device is new enough to handle PCI 2.3) or use MSI/MSI-X based devices (that works best ;) Neither of those options are readily available in your case. thanks, -chris ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 1:32 ` Chris Wright @ 2010-03-31 10:07 ` Kenni Lund 2010-03-31 15:15 ` Chris Wright 0 siblings, 1 reply; 39+ messages in thread From: Kenni Lund @ 2010-03-31 10:07 UTC (permalink / raw) To: Chris Wright; +Cc: Alexander Graf, kvm list 2010/3/31 Chris Wright <chrisw@redhat.com>: > * Kenni Lund (kenni@kelu.dk) wrote: >> 2010/3/31 Chris Wright <chrisw@redhat.com>: >> >> So I suppose I'll need to get rid of this shared IRQ before I can >> >> conclude anything on the patch in git. Hmm, is there some cleaver way >> >> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ >> >> settings, disabling hardware and/or moving the hardware around in >> >> various PCI slots? >> > >> > The way I typically work around this is simply unbinding the driver from >> > the device in the host (and thus freeing the irq). >> >> Doh...anyway, I went all the way, found a USB->PS2 adaptor, disabled >> the onboard USB controller and PATA controller in BIOS, and now got >> "kvm_assigned_intx_device" for all 4 IRQs :) > > A little extreme...but hey, that works ;-) I'm still curious what's > going on w/ your PCI card and the irq routing. Something is suspect. I'll not be able to do any further testing for the next two weeks, but if you want me to perform some tests, test patches etc., just write what you want me to do and I'll do it in two weeks. Best Regards Kenni ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 10:07 ` Kenni Lund @ 2010-03-31 15:15 ` Chris Wright 0 siblings, 0 replies; 39+ messages in thread From: Chris Wright @ 2010-03-31 15:15 UTC (permalink / raw) To: Kenni Lund; +Cc: Chris Wright, Alexander Graf, kvm list * Kenni Lund (kenni@kelu.dk) wrote: > 2010/3/31 Chris Wright <chrisw@redhat.com>: > > * Kenni Lund (kenni@kelu.dk) wrote: > >> 2010/3/31 Chris Wright <chrisw@redhat.com>: > >> >> So I suppose I'll need to get rid of this shared IRQ before I can > >> >> conclude anything on the patch in git. Hmm, is there some cleaver way > >> >> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ > >> >> settings, disabling hardware and/or moving the hardware around in > >> >> various PCI slots? > >> > > >> > The way I typically work around this is simply unbinding the driver from > >> > the device in the host (and thus freeing the irq). > >> > >> Doh...anyway, I went all the way, found a USB->PS2 adaptor, disabled > >> the onboard USB controller and PATA controller in BIOS, and now got > >> "kvm_assigned_intx_device" for all 4 IRQs :) > > > > A little extreme...but hey, that works ;-) I'm still curious what's > > going on w/ your PCI card and the irq routing. Something is suspect. > > I'll not be able to do any further testing for the next two weeks, but > if you want me to perform some tests, test patches etc., just write > what you want me to do and I'll do it in two weeks. Thanks for your willingness to do some tests. I'll poke around a bit and see if I can come up w/ something useful for you to try out. thanks, -chris ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 0:47 ` Kenni Lund 2010-03-31 1:32 ` Chris Wright @ 2010-03-31 11:43 ` Kenni Lund 2010-03-31 12:24 ` Alexander Graf 1 sibling, 1 reply; 39+ messages in thread From: Kenni Lund @ 2010-03-31 11:43 UTC (permalink / raw) To: Chris Wright; +Cc: Alexander Graf, kvm list 2010/3/31 Kenni Lund <kenni@kelu.dk>: > Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks > a lot for your help...I have one more question, though: If I have two > devices (like the ivtv tuner and the USB card) and they share an IRQ, > if I then assign BOTH of them to the same guest, will it then work? > > Alexander, the patch works, I hope to see it in a stable release in > the near future ;) Unfortunately, I have a correction to this :( It _almost_ works. I had some video/audio artifacts which I though was caused by bad reception, but after switching the DVB-T tuner back and forth between the PCI USB card and a laptop, it got clear to me, that this was a passthrough issue. I get no errors in dmesg on the host or in the guest. I recorded a short videoclip which illustrates the issue: https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en Best Regards Kenni ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 11:43 ` Kenni Lund @ 2010-03-31 12:24 ` Alexander Graf 2010-03-31 13:04 ` Kenni Lund 2010-03-31 15:18 ` Chris Wright 0 siblings, 2 replies; 39+ messages in thread From: Alexander Graf @ 2010-03-31 12:24 UTC (permalink / raw) To: Kenni Lund; +Cc: Chris Wright, kvm list Kenni Lund wrote: > 2010/3/31 Kenni Lund <kenni@kelu.dk>: > >> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks >> a lot for your help...I have one more question, though: If I have two >> devices (like the ivtv tuner and the USB card) and they share an IRQ, >> if I then assign BOTH of them to the same guest, will it then work? >> >> Alexander, the patch works, I hope to see it in a stable release in >> the near future ;) >> > > Unfortunately, I have a correction to this :( It _almost_ works. I had > some video/audio artifacts which I though was caused by bad reception, > but after switching the DVB-T tuner back and forth between the PCI USB > card and a laptop, it got clear to me, that this was a passthrough > issue. I get no errors in dmesg on the host or in the guest. > > I recorded a short videoclip which illustrates the issue: > https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en > Hrm, I'm not sure these would be related to the small BAR region patch. It looks more like a timing issue. Alex ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 12:24 ` Alexander Graf @ 2010-03-31 13:04 ` Kenni Lund 2010-03-31 15:18 ` Chris Wright 1 sibling, 0 replies; 39+ messages in thread From: Kenni Lund @ 2010-03-31 13:04 UTC (permalink / raw) To: Alexander Graf; +Cc: Chris Wright, kvm list 2010/3/31 Alexander Graf <agraf@suse.de>: > Kenni Lund wrote: >> 2010/3/31 Kenni Lund <kenni@kelu.dk>: >> >>> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks >>> a lot for your help...I have one more question, though: If I have two >>> devices (like the ivtv tuner and the USB card) and they share an IRQ, >>> if I then assign BOTH of them to the same guest, will it then work? >>> >>> Alexander, the patch works, I hope to see it in a stable release in >>> the near future ;) >>> >> >> Unfortunately, I have a correction to this :( It _almost_ works. I had >> some video/audio artifacts which I though was caused by bad reception, >> but after switching the DVB-T tuner back and forth between the PCI USB >> card and a laptop, it got clear to me, that this was a passthrough >> issue. I get no errors in dmesg on the host or in the guest. >> >> I recorded a short videoclip which illustrates the issue: >> https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en >> > > Hrm, I'm not sure these would be related to the small BAR region patch. > It looks more like a timing issue. For what it's worth, I quickly tried to passthrough one of the onboard USB controllers instead (Intel Corporation 82801JD/DO (ICH10 Family), which is also affected by the 4k limitation). It improves the artifact problems, bringing the artifacts down to one artifact every 5 seconds. I'll be back in ~1½ week, please tell me if you want me to test something when I get back. Thanks for your help :) Best Regards Kenni ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 12:24 ` Alexander Graf 2010-03-31 13:04 ` Kenni Lund @ 2010-03-31 15:18 ` Chris Wright 2010-03-31 15:23 ` Alexander Graf 2010-04-07 5:52 ` Avi Kivity 1 sibling, 2 replies; 39+ messages in thread From: Chris Wright @ 2010-03-31 15:18 UTC (permalink / raw) To: Alexander Graf; +Cc: Kenni Lund, Chris Wright, kvm list * Alexander Graf (agraf@suse.de) wrote: > Kenni Lund wrote: > > 2010/3/31 Kenni Lund <kenni@kelu.dk>: > > > >> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks > >> a lot for your help...I have one more question, though: If I have two > >> devices (like the ivtv tuner and the USB card) and they share an IRQ, > >> if I then assign BOTH of them to the same guest, will it then work? > >> > >> Alexander, the patch works, I hope to see it in a stable release in > >> the near future ;) > >> > > > > Unfortunately, I have a correction to this :( It _almost_ works. I had > > some video/audio artifacts which I though was caused by bad reception, > > but after switching the DVB-T tuner back and forth between the PCI USB > > card and a laptop, it got clear to me, that this was a passthrough > > issue. I get no errors in dmesg on the host or in the guest. > > > > I recorded a short videoclip which illustrates the issue: > > https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en > > > > Hrm, I'm not sure these would be related to the small BAR region patch. > It looks more like a timing issue. small BAR == slow path == timing issue? thanks, -chris ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 15:18 ` Chris Wright @ 2010-03-31 15:23 ` Alexander Graf 2010-04-07 5:52 ` Avi Kivity 1 sibling, 0 replies; 39+ messages in thread From: Alexander Graf @ 2010-03-31 15:23 UTC (permalink / raw) To: Chris Wright; +Cc: Kenni Lund, kvm list Chris Wright wrote: > * Alexander Graf (agraf@suse.de) wrote: > >> Kenni Lund wrote: >> >>> 2010/3/31 Kenni Lund <kenni@kelu.dk>: >>> >>> >>>> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks >>>> a lot for your help...I have one more question, though: If I have two >>>> devices (like the ivtv tuner and the USB card) and they share an IRQ, >>>> if I then assign BOTH of them to the same guest, will it then work? >>>> >>>> Alexander, the patch works, I hope to see it in a stable release in >>>> the near future ;) >>>> >>>> >>> Unfortunately, I have a correction to this :( It _almost_ works. I had >>> some video/audio artifacts which I though was caused by bad reception, >>> but after switching the DVB-T tuner back and forth between the PCI USB >>> card and a laptop, it got clear to me, that this was a passthrough >>> issue. I get no errors in dmesg on the host or in the guest. >>> >>> I recorded a short videoclip which illustrates the issue: >>> https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en >>> >>> >> Hrm, I'm not sure these would be related to the small BAR region patch. >> It looks more like a timing issue. >> > > small BAR == slow path == timing issue? > Could be, yeah. The only chance I see to avoid that is to have in-kernel small BAR code that would issue the MMIO accesses without the userspace churn. One thing I could imagine would be < PAGE_SIZE memory slots. Then userspace just maps the 4K to the BAR, but passes only a part of it on as slot. Alex ^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: PCI passthrough resource remapping 2010-03-31 15:18 ` Chris Wright 2010-03-31 15:23 ` Alexander Graf @ 2010-04-07 5:52 ` Avi Kivity 1 sibling, 0 replies; 39+ messages in thread From: Avi Kivity @ 2010-04-07 5:52 UTC (permalink / raw) To: Chris Wright; +Cc: Alexander Graf, Kenni Lund, kvm list On 03/31/2010 06:18 PM, Chris Wright wrote: > >> Hrm, I'm not sure these would be related to the small BAR region patch. >> It looks more like a timing issue. >> > small BAR == slow path == timing issue? > Would be interesting to verify using perf with the 'kvm:kvm_mmio' software event, see how many happen per second. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. ^ permalink raw reply [flat|nested] 39+ messages in thread
end of thread, other threads:[~2010-04-07 5:53 UTC | newest] Thread overview: 39+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-01-09 2:45 PCI passthrough resource remapping Ryan C. Underwood 2010-01-09 3:22 ` Alexander Graf 2010-01-10 21:53 ` Ryan C. Underwood 2010-01-10 22:15 ` Ryan C. Underwood 2010-01-14 13:59 ` Avi Kivity 2010-01-14 15:26 ` Ryan C. Underwood 2010-01-14 15:34 ` Avi Kivity 2010-01-14 15:47 ` Michael S. Tsirkin 2010-01-14 15:54 ` Avi Kivity 2010-01-14 18:31 ` Ryan C. Underwood 2010-01-14 19:09 ` Avi Kivity 2010-01-14 19:34 ` Ryan C. Underwood 2010-01-16 9:23 ` Avi Kivity 2010-01-15 13:11 ` Pasi Kärkkäinen 2010-01-15 13:15 ` Alexander Graf 2010-03-26 2:37 ` Kenni Lund 2010-03-26 3:00 ` Brian Jackson 2010-03-29 17:23 ` Kenni Lund 2010-03-29 19:17 ` Alexander Graf 2010-03-29 23:00 ` Kenni Lund 2010-03-29 23:12 ` Alexander Graf 2010-03-29 23:47 ` Chris Wright 2010-03-30 0:21 ` Kenni Lund 2010-03-30 2:08 ` Chris Wright 2010-03-30 22:27 ` Kenni Lund 2010-03-30 22:29 ` Alexander Graf 2010-03-30 23:52 ` Kenni Lund 2010-03-31 0:59 ` Chris Wright 2010-03-30 23:58 ` Chris Wright 2010-03-31 0:47 ` Kenni Lund 2010-03-31 1:32 ` Chris Wright 2010-03-31 10:07 ` Kenni Lund 2010-03-31 15:15 ` Chris Wright 2010-03-31 11:43 ` Kenni Lund 2010-03-31 12:24 ` Alexander Graf 2010-03-31 13:04 ` Kenni Lund 2010-03-31 15:18 ` Chris Wright 2010-03-31 15:23 ` Alexander Graf 2010-04-07 5:52 ` Avi Kivity
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.