PCI passthrough resource remapping

PCI passthrough resource remapping

[Ryan C. Underwood]

I have a multifunction PCI device that I'd like to pass through to KVM.
In order to do that, I'm reading that the PCI memory region must be 4K-page
aligned and the PCI memory resources itself must also be exact multiples
of 4K pages.

I have added the following on my kernel command line:
reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4

But I don't know if it has any effect.  The resources are still not
sized in 4K pages.  Also, this seems to screw up the last device.

Am I doing the right thing.  I am on amd64 and using the latest qemu-kvm
release 0.12.1.2 with kernel 2.6.31.  It seems very hard to find
documentation on the kernel parameters related to pci-passthrough.


08:09.0 0c00: 1180:0832 (rev 05) (prog-if 10)
        Subsystem: 103c:30cd
        Flags: bus master, medium devsel, latency 32, IRQ 16
        Memory at f4400000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: <access denied>
        Kernel driver in use: ohci1394
        Kernel modules: firewire-ohci, ohci1394

08:09.1 0805: 1180:0822 (rev 22)
        Subsystem: 103c:30cd
        Flags: bus master, medium devsel, latency 32, IRQ 18
        Memory at f4400800 (32-bit, non-prefetchable) [size=256]
        Capabilities: <access denied>
        Kernel driver in use: sdhci-pci
        Kernel modules: sdhci-pci

08:09.2 0880: 1180:0843 (rev 12)
        Subsystem: 103c:30cd
        Flags: bus master, medium devsel, latency 32, IRQ 10
        Memory at f4400c00 (32-bit, non-prefetchable) [size=256]
        Capabilities: <access denied>
        Kernel driver in use: ricoh-mmc
        Kernel modules: ricoh_mmc

08:09.3 0880: 1180:0592 (rev 12)
        Subsystem: 103c:30cd
        Flags: bus master, medium devsel, latency 32, IRQ 10
        Memory at f4401000 (32-bit, non-prefetchable) [size=256]
        Capabilities: <access denied>

08:09.4 0880: 1180:0852 (rev ff) (prog-if ff)
        !!! Unknown header type 7f


-- 
Ryan C. Underwood, <nemesis@icequake.net>

Re: PCI passthrough resource remapping

[Alexander Graf]

On 09.01.2010, at 03:45, Ryan C. Underwood wrote:

> 
> I have a multifunction PCI device that I'd like to pass through to KVM.
> In order to do that, I'm reading that the PCI memory region must be 4K-page
> aligned and the PCI memory resources itself must also be exact multiples
> of 4K pages.
> 
> I have added the following on my kernel command line:
> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
> 
> But I don't know if it has any effect.  The resources are still not
> sized in 4K pages.  Also, this seems to screw up the last device.

I submitted a patch to qemu-kvm recently that got rid of that limitation. Please try out if the current git head works for you.

Alex

Re: PCI passthrough resource remapping

[Ryan C. Underwood]

> > I have a multifunction PCI device that I'd like to pass through to KVM.
> > In order to do that, I'm reading that the PCI memory region must be 4K-page
> > aligned and the PCI memory resources itself must also be exact multiples
> > of 4K pages.
> > 
> > I have added the following on my kernel command line:
> > reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
> > 
> > But I don't know if it has any effect.  The resources are still not
> > sized in 4K pages.  Also, this seems to screw up the last device.
> 
> I submitted a patch to qemu-kvm recently that got rid of that limitation. Please try out if the current git head works for you.

This works around that particular limitation, but now it looks like I
have to have VT-D for this to work at all.  (It seems so hard to find
documentation about PCI passthrough, probably because things are
changing all the time.)

device: 08:09.0: driver="pci-assign" host="08:09.0"
device: 08:09.1: driver="pci-assign" host="08:09.1"
device: 08:09.2: driver="pci-assign" host="08:09.2"
device: 08:09.3: driver="pci-assign" host="08:09.3"
device: 08:09.4: driver="pci-assign" host="08:09.4"
PCI region 0 at address 0xf4400000 has size 0x800, which is not a multiple of 4K. You might experience some performance hit due to that.
No IOMMU found.  Unable to assign device "08:09.0"
Failed to deassign device "08:09.0" : Invalid argument
Error initializing device pci-assign

-- 
Ryan C. Underwood, <nemesis@icequake.net>

Re: PCI passthrough resource remapping

[Ryan C. Underwood]

I guess I'll run the things I've found by the list to see if I'm off
track or not.

There is this patch:
http://marc.info/?l=xen-devel&m=124748015304566&w=4

which would seem to be related to what I'm doing, trying to pass through
a multifunction device (a Ricoh firewire controller which has digital
media card reader subfunctions).

Then there is this patch:
http://patchwork.kernel.org/patch/20195/

Which is related to PCI passthrough without VT-d, but I have no idea if
this patch is necessary or sufficient.

Also, just for further complication, the Ricoh chip does not support
MSI and shares an IRQ on the system board with the USB host controller.
I have rebound the USB host controller to pci-stub, but I'm not sure if
that totally takes care of the IRQ-sharing-without-MSI issue.

-- 
Ryan C. Underwood, <nemesis@icequake.net>

Re: PCI passthrough resource remapping

[Avi Kivity]

On 01/11/2010 12:15 AM, Ryan C. Underwood wrote:
> I guess I'll run the things I've found by the list to see if I'm off
> track or not.
>
> There is this patch:
> http://marc.info/?l=xen-devel&m=124748015304566&w=4
>
> which would seem to be related to what I'm doing, trying to pass through
> a multifunction device (a Ricoh firewire controller which has digital
> media card reader subfunctions).
>
> Then there is this patch:
> http://patchwork.kernel.org/patch/20195/
>
> Which is related to PCI passthrough without VT-d, but I have no idea if
> this patch is necessary or sufficient.
>    

That effort has been discontinues.

> Also, just for further complication, the Ricoh chip does not support
> MSI and shares an IRQ on the system board with the USB host controller.
> I have rebound the USB host controller to pci-stub, but I'm not sure if
> that totally takes care of the IRQ-sharing-without-MSI issue.
>    

Can you post lspci -vv output for that card?  If it is pci 2.3 compliant 
we might be able to handle the sharing.

-- 
error compiling committee.c: too many arguments to function

Re: PCI passthrough resource remapping

[Ryan C. Underwood]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]


On Thu, Jan 14, 2010 at 03:59:53PM +0200, Avi Kivity wrote:
> 
> >Also, just for further complication, the Ricoh chip does not support
> >MSI and shares an IRQ on the system board with the USB host controller.
> >I have rebound the USB host controller to pci-stub, but I'm not sure if
> >that totally takes care of the IRQ-sharing-without-MSI issue.
> 
> Can you post lspci -vv output for that card?  If it is pci 2.3
> compliant we might be able to handle the sharing.

It claims to be PCI 3.0 compliant.  But I don't see any mention MSI
anywhere in the specification:
http://www.aeneas.com.cn/PDF/Ricoh/2005/R5C832E1%5B1%5D.00.pdf

Attached is lspci -vv, thanks.

-- 
Ryan C. Underwood, <nemesis@icequake.net>

[-- Attachment #2: lspci.txt --]
[-- Type: text/plain, Size: 14993 bytes --]

00:00.0 Host bridge: Intel Corporation Mobile PM965/GM965/GL960 Memory Controller Hub (rev 0c)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: <access denied>
	Kernel driver in use: agpgart-intel
	Kernel modules: intel-agp

00:02.0 VGA compatible controller: Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller (rev 0c)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 30
	Region 0: Memory at f4000000 (64-bit, non-prefetchable) [size=1M]
	Region 2: Memory at d0000000 (64-bit, prefetchable) [size=256M]
	Region 4: I/O ports at 1800 [size=8]
	Capabilities: <access denied>
	Kernel driver in use: i915
	Kernel modules: i915

00:02.1 Display controller: Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller (rev 0c)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Region 0: Memory at f4100000 (64-bit, non-prefetchable) [size=1M]
	Capabilities: <access denied>

00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #4 (rev 03)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 16
	Region 4: I/O ports at 1840 [size=32]
	Kernel driver in use: uhci_hcd

00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #5 (rev 03)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 21
	Region 4: I/O ports at 1860 [size=32]
	Kernel driver in use: uhci_hcd

00:1a.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #2 (rev 03) (prog-if 20)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin C routed to IRQ 18
	Region 0: Memory at f4705c00 (32-bit, non-prefetchable) [size=1K]
	Capabilities: <access denied>
	Kernel driver in use: ehci_hcd

00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 03)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 22
	Region 0: Memory at f4500000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: <access denied>
	Kernel driver in use: HDA Intel
	Kernel modules: snd-hda-intel

00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 (rev 03)
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=02, subordinate=03, sec-latency=0
	I/O behind bridge: 00002000-00002fff
	Memory behind bridge: f2000000-f3ffffff
	Prefetchable memory behind bridge: 00000000f0000000-00000000f1ffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: <access denied>
	Kernel driver in use: pcieport-driver
	Kernel modules: shpchp

00:1c.1 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 2 (rev 03)
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: f4200000-f42fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: <access denied>
	Kernel driver in use: pcieport-driver
	Kernel modules: shpchp

00:1c.2 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 3 (rev 03)
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: <access denied>
	Kernel driver in use: pcieport-driver
	Kernel modules: shpchp

00:1c.3 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 4 (rev 03)
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=07, subordinate=07, sec-latency=0
	Memory behind bridge: f4300000-f43fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: <access denied>
	Kernel driver in use: pcieport-driver
	Kernel modules: shpchp

00:1d.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #1 (rev 03)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 23
	Region 4: I/O ports at 1880 [size=32]
	Kernel driver in use: uhci_hcd

00:1d.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #2 (rev 03)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 19
	Region 4: I/O ports at 18a0 [size=32]
	Kernel driver in use: uhci_hcd

00:1d.2 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #3 (rev 03)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin C routed to IRQ 18
	Region 4: I/O ports at 18c0 [size=32]
	Kernel driver in use: uhci_hcd

00:1d.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #1 (rev 03) (prog-if 20)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 23
	Region 0: Memory at f4706000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: <access denied>
	Kernel driver in use: ehci_hcd

00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev f3) (prog-if 01)
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Bus: primary=00, secondary=08, subordinate=08, sec-latency=32
	Memory behind bridge: f4400000-f44fffff
	Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: <access denied>

00:1f.0 ISA bridge: Intel Corporation 82801HEM (ICH8M) LPC Interface Controller (rev 03)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: <access denied>
	Kernel modules: iTCO_wdt

00:1f.1 IDE interface: Intel Corporation 82801HBM/HEM (ICH8M/ICH8M-E) IDE Controller (rev 03) (prog-if 8a [Master SecP PriP])
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 19
	Region 0: I/O ports at 01f0 [size=8]
	Region 1: I/O ports at 03f4 [size=1]
	Region 2: I/O ports at 0170 [size=8]
	Region 3: I/O ports at 0374 [size=1]
	Region 4: I/O ports at 1830 [size=16]
	Kernel driver in use: ata_piix

00:1f.2 SATA controller: Intel Corporation 82801HBM/HEM (ICH8M/ICH8M-E) SATA AHCI Controller (rev 03) (prog-if 01)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 28
	Region 0: I/O ports at 18f8 [size=8]
	Region 1: I/O ports at 18ec [size=4]
	Region 2: I/O ports at 18f0 [size=8]
	Region 3: I/O ports at 18e8 [size=4]
	Region 4: I/O ports at 1c00 [size=32]
	Region 5: Memory at f4705000 (32-bit, non-prefetchable) [size=2K]
	Capabilities: <access denied>
	Kernel driver in use: ahci

00:1f.3 SMBus: Intel Corporation 82801H (ICH8 Family) SMBus Controller (rev 03)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin C routed to IRQ 11
	Region 0: Memory at c0000000 (32-bit, non-prefetchable) [size=256]
	Region 4: I/O ports at 1c20 [size=32]
	Kernel modules: i2c-i801

07:00.0 Network controller: Intel Corporation PRO/Wireless 4965 AG or AGN [Kedron] Network Connection (rev 61)
	Subsystem: Intel Corporation Device 1100
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 29
	Region 0: Memory at f4300000 (64-bit, non-prefetchable) [size=8K]
	Capabilities: <access denied>
	Kernel driver in use: iwlagn
	Kernel modules: iwlagn

08:09.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 05) (prog-if 10)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 64 (500ns min, 1000ns max)
	Interrupt: pin A routed to IRQ 16
	Region 0: [virtual] Memory at f4400000 (32-bit, non-prefetchable) [size=2K]
	Capabilities: <access denied>
	Kernel driver in use: pci-stub
	Kernel modules: firewire-ohci, ohci1394

08:09.1 SD Host controller: Ricoh Co Ltd R5C822 SD/SDIO/MMC/MS/MSPro Host Adapter (rev 22)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin B routed to IRQ 18
	Region 0: [virtual] Memory at f4400800 (32-bit, non-prefetchable) [disabled] [size=256]
	Capabilities: <access denied>
	Kernel driver in use: pci-stub
	Kernel modules: sdhci-pci

08:09.2 System peripheral: Ricoh Co Ltd R5C843 MMC Host Controller (rev 12)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin B routed to IRQ 10
	Region 0: [virtual] Memory at f4400c00 (32-bit, non-prefetchable) [disabled] [size=256]
	Capabilities: <access denied>
	Kernel driver in use: pci-stub
	Kernel modules: ricoh_mmc

08:09.3 System peripheral: Ricoh Co Ltd R5C592 Memory Stick Bus Host Adapter (rev 12)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin B routed to IRQ 10
	Region 0: [virtual] Memory at f4401000 (32-bit, non-prefetchable) [disabled] [size=256]
	Capabilities: <access denied>
	Kernel driver in use: pci-stub

08:09.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev 12)
	Subsystem: Hewlett-Packard Company Device 30cd
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 248, Cache Line Size: 1020 bytes
	Interrupt: pin B routed to IRQ 10
	Region 0: Memory at f4401400 (32-bit, non-prefetchable) [size=256]
	Capabilities: <access denied>
	Kernel driver in use: pci-stub

Re: PCI passthrough resource remapping

[Avi Kivity]

On 01/14/2010 05:26 PM, Ryan C. Underwood wrote:
> On Thu, Jan 14, 2010 at 03:59:53PM +0200, Avi Kivity wrote:
>    
>> >  
>>      
>>> >  >Also, just for further complication, the Ricoh chip does not support
>>> >  >MSI and shares an IRQ on the system board with the USB host controller.
>>> >  >I have rebound the USB host controller to pci-stub, but I'm not sure if
>>> >  >that totally takes care of the IRQ-sharing-without-MSI issue.
>>>        
>> >  
>> >  Can you post lspci -vv output for that card?  If it is pci 2.3
>> >  compliant we might be able to handle the sharing.
>>      
> It claims to be PCI 3.0 compliant.  But I don't see any mention MSI
> anywhere in the specification:
> http://www.aeneas.com.cn/PDF/Ricoh/2005/R5C832E1%5B1%5D.00.pdf
>
> Attached is lspci -vv, thanks.
>
>    
>
> 08:09.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 05) (prog-if 10)
> 	Subsystem: Hewlett-Packard Company Device 30cd
> 	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx-
> 	Latency: 64 (500ns min, 1000ns max)
> 	Interrupt: pin A routed to IRQ 16
> 	Region 0: [virtual] Memory at f4400000 (32-bit, non-prefetchable) [size=2K]
> 	Capabilities:<access denied>
> 	Kernel driver in use: pci-stub
> 	Kernel modules: firewire-ohci, ohci1394
>
>    

Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant?

> 08:09.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev 12)
> 	Subsystem: Hewlett-Packard Company Device 30cd
> 	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx-
> 	Latency: 248, Cache Line Size: 1020 bytes
> 	Interrupt: pin B routed to IRQ 10
> 	Region 0: Memory at f4401400 (32-bit, non-prefetchable) [size=256]
> 	Capabilities:<access denied>
> 	Kernel driver in use: pci-stub
>    

But another function has this feature?

-- 
error compiling committee.c: too many arguments to function

Re: PCI passthrough resource remapping

[Michael S. Tsirkin]

On Thu, Jan 14, 2010 at 05:34:42PM +0200, Avi Kivity wrote:
> On 01/14/2010 05:26 PM, Ryan C. Underwood wrote:
>> On Thu, Jan 14, 2010 at 03:59:53PM +0200, Avi Kivity wrote:
>>    
>>> >       
>>>> >  >Also, just for further complication, the Ricoh chip does not support
>>>> >  >MSI and shares an IRQ on the system board with the USB host controller.
>>>> >  >I have rebound the USB host controller to pci-stub, but I'm not sure if
>>>> >  >that totally takes care of the IRQ-sharing-without-MSI issue.
>>>>        
>>> >  >  Can you post lspci -vv output for that card?  If it is pci 2.3
>>> >  compliant we might be able to handle the sharing.
>>>      
>> It claims to be PCI 3.0 compliant.  But I don't see any mention MSI
>> anywhere in the specification:
>> http://www.aeneas.com.cn/PDF/Ricoh/2005/R5C832E1%5B1%5D.00.pdf
>>
>> Attached is lspci -vv, thanks.
>>
>>    
>>
>> 08:09.0 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 IEEE 1394 Controller (rev 05) (prog-if 10)
>> 	Subsystem: Hewlett-Packard Company Device 30cd
>> 	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx-
>> 	Latency: 64 (500ns min, 1000ns max)
>> 	Interrupt: pin A routed to IRQ 16
>> 	Region 0: [virtual] Memory at f4400000 (32-bit, non-prefetchable) [size=2K]
>> 	Capabilities:<access denied>
>> 	Kernel driver in use: pci-stub
>> 	Kernel modules: firewire-ohci, ohci1394
>>
>>    
>
> Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant?

No it doesn't, just that interrupt disable bit is not set.

>> 08:09.4 System peripheral: Ricoh Co Ltd xD-Picture Card Controller (rev 12)
>> 	Subsystem: Hewlett-Packard Company Device 30cd
>> 	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
>> 	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium>TAbort-<TAbort-<MAbort->SERR-<PERR- INTx-
>> 	Latency: 248, Cache Line Size: 1020 bytes
>> 	Interrupt: pin B routed to IRQ 10
>> 	Region 0: Memory at f4401400 (32-bit, non-prefetchable) [size=256]
>> 	Capabilities:<access denied>
>> 	Kernel driver in use: pci-stub
>>    
>
> But another function has this feature?

Another function has this bit set.

> -- 
> error compiling committee.c: too many arguments to function

Re: PCI passthrough resource remapping

[Avi Kivity]

On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote:
>
>> Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant?
>>      
> No it doesn't, just that interrupt disable bit is not set.
>    

Thanks.  Ryan, while kvm doesn't support assigning a device with shared 
interrupts now, in the future it will likely be possible to share it.  
You'll still need an iommu.

-- 
error compiling committee.c: too many arguments to function

Re: PCI passthrough resource remapping

[Ryan C. Underwood]

On Thu, Jan 14, 2010 at 05:54:51PM +0200, Avi Kivity wrote:
> On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote:
> >
> >>Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant?
> >No it doesn't, just that interrupt disable bit is not set.
> 
> Thanks.  Ryan, while kvm doesn't support assigning a device with
> shared interrupts now, in the future it will likely be possible to
> share it.  You'll still need an iommu.

No IOMMU on this machine and this is all integrated hardware.

This IOMMU requirement seems so strange.  I used to pass through PCI
devices ages ago when using the DOSEMU emulator.  It emulated PCI BIOS
functions and mapped the PCI config space and memory regions into the
emulator process.  The device interrupt was grabbed and handled in the
emulator's kernel support, waking up the emulator when an interrupt came
in.

I don't really know anything about kvm internals, but I'd like to
understand more about the particulars of the IOMMU requirement if you
don't mind.

-- 
Ryan C. Underwood, <nemesis@icequake.net>

Re: PCI passthrough resource remapping

[Avi Kivity]

On 01/14/2010 08:31 PM, Ryan C. Underwood wrote:
>
> On Thu, Jan 14, 2010 at 05:54:51PM +0200, Avi Kivity wrote:
>    
>> On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote:
>>      
>>>        
>>>> Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant?
>>>>          
>>> No it doesn't, just that interrupt disable bit is not set.
>>>        
>> Thanks.  Ryan, while kvm doesn't support assigning a device with
>> shared interrupts now, in the future it will likely be possible to
>> share it.  You'll still need an iommu.
>>      
> No IOMMU on this machine and this is all integrated hardware.
>
> This IOMMU requirement seems so strange.  I used to pass through PCI
> devices ages ago when using the DOSEMU emulator.  It emulated PCI BIOS
> functions and mapped the PCI config space and memory regions into the
> emulator process.  The device interrupt was grabbed and handled in the
> emulator's kernel support, waking up the emulator when an interrupt came
> in.
>
> I don't really know anything about kvm internals, but I'd like to
> understand more about the particulars of the IOMMU requirement if you
> don't mind.
>    

PCI cards can access system memory directly.  If you assign a card to a 
guest, the guest will program the card to transfer data to system memory 
using guest addresses; since guest addresses don't correspond to host 
addresses, memory corruption will ensue.

Not sure about your experience with DOSEMU; maybe these devices were not 
dma capable.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: PCI passthrough resource remapping

[Ryan C. Underwood]

On Thu, Jan 14, 2010 at 09:09:32PM +0200, Avi Kivity wrote:
> 
> PCI cards can access system memory directly.  If you assign a card
> to a guest, the guest will program the card to transfer data to
> system memory using guest addresses; since guest addresses don't
> correspond to host addresses, memory corruption will ensue.

I see, so the only way to fix this would be either with a special guest
driver for the device that does not perform DMA, or if that is
impossible (due to no docs), to trap and rewrite any command writes to
the device's MMIO region that reference a DMA write target buffer.

Forgive my ignorance, but is it possible that the latter is already
possible with qemu-kvm (somewhat like hardware memory breakpoints in
Soft-ICE)?

If qemu-kvm can be made to break and log on PCI memory accesses, I would
then hack around the safety limitations, assuming that's all they are,
and analyze the PCI writes one by one to find the cases where a physical
address is passed to the card.

Then I would perform the IOMMU translation myself in software whenever a
physmem address shows up in the command stream.  (Somewhat like the
security validation of a 3D graphics card command stream in the DRM.)

> Not sure about your experience with DOSEMU; maybe these devices were
> not dma capable.

Well, more likely the MS-DOS drivers were simply not using PCI DMA
transfers, since there is very little point to add such complexity to
the driver in a non-multitasking OS.  Of course that makes perfect sense
now!

-- 
Ryan C. Underwood, <nemesis@icequake.net>

Re: PCI passthrough resource remapping

[Pasi Kärkkäinen]

On Thu, Jan 14, 2010 at 12:31:32PM -0600, Ryan C. Underwood wrote:
> 
> 
> On Thu, Jan 14, 2010 at 05:54:51PM +0200, Avi Kivity wrote:
> > On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote:
> > >
> > >>Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant?
> > >No it doesn't, just that interrupt disable bit is not set.
> > 
> > Thanks.  Ryan, while kvm doesn't support assigning a device with
> > shared interrupts now, in the future it will likely be possible to
> > share it.  You'll still need an iommu.
> 
> No IOMMU on this machine and this is all integrated hardware.
> 
> This IOMMU requirement seems so strange.  I used to pass through PCI
> devices ages ago when using the DOSEMU emulator.  It emulated PCI BIOS
> functions and mapped the PCI config space and memory regions into the
> emulator process.  The device interrupt was grabbed and handled in the
> emulator's kernel support, waking up the emulator when an interrupt came
> in.
> 
> I don't really know anything about kvm internals, but I'd like to
> understand more about the particulars of the IOMMU requirement if you
> don't mind.
> 

Xen supports PCI passthrough to PV guests without IOMMU. This can create
security problems, since the guests get DMA access to physical hardware,
but that's usually OK in the situations where you want to use PCI
passthrough on your desktop or on your development box.

-- Pasi

Re: PCI passthrough resource remapping

[Alexander Graf]

On 15.01.2010, at 14:11, Pasi Kärkkäinen wrote:

> On Thu, Jan 14, 2010 at 12:31:32PM -0600, Ryan C. Underwood wrote:
>> 
>> 
>> On Thu, Jan 14, 2010 at 05:54:51PM +0200, Avi Kivity wrote:
>>> On 01/14/2010 05:47 PM, Michael S. Tsirkin wrote:
>>>> 
>>>>> Michael, I think 'DisINTx-' means the device is not PCI 2.3 compliant?
>>>> No it doesn't, just that interrupt disable bit is not set.
>>> 
>>> Thanks.  Ryan, while kvm doesn't support assigning a device with
>>> shared interrupts now, in the future it will likely be possible to
>>> share it.  You'll still need an iommu.
>> 
>> No IOMMU on this machine and this is all integrated hardware.
>> 
>> This IOMMU requirement seems so strange.  I used to pass through PCI
>> devices ages ago when using the DOSEMU emulator.  It emulated PCI BIOS
>> functions and mapped the PCI config space and memory regions into the
>> emulator process.  The device interrupt was grabbed and handled in the
>> emulator's kernel support, waking up the emulator when an interrupt came
>> in.
>> 
>> I don't really know anything about kvm internals, but I'd like to
>> understand more about the particulars of the IOMMU requirement if you
>> don't mind.
>> 
> 
> Xen supports PCI passthrough to PV guests without IOMMU. This can create
> security problems, since the guests get DMA access to physical hardware,
> but that's usually OK in the situations where you want to use PCI
> passthrough on your desktop or on your development box.

That's why there way PV support for DMA in KVM too, but it turned out to be rather unmaintained and hard to detect if it actually works. Because if the guest then just didn't use the PV parts to remap its DMA regions, your PCI card ended up writing into random host memory regions. WIthout you knowing.

Xen doesn't have that problem as badly as we do, because it can guarantee that a PV guest is PV aware. On KVM PV is an optional add-in. All guests start off being fully virtualized.

So we voted for dropping PV DMA support in KVM and just went with the IOMMU only approach. In the long run that's a pretty straight-forward hardware requirement. And if you're using KVM you're used to hardware requirements already anyways ;-).

Alex

Re: PCI passthrough resource remapping

[Avi Kivity]

On 01/14/2010 09:34 PM, Ryan C. Underwood wrote:
> On Thu, Jan 14, 2010 at 09:09:32PM +0200, Avi Kivity wrote:
>    
>> PCI cards can access system memory directly.  If you assign a card
>> to a guest, the guest will program the card to transfer data to
>> system memory using guest addresses; since guest addresses don't
>> correspond to host addresses, memory corruption will ensue.
>>      
> I see, so the only way to fix this would be either with a special guest
> driver for the device that does not perform DMA, or if that is
> impossible (due to no docs), to trap and rewrite any command writes to
> the device's MMIO region that reference a DMA write target buffer.
>
> Forgive my ignorance, but is it possible that the latter is already
> possible with qemu-kvm (somewhat like hardware memory breakpoints in
> Soft-ICE)?
>
>    

Yes, you can easily trap mmio writes to a device, and in fact kvm does 
this in some non-default scenarios.

> If qemu-kvm can be made to break and log on PCI memory accesses, I would
> then hack around the safety limitations, assuming that's all they are,
> and analyze the PCI writes one by one to find the cases where a physical
> address is passed to the card.
>
> Then I would perform the IOMMU translation myself in software whenever a
> physmem address shows up in the command stream.  (Somewhat like the
> security validation of a 3D graphics card command stream in the DRM.)
>
>    

That's definitely doable, but you would need to know exactly how the 
device does dma.  You would also need to lock all pages into memory 
(mlockall()) and how pages are mapped (/proc/$pid/pagemap?)

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/1/9 Alexander Graf <agraf@suse.de>:
>
> On 09.01.2010, at 03:45, Ryan C. Underwood wrote:
>
>>
>> I have a multifunction PCI device that I'd like to pass through to KVM.
>> In order to do that, I'm reading that the PCI memory region must be 4K-page
>> aligned and the PCI memory resources itself must also be exact multiples
>> of 4K pages.
>>
>> I have added the following on my kernel command line:
>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
>>
>> But I don't know if it has any effect.  The resources are still not
>> sized in 4K pages.  Also, this seems to screw up the last device.
>
> I submitted a patch to qemu-kvm recently that got rid of that limitation. Please try out if the current git head works for you.
>
> Alex--

I just upgraded to kernel 2.6.32.10 with qemu-kvm  0.12.3 and I still
get the following error when trying to pass through a dedicated PCI
USB card:

"Unable to assign device: PCI region 0 at address 0xe9403000 has size
0x100,  which is not a multiple of 4K
Error initializing device pci-assign"

Didn't the above patch make it into qemu-kvm? I don't know why, but I
was under the impression that this was fixed when I upgraded to
qemu-kvm 0.12.3.

Thanks

Best Regards
Kenni

Re: PCI passthrough resource remapping

[Brian Jackson]

It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if  
there is one

Sent from my iPhone

On Mar 25, 2010, at 9:37 PM, Kenni Lund <kenni@kelu.dk> wrote:

> 2010/1/9 Alexander Graf <agraf@suse.de>:
>>
>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote:
>>
>>>
>>> I have a multifunction PCI device that I'd like to pass through to  
>>> KVM.
>>> In order to do that, I'm reading that the PCI memory region must  
>>> be 4K-page
>>> aligned and the PCI memory resources itself must also be exact  
>>> multiples
>>> of 4K pages.
>>>
>>> I have added the following on my kernel command line:
>>> reassign_resources  
>>> reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
>>>
>>> But I don't know if it has any effect.  The resources are still not
>>> sized in 4K pages.  Also, this seems to screw up the last device.
>>
>> I submitted a patch to qemu-kvm recently that got rid of that  
>> limitation. Please try out if the current git head works for you.
>>
>> Alex--
>
> I just upgraded to kernel 2.6.32.10 with qemu-kvm  0.12.3 and I still
> get the following error when trying to pass through a dedicated PCI
> USB card:
>
> "Unable to assign device: PCI region 0 at address 0xe9403000 has size
> 0x100,  which is not a multiple of 4K
> Error initializing device pci-assign"
>
> Didn't the above patch make it into qemu-kvm? I don't know why, but I
> was under the impression that this was fixed when I upgraded to
> qemu-kvm 0.12.3.
>
> Thanks
>
> Best Regards
> Kenni
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: PCI passthrough resource remapping

[Kenni Lund]

>> 2010/1/9 Alexander Graf <agraf@suse.de>:
>>>
>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote:
>>>
>>>>
>>>> I have a multifunction PCI device that I'd like to pass through to KVM.
>>>> In order to do that, I'm reading that the PCI memory region must be
>>>> 4K-page
>>>> aligned and the PCI memory resources itself must also be exact multiples
>>>> of 4K pages.
>>>>
>>>> I have added the following on my kernel command line:
>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
>>>>
>>>> But I don't know if it has any effect.  The resources are still not
>>>> sized in 4K pages.  Also, this seems to screw up the last device.
>>>
>>> I submitted a patch to qemu-kvm recently that got rid of that limitation.
>>> Please try out if the current git head works for you.
>>>
>>> Alex--
>>
>> I just upgraded to kernel 2.6.32.10 with qemu-kvm  0.12.3 and I still
>> get the following error when trying to pass through a dedicated PCI
>> USB card:
>>
>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size
>> 0x100,  which is not a multiple of 4K
>> Error initializing device pci-assign"
>>
>> Didn't the above patch make it into qemu-kvm? I don't know why, but I
>> was under the impression that this was fixed when I upgraded to
>> qemu-kvm 0.12.3.
>>
> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there
> is one

That would be highly appriciated...with the current USB support in
QEMU, PCI passthrough is the only way to get USB 2.0 support. I've
bought two dedicated PCI USB cards for this, but none of them works
due to the above limitation.

Perhaps a developer can comment on this? Are there any plans on
including this patch in the stable releases in the near future?

Thanks :)

Best Regards
Kenni Lund

Re: PCI passthrough resource remapping

[Alexander Graf]

On 29.03.2010, at 19:23, Kenni Lund wrote:

>>> 2010/1/9 Alexander Graf <agraf@suse.de>:
>>>> 
>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote:
>>>> 
>>>>> 
>>>>> I have a multifunction PCI device that I'd like to pass through to KVM.
>>>>> In order to do that, I'm reading that the PCI memory region must be
>>>>> 4K-page
>>>>> aligned and the PCI memory resources itself must also be exact multiples
>>>>> of 4K pages.
>>>>> 
>>>>> I have added the following on my kernel command line:
>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
>>>>> 
>>>>> But I don't know if it has any effect.  The resources are still not
>>>>> sized in 4K pages.  Also, this seems to screw up the last device.
>>>> 
>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation.
>>>> Please try out if the current git head works for you.
>>>> 
>>>> Alex--
>>> 
>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm  0.12.3 and I still
>>> get the following error when trying to pass through a dedicated PCI
>>> USB card:
>>> 
>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size
>>> 0x100,  which is not a multiple of 4K
>>> Error initializing device pci-assign"
>>> 
>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I
>>> was under the impression that this was fixed when I upgraded to
>>> qemu-kvm 0.12.3.
>>> 
>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there
>> is one
> 
> That would be highly appriciated...with the current USB support in
> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've
> bought two dedicated PCI USB cards for this, but none of them works
> due to the above limitation.
> 
> Perhaps a developer can comment on this? Are there any plans on
> including this patch in the stable releases in the near future?

Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable.

I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-).


Alex

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/3/29 Alexander Graf <agraf@suse.de>:
>
> On 29.03.2010, at 19:23, Kenni Lund wrote:
>
>>>> 2010/1/9 Alexander Graf <agraf@suse.de>:
>>>>>
>>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote:
>>>>>
>>>>>>
>>>>>> I have a multifunction PCI device that I'd like to pass through to KVM.
>>>>>> In order to do that, I'm reading that the PCI memory region must be
>>>>>> 4K-page
>>>>>> aligned and the PCI memory resources itself must also be exact multiples
>>>>>> of 4K pages.
>>>>>>
>>>>>> I have added the following on my kernel command line:
>>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
>>>>>>
>>>>>> But I don't know if it has any effect.  The resources are still not
>>>>>> sized in 4K pages.  Also, this seems to screw up the last device.
>>>>>
>>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation.
>>>>> Please try out if the current git head works for you.
>>>>>
>>>>> Alex--
>>>>
>>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm  0.12.3 and I still
>>>> get the following error when trying to pass through a dedicated PCI
>>>> USB card:
>>>>
>>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size
>>>> 0x100,  which is not a multiple of 4K
>>>> Error initializing device pci-assign"
>>>>
>>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I
>>>> was under the impression that this was fixed when I upgraded to
>>>> qemu-kvm 0.12.3.
>>>>
>>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there
>>> is one
>>
>> That would be highly appriciated...with the current USB support in
>> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've
>> bought two dedicated PCI USB cards for this, but none of them works
>> due to the above limitation.
>>
>> Perhaps a developer can comment on this? Are there any plans on
>> including this patch in the stable releases in the near future?
>
> Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable.
>
> I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-).

Sure, I have compiled the current git snapshot and performed some
tests...It's at least mostly working, so I'm a bit unsure if this is a
bug related to this or to something else.

Here's my test results on trying to passthrough a PCI USB card (I've
copy-pasted the text below into http://pastebin.com/8RJE36wG in case
formatting is lost below):

--------

qemu-kvm complete command line:

qemu-kvm -usbdevice tablet -net
nic,macaddr=52:54:00:00:00:01,model=virtio -net tap,ifname=tap0 -vnc
:1 -smp 2 -m 2048 -cdrom
/data/server/Linux/mythbuntu-9.10-desktop-amd64.iso -drive
file=/data/virtualization/01_Mythbuntu.img,if=virtio,boot=on -boot c
-localtime -daemonize -pcidevice host=02:01.0 -pcidevice host=02:01.1
-pcidevice host=02:01.2 -pcidevice host=02:01.3 -monitor
unix:/var/run/kvm/01.socket,server,nowait -k da
--------

uname -a on host

Linux mediaserver 2.6.32-ARCH #1 SMP PREEMPT Fri Mar 26 02:03:53 CET
2010 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel
GNU/Linux

Exact kernel version is 2.6.32.10.

--------

lspci -v on host, only for the USB PCI card:

02:01.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03)
(prog-if 10 [OHCI])
        Subsystem: ALi Corporation ASRock 939Dual-SATA2 Motherboard
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 18
        Memory at e9400000 (32-bit, non-prefetchable) [size=4K]
        Capabilities: [60] Power Management version 2
        Kernel driver in use: pci-stub
        Kernel modules: ohci-hcd

02:01.1 USB Controller: ALi Corporation USB 1.1 Controller (rev 03)
(prog-if 10 [OHCI])
        Subsystem: ALi Corporation ASRock 939Dual-SATA2 Motherboard
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 16
        Memory at e9401000 (32-bit, non-prefetchable) [size=4K]
        Capabilities: [60] Power Management version 2
        Kernel driver in use: pci-stub
        Kernel modules: ohci-hcd

02:01.2 USB Controller: ALi Corporation USB 1.1 Controller (rev 03)
(prog-if 10 [OHCI])
        Subsystem: ALi Corporation ASRock 939Dual-SATA2 Motherboard
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 20
        Memory at e9402000 (32-bit, non-prefetchable) [size=4K]
        Capabilities: [60] Power Management version 2
        Kernel driver in use: pci-stub
        Kernel modules: ohci-hcd

02:01.3 USB Controller: ALi Corporation USB 2.0 Controller (rev 01)
(prog-if 20 [EHCI])
        Subsystem: Device 2020:8888
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 19
        Memory at e9403000 (32-bit, non-prefetchable) [size=256]
        Capabilities: [50] Power Management version 2
        Capabilities: [58] Debug port: BAR=1 offset=0090
        Kernel driver in use: pci-stub
        Kernel modules: ehci-hcd
--------

Commands on host used to prepare PCI USB card:

echo "10b9 5237" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:02:01.0" > /sys/bus/pci/drivers/ohci_hcd/unbind
echo "0000:02:01.1" > /sys/bus/pci/drivers/ohci_hcd/unbind
echo "0000:02:01.2" > /sys/bus/pci/drivers/ohci_hcd/unbind
echo "0000:02:01.0" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:02:01.1" > /sys/bus/pci/drivers/pci-stub/bind
echo "0000:02:01.2" > /sys/bus/pci/drivers/pci-stub/bind
echo "10b9 5237" > /sys/bus/pci/drivers/pci-stub/remove_id

echo "10b9 5239" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:02:01.3" > /sys/bus/pci/drivers/ehci_hcd/unbind
echo "0000:02:01.3" > /sys/bus/pci/drivers/pci-stub/bind
echo "10b9 5239" > /sys/bus/pci/drivers/pci-stub/remove_id
--------

Starting qemu-kvm 0.12.3 (fails):

device: 02:01.0: driver="pci-assign" host="02:01.0"
device: 02:01.1: driver="pci-assign" host="02:01.1"
device: 02:01.2: driver="pci-assign" host="02:01.2"
device: 02:01.3: driver="pci-assign" host="02:01.3"
Unable to assign device: PCI region 0 at address 0xe9403000 has size
0x100,  which is not a multiple of 4K
Error initializing device pci-assign
--------

Starting qemu-kvm git (succeeds):

device: 02:01.0: driver="pci-assign" host="02:01.0"
device: 02:01.1: driver="pci-assign" host="02:01.1"
device: 02:01.2: driver="pci-assign" host="02:01.2"
device: 02:01.3: driver="pci-assign" host="02:01.3"
PCI region 0 at address 0xe9403000 has size 0x100, which is not a
multiple of 4K. You might experience some performance hit due to that.
--------

lspci -v in guest with qemu-kvm git (seems fine?):

00:05.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10)
        Subsystem: ALi Corporation USB 1.1 Controller
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 10
        Memory at f2041000 (32-bit, non-prefetchable) [size=4K]
        Kernel driver in use: ohci_hcd

00:06.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10)
        Subsystem: ALi Corporation USB 1.1 Controller
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 11
        Memory at f2042000 (32-bit, non-prefetchable) [size=4K]
        Kernel driver in use: ohci_hcd

00:07.0 USB Controller: ALi Corporation USB 1.1 Controller (rev 03) (prog-if 10)
        Subsystem: ALi Corporation USB 1.1 Controller
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 10
        Memory at f2043000 (32-bit, non-prefetchable) [size=4K]
        Kernel driver in use: ohci_hcd

00:08.0 USB Controller: ALi Corporation USB 2.0 Controller (rev 01) (prog-if 20)
        Subsystem: Device 2020:8888
        Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 11
        Memory at f2044000 (32-bit, non-prefetchable) [size=256]
        Kernel driver in use: ehci_hcd

--------

dmesg in guest when inserting a DVB-T USB tuner into the PCI USB card:

[  190.900076] usb 1-6: new high speed USB device using ehci_hcd and address 2
[  191.223314] usb 1-6: configuration #1 chosen from 1 choice
[  191.505921] dvb-usb: found a 'Intel CE9500 reference design' in warm state.
[  191.523481] dvb-usb: will pass the complete MPEG2 transport stream
to the software demuxer.
[  191.537924] DVB: registering new adapter (Intel CE9500 reference design)
[  191.647202] DVB: registering adapter 0 frontend 0 (Zarlink ZL10353 DVB-T)...
[  191.716542] MXL5005S: Attached at address 0xc6
[  191.719948] dvb-usb: Intel CE9500 reference design successfully
initialized and connected.
[  191.719989] usbcore: registered new interface driver dvb_usb_ce6230

So far, so good...I'm actually able to scan and find channels with
this USB tuner, but once I try to actually watch a channel, it'll only
give me garbage, eg. artifacts. Probably at the same time the
following message is written to the host:

irq 19: nobody cared (try booting with the "irqpoll" option)
Pid: 0, comm: swapper Not tainted 2.6.32-ARCH #1
Call Trace:
 <IRQ>  [<ffffffff810ab7ee>] ? __report_bad_irq+0x1e/0x90
 [<ffffffff810ab9eb>] ? note_interrupt+0x18b/0x1d0
 [<ffffffff810ac2fd>] ? handle_fasteoi_irq+0xcd/0xf0
 [<ffffffff810153e5>] ? handle_irq+0x15/0x20
 [<ffffffff81014902>] ? do_IRQ+0x62/0xe0
 [<ffffffff81012a53>] ? ret_from_intr+0x0/0x11
 <EOI>  [<ffffffff81033d22>] ? native_safe_halt+0x2/0x10
 [<ffffffffa01d2233>] ? acpi_idle_enter_c1+0x9b/0x112 [processor]
 [<ffffffff81285342>] ? menu_select+0x102/0x290
 [<ffffffff8128422f>] ? cpuidle_idle_call+0x9f/0x170
 [<ffffffff81011202>] ? cpu_idle+0xb2/0x110
 [<ffffffff81516d33>] ? start_kernel+0x3c6/0x3d1
 [<ffffffff81516430>] ? x86_64_start_kernel+0xfd/0x10c
handlers:
[<ffffffffa009a7f0>] (usb_hcd_irq+0x0/0x80 [usbcore])
Disabling IRQ #19

...after which the PCI USB card stops working in the guest. Is this
issue related to the issue the other person had? I'm more than willing
to help out with testing, if you or another developer are interested
in fixing this passthrough support.

Thanks :)

Best Regards
Kenni Lund

Re: PCI passthrough resource remapping

[Alexander Graf]

On 30.03.2010, at 01:00, Kenni Lund wrote:

> 2010/3/29 Alexander Graf <agraf@suse.de>:
>> 
>> On 29.03.2010, at 19:23, Kenni Lund wrote:
>> 
>>>>> 2010/1/9 Alexander Graf <agraf@suse.de>:
>>>>>> 
>>>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote:
>>>>>> 
>>>>>>> 
>>>>>>> I have a multifunction PCI device that I'd like to pass through to KVM.
>>>>>>> In order to do that, I'm reading that the PCI memory region must be
>>>>>>> 4K-page
>>>>>>> aligned and the PCI memory resources itself must also be exact multiples
>>>>>>> of 4K pages.
>>>>>>> 
>>>>>>> I have added the following on my kernel command line:
>>>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
>>>>>>> 
>>>>>>> But I don't know if it has any effect.  The resources are still not
>>>>>>> sized in 4K pages.  Also, this seems to screw up the last device.
>>>>>> 
>>>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation.
>>>>>> Please try out if the current git head works for you.
>>>>>> 
>>>>>> Alex--
>>>>> 
>>>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm  0.12.3 and I still
>>>>> get the following error when trying to pass through a dedicated PCI
>>>>> USB card:
>>>>> 
>>>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size
>>>>> 0x100,  which is not a multiple of 4K
>>>>> Error initializing device pci-assign"
>>>>> 
>>>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I
>>>>> was under the impression that this was fixed when I upgraded to
>>>>> qemu-kvm 0.12.3.
>>>>> 
>>>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there
>>>> is one
>>> 
>>> That would be highly appriciated...with the current USB support in
>>> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've
>>> bought two dedicated PCI USB cards for this, but none of them works
>>> due to the above limitation.
>>> 
>>> Perhaps a developer can comment on this? Are there any plans on
>>> including this patch in the stable releases in the near future?
>> 
>> Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable.
>> 
>> I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-).
> 
> Sure, I have compiled the current git snapshot and performed some
> tests...It's at least mostly working, so I'm a bit unsure if this is a
> bug related to this or to something else.

Chris, any idea on this? Looks like something's going wrong with function assignment.


Alex

Re: PCI passthrough resource remapping

[Chris Wright]

* Alexander Graf (agraf@suse.de) wrote:
> On 30.03.2010, at 01:00, Kenni Lund wrote:
> 
> > 2010/3/29 Alexander Graf <agraf@suse.de>:
> >> 
> >> On 29.03.2010, at 19:23, Kenni Lund wrote:
> >> 
> >>>>> 2010/1/9 Alexander Graf <agraf@suse.de>:
> >>>>>> 
> >>>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote:
> >>>>>> 
> >>>>>>> 
> >>>>>>> I have a multifunction PCI device that I'd like to pass through to KVM.
> >>>>>>> In order to do that, I'm reading that the PCI memory region must be
> >>>>>>> 4K-page
> >>>>>>> aligned and the PCI memory resources itself must also be exact multiples
> >>>>>>> of 4K pages.
> >>>>>>> 
> >>>>>>> I have added the following on my kernel command line:
> >>>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
> >>>>>>> 
> >>>>>>> But I don't know if it has any effect.  The resources are still not
> >>>>>>> sized in 4K pages.  Also, this seems to screw up the last device.
> >>>>>> 
> >>>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation.
> >>>>>> Please try out if the current git head works for you.
> >>>>>> 
> >>>>>> Alex--
> >>>>> 
> >>>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm  0.12.3 and I still
> >>>>> get the following error when trying to pass through a dedicated PCI
> >>>>> USB card:
> >>>>> 
> >>>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size
> >>>>> 0x100,  which is not a multiple of 4K
> >>>>> Error initializing device pci-assign"
> >>>>> 
> >>>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I
> >>>>> was under the impression that this was fixed when I upgraded to
> >>>>> qemu-kvm 0.12.3.
> >>>>> 
> >>>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there
> >>>> is one
> >>> 
> >>> That would be highly appriciated...with the current USB support in
> >>> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've
> >>> bought two dedicated PCI USB cards for this, but none of them works
> >>> due to the above limitation.
> >>> 
> >>> Perhaps a developer can comment on this? Are there any plans on
> >>> including this patch in the stable releases in the near future?
> >> 
> >> Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable.
> >> 
> >> I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-).
> > 
> > Sure, I have compiled the current git snapshot and performed some
> > tests...It's at least mostly working, so I'm a bit unsure if this is a
> > bug related to this or to something else.
> 
> Chris, any idea on this? Looks like something's going wrong with function assignment.

Hmm, one thing that sticks out to me is the debug port.  Kenni, can you
post full dmesg on both host and guest, nothing is obviously broken (and
in fact the guest should never "see" the debug port).

thanks,
-chris

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/3/30 Chris Wright <chrisw@redhat.com>:
> * Alexander Graf (agraf@suse.de) wrote:
>> On 30.03.2010, at 01:00, Kenni Lund wrote:
>>
>> > 2010/3/29 Alexander Graf <agraf@suse.de>:
>> >>
>> >> On 29.03.2010, at 19:23, Kenni Lund wrote:
>> >>
>> >>>>> 2010/1/9 Alexander Graf <agraf@suse.de>:
>> >>>>>>
>> >>>>>> On 09.01.2010, at 03:45, Ryan C. Underwood wrote:
>> >>>>>>
>> >>>>>>>
>> >>>>>>> I have a multifunction PCI device that I'd like to pass through to KVM.
>> >>>>>>> In order to do that, I'm reading that the PCI memory region must be
>> >>>>>>> 4K-page
>> >>>>>>> aligned and the PCI memory resources itself must also be exact multiples
>> >>>>>>> of 4K pages.
>> >>>>>>>
>> >>>>>>> I have added the following on my kernel command line:
>> >>>>>>> reassign_resources reassigndev=08:09.0,08:09.1,08:09.2,08:09.3,08:09.4
>> >>>>>>>
>> >>>>>>> But I don't know if it has any effect.  The resources are still not
>> >>>>>>> sized in 4K pages.  Also, this seems to screw up the last device.
>> >>>>>>
>> >>>>>> I submitted a patch to qemu-kvm recently that got rid of that limitation.
>> >>>>>> Please try out if the current git head works for you.
>> >>>>>>
>> >>>>>> Alex--
>> >>>>>
>> >>>>> I just upgraded to kernel 2.6.32.10 with qemu-kvm  0.12.3 and I still
>> >>>>> get the following error when trying to pass through a dedicated PCI
>> >>>>> USB card:
>> >>>>>
>> >>>>> "Unable to assign device: PCI region 0 at address 0xe9403000 has size
>> >>>>> 0x100,  which is not a multiple of 4K
>> >>>>> Error initializing device pci-assign"
>> >>>>>
>> >>>>> Didn't the above patch make it into qemu-kvm? I don't know why, but I
>> >>>>> was under the impression that this was fixed when I upgraded to
>> >>>>> qemu-kvm 0.12.3.
>> >>>>>
>> >>>> It's only in qemu-kvm.git. Maybe it should go into qemu-kvm-0.12.4 if there
>> >>>> is one
>> >>>
>> >>> That would be highly appriciated...with the current USB support in
>> >>> QEMU, PCI passthrough is the only way to get USB 2.0 support. I've
>> >>> bought two dedicated PCI USB cards for this, but none of them works
>> >>> due to the above limitation.
>> >>>
>> >>> Perhaps a developer can comment on this? Are there any plans on
>> >>> including this patch in the stable releases in the near future?
>> >>
>> >> Please first try out to build the current git snapshot of qemu-kvm. If it works properly for you then I agree that we should take this into 0.12-stable.
>> >>
>> >> I wrote the support for a card that still didn't work even with this patch. So having someone say it makes things work for him is definitely a must :-).
>> >
>> > Sure, I have compiled the current git snapshot and performed some
>> > tests...It's at least mostly working, so I'm a bit unsure if this is a
>> > bug related to this or to something else.
>>
>> Chris, any idea on this? Looks like something's going wrong with function assignment.
>
> Hmm, one thing that sticks out to me is the debug port.  Kenni, can you
> post full dmesg on both host and guest, nothing is obviously broken (and
> in fact the guest should never "see" the debug port).
>

Uploaded here:
Client dmesg: http://pastebin.com/uNG4QK5j
Host dmesg: http://pastebin.com/jZu3WKZW

I just verified it and I do get the call trace in the host (which
disables IRQ 19, used by the PCI USB card), exactly at the same second
I ask the DVB-T tuner to view a channel in the guest.

Thanks..

Best Regards
Kenni Lund

Re: PCI passthrough resource remapping

[Chris Wright]

* Kenni Lund (kenni@kelu.dk) wrote:
> Client dmesg: http://pastebin.com/uNG4QK5j
> Host dmesg: http://pastebin.com/jZu3WKZW
> 
> I just verified it and I do get the call trace in the host (which
> disables IRQ 19, used by the PCI USB card), exactly at the same second

It looks like IRQ 19 is shared between the ehci controller and the
ivtv tuner.  What do you see in /proc/interrupts on the host (before
you unbind and after you bind to pci stub)?

The host dmesg shows other possible devices sharing that interrupt:

$ grep "IRQ 19" jZu3WKZW
   482. ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
   578. ehci_hcd 0000:02:01.3: PCI INT A -> GSI 19 (level, low) -> IRQ 19
   623. uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
   795. ivtv 0000:03:09.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
   814. IRQ 19/ivtv1: IRQF_DISABLED is not guaranteed on shared IRQs
   955. pci-stub 0000:02:01.3: PCI INT A -> GSI 19 (level, low) -> IRQ 19

thanks,
-chris

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/3/30 Chris Wright <chrisw@redhat.com>:
> * Kenni Lund (kenni@kelu.dk) wrote:
>> Client dmesg: http://pastebin.com/uNG4QK5j
>> Host dmesg: http://pastebin.com/jZu3WKZW
>>
>> I just verified it and I do get the call trace in the host (which
>> disables IRQ 19, used by the PCI USB card), exactly at the same second
>
> It looks like IRQ 19 is shared between the ehci controller and the
> ivtv tuner.  What do you see in /proc/interrupts on the host (before
> you unbind and after you bind to pci stub)?

Ahh, and even if the ivtv module is not loaded, I will still have a
shared IRQ, right? I didn't see ivtv in /proc/interrupts before, as I
unload the ivtv driver on boot in /etc/rc.local, before unbinding the
ivtv tuner and binding it to pci stub. (the ivtv tuner is normally
assigned to the same guest, but not now while testing the PCI USB
card).

If I don't unload (and unbind/bind) the ivtv driver/tuner on boot in
/etc/local, I get the following in /proc/interrupts on a clean boot:
http://pastebin.com/SFQj58LC

If I now unbind and bind the PCI USB card to pci stub, I get no
changes in /proc/interrupts.

So I suppose I'll need to get rid of this shared IRQ before I can
conclude anything on the patch in git. Hmm, is there some cleaver way
of fixing this in Linux, or do I have to fix it by changing BIOS IRQ
settings, disabling hardware and/or moving the hardware around in
various PCI slots?

Best Regards
Kenni

Re: PCI passthrough resource remapping

[Alexander Graf]

On 31.03.2010, at 00:27, Kenni Lund wrote:

> 2010/3/30 Chris Wright <chrisw@redhat.com>:
>> * Kenni Lund (kenni@kelu.dk) wrote:
>>> Client dmesg: http://pastebin.com/uNG4QK5j
>>> Host dmesg: http://pastebin.com/jZu3WKZW
>>> 
>>> I just verified it and I do get the call trace in the host (which
>>> disables IRQ 19, used by the PCI USB card), exactly at the same second
>> 
>> It looks like IRQ 19 is shared between the ehci controller and the
>> ivtv tuner.  What do you see in /proc/interrupts on the host (before
>> you unbind and after you bind to pci stub)?
> 
> Ahh, and even if the ivtv module is not loaded, I will still have a
> shared IRQ, right? I didn't see ivtv in /proc/interrupts before, as I
> unload the ivtv driver on boot in /etc/rc.local, before unbinding the
> ivtv tuner and binding it to pci stub. (the ivtv tuner is normally
> assigned to the same guest, but not now while testing the PCI USB
> card).
> 
> If I don't unload (and unbind/bind) the ivtv driver/tuner on boot in
> /etc/local, I get the following in /proc/interrupts on a clean boot:
> http://pastebin.com/SFQj58LC
> 
> If I now unbind and bind the PCI USB card to pci stub, I get no
> changes in /proc/interrupts.
> 
> So I suppose I'll need to get rid of this shared IRQ before I can
> conclude anything on the patch in git. Hmm, is there some cleaver way
> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ
> settings, disabling hardware and/or moving the hardware around in
> various PCI slots?

The easiest thing coming to mind is to unplug the ivtv card for now. It's really only to verify that the patch does something useful :-).


Alex

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/3/31 Alexander Graf <agraf@suse.de>:
>
> On 31.03.2010, at 00:27, Kenni Lund wrote:
>
>> 2010/3/30 Chris Wright <chrisw@redhat.com>:
>>> * Kenni Lund (kenni@kelu.dk) wrote:
>>>> Client dmesg: http://pastebin.com/uNG4QK5j
>>>> Host dmesg: http://pastebin.com/jZu3WKZW
>>>>
>>>> I just verified it and I do get the call trace in the host (which
>>>> disables IRQ 19, used by the PCI USB card), exactly at the same second
>>>
>>> It looks like IRQ 19 is shared between the ehci controller and the
>>> ivtv tuner.  What do you see in /proc/interrupts on the host (before
>>> you unbind and after you bind to pci stub)?
>>
>> Ahh, and even if the ivtv module is not loaded, I will still have a
>> shared IRQ, right? I didn't see ivtv in /proc/interrupts before, as I
>> unload the ivtv driver on boot in /etc/rc.local, before unbinding the
>> ivtv tuner and binding it to pci stub. (the ivtv tuner is normally
>> assigned to the same guest, but not now while testing the PCI USB
>> card).
>>
>> If I don't unload (and unbind/bind) the ivtv driver/tuner on boot in
>> /etc/local, I get the following in /proc/interrupts on a clean boot:
>> http://pastebin.com/SFQj58LC
>>
>> If I now unbind and bind the PCI USB card to pci stub, I get no
>> changes in /proc/interrupts.
>>
>> So I suppose I'll need to get rid of this shared IRQ before I can
>> conclude anything on the patch in git. Hmm, is there some cleaver way
>> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ
>> settings, disabling hardware and/or moving the hardware around in
>> various PCI slots?
>
> The easiest thing coming to mind is to unplug the ivtv card for now. It's really only to verify that the patch does something useful :-).

This was not sufficient. Same issue with the ivtv card unplugged...if
I interpret the content of /proc/interrupt correctly, the USB PCI
cards uses 4 IRQs and out of these three IRQs are still shared with
other components.

The card uses IRQ 16, 18, 19, 20:
pci-stub 0000:02:01.0: PCI INT B -> GSI 18 (level, low) -> IRQ 18
pci-stub 0000:02:01.1: PCI INT C -> GSI 16 (level, low) -> IRQ 16
pci-stub 0000:02:01.2: PCI INT D -> GSI 20 (level, low) -> IRQ 20
pci-stub 0000:02:01.3: PCI INT A -> GSI 19 (level, low) -> IRQ 19

Only IRQ 20 gets "kvm_assigned_intx_device" after
unbinding+binding+qemu-kvm launch:
interrupts before: http://pastebin.com/DdJEv29z
interrupts after: http://pastebin.com/zasWEZsL

It seems like i still have shared IRQs with the onboard USB controller
as well as the PATA controller. In the BIOS i have the option of
setting the IRQ of "PCI Card 1" and "PCI Card 2". I tried changing
these into IRQ 10 and 11 (which are free according to
/proc/interrupts), but it changed absolutely nothing in
/proc/interrupts after booting(?) :-(

Any kind of hint which could lead me in hte right direction, would be
highly appreciated...thanks.

Best Regards
Kenni

Re: PCI passthrough resource remapping

[Chris Wright]

* Kenni Lund (kenni@kelu.dk) wrote:
> 2010/3/30 Chris Wright <chrisw@redhat.com>:
> > * Kenni Lund (kenni@kelu.dk) wrote:
> >> Client dmesg: http://pastebin.com/uNG4QK5j
> >> Host dmesg: http://pastebin.com/jZu3WKZW
> >>
> >> I just verified it and I do get the call trace in the host (which
> >> disables IRQ 19, used by the PCI USB card), exactly at the same second
> >
> > It looks like IRQ 19 is shared between the ehci controller and the
> > ivtv tuner.  What do you see in /proc/interrupts on the host (before
> > you unbind and after you bind to pci stub)?
> 
> Ahh, and even if the ivtv module is not loaded, I will still have a
> shared IRQ, right? I didn't see ivtv in /proc/interrupts before, as I
> unload the ivtv driver on boot in /etc/rc.local, before unbinding the
> ivtv tuner and binding it to pci stub. (the ivtv tuner is normally
> assigned to the same guest, but not now while testing the PCI USB
> card).
> 
> If I don't unload (and unbind/bind) the ivtv driver/tuner on boot in
> /etc/local, I get the following in /proc/interrupts on a clean boot:
> http://pastebin.com/SFQj58LC
> 
> If I now unbind and bind the PCI USB card to pci stub, I get no
> changes in /proc/interrupts.

Sorry, I meant bind to pci_stub and launch guest.  IOW, you should see
kvm_assigned_{msi,msix,intx}_device (from your lspci, I'd expect intx).

What's odd is a device is asserting an interrupt to a line w/ no handler
acking.  The IRQ 19, should have kvm handling the interrupt, in which
case it'd always return IRQ_HANDLED.  And for the case of shared
interrupts, we won't let you start the guest with an assigned device
that's sharing an interrupt.  IOW, we do request_irq() w/out specifying
IRQF_SHARED (meaning we want an exclusive irq).

> So I suppose I'll need to get rid of this shared IRQ before I can
> conclude anything on the patch in git. Hmm, is there some cleaver way
> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ
> settings, disabling hardware and/or moving the hardware around in
> various PCI slots?

The way I typically work around this is simply unbinding the driver from
the device in the host (and thus freeing the irq).

thanks,
-chris

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/3/31 Chris Wright <chrisw@redhat.com>:
>> So I suppose I'll need to get rid of this shared IRQ before I can
>> conclude anything on the patch in git. Hmm, is there some cleaver way
>> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ
>> settings, disabling hardware and/or moving the hardware around in
>> various PCI slots?
>
> The way I typically work around this is simply unbinding the driver from
> the device in the host (and thus freeing the irq).

Doh...anyway, I went all the way, found a USB->PS2 adaptor, disabled
the onboard USB controller and PATA controller in BIOS, and now got
"kvm_assigned_intx_device" for all 4 IRQs :)

Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks
a lot for your help...I have one more question, though: If I have two
devices (like the ivtv tuner and the USB card) and they share an IRQ,
if I then assign BOTH of them to the same guest, will it then work?

Alexander, the patch works, I hope to see it in a stable release in
the near future ;)

Best Regards
Kenni

Re: PCI passthrough resource remapping

[Chris Wright]

* Kenni Lund (kenni@kelu.dk) wrote:
> 2010/3/31 Alexander Graf <agraf@suse.de>:
> > The easiest thing coming to mind is to unplug the ivtv card for now. It's really only to verify that the patch does something useful :-).
> 
> This was not sufficient. Same issue with the ivtv card unplugged...if
> I interpret the content of /proc/interrupt correctly, the USB PCI
> cards uses 4 IRQs and out of these three IRQs are still shared with
> other components.
> 
> The card uses IRQ 16, 18, 19, 20:
> pci-stub 0000:02:01.0: PCI INT B -> GSI 18 (level, low) -> IRQ 18
> pci-stub 0000:02:01.1: PCI INT C -> GSI 16 (level, low) -> IRQ 16
> pci-stub 0000:02:01.2: PCI INT D -> GSI 20 (level, low) -> IRQ 20
> pci-stub 0000:02:01.3: PCI INT A -> GSI 19 (level, low) -> IRQ 19
> 
> Only IRQ 20 gets "kvm_assigned_intx_device" after
> unbinding+binding+qemu-kvm launch:

What is your qemu command line? It's acting as if function .2 (one of
the ochi controllers) is generating the interrupt rather than .3 (the
ehci controller).  And indeed, there is no handler to ack that interrupt
(just the likely on-board uhci one).

before:

19:          0          0          0          0   IO-APIC-fasteoi   ehci_hcd:usb3, uhci_hcd:usb9
20:          0          0          0          0   IO-APIC-fasteoi   ohci_hcd:usb12

after:
19:      75010      73438      76479      75074   IO-APIC-fasteoi   uhci_hcd:usb9
20:          0          0          0          0   IO-APIC-fasteoi   kvm_assigned_intx_device

Can you unbind uhci_hcd:usb9 and then give both .2 and .3 to the guest?

thanks,
-chris

Re: PCI passthrough resource remapping

[Chris Wright]

* Kenni Lund (kenni@kelu.dk) wrote:
> 2010/3/31 Chris Wright <chrisw@redhat.com>:
> >> So I suppose I'll need to get rid of this shared IRQ before I can
> >> conclude anything on the patch in git. Hmm, is there some cleaver way
> >> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ
> >> settings, disabling hardware and/or moving the hardware around in
> >> various PCI slots?
> >
> > The way I typically work around this is simply unbinding the driver from
> > the device in the host (and thus freeing the irq).
> 
> Doh...anyway, I went all the way, found a USB->PS2 adaptor, disabled
> the onboard USB controller and PATA controller in BIOS, and now got
> "kvm_assigned_intx_device" for all 4 IRQs :)

A little extreme...but hey, that works ;-)  I'm still curious what's
going on w/ your PCI card and the irq routing.  Something is suspect.

> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks
> a lot for your help...I have one more question, though: If I have two
> devices (like the ivtv tuner and the USB card) and they share an IRQ,
> if I then assign BOTH of them to the same guest, will it then work?

No, it won't.  The first one will request an exclusive interrupt, and the
second one will fail its request_irq.  There's two options, one is to
use a different driver for kvm (if the device is new enough to handle
PCI 2.3) or use MSI/MSI-X based devices (that works best ;)  Neither of
those options are readily available in your case.

thanks,
-chris

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/3/31 Chris Wright <chrisw@redhat.com>:
> * Kenni Lund (kenni@kelu.dk) wrote:
>> 2010/3/31 Chris Wright <chrisw@redhat.com>:
>> >> So I suppose I'll need to get rid of this shared IRQ before I can
>> >> conclude anything on the patch in git. Hmm, is there some cleaver way
>> >> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ
>> >> settings, disabling hardware and/or moving the hardware around in
>> >> various PCI slots?
>> >
>> > The way I typically work around this is simply unbinding the driver from
>> > the device in the host (and thus freeing the irq).
>>
>> Doh...anyway, I went all the way, found a USB->PS2 adaptor, disabled
>> the onboard USB controller and PATA controller in BIOS, and now got
>> "kvm_assigned_intx_device" for all 4 IRQs :)
>
> A little extreme...but hey, that works ;-)  I'm still curious what's
> going on w/ your PCI card and the irq routing.  Something is suspect.

I'll not be able to do any further testing for the next two weeks, but
if you want me to perform some tests, test patches etc., just write
what you want me to do and I'll do it in two weeks.

Best Regards
Kenni

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/3/31 Kenni Lund <kenni@kelu.dk>:
> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks
> a lot for your help...I have one more question, though: If I have two
> devices (like the ivtv tuner and the USB card) and they share an IRQ,
> if I then assign BOTH of them to the same guest, will it then work?
>
> Alexander, the patch works, I hope to see it in a stable release in
> the near future ;)

Unfortunately, I have a correction to this :( It _almost_ works. I had
some video/audio artifacts which I though was caused by bad reception,
but after switching the DVB-T tuner back and forth between the PCI USB
card and a laptop, it got clear to me, that this was a passthrough
issue. I get no errors in dmesg on the host or in the guest.

I recorded a short videoclip which illustrates the issue:
https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en

Best Regards
Kenni

Re: PCI passthrough resource remapping

[Alexander Graf]

Kenni Lund wrote:
> 2010/3/31 Kenni Lund <kenni@kelu.dk>:
>   
>> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks
>> a lot for your help...I have one more question, though: If I have two
>> devices (like the ivtv tuner and the USB card) and they share an IRQ,
>> if I then assign BOTH of them to the same guest, will it then work?
>>
>> Alexander, the patch works, I hope to see it in a stable release in
>> the near future ;)
>>     
>
> Unfortunately, I have a correction to this :( It _almost_ works. I had
> some video/audio artifacts which I though was caused by bad reception,
> but after switching the DVB-T tuner back and forth between the PCI USB
> card and a laptop, it got clear to me, that this was a passthrough
> issue. I get no errors in dmesg on the host or in the guest.
>
> I recorded a short videoclip which illustrates the issue:
> https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en
>   

Hrm, I'm not sure these would be related to the small BAR region patch.
It looks more like a timing issue.


Alex

Re: PCI passthrough resource remapping

[Kenni Lund]

2010/3/31 Alexander Graf <agraf@suse.de>:
> Kenni Lund wrote:
>> 2010/3/31 Kenni Lund <kenni@kelu.dk>:
>>
>>> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks
>>> a lot for your help...I have one more question, though: If I have two
>>> devices (like the ivtv tuner and the USB card) and they share an IRQ,
>>> if I then assign BOTH of them to the same guest, will it then work?
>>>
>>> Alexander, the patch works, I hope to see it in a stable release in
>>> the near future ;)
>>>
>>
>> Unfortunately, I have a correction to this :( It _almost_ works. I had
>> some video/audio artifacts which I though was caused by bad reception,
>> but after switching the DVB-T tuner back and forth between the PCI USB
>> card and a laptop, it got clear to me, that this was a passthrough
>> issue. I get no errors in dmesg on the host or in the guest.
>>
>> I recorded a short videoclip which illustrates the issue:
>> https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en
>>
>
> Hrm, I'm not sure these would be related to the small BAR region patch.
> It looks more like a timing issue.

For what it's worth, I quickly tried to passthrough one of the onboard
USB controllers instead (Intel Corporation 82801JD/DO (ICH10 Family),
which is also affected by the 4k limitation). It improves the artifact
problems, bringing the artifacts down to one artifact every 5 seconds.

I'll be back in ~1½ week, please tell me if you want me to test
something when I get back.

Thanks for your help :)

Best Regards
Kenni

Re: PCI passthrough resource remapping

[Chris Wright]

* Kenni Lund (kenni@kelu.dk) wrote:
> 2010/3/31 Chris Wright <chrisw@redhat.com>:
> > * Kenni Lund (kenni@kelu.dk) wrote:
> >> 2010/3/31 Chris Wright <chrisw@redhat.com>:
> >> >> So I suppose I'll need to get rid of this shared IRQ before I can
> >> >> conclude anything on the patch in git. Hmm, is there some cleaver way
> >> >> of fixing this in Linux, or do I have to fix it by changing BIOS IRQ
> >> >> settings, disabling hardware and/or moving the hardware around in
> >> >> various PCI slots?
> >> >
> >> > The way I typically work around this is simply unbinding the driver from
> >> > the device in the host (and thus freeing the irq).
> >>
> >> Doh...anyway, I went all the way, found a USB->PS2 adaptor, disabled
> >> the onboard USB controller and PATA controller in BIOS, and now got
> >> "kvm_assigned_intx_device" for all 4 IRQs :)
> >
> > A little extreme...but hey, that works ;-)  I'm still curious what's
> > going on w/ your PCI card and the irq routing.  Something is suspect.
> 
> I'll not be able to do any further testing for the next two weeks, but
> if you want me to perform some tests, test patches etc., just write
> what you want me to do and I'll do it in two weeks.

Thanks for your willingness to do some tests.  I'll poke around a bit
and see if I can come up w/ something useful for you to try out.

thanks,
-chris

Re: PCI passthrough resource remapping

[Chris Wright]

* Alexander Graf (agraf@suse.de) wrote:
> Kenni Lund wrote:
> > 2010/3/31 Kenni Lund <kenni@kelu.dk>:
> >   
> >> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks
> >> a lot for your help...I have one more question, though: If I have two
> >> devices (like the ivtv tuner and the USB card) and they share an IRQ,
> >> if I then assign BOTH of them to the same guest, will it then work?
> >>
> >> Alexander, the patch works, I hope to see it in a stable release in
> >> the near future ;)
> >>     
> >
> > Unfortunately, I have a correction to this :( It _almost_ works. I had
> > some video/audio artifacts which I though was caused by bad reception,
> > but after switching the DVB-T tuner back and forth between the PCI USB
> > card and a laptop, it got clear to me, that this was a passthrough
> > issue. I get no errors in dmesg on the host or in the guest.
> >
> > I recorded a short videoclip which illustrates the issue:
> > https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en
> >   
> 
> Hrm, I'm not sure these would be related to the small BAR region patch.
> It looks more like a timing issue.

small BAR == slow path == timing issue?

thanks,
-chris

Re: PCI passthrough resource remapping

[Alexander Graf]

Chris Wright wrote:
> * Alexander Graf (agraf@suse.de) wrote:
>   
>> Kenni Lund wrote:
>>     
>>> 2010/3/31 Kenni Lund <kenni@kelu.dk>:
>>>   
>>>       
>>>> Booting the guest and tuning to a DVB-T channel _works_ !!! :-D Thanks
>>>> a lot for your help...I have one more question, though: If I have two
>>>> devices (like the ivtv tuner and the USB card) and they share an IRQ,
>>>> if I then assign BOTH of them to the same guest, will it then work?
>>>>
>>>> Alexander, the patch works, I hope to see it in a stable release in
>>>> the near future ;)
>>>>     
>>>>         
>>> Unfortunately, I have a correction to this :( It _almost_ works. I had
>>> some video/audio artifacts which I though was caused by bad reception,
>>> but after switching the DVB-T tuner back and forth between the PCI USB
>>> card and a laptop, it got clear to me, that this was a passthrough
>>> issue. I get no errors in dmesg on the host or in the guest.
>>>
>>> I recorded a short videoclip which illustrates the issue:
>>> https://docs.google.com/leaf?id=0B-_nZameGeN-NDk4YzQ3N2EtMmEzMi00NTU4LWFjMjgtNzkxMzcxYzg2MTM1&hl=en
>>>   
>>>       
>> Hrm, I'm not sure these would be related to the small BAR region patch.
>> It looks more like a timing issue.
>>     
>
> small BAR == slow path == timing issue?
>   

Could be, yeah. The only chance I see to avoid that is to have in-kernel
small BAR code that would issue the MMIO accesses without the userspace
churn. One thing I could imagine would be < PAGE_SIZE memory slots. Then
userspace just maps the 4K to the BAR, but passes only a part of it on
as slot.


Alex

Re: PCI passthrough resource remapping

[Avi Kivity]

On 03/31/2010 06:18 PM, Chris Wright wrote:
>
>> Hrm, I'm not sure these would be related to the small BAR region patch.
>> It looks more like a timing issue.
>>      
> small BAR == slow path == timing issue?
>    

Would be interesting to verify using perf with the 'kvm:kvm_mmio' 
software event, see how many happen per second.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.