From mboxrd@z Thu Jan 1 00:00:00 1970 From: James Neave Subject: Re: PCI Passthrough, error: The driver 'pci-stub' is occupying your device 0000:08:06.2 Date: Mon, 21 Feb 2011 22:55:44 +0000 Message-ID: References: <20110207132641.GD2665@redhat.com> <1297705728.14733.50.camel@x201> <1298322078.5764.45.camel@x201> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: kvm@vger.kernel.org To: Alex Williamson Return-path: Received: from mail-qy0-f174.google.com ([209.85.216.174]:59311 "EHLO mail-qy0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750836Ab1BUW4F convert rfc822-to-8bit (ORCPT ); Mon, 21 Feb 2011 17:56:05 -0500 Received: by qyk7 with SMTP id 7so2399566qyk.19 for ; Mon, 21 Feb 2011 14:56:04 -0800 (PST) In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Mon, Feb 21, 2011 at 10:49 PM, James Neave wrote= : > On Mon, Feb 21, 2011 at 10:25 PM, James Neave wro= te: >> On Mon, Feb 21, 2011 at 9:01 PM, Alex Williamson >> wrote: >>> On Mon, 2011-02-21 at 20:31 +0000, James Neave wrote: >>>> On Mon, Feb 14, 2011 at 5:48 PM, Alex Williamson >>>> wrote: >>>> > On Sat, 2011-02-12 at 16:04 +0000, James Neave wrote: >>>> >> On Tue, Feb 8, 2011 at 10:17 AM, James Neave wrote: >>>> >> > On Tue, Feb 8, 2011 at 9:59 AM, Kenni Lund wr= ote: >>>> >> >> 2011/2/7 Daniel P. Berrange : >>>> >> >>> On Sat, Feb 05, 2011 at 04:34:01PM +0000, James Neave wrote= : >>>> >> >>>> Hi, >>>> >> >>>> >>>> >> >>>> I'm trying to pass a NOVA-T-500 TV Tuner card through to a= gust VM. >>>> >> >>>> I'm getting the error "The driver 'pci-stub' is occupying = your device >>>> >> >>>> 0000:08:06.2" >>>> >> >>> >>>> >> >>> This is a rather misleading error message. It is *expected*= that >>>> >> >>> pci-stub will occupy the device. Unfortunately the rest of = the >>>> >> >>> error messages QEMU is printing aren't much help either, bu= t >>>> >> >>> ultimately something is returning -EBUSY in the PCI device = assign >>>> >> >>> step >>>> >> >> >>>> >> >> James, as far as I remember, I had the same issue when I set= up my >>>> >> >> system. Looking at my current (working) boot-script, apparen= tly I've >>>> >> >> added a 4th line which removes the pci-stub again as a >>>> >> >> workaround....and it works: >>>> >> >> >>>> >> >> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/new_id >>>> >> >> echo "0000:04:08.0" > /sys/bus/pci/drivers/ivtv/unbind >>>> >> >> echo "0000:04:08.0" > /sys/bus/pci/drivers/pci-stub/bind >>>> >> >> echo "4444 0016" > /sys/bus/pci/drivers/pci-stub/remove_id >>>> >> >> >>>> >> >> Best regards >>>> >> >> Kenni >>>> >> >> >>>> >> > >>>> >> > Hi Kenni, >>>> >> > >>>> >> > Can I get a bit more information on "boot-script" please? Whi= ch file >>>> >> > exaclty have you put this in? Did you write your own service = script >>>> >> > and put it in init.d? >>>> >> > >>>> >> > I've tried this: >>>> >> > >>>> >> > echo "8086 10b9" > /sys/bus/pci/drivers/pci-stub/new_id >>>> >> > echo 0000:01:00.0 > /sys/bus/pci/devices/0000:01:00.0/driver/= unbind >>>> >> > echo 0000:01:00.0 > /sys/bus/pci/drivers/pci-stub/bind >>>> >> > >>>> >> > I'll try it again with the fourth line added, manually before= I start the VM. >>>> >> > >>>> >> > How come yours is 'echo "PCI" > /sys/bus/drivers/DRIVERNAME/u= nbind' >>>> >> > and mine is echo "PCI" > /sys/bus/pci/devices/PCI/driver/unbi= nd >>>> >> > >>>> >> > Obviously one looks up which driver is being used by the PCI = id, but >>>> >> > how do I look up which driver my PCI card is using? >>>> >> > >>>> >> > Thanks, >>>> >> > >>>> >> > James. >>>> >> > >>>> >> >>>> >> Hi, >>>> >> >>>> >> OK, adding the fourth line does nothing, changing the second li= ne to >>>> >> "driver" rather than "device" does nothing, including in combin= ation >>>> >> with fourth line there/not there. >>>> > >>>> > Yep, the last line is just removing the id from pci-stub, it doe= sn't >>>> > change anything about devices that are already bound to it. =C2=A0= driver vs >>>> > device are just different ways to get to the same thing. >>>> > >>>> >> So I'm still stuck, can anybody else help? >>>> >> Perhaps point me to a guide on how to compile the latest qemu-k= vm >>>> >> against my kernel? >>>> > >>>> > I don't know why you're getting -EBUSY for this device, but mayb= e we can >>>> > start from a clean slate and see if it helps. =C2=A0Here's what = I would >>>> > suggest: >>>> > >>>> > echo "0000:08:06.0" > /sys/bus/pci/devices/0000:08:06.0/driver/u= nbind >>>> > echo "0000:08:06.1" > /sys/bus/pci/devices/0000:08:06.1/driver/u= nbind >>>> > echo "0000:08:06.2" > /sys/bus/pci/devices/0000:08:06.2/driver/u= nbind >>>> > echo "0000:08:0e.0" > /sys/bus/pci/devices/0000:08:0e.0/driver/u= nbind >>>> > >>>> > Note we have to knock out the firewire because it shares an inte= rrupt >>>> > with the ehci device you're trying to assign. =C2=A0We want to r= emove the USB >>>> > controller entirely from the host. =C2=A0Your dmesg indicates th= e host is >>>> > still seeing the device via the uhci ports, and isn't happy abou= t it. >>>> > You can ignore pci-stub for the moment, it's just a way to keep = drivers >>>> > from claiming the device, it's not required for device assignmen= t. =C2=A0Now, >>>> > instead of only trying to assign the ehci, let's move the whole = usb >>>> > controller to the guest: >>>> > >>>> > -device pci-assign,host=3D08:06.0,addr=3D5.0 \ >>>> > -device pci-assign,host=3D08:06.1,addr=3D5.1 \ >>>> > -device pci-assign,host=3D08:06.2,addr=3D5.2 >>>> > >>>> > (slot 5 on the guest is arbitrary, pick something else if you ne= ed to) >>>> > If that works, then you can bind all those devices to pci-stub a= nd it >>>> > should still work. >>>> > >>>> > Alex >>>> >>>> Hi, >>>> >>>> Sorry about the slow reply, I hosed all of my PCs fiddling with >>>> compiling the latest qemu, took me a while to put it all back toge= ther >>>> again in between work! >>>> >>>> I'm afraid I use virtmanager, although I guess using the "add pci >>>> device" function is the same as -device pci-assign? It does seem t= o >>>> add it to the command line that gets written out to the log files = in >>>> /var/log/libvirt/qemu. >>>> >>>> Nevertheless, I've tried that and still not luck, the log output i= s >>>> similar, with one extra line: >>>> >>>> http://pastebin.com/MJ6aqjNq >>> >>> You actually ended up with: >>> >>> -device pci-assign,host=3D08:06.0,id=3Dhostdev0,configfd=3D58,bus=3D= pci.0,addr=3D0x6 \ >>> -device pci-assign,host=3D08:06.1,id=3Dhostdev1,configfd=3D59,bus=3D= pci.0,addr=3D0x7 \ >>> -device pci-assign,host=3D08:06.2,id=3Dhostdev2,configfd=3D60,bus=3D= pci.0,addr=3D0x8 >>> >>> Which isn't quite what I was suggesting. =C2=A0You probably have xm= l that >>> looks like this: >>> >>> =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>>> =C2=A0 =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0... >>> >>> Try adding an address line, so you get something more like this: >>> >>> =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>>> =C2=A0 =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0
>>> =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>>> =C2=A0 =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0
>>> =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>>> =C2=A0 =C2=A0 =C2=A0 >>> =C2=A0 =C2=A0 =C2=A0
>>> =C2=A0 =C2=A0 >>> >>> >>>> Using raw in/out ioport access (sysfs - Input/output error) >>> >>> This is just an informational line to let us know whether pci-sysfs >>> supports read/write on the ioport resource files. =C2=A0If it does,= we use >>> those rather than doing raw in/out for access to the device. =C2=A0= This does >>> highlight another potential problem. =C2=A0Your distro probably doe= sn't have >>> all the patches in place for non-privileged device assignment, whic= h >>> could be why you're having strange issues. =C2=A0Check >>> your /etc/libvirt/qemu.conf for the 'user =3D' and 'group =3D' line= s. =C2=A0If >>> they're not already, try setting them to 'root', restart libvirtd a= nd >>> see if anything improves. >>> >>>> Here's the dmesg output: >>>> http://pastebin.com/AE1euUN1 >>> >>> If still issues after the above, it might be useful to pastbin the >>> entire dmesg so we can make sure the iommu is really on. =C2=A0Than= ks, >>> >>> Alex >> >> Hi, >> >> No such luck I'm afraid. >> >> Here is the original XML: >> >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0
> function=3D'0x0'/> >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0
> function=3D'0x0'/> >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0
> function=3D'0x0'/> >> =C2=A0 =C2=A0 >> >> The only difference from your recommended change is the target (I ta= ke >> it they are target addresses for the VM?) addresses run: >> 0000:00:06.0 >> 0000:00:07.0 >> 0000:00:08.0 >> >> Instead of: >> 0000:00:06.0 >> 0000:00:06.1 >> 0000:00:06.2 >> >> Regardless, I still changed test.xml to: >> >> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0
> function=3D'0x0'/> >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0
> function=3D'0x1'/> >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0 =C2=A0
>> =C2=A0 =C2=A0 =C2=A0 >> =C2=A0 =C2=A0 =C2=A0
> function=3D'0x2'/> >> =C2=A0 =C2=A0 >> >> To no effect. >> >> In qemu.conf, user and group were commented out, I uncommented both >> and they were both already set to root. >> >> After both a restart of libvirt-bin and the pc itself, no change. >> >> Finally, here is the very latest dmesg: >> http://pastebin.com/9HE61K62 >> >> Does anybody know the debug kernel switches for iommu? >> >> Many Thanks, >> >> James. >> > > Hi, > > A ray of sunshine (ish) > > I can apparently pass through this device: > > 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (= rev 40) > =C2=A0 =C2=A0 =C2=A0 =C2=A0Subsystem: Giga-byte Technology Device a10= 2 > =C2=A0 =C2=A0 =C2=A0 =C2=A0Control: I/O- Mem+ BusMaster+ SpecCycle- M= emWINV- VGASnoop- > ParErr- Stepping- SERR- FastB2B- DisINTx- > =C2=A0 =C2=A0 =C2=A0 =C2=A0Status: Cap+ 66MHz- UDF- FastB2B- ParErr- = DEVSEL=3Dslow >TAbort- > SERR- =C2=A0 =C2=A0 =C2=A0 =C2=A0Latency: 32, Cache Line Size: 64 bytes > =C2=A0 =C2=A0 =C2=A0 =C2=A0Interrupt: pin ? routed to IRQ 16 > =C2=A0 =C2=A0 =C2=A0 =C2=A0Region 0: Memory at fdff4000 (64-bit, non-= prefetchable) [size=3D16K] > =C2=A0 =C2=A0 =C2=A0 =C2=A0Capabilities: > =C2=A0 =C2=A0 =C2=A0 =C2=A0Kernel driver in use: HDA Intel > =C2=A0 =C2=A0 =C2=A0 =C2=A0Kernel modules: snd-hda-intel > > From test.log: > > 2011-02-21 22:40:58.888: starting up > LC_ALL=3DC PATH=3D/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/= sbin:/bin > QEMU_AUDIO_DRV=3Dnone /usr/bin/kvm -S -M pc-0.14 -enable-kvm -m 512 -= smp > 3,sockets=3D3,cores=3D1,threads=3D1 -name test -uuid > 307bfcd2-9dec-29b7-1b4d-c46cd9d7cdbc -nodefconfig -nodefaults -charde= v > socket,id=3Dcharmonitor,path=3D/var/lib/libvirt/qemu/test.monitor,ser= ver,nowait > -mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dreadline -rtc base=3Du= tc -boot > c -drive file=3D/var/lib/libvirt/images/test.img,if=3Dnone,id=3Ddrive= -virtio-disk0,boot=3Don,format=3Draw > -device virtio-blk-pci,bus=3Dpci.0,addr=3D0x4,drive=3Ddrive-virtio-di= sk0,id=3Dvirtio-disk0 > -drive file=3D/home/james/ubuntu_10.10_x86.iso,if=3Dnone,media=3Dcdro= m,id=3Ddrive-ide0-1-0,readonly=3Don,format=3Draw > -device ide-drive,bus=3Dide.1,unit=3D0,drive=3Ddrive-ide0-1-0,id=3Did= e0-1-0 > -netdev tap,fd=3D59,id=3Dhostnet0 -device > virtio-net-pci,netdev=3Dhostnet0,id=3Dnet0,mac=3D52:54:00:7d:32:7c,bu= s=3Dpci.0,addr=3D0x3 > -chardev pty,id=3Dcharserial0 -device > isa-serial,chardev=3Dcharserial0,id=3Dserial0 -usb -vnc 127.0.0.1:1 -= vga > cirrus -device pci-assign,host=3D00:14.2,id=3Dhostdev0,configfd=3D60,= bus=3Dpci.0,addr=3D0x6 > -device virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x5 > char device redirected to /dev/pts/1 > kvm: -device virtio-net-pci,netdev=3Dhostnet0,id=3Dnet0,mac=3D52:54:0= 0:7d:32:7c,bus=3Dpci.0,addr=3D0x3: > pci_add_option_rom: failed to find romfile "pxe-virtio.bin" > 2011-02-21 22:41:48.090: shutting down > > Specifically: -device > pci-assign,host=3D00:14.2,id=3Dhostdev0,configfd=3D60,bus=3Dpci.0,add= r=3D0x6 > > Unfortunately I can't get it to boot an ISO through virtmanager! >< > Well, not without deleting the VM and recreating it again. > I shall try getting the latest virt-manager. > > Regards, > > James. > Or I could just try pressing the Apply button... =C2=AC.=C2=AC;; J.