From: Andreas Hartmann <andihartmann@01019freenet.de>
To: Andreas Hartmann <andihartmann@01019freenet.de>
Cc: Alex Williamson <alex.williamson@redhat.com>,
Joerg.Roedel@amd.com, kvm@vger.kernel.org
Subject: Re: AMD KVM Pci Passthrough reports device busy
Date: Wed, 6 Jun 2012 10:12:27 +0200 [thread overview]
Message-ID: <201206060812.q568CSl6002668@mail.maya.org> (raw)
In-Reply-To: <4FCE3A0E.5020806@01019freenet.de>
On Tue, 05 Jun 2012 18:55:42 +0200
Andreas Hartmann <andihartmann@01019freenet.de> wrote:
> Alex Williamson wrote:
> [...]
> > Yep, I think the previous suggestion about reloading vfio_iommu_type1
> > with allow_unsafe_interrupts=1 will solve it.
>
> Yes! Works now. Success!!!!!
>
> Works means: Device is seen in VM. I couldn't test it up to now, because
> I don't have any driver in the VM for this device.
Meanwhile, I enabled the drivers in the VM for the device I passed
through. Unfortunately, it doesn't work :-(. I'm getting this entry in
messages at the moment, the module rt2800pci is used by hostapd:
Jun 6 09:25:02 host kernel: [ 201.895812] irq 21: nobody cared (try booting with the "irqpoll" option)
Jun 6 09:25:02 host kernel: [ 201.895819] Pid: 0, comm: swapper/1 Not tainted 3.4.0-next-20120529-16.1-desktop #6
Jun 6 09:25:02 host kernel: [ 201.895822] Call Trace:
Jun 6 09:25:02 host kernel: [ 201.895836] <IRQ> [<ffffffff810d37a8>] __report_bad_irq+0x38/0xe0
Jun 6 09:25:02 host kernel: [ 201.895842] [<ffffffff810d3a6d>] note_interrupt+0x16d/0x220
Jun 6 09:25:02 host kernel: [ 201.895849] [<ffffffff810d12d6>] handle_irq_event_percpu+0xc6/0x270
Jun 6 09:25:02 host kernel: [ 201.895855] [<ffffffff810d14c9>] handle_irq_event+0x49/0x70
Jun 6 09:25:02 host kernel: [ 201.895860] [<ffffffff810d45d2>] handle_fasteoi_irq+0x82/0x130
Jun 6 09:25:02 host kernel: [ 201.895865] [<ffffffff81004460>] handle_irq+0x20/0x30
Jun 6 09:25:02 host kernel: [ 201.895869] [<ffffffff81004098>] do_IRQ+0x58/0xe0
Jun 6 09:25:02 host kernel: [ 201.895876] [<ffffffff815f112a>] common_interrupt+0x6a/0x6a
Jun 6 09:25:02 host kernel: [ 201.895907] <EOI> [<ffffffffa0029077>] ? arch_local_irq_enable+0x8/0xd [processor]
Jun 6 09:25:02 host kernel: [ 201.895915] [<ffffffff8107a37a>] ? sched_clock_idle_wakeup_event+0x1a/0x20
Jun 6 09:25:02 host kernel: [ 201.895929] [<ffffffffa002a046>] acpi_idle_enter_simple+0xd0/0x111 [processor]
Jun 6 09:25:02 host kernel: [ 201.895939] [<ffffffff814915f9>] cpuidle_enter+0x19/0x20
Jun 6 09:25:02 host kernel: [ 201.895943] [<ffffffff81491d81>] cpuidle_idle_call+0xc1/0x1e0
Jun 6 09:25:02 host kernel: [ 201.895949] [<ffffffff8100bd45>] cpu_idle+0x85/0xd0
Jun 6 09:25:02 host kernel: [ 201.895955] [<ffffffff815e63d5>] start_secondary+0x8a/0x8c
Jun 6 09:25:02 host kernel: [ 201.895958] handlers:
Jun 6 09:25:02 host kernel: [ 201.895967] [<ffffffffa0488230>] vfio_intx_handler [vfio_pci] threaded [<ffffffffa04884e0>] vfio_intx_thread [vfio_pci]
Jun 6 09:25:02 host kernel: [ 201.895969] Disabling IRQ #21
I tried with irqpoll, but this didn't help. BTW: IRQ 21 isn't a shared
interrupt! Did I miss an option during kernel configuration?
Just to remember:
The passed through device is a WLAN-device and it's driven by rt2800pci:
06:07.0 Network controller: Ralink corp. RT2800 802.11n PCI
I unbound all the devices in the same group:
echo "1002 4385" > /sys/bus/pci/drivers/pci-stub/new_id
echo 0000:00:14.0 > /sys/bus/pci/devices/0000:00:14.0/driver/unbind
echo 0000:00:14.0 > /sys/bus/pci/drivers/pci-stub/bind
echo "1002 439c " > /sys/bus/pci/drivers/pci-stub/new_id
echo 0000:00:14.1 > /sys/bus/pci/devices/0000:00:14.1/driver/unbind
echo 0000:00:14.1 > /sys/bus/pci/drivers/pci-stub/bind
echo "1002 4383" > /sys/bus/pci/drivers/pci-stub/new_id
echo 0000:00:14.2 > /sys/bus/pci/devices/0000:00:14.2/driver/unbind
echo 0000:00:14.2 > /sys/bus/pci/drivers/pci-stub/bind
echo "1002 439d" > /sys/bus/pci/drivers/pci-stub/new_id
echo 0000:00:14.3 > /sys/bus/pci/devices/0000:00:14.3/driver/unbind
echo 0000:00:14.3 > /sys/bus/pci/drivers/pci-stub/bind
echo "1002 4384" > /sys/bus/pci/drivers/pci-stub/new_id
echo 0000:00:14.4 > /sys/bus/pci/devices/0000:00:14.4/driver/unbind
echo 0000:00:14.4 > /sys/bus/pci/drivers/pci-stub/bind
echo "1002 4399" > /sys/bus/pci/drivers/pci-stub/new_id
echo 0000:00:14.5 > /sys/bus/pci/devices/0000:00:14.5/driver/unbind
echo 0000:00:14.5 > /sys/bus/pci/drivers/pci-stub/bind
modprobe vfio-pci
echo "1814 0601" > /sys/bus/pci/drivers/vfio-pci/new_id
echo 0000:06:07.0 > /sys/bus/pci/devices/0000:06:07.0/driver/unbind
echo 0000:06:07.0 > /sys/bus/pci/drivers/vfio-pci/bind
The corresponding entries in messages are:
Jun 6 09:23:01 host kernel: [ 81.777453] pci-stub 0000:00:14.0: claimed by stub
Jun 6 09:23:01 host kernel: [ 81.777788] pci-stub 0000:00:14.0: claimed by stub
Jun 6 09:23:02 host kernel: [ 81.782465] pci-stub 0000:00:14.1: claimed by stub
Jun 6 09:23:02 host kernel: [ 81.801572] AMD-Vi: Event logged [IO_PAGE_FAULT device=00:14.0 domain=0x0000 address=0x000000fdf9103300 flags=0x0600]
Jun 6 09:23:02 host kernel: [ 81.950388] pci-stub 0000:00:14.2: claimed by stub
Jun 6 09:23:02 host kernel: [ 81.950505] pci-stub 0000:00:14.3: claimed by stub
Jun 6 09:23:02 host kernel: [ 81.950723] pci-stub 0000:00:14.3: claimed by stub
Jun 6 09:23:02 host kernel: [ 81.950828] pci-stub 0000:00:14.4: claimed by stub
Jun 6 09:23:02 host kernel: [ 81.951031] pci-stub 0000:00:14.4: claimed by stub
Jun 6 09:23:02 host kernel: [ 81.951280] ohci_hcd 0000:00:14.5: remove, state 4
Jun 6 09:23:02 host kernel: [ 81.951293] usb usb6: USB disconnect, device number 1
Jun 6 09:23:02 host kernel: [ 81.951840] ohci_hcd 0000:00:14.5: USB bus 6 deregistered
Jun 6 09:23:02 host kernel: [ 81.951993] pci-stub 0000:00:14.5: claimed by stub
Jun 6 09:23:02 host kernel: [ 81.957543] VFIO - User Level meta-driver version: 0.3
Jun 6 09:23:02 host kernel: [ 81.960441] vfio_iommu_group_notifier: Device 0000:06:07.0, group 9 bound to driver vfio-pci
Jun 6 09:23:02 host kernel: [ 81.960617] vfio_iommu_group_notifier: Device 0000:06:07.0, group 9 unbinding from driver vfio-pci
Jun 6 09:23:02 host kernel: [ 81.961002] vfio_iommu_group_notifier: Device 0000:06:07.0, group 9 bound to driver vfio-pci
The result with lspci -vs14 is:
00:14.0 SMBus: Advanced Micro Devices [AMD] nee ATI SBx00 SMBus Controller (rev 42)
Flags: 66MHz, medium devsel
Kernel driver in use: pci-stub
00:14.1 IDE interface: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 IDE Controller (rev 40) (prog-if 8a [Master SecP PriP])
Subsystem: Giga-byte Technology Device 5002
Flags: 66MHz, medium devsel, IRQ 17
I/O ports at 01f0 [size=8]
I/O ports at 03f4 [size=1]
I/O ports at 0170 [size=8]
I/O ports at 0374 [size=1]
I/O ports at fa00 [size=16]
Kernel driver in use: pci-stub
00:14.2 Audio device: Advanced Micro Devices [AMD] nee ATI SBx00 Azalia (Intel HDA) (rev 40)
Subsystem: Giga-byte Technology Device a132
Flags: slow devsel, IRQ 16
Memory at fdff4000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 2
Kernel driver in use: pci-stub
00:14.3 ISA bridge: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
Subsystem: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 LPC host controller
Flags: bus master, 66MHz, medium devsel, latency 0
Kernel driver in use: pci-stub
00:14.4 PCI bridge: Advanced Micro Devices [AMD] nee ATI SBx00 PCI to PCI Bridge (rev 40) (prog-if 01 [Subtractive decode])
Flags: bus master, VGA palette snoop, 66MHz, medium devsel, latency 64
Bus: primary=00, secondary=06, subordinate=06, sec-latency=64
I/O behind bridge: 00009000-00009fff
Memory behind bridge: fd800000-fd8fffff
Prefetchable memory behind bridge: fd700000-fd7fffff
Kernel driver in use: pci-stub
00:14.5 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI2 Controller (prog-if 10 [OHCI])
Subsystem: Giga-byte Technology Device 5004
Flags: 66MHz, medium devsel, IRQ 18
Memory at fdffa000 (32-bit, non-prefetchable) [size=4K]
Kernel driver in use: pci-stub
lspci -vs7
06:07.0 Network controller: Ralink corp. RT2800 802.11n PCI
Subsystem: Linksys Device 0067
Flags: bus master, slow devsel, latency 32, IRQ 5
Memory at fd8e0000 (32-bit, non-prefetchable) [size=64K]
Capabilities: [40] Power Management version 3
Kernel driver in use: vfio-pci
On starting the VM, the following entries came up in messages:
Jun 6 09:23:51 host kernel: [ 131.397055] kvm: 8175: cpu0 unhandled rdmsr: 0xc0010001
The VM is started as root with this command:
/usr/local/bin/qemu-system-x86_64 \
-enable-kvm \
-m 1024 \
-rtc base=utc \
-drive file=/my.hd,if=none,id=drive-virtio-disk0,format=qcow2,cache=writeback \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-netdev tap,ifname=tap5,script=no,id=hostnet0 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=12:34:56:78:90:12,bus=pci.0,addr=0x3 \
-device vfio-pci,host=06:07.0,id=hostdev0,bus=pci.0,addr=0x5
qemu-system-x86_64 prints out the following on startup:
Warning, device 0000:06:07.0 does not support reset (-> only the first time the VM is started)
sometimes:
qemu-system-x86_64: vfio_dma_map(0x7fb10a848170, 0x00000000000cc000, 0x4000, 0x7fb0bfacc000) = -16 (Device or resource busy)
qemu-system-x86_64: vfio_dma_map(0x7fb10a848170, 0x00000000000cc000, 0x4000, 0x7fb0bfacc000) = -16 (Device or resource busy)
Do you have any idea what's going wrong here?
Thanks,
kind regards,
Andreas
next prev parent reply other threads:[~2012-06-06 8:15 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-04 21:11 AMD KVM Pci Passthrough reports device busy Chris Sanders
2012-06-05 3:44 ` Alex Williamson
2012-06-05 10:39 ` Andreas Hartmann
2012-06-05 14:27 ` Alex Williamson
2012-06-05 15:17 ` Andreas Hartmann
2012-06-05 15:48 ` Alex Williamson
2012-06-05 15:58 ` Andreas Hartmann
2012-06-05 16:19 ` Alex Williamson
2012-06-05 16:55 ` Andreas Hartmann
2012-06-05 18:43 ` Alex Williamson
2012-06-05 20:37 ` Andreas Hartmann
2012-06-05 21:09 ` Alex Williamson
2012-06-05 22:02 ` Andreas Hartmann
2012-06-06 8:12 ` Andreas Hartmann [this message]
2012-06-06 8:46 ` Andreas Hartmann
2012-06-06 9:35 ` Andreas Hartmann
2012-06-06 16:39 ` Alex Williamson
2012-06-06 19:17 ` Andreas Hartmann
2012-06-06 10:11 ` Joerg Roedel
2012-06-25 5:55 ` Andreas Hartmann
2012-06-25 11:22 ` Joerg Roedel
2012-07-11 14:26 ` Andreas Hartmann
2012-07-11 16:58 ` Joerg Roedel
2012-07-11 19:32 ` Andreas Hartmann
2012-07-11 20:01 ` Alex Williamson
2012-06-06 1:32 ` sheng qiu
2012-06-06 3:07 ` Chris Sanders
2012-06-06 3:25 ` Alex Williamson
2012-06-06 3:31 ` Chris Sanders
2012-06-06 5:27 ` Alex Williamson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201206060812.q568CSl6002668@mail.maya.org \
--to=andihartmann@01019freenet.de \
--cc=Joerg.Roedel@amd.com \
--cc=alex.williamson@redhat.com \
--cc=kvm@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).