From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: AMD KVM Pci Passthrough reports device busy Date: Tue, 05 Jun 2012 10:19:19 -0600 Message-ID: <1338913159.23475.212.camel@bling.home> References: <1338867845.23475.168.camel@bling.home> <201206051039.q55AdXda002609@mail.maya.org> <1338906425.23475.186.camel@bling.home> <201206051517.q55FHU5B002790@mail.maya.org> <4FCE2C89.90209@01019freenet.de> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: Joerg.Roedel@amd.com, Chris Sanders , kvm@vger.kernel.org To: Andreas Hartmann Return-path: Received: from mx1.redhat.com ([209.132.183.28]:10040 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751727Ab2FEQTb (ORCPT ); Tue, 5 Jun 2012 12:19:31 -0400 In-Reply-To: <4FCE2C89.90209@01019freenet.de> Sender: kvm-owner@vger.kernel.org List-ID: On Tue, 2012-06-05 at 17:58 +0200, Andreas Hartmann wrote: > Andreas Hartmann wrote: > [...] > > I tried to run qemu-system-x86_64 but got this error on startup: > > > > qemu-system-x86_64: -device vfio-pci,host=06:07.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to set iommu for container: Operation not permitted > > > > qemu-system-x86_64: -device vfio-pci,host=06:07.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to setup container for group 9 > > > > qemu-system-x86_64: -device vfio-pci,host=06:07.0,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to get group 9 > > ** > > ERROR:qom/object.c:389:object_delete: assertion failed: (obj->ref == 0) > > > > > > I started qemu-system-x86_64 with this option among others > > > > -device vfio-pci,host=06:07.0,id=hostdev0,bus=pci.0,addr=0x5 > > > > after all of the devices have been added to pci-stub but 06:07.0, which was added to vfio-pci. > > > > > > Could you please tell me, why the operation isn't permitted? I started > > qemu-system-x86_64 as root. > > I straced the call with strace and got the following error: > > ... > 8048 open("/usr/local/share/qemu/pxe-virtio.rom", O_RDONLY) = 14 > 8048 lseek(14, 0, SEEK_END) = 60416 > 8048 lseek(14, 0, SEEK_SET) = 0 > 8048 read(14, "U\252v\351\217\0z\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\177\0 > \0<\0\365\274\266\16"..., 60416) = 60416 > 8048 close(14) = 0 > 8048 stat("/sys/bus/pci/devices/0000:06:07.0/", {st_mode=S_IFDIR|0755, > st_size=0, ...}) = 0 > 8048 readlink("/sys/bus/pci/devices/0000:06:07.0/iommu_group", > "../../../../kernel/iommu_groups/9"..., 4096) = 33 > 8048 open("/dev/vfio/9", O_RDWR) = 14 > 8048 ioctl(14, 0x3b67, 0x7fff237d5ac0) = 0 > 8048 open("/dev/vfio/vfio", O_RDWR) = 15 > 8048 ioctl(15, 0x3b64, 0xf) = 0 > 8048 ioctl(15, 0x3b65, 0x1) = 1 > 8048 ioctl(14, 0x3b68, 0x7fff237d5ad8) = 0 > 8048 ioctl(15, 0x3b66, 0x1) = -1 EPERM (Operation not permitted) > ... Yep, I think the previous suggestion about reloading vfio_iommu_type1 with allow_unsafe_interrupts=1 will solve it. It has nothing to do with file permissions, you're getting EPERM at the point where we set the iommu type, which enables access to devices. By default we want an iommu which protects against malicious MSI attacks, which requires interrupt remapping on x86. Joerg has been working to add this for AMD since the hardware supports it, but for now, the above options lets us bypass the check. Thanks, Alex