From mboxrd@z Thu Jan  1 00:00:00 1970
From: Alex Williamson <alex.williamson@redhat.com>
Subject: Re: AMD KVM Pci Passthrough reports device busy
Date: Wed, 06 Jun 2012 10:39:30 -0600
Message-ID: <1339000770.23475.281.camel@bling.home>
References: <CALH4D1EJTOBLMh=YaC0AggeddpSchbjjBmqMSmCJs7TTOiscFA@mail.gmail.com>
	 <1338867845.23475.168.camel@bling.home>
	 <201206051039.q55AdXda002609@mail.maya.org>
	 <1338906425.23475.186.camel@bling.home>
	 <201206051517.q55FHU5B002790@mail.maya.org>
	 <4FCE2C89.90209@01019freenet.de> <1338913159.23475.212.camel@bling.home>
	 <4FCE3A0E.5020806@01019freenet.de>
	 <201206060812.q568CSl6002668@mail.maya.org>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Cc: Joerg.Roedel@amd.com, kvm@vger.kernel.org
To: Andreas Hartmann <andihartmann@01019freenet.de>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mx1.redhat.com ([209.132.183.28]:18907 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1757902Ab2FFSF0 (ORCPT <rfc822;kvm@vger.kernel.org>);
	Wed, 6 Jun 2012 14:05:26 -0400
In-Reply-To: <201206060812.q568CSl6002668@mail.maya.org>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Wed, 2012-06-06 at 10:12 +0200, Andreas Hartmann wrote:
> On Tue, 05 Jun 2012 18:55:42 +0200
> Andreas Hartmann <andihartmann@01019freenet.de> wrote:
> 
> > Alex Williamson wrote:
> > [...]
> > > Yep, I think the previous suggestion about reloading vfio_iommu_type1
> > > with allow_unsafe_interrupts=1 will solve it.
> > 
> > Yes! Works now. Success!!!!!
> > 
> > Works means: Device is seen in VM. I couldn't test it up to now, because
> > I don't have any driver in the VM for this device.
> 
> Meanwhile, I enabled the drivers in the VM for the device I passed
> through. Unfortunately, it doesn't work :-(. I'm getting this entry in
> messages at the moment, the module rt2800pci is used by hostapd:
> 
> Jun  6 09:25:02 host kernel: [  201.895812] irq 21: nobody cared (try booting with the "irqpoll" option)
> Jun  6 09:25:02 host kernel: [  201.895819] Pid: 0, comm: swapper/1 Not tainted 3.4.0-next-20120529-16.1-desktop #6
> Jun  6 09:25:02 host kernel: [  201.895822] Call Trace:
> Jun  6 09:25:02 host kernel: [  201.895836]  <IRQ>  [<ffffffff810d37a8>] __report_bad_irq+0x38/0xe0
> Jun  6 09:25:02 host kernel: [  201.895842]  [<ffffffff810d3a6d>] note_interrupt+0x16d/0x220
> Jun  6 09:25:02 host kernel: [  201.895849]  [<ffffffff810d12d6>] handle_irq_event_percpu+0xc6/0x270
> Jun  6 09:25:02 host kernel: [  201.895855]  [<ffffffff810d14c9>] handle_irq_event+0x49/0x70
> Jun  6 09:25:02 host kernel: [  201.895860]  [<ffffffff810d45d2>] handle_fasteoi_irq+0x82/0x130
> Jun  6 09:25:02 host kernel: [  201.895865]  [<ffffffff81004460>] handle_irq+0x20/0x30
> Jun  6 09:25:02 host kernel: [  201.895869]  [<ffffffff81004098>] do_IRQ+0x58/0xe0
> Jun  6 09:25:02 host kernel: [  201.895876]  [<ffffffff815f112a>] common_interrupt+0x6a/0x6a
> Jun  6 09:25:02 host kernel: [  201.895907]  <EOI>  [<ffffffffa0029077>] ? arch_local_irq_enable+0x8/0xd [processor]
> Jun  6 09:25:02 host kernel: [  201.895915]  [<ffffffff8107a37a>] ? sched_clock_idle_wakeup_event+0x1a/0x20
> Jun  6 09:25:02 host kernel: [  201.895929]  [<ffffffffa002a046>] acpi_idle_enter_simple+0xd0/0x111 [processor]
> Jun  6 09:25:02 host kernel: [  201.895939]  [<ffffffff814915f9>] cpuidle_enter+0x19/0x20
> Jun  6 09:25:02 host kernel: [  201.895943]  [<ffffffff81491d81>] cpuidle_idle_call+0xc1/0x1e0
> Jun  6 09:25:02 host kernel: [  201.895949]  [<ffffffff8100bd45>] cpu_idle+0x85/0xd0
> Jun  6 09:25:02 host kernel: [  201.895955]  [<ffffffff815e63d5>] start_secondary+0x8a/0x8c
> Jun  6 09:25:02 host kernel: [  201.895958] handlers:
> Jun  6 09:25:02 host kernel: [  201.895967] [<ffffffffa0488230>] vfio_intx_handler [vfio_pci] threaded [<ffffffffa04884e0>] vfio_intx_thread [vfio_pci]
> Jun  6 09:25:02 host kernel: [  201.895969] Disabling IRQ #21

If there's nothing else on irq 21, this might indicate that we're using
the wrong mechanism to disable interrupts from the device.  Please try
the debug patch below and report what you get for the printk in dmesg
and whether or not the problem goes away.  This may be a candidate for a
device that needs to be blacklisted from reporting that it supports pci
2.3 interrupt disabling.  Please also report 'sudo lspci -vvvxxx -s
06:07.0'.  Thanks,

Alex

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 1e5315c..1aa45b5 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -49,7 +49,8 @@ static int vfio_pci_enable(struct vfio_pci_device *vdev)
 	if (ret)
 		goto out;
 
-	vdev->pci_2_3 = pci_intx_mask_supported(pdev);
+	printk("%s(%s) supports intx mask: %d\n", __func__, dev_name(&vdev->pdev->dev), pci_intx_mask_supported(pdev));
+	//vdev->pci_2_3 = pci_intx_mask_supported(pdev);
 
 	pci_read_config_word(pdev, PCI_COMMAND, &cmd);
 	if (vdev->pci_2_3 && (cmd & PCI_COMMAND_INTX_DISABLE)) {