Re: AMD KVM Pci Passthrough reports device busy

From: Andreas Hartmann <andihartmann@01019freenet.de>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: Andreas Hartmann <andihartmann@01019freenet.de>,
	Joerg.Roedel@amd.com, kvm@vger.kernel.org
Subject: Re: AMD KVM Pci Passthrough reports device busy
Date: Wed, 6 Jun 2012 21:17:49 +0200	[thread overview]
Message-ID: <201206061917.q56JHqAe003044@mail.maya.org> (raw)
In-Reply-To: <1339000770.23475.281.camel@bling.home>

On Wed, 06 Jun 2012 10:39:30 -0600
Alex Williamson <alex.williamson@redhat.com> wrote:

> On Wed, 2012-06-06 at 10:12 +0200, Andreas Hartmann wrote:
> > On Tue, 05 Jun 2012 18:55:42 +0200
> > Andreas Hartmann <andihartmann@01019freenet.de> wrote:
> > 
> > > Alex Williamson wrote:
> > > [...]
> > > > Yep, I think the previous suggestion about reloading vfio_iommu_type1
> > > > with allow_unsafe_interrupts=1 will solve it.
> > > 
> > > Yes! Works now. Success!!!!!
> > > 
> > > Works means: Device is seen in VM. I couldn't test it up to now, because
> > > I don't have any driver in the VM for this device.
> > 
> > Meanwhile, I enabled the drivers in the VM for the device I passed
> > through. Unfortunately, it doesn't work :-(. I'm getting this entry in
> > messages at the moment, the module rt2800pci is used by hostapd:
> > 
> > Jun  6 09:25:02 host kernel: [  201.895812] irq 21: nobody cared (try booting with the "irqpoll" option)
> > Jun  6 09:25:02 host kernel: [  201.895819] Pid: 0, comm: swapper/1 Not tainted 3.4.0-next-20120529-16.1-desktop #6
> > Jun  6 09:25:02 host kernel: [  201.895822] Call Trace:
> > Jun  6 09:25:02 host kernel: [  201.895836]  <IRQ>  [<ffffffff810d37a8>] __report_bad_irq+0x38/0xe0
> > Jun  6 09:25:02 host kernel: [  201.895842]  [<ffffffff810d3a6d>] note_interrupt+0x16d/0x220
> > Jun  6 09:25:02 host kernel: [  201.895849]  [<ffffffff810d12d6>] handle_irq_event_percpu+0xc6/0x270
> > Jun  6 09:25:02 host kernel: [  201.895855]  [<ffffffff810d14c9>] handle_irq_event+0x49/0x70
> > Jun  6 09:25:02 host kernel: [  201.895860]  [<ffffffff810d45d2>] handle_fasteoi_irq+0x82/0x130
> > Jun  6 09:25:02 host kernel: [  201.895865]  [<ffffffff81004460>] handle_irq+0x20/0x30
> > Jun  6 09:25:02 host kernel: [  201.895869]  [<ffffffff81004098>] do_IRQ+0x58/0xe0
> > Jun  6 09:25:02 host kernel: [  201.895876]  [<ffffffff815f112a>] common_interrupt+0x6a/0x6a
> > Jun  6 09:25:02 host kernel: [  201.895907]  <EOI>  [<ffffffffa0029077>] ? arch_local_irq_enable+0x8/0xd [processor]
> > Jun  6 09:25:02 host kernel: [  201.895915]  [<ffffffff8107a37a>] ? sched_clock_idle_wakeup_event+0x1a/0x20
> > Jun  6 09:25:02 host kernel: [  201.895929]  [<ffffffffa002a046>] acpi_idle_enter_simple+0xd0/0x111 [processor]
> > Jun  6 09:25:02 host kernel: [  201.895939]  [<ffffffff814915f9>] cpuidle_enter+0x19/0x20
> > Jun  6 09:25:02 host kernel: [  201.895943]  [<ffffffff81491d81>] cpuidle_idle_call+0xc1/0x1e0
> > Jun  6 09:25:02 host kernel: [  201.895949]  [<ffffffff8100bd45>] cpu_idle+0x85/0xd0
> > Jun  6 09:25:02 host kernel: [  201.895955]  [<ffffffff815e63d5>] start_secondary+0x8a/0x8c
> > Jun  6 09:25:02 host kernel: [  201.895958] handlers:
> > Jun  6 09:25:02 host kernel: [  201.895967] [<ffffffffa0488230>] vfio_intx_handler [vfio_pci] threaded [<ffffffffa04884e0>] vfio_intx_thread [vfio_pci]
> > Jun  6 09:25:02 host kernel: [  201.895969] Disabling IRQ #21
> 
> If there's nothing else on irq 21, this might indicate that we're using
> the wrong mechanism to disable interrupts from the device.  Please try
> the debug patch below and report what you get for the printk in dmesg
> and whether or not the problem goes away.

Jun  6 21:05:43 host kernel: [  186.133235] vfio_pci_enable(0000:06:07.0) supports intx mask: 1

The device works fine with this patch :-) !!

> This may be a candidate for a
> device that needs to be blacklisted from reporting that it supports pci
> 2.3 interrupt disabling.  Please also report 'sudo lspci -vvvxxx -s
> 06:07.0'.

06:07.0 Network controller: Ralink corp. RT2800 802.11n PCI
        Subsystem: Linksys Device 0067
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 32 (500ns min, 1000ns max), Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 21
        Region 0: Memory at fd8e0000 (32-bit, non-prefetchable) [size=64K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: vfio-pci
00: 14 18 01 06 17 00 10 04 00 00 80 02 10 20 00 00
10: 00 00 8e fd 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 01 80 00 00 37 17 67 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 05 01 02 04
40: 01 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

> Thanks,
> 
> Alex

I've to say thank you! 
If you have the final patch, I would like to test it again!

Kind regards,
Andreas