From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41385) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YXW6U-0003db-JA for qemu-devel@nongnu.org; Mon, 16 Mar 2015 10:34:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YXW6J-0002Y8-21 for qemu-devel@nongnu.org; Mon, 16 Mar 2015 10:34:50 -0400 Received: from e28smtp06.in.ibm.com ([122.248.162.6]:56117) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YXW6I-0002Wj-En for qemu-devel@nongnu.org; Mon, 16 Mar 2015 10:34:38 -0400 Received: from /spool/local by e28smtp06.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 16 Mar 2015 20:04:35 +0530 Date: Tue, 17 Mar 2015 01:34:25 +1100 From: Gavin Shan Message-ID: <20150316143425.GA6946@shangw> References: <1426054314-19564-1-git-send-email-gwshan@linux.vnet.ibm.com> <1426054314-19564-2-git-send-email-gwshan@linux.vnet.ibm.com> <1426283487.3643.132.camel@redhat.com> <20150316010459.GA13680@shangw> <1426478732.17565.227.camel@kernel.crashing.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1426478732.17565.227.camel@kernel.crashing.org> Subject: Re: [Qemu-devel] [Qemu-ppc] [PATCH 2/3] VFIO: Clear INTx pending state on EEH reset Reply-To: Gavin Shan List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Benjamin Herrenschmidt Cc: Alex Williamson , david@gibson.dropbear.id.au, qemu-ppc@nongnu.org, Gavin Shan , qemu-devel@nongnu.org On Mon, Mar 16, 2015 at 03:05:32PM +1100, Benjamin Herrenschmidt wrote: >On Mon, 2015-03-16 at 12:04 +1100, Gavin Shan wrote: >> >> >> (2) QEMU sends IOCTL commands to host to disable MSIx and enable INTx. At >> this stage the INTx is still masked. At later point, the guest is requesting >> unmasking INTx, which is captured by host. Host checks and founds pending >> INTx, which is sent to QEMU. In QEMU INTx handler (vfio_intx_interrupt()), >> the mmap'ed regions are disabled, "intx.pending" is set and a timer is started >> to reenable mmap'ed regions if "intx.pending" is cleared there. However, >> "intx.pending" is only cleared upon BAR access in slow path, which is never >> happing. >> >> (3) After guest disables MSIx and issue EEH reset, the device driver starts >> to check its firmware state by reading MMIO register, which isn't completed >> by QEMU VFIO BAR slow path (Note: fast path supported by mmaped regions have >> been disabled). Eventually, the guest hangs on reading MMIO register. With >> this patch applied to QEMU, I didn't see the problem again. > >Note that it might be a good idea to disable INTx (and synchronize with a cfg >read of some sort) around resetting a device. > >Otherwise, you may hit a known issue if the device is behind a switch and has >sent the INTx "assert" message, and not the "deassert" one before it gets reset. > >That can cause the INTx to effectively be "stuck" in the switch preventing a >subsequent one from being delivered. > Yeah, It makes more sense to disable INTx before issuing EEH reset. I verified that disabling INTx interrupt upon EEH reset can avoid the issue as well. I'll post updated patch accordingly if Alex Williamson doesn't object. Thanks, Gavin >Cheers, >Ben. > >