From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53849) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1av3pV-0000PQ-Un for qemu-devel@nongnu.org; Tue, 26 Apr 2016 10:19:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1av3pR-0002sd-TG for qemu-devel@nongnu.org; Tue, 26 Apr 2016 10:19:09 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37954) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1av3pR-0002sY-Lc for qemu-devel@nongnu.org; Tue, 26 Apr 2016 10:19:05 -0400 Date: Tue, 26 Apr 2016 16:19:00 +0200 From: Radim =?utf-8?B?S3LEjW3DocWZ?= Message-ID: <20160426141859.GB19789@potion> References: <1461055122-32378-1-git-send-email-peterx@redhat.com> <571DA823.1030003@web.de> <20160425071806.GF3261@pxdev.xzpeter.org> <571DC61C.9020006@web.de> <20160426073426.GD28545@pxdev.xzpeter.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160426073426.GD28545@pxdev.xzpeter.org> Subject: Re: [Qemu-devel] [PATCH v4 00/16] IOMMU: Enable interrupt remapping for Intel IOMMU List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Peter Xu Cc: Jan Kiszka , qemu-devel@nongnu.org, imammedo@redhat.com, rth@twiddle.net, ehabkost@redhat.com, jasowang@redhat.com, marcel@redhat.com, mst@redhat.com, pbonzini@redhat.com, alex.williamson@redhat.com, wexu@redhat.com 2016-04-26 15:34+0800, Peter Xu: > Hi, Jan, > > The above issue should be caused by EOI missing of level-triggered > interrupts. Before that, I was always using edge-triggered > interrupts for test, so didn't encounter this one. Would you please > help try below patch? It can be applied directly onto the series, > and should solve the issue (it works on my test vm, and I'll take it > in v5 as well if it also works for you): > > ------------------------- > > diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c > @@ -281,6 +281,36 @@ ioapic_mem_read(void *opaque, hwaddr addr, unsigned int size) > +/* > + * This is to satisfy the hack in Linux kernel. One hack of it is to > + * simulate clearing the Remote IRR bit of IOAPIC entry using the > + * following: > + * > + * "For IO-APIC's with EOI register, we use that to do an explicit EOI. > + * Otherwise, we simulate the EOI message manually by changing the trigger > + * mode to edge and then back to level, with RTE being masked during > + * this." > + * > + * (See linux kernel __eoi_ioapic_pin() comment in commit c0205701) > + * > + * This is based on the assumption that, Remote IRR bit will be > + * cleared by IOAPIC hardware for edge-triggered interrupts (I > + * believe that's what the IOAPIC version 0x1X hardware does). I thought that Linux doesn't use explicit "EOI" to IO-APIC, but relies on EOI broadcast from LAPIC -- does that change with IR? > + * So > + * if we are emulating it, we'd better do it the same here, so that > + * the guest kernel hack will work as well on QEMU. Totally. > + * Without this, level-triggered interrupts in IR mode might fail to > + * work correctly. (I don't really understand why it worked before.) > + */ > +static inline void > +ioapic_fix_edge_remote_irr(uint64_t *entry) > +{ > + if (*entry & IOAPIC_LVT_TRIGGER_MODE) { > + /* Level triggered interrupts, make sure remote IRR is zero */ > + *entry &= ~((uint64_t)IOAPIC_LVT_REMOTE_IRR); (You can just unconditionally zero it, edge doesn't care.) > + } > +} > + > @@ -314,6 +344,7 @@ ioapic_mem_write(void *opaque, hwaddr addr, uint64_t val, > s->ioredtbl[index] &= ~0xffffffffULL; > s->ioredtbl[index] |= val; > } > + ioapic_fix_edge_remote_irr(&s->ioredtbl[index]); I think this can be done only in the else branch of (s->ioregsel & 1). (If the guest kernel does level->edge->level, then remote_irr probably should be cleared only on edge->level transition and not on level->level, but I haven't seen that in the spec ...) > ioapic_service(s); > ------------------------ > > I am still looking into guest part codes. Although the above patch > should solve the issue, there are still issues in guest codes when > IR is enabled: > > - mismatched "vector" in IOAPIC entry and IRTE entry (this is > required in vt-d spec 5.1.5.1, and required to correctly deliver > EOI broadcast I guess). See intel_irq_remapping_prepare_irte(): "required" is a way of saying that the opposite is undefined. No need to think about it in IOMMU. > - I encountered that level-triggered entries in IOAPIC is marked as > edge-triggered interrupt in APIC (which is strange)... What/where do you mean? (The only difference I know of is that level triggered vectors in LAPIC have their respective TMR bit set while edge do not.) Thanks.