Re: [Qemu-devel] [PATCH v4 00/16] IOMMU: Enable interrupt remapping for Intel IOMMU

[Peter Xu <peterx@redhat.com>]

On Wed, Apr 27, 2016 at 04:31:13PM +0200, Radim Krčmář wrote:

[...]

> >> > I am still looking into guest part codes. Although the above patch
> >> > should solve the issue, there are still issues in guest codes when
> >> > IR is enabled:
> >> > 
> >> > - mismatched "vector" in IOAPIC entry and IRTE entry (this is
> >> >   required in vt-d spec 5.1.5.1, and required to correctly deliver
> >> >   EOI broadcast I guess). See intel_irq_remapping_prepare_irte():
> >> 
> >> "required" is a way of saying that the opposite is undefined.
> >> No need to think about it in IOMMU.
> > 
> > Why? Without correct vector information, IOAPIC will not be able to
> > know which entry to clear the Remote IRR bit (please check
> > ioapic_eoi_broadcast())?
> 
> IOAPIC won't get correct EOI and Intel made it into an OS bug, because
> there was no good action that the hardware could take.  (We have a lot
> more freedom, but I think that partially fixing something that doesn't
> work on real hardware is a wasted effort.)

To make sure I understand this correctly... Do you mean that real
IOAPIC hardware will not handle this EOI broadcast correctly even if
we fill in matched vector in the IOAPIC entry with IRTE one (when IR
is enabled)?

I'd appreciate if there is any link or anything that can provide me
more background on this matter.. TIA.

> 
> Or did you mean that mismatched vector is a possible source of the fixed
> bug?  (I originally dismissed it, because real hardware works.)

Nop. The above patch fixes the hack for "explicit IOAPIC EOI", and I
suppose mismatched vector issue will cause "EOI broadcast" problem.
But IIUC from your above comment, we can temporarily skip this
"issue" for now, if it won't work even on real hardwares and even
vectors are matched.

Anyway, as long as the explicit EOI works, we can survive. And this
gives me the reason to send v5 first.

> 
> >> > - I encountered that level-triggered entries in IOAPIC is marked as
> >> >   edge-triggered interrupt in APIC (which is strange)...
> >> 
> >> What/where do you mean?
> >> (The only difference I know of is that level triggered vectors in LAPIC
> >>  have their respective TMR bit set while edge do not.)
> > 
> > Exactly. Here is what I mean:
> > 
> > static void apic_eoi(APICCommonState *s)
> > {
> >     int isrv;
> >     isrv = get_highest_priority_int(s->isr);
> >     if (isrv < 0)
> >         return;
> >     apic_reset_bit(s->isr, isrv);
> >     if (!(s->spurious_vec & APIC_SV_DIRECTED_IO) && apic_get_bit(s->tmr, isrv)) {
> >         ioapic_eoi_broadcast(isrv);
> >     }
> >     apic_sync_vapic(s, SYNC_FROM_VAPIC | SYNC_TO_VAPIC);
> >     apic_update_irq(s);
> > }
> > 
> > APIC will notify IOAPIC only if the corresponding vector in TMR bit
> > is set (in "apic_get_bit(s->tmr, isrv)", or say, it's a
> > level-triggered interrupt in APIC registers). What I have traced is
> > that, the EOI broadcast is missing because this bit is cleared in
> > APIC TMR while it should be set. I need some more tests to double
> > confirm this though, in case I made any mistake.
> 
> (There are two "legal" situations where TMR can be 0 and IOAPIC sets
>  remote IRR -- if edge and level interrupts are assigned to the same
>  vector and if IOAPIC is level while IR and OS edge, both would bug on
>  real hardware too ...)
> 
> Does QEMU bug with TCG?

Gave it a shot today. It happens as well.

Thanks,

-- peterx