Re: KVM devices assignment; PCIe AER?

From: Etienne Martineau <etmartin101@gmail.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: Chris Wright <chrisw@sous-sol.org>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	kvm@vger.kernel.org
Subject: Re: KVM devices assignment; PCIe AER?
Date: Wed, 27 Oct 2010 14:43:51 -0700 (PDT)	[thread overview]
Message-ID: <alpine.DEB.2.00.1010271228001.1805@ubuntu.ubuntu-domain> (raw)
In-Reply-To: <1288206999.5129.203.camel@x201>

On Wed, 27 Oct 2010, Alex Williamson wrote:
> No, emulated devices trigger interrupts directly with qemu_set_irq.
> irqfds are currently only used by vhost afaik, since it's being
> interrupted externally, much like pass through devices are.

Fair enough. Thanks for the clarification.

> Sort of.  When the VFIO device triggers an interrupt, we get notified
> via the eventfd we've registered for that interrupt.  We can then call
> qemu_set_irq directly to raise that interrupt in the KVM kernel APIC.
> That much works today.

Understood but performance wise this is no good for KVM right?

> The irqfd mechanism is simply a way for KVM to
> directly consume the eventfd and raise an interrupt via a pre-setup
> vector.  That's yet to be implemented for INTx on VFIO, but should
> mostly be a matter of connecting existing pieces together.  It's working
> for MSI-X.

OK, I was on the impression you already had irqfd 'connected' to KVM from 
VFIO... This is why I was asking about the nature of the changed in VFIO.

> When VFIO sends an interrupt, it disables the physical device from
> generating more interrupts (this is where VFIO requires PCI 2.3
> compliant devices for the INTx disable bit int he status register).
> When the guest services the interrupt, we can detect this by catching
> the EOI of the IOAPIC.  At that point, we can re-eanble interrupts on
> the device.  Wash, rinse, repeat.
>
> To do this in qemu, I created a callback on the ioapic where drivers can
> register for the interrupt they care about.  Since KVM moves the ioapic
> into the kernel, we need to extend this into KVM and have yet another
> eventfd mechanism.  It's possible that we could have the VFIO kernel
> module also receive this eventfd, re-enabling interrupts on the device,
> in much the same way as above.

In the cases of KVM where are you going to catch the EIO? For some 
reason I'm on the impression that this is part of KVM. If so then how are 
you going to 'signal' to VFIO? Cannot use eventfd here right?

> Yes, none of this requires KVM specific modifications to VFIO.  VFIO is
> still just triggering eventfds, and hopefully receiving one via an
> irqfd-like mechanism for EOI.

Thanks for your reply.
-Etienne