Re: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding

From: Joerg Roedel <joro@8bytes.org>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: "Wu, Feng" <feng.wu@intel.com>,
	Eric Auger <eric.auger@linaro.org>,
	Avi Kivity <avi.kivity@gmail.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"pbonzini@redhat.com" <pbonzini@redhat.com>,
	"mtosatti@redhat.com" <mtosatti@redhat.com>
Subject: Re: [v4 08/16] KVM: kvm-vfio: User API for IRQ forwarding
Date: Wed, 24 Jun 2015 17:46:16 +0200	[thread overview]
Message-ID: <20150624154616.GB18569@8bytes.org> (raw)
In-Reply-To: <1434657848.3700.83.camel@redhat.com>

On Thu, Jun 18, 2015 at 02:04:08PM -0600, Alex Williamson wrote:
> There are plenty of details to be filled in,

I also need to fill plenty of details in my head first, so here are some
suggestions based on my current understanding. Please don't hesitate to
correct me if where I got something wrong.

So first I totally agree that the handling of PI/non-PI configurations
should be transparent to user-space.

I read a bit through the VT-d spec, and my understanding of posted
interrupts so far is that:

	1) Each VCPU gets a PI-Descriptor with its pending Posted
	   Interrupts. This descriptor needs to be updated when a VCPU
	   is migrated to another PCPU and should thus be under control
	   of KVM.

	   This is similar to the vAPIC backing page in the AMD version
	   of this, except that the PCPU routing information is stored
	   somewhere else on AMD.

	2) As long as the VCPU runs the IRTEs are configured for
	   posting, when the VCPU goes to sleep the old remapped entry is
	   established again. So when the VCPU sleeps the interrupt
	   would get routed to VFIO and forwarded through the eventfd.

	   This would be different to the AMD version, where we have a
	   running bit. When this is clear the IOMMU will trigger an event
	   in its event-log. This might need special handling in VFIO
	   ('might' because VFIO does not need to forward the interrupt,
	    it just needs to make sure the VCPU wakes up).

	   Please correct me if my understanding of the Intel version is
	   wrong.

So most of the data structures the IOMMU reads for this need to be
updated from KVM code (either x86-generic or AMD/Intel specific code),
as KVM has the information about VCPU load/unload and the IRQ routing.

What KVM needs from VFIO are the informations about the physical
interrupts, and it makes total sense to attach them as metadata to the
eventfd.

But the problems start at how this metadata should look like. It would
be good to have some generic description, but not sure if this is
possible. Otherwise this metadata would need to be requested by VFIO
from the IOMMU driver and passed on to KVM, which it then passes back to
the IOMMU driver. Or something like that.

	Joerg