From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steve Rutherford Subject: Re: [RFC PATCH 3/4] KVM: x86: Add EOI exit bitmap inference Date: Thu, 14 May 2015 16:21:20 -0700 Message-ID: <20150514232120.GB21835@google.com> References: <1431481652-27268-1-git-send-email-srutherford@google.com> <1431481652-27268-3-git-send-email-srutherford@google.com> <5553028F.1010204@redhat.com> <20150513222406.GA24121@google.com> <555468C8.8050303@redhat.com> <1431616989.3625.115.camel@redhat.com> <5554C35E.5060507@redhat.com> <1431619480.3625.137.camel@redhat.com> <20150514221041.GA21835@google.com> <1431642919.3625.148.camel@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Paolo Bonzini , kvm@vger.kernel.org, ahonig@google.com To: Alex Williamson Return-path: Received: from mail-ie0-f178.google.com ([209.85.223.178]:34819 "EHLO mail-ie0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932871AbbENXVY (ORCPT ); Thu, 14 May 2015 19:21:24 -0400 Received: by iebpz10 with SMTP id pz10so75469418ieb.2 for ; Thu, 14 May 2015 16:21:23 -0700 (PDT) Content-Disposition: inline In-Reply-To: <1431642919.3625.148.camel@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On Thu, May 14, 2015 at 04:35:19PM -0600, Alex Williamson wrote: > On Thu, 2015-05-14 at 15:10 -0700, Steve Rutherford wrote: > > On Thu, May 14, 2015 at 10:04:40AM -0600, Alex Williamson wrote: > > > On Thu, 2015-05-14 at 17:46 +0200, Paolo Bonzini wrote: > > > > > > > > On 14/05/2015 17:23, Alex Williamson wrote: > > > > > > Would irq_ack_notifiers be used at all with this patch set? Resampling > > > > > > of IOAPIC level-triggered interrupts would be implemented in userspace. > > > > > > For the same reason, assigned devices using legacy device assignment > > > > > > probably would not be able to use INTX (so this feature should depend on > > > > > > !KVM_DEVICE_ASSIGNMENT). Add the emulated i8254 and the last user of > > > > > > irq_ack_notifiers goes away. > > > > > > > > > > > > Alex, how does VFIO do INTX resampling if you're using TCG or -machine > > > > > > kernel_irqchip=off? (Context: this series keeps the local APIC > > > > > > emulation in the kernel, thus including MSI, but moves the IOAPIC > > > > > > emualtion to userspace). > > > > > > > > > > Without KVM irqchip, we use a rudimentary approach where we disable > > > > > mmaps to the device when an interrupt occurs. This makes us trap all > > > > > accesses to the device. We then handle any device access from the guest > > > > > as a potential EOI and unmask the interrupt. If the interrupt re-fires, > > > > > we've at least rate-limited it. If it doesn't, a timer eventually > > > > > re-enables devices mmaps. It's not very efficient, but it works (for > > > > > both PCI and platform devices) and avoids IRQ APIs through QEMU that get > > > > > very platform/machine specific. Thanks, > > > > > > > > If we move the IOAPIC back to userspace, it probably would be easier to > > > > implement irqfd/resamplefd in userspace as well. Let the bikeshedding > > > > start! > > > > > > The current solution is merely functional with the benefit of being > > > architecture and machine-type independent. If someone actually cared > > > about vfio assigned device performance without KVM irqchip, we'd > > > certainly need a more deterministic and efficient EOI path. Thanks, > > > > > > Alex > > > > > > > I'd like this feature to not preclude fast /modern/ device assignment > > (i.e. devices using MSIs). A perf hit (or even incompatibility) for > > legacy devices is fine [that's sort of the premise of this patchset]. > > What's necessary for the device assignment to work with the split > > irqchip? > > The current code will attempt to use kvm_irqchip_add_msi_route() and > kvm_irqchip_add_irqfd_notifier() to route the MSI through the KVM > irqchip. Without that, QEMU will receive the MSI eventfd and call > msi_notify() or msix_notify(), which would be significantly slower. > This should be pretty much the same as virtio, which you've hopefully > already taken into consideration. Thanks, > > Alex > Those will be easy to update, given that the routing table is staying in the kernel. The notion of "kvm_irqchip_in_kernel" will need to be updated when support for this is added to QEMU, but that's not a huge deal.