From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Gabriel L. Somlo" Subject: Re: [PATCH RFC] kvm: ignore apic polarity Date: Thu, 27 Feb 2014 18:13:12 -0500 Message-ID: <20140227231312.GH17184@ERROL.INI.CMU.EDU> References: <20140214211311.GH29329@ERROL.INI.CMU.EDU> <20140214220600.GI29329@ERROL.INI.CMU.EDU> <2CEB9F8C-E983-4182-A514-44EC568E18D8@suse.de> <20140216114151.GB30056@redhat.com> <1392562020.15608.437.camel@ul30vt.home> <20140216162300.GI30056@redhat.com> <20140227170549.GA23037@redhat.com> <20140227214102.GG17184@ERROL.INI.CMU.EDU> <530FBC9F.8080800@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: "kvm@vger.kernel.org" , "Michael S. Tsirkin" , "eddie.dong@intel.com" , "qemu-devel@nongnu.org" , Alexander Graf , Alex Williamson To: Paolo Bonzini Return-path: Content-Disposition: inline In-Reply-To: <530FBC9F.8080800@redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org List-Id: kvm.vger.kernel.org On Thu, Feb 27, 2014 at 11:30:55PM +0100, Paolo Bonzini wrote: > Il 27/02/2014 22:41, Gabriel L. Somlo ha scritto: > >On Thu, Feb 27, 2014 at 07:05:49PM +0200, Michael S. Tsirkin wrote: > >>apic polarity in KVM does not work: too many things assume active high. > >>Let's not pretend it works, let's just ignore polarity flag. If we ever > >>want to emulate it exactly, this will need a feature flag anyway. > >> > >>Also report this to userspace: this makes it > >>possible to report the interrupt active-low > >>in ACPI, this way we are closer to real hardware. > >> > >>This patch fixes OSX running on KVM. > >> > >>Reported-by: "Gabriel L. Somlo" > >>Signed-off-by: Michael S. Tsirkin > >> > >>--- > > > >So, the way I understand this (and I'm writing this mainly for myself, > >to make sure I understand correctly, so please kick me if I got it > >wrong), ACPI tells the guest OS how to configure "physical" ioapic polarity. > > > >With ActiveHigh, "physical" == "logical", i.e. "high" == "asserted" > >and "low" == "deasserted". > > > >With ActiveLow, "physical" == !"logical", so the other way around. > > > >QEMU being hard-coded to ActiveHigh is the moral equivalent of always > >sending the kernel (KVM) "logical" line states, rather than "physical" > >ones. > > > >Assuming KVM's userland clients are all coded for ActiveHigh, we can > >(should, for sanity's sake) just assume line states from userland are > >logical, and stop paying attention the polarity bits. That way, > >misbehaving guests [*] can configure their ioapics as they please, and > >things will just work OK regardless. > > > >As you pointed out earlier, even KVM itself already kind-of assumes > >ActiveHigh (e.g. in __kvm_irq_line_state(), which should be coded > >differently if ActiveLow were a serious possibility, and, BTW, > >irq_states[irq] would probably have to be initialized to all-1's if > >ActiveLow wre used, etc, etc). > > This is a much better description. Can you turn it into a patch to > Documentation/virtual/kvm/api.txt and a more complete commit > message? Do you mean one patch to change both virt/kvm/ioapic.c and Documentation/virtual/kvm/api.txt ? Or a separate documentation patch ? (sorry for my ignorance, I'm new to being a KVM contributor :) ) > >With Fedora 20 Live x86_64: > > > >If all I do is 's/ActiveHigh/ActiveLow/' in hw/i386/[q36-]acpi-dsdt.dsl, > >but otherwise don't try to change how QEMU deals with "logical" vs. > >"physical" ioapic polarity, things work great. Printk's show polarity > >set to 1, but with the ignore-polarity patch things work fine. > > > >With normal (ActiveHigh) ACPI, printk reports polarity set to 0, and > >things *still* work exactly the same. > > Also, there is a problem in this: we definitely do not want to have > different ACPI tables for TCG vs. KVM. Have you guys tested what > happens with Linux guests + TCG if interrupts are declared > active-low? I think removing the polarity xor from KVM is about giving up on trying to add ActiveLow support to QEMU altogether. What I tested was what would happen if Linux (which pays attention to ACPI) were told to use ActiveLow, but thre rest of QEMU continued being hardcoded as ActiveHigh. Basically, another datapoint similar to what happens with OS X, which completely ignores ACPI and configures the ioapic as ActiveLow (even while running on ActiveHigh "hardware", i.e. QEMU). With KVM no longer paying attention to the polarity bit, things work fine, both with Linux-thinking-it's-ActiveLow, and with OS X. But, since QEMU will stay ActiveHigh, I don't think TCG will be impacted in any way by this change. (Hmmm, maybe this one of the reasons I never got OS X to boot without -enable-kvm... I should look at the QEMU hw/intc/ioapic*.c, and see if *it* cares about guest-configured polarity, and maybe get it to *stop* caring :) Thanks, --Gabriel > > QEMU likely has many other places that hard-code active-high. One > approach could be to add a QOM property to the ioapic that is a > bitmask of which GSIs are active-low. The ioapic can consult it > like this: > > if (vector >= 0 && vector < IOAPIC_NUM_PINS) { > uint32_t mask = 1 << vector; > uint64_t entry = s->ioredtbl[vector]; > > if (entry & (1 << IOAPIC_LVT_POLARITY_SHIFT)) { > level = !level; > } > + if (s->active_low_mask & (1 << vector)) { > + level = !level; > + } > if (((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1) == > IOAPIC_TRIGGER_LEVEL) { > /* level triggered */ > > etc. so that the two NOTs undo each other, making the input to > QEMU's ioapic also "logical" rather than "physical". > > Paolo From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58130) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WJABA-0005ZR-QN for qemu-devel@nongnu.org; Thu, 27 Feb 2014 18:15:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WJAB2-0000QV-0x for qemu-devel@nongnu.org; Thu, 27 Feb 2014 18:15:48 -0500 Received: from mail-qa0-x22b.google.com ([2607:f8b0:400d:c00::22b]:53063) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WJAB1-0000QD-R2 for qemu-devel@nongnu.org; Thu, 27 Feb 2014 18:15:39 -0500 Received: by mail-qa0-f43.google.com with SMTP id o15so4807679qap.2 for ; Thu, 27 Feb 2014 15:15:39 -0800 (PST) Date: Thu, 27 Feb 2014 18:13:12 -0500 From: "Gabriel L. Somlo" Message-ID: <20140227231312.GH17184@ERROL.INI.CMU.EDU> References: <20140214211311.GH29329@ERROL.INI.CMU.EDU> <20140214220600.GI29329@ERROL.INI.CMU.EDU> <2CEB9F8C-E983-4182-A514-44EC568E18D8@suse.de> <20140216114151.GB30056@redhat.com> <1392562020.15608.437.camel@ul30vt.home> <20140216162300.GI30056@redhat.com> <20140227170549.GA23037@redhat.com> <20140227214102.GG17184@ERROL.INI.CMU.EDU> <530FBC9F.8080800@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <530FBC9F.8080800@redhat.com> Subject: Re: [Qemu-devel] [PATCH RFC] kvm: ignore apic polarity List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: "kvm@vger.kernel.org" , "Michael S. Tsirkin" , "eddie.dong@intel.com" , "qemu-devel@nongnu.org" , Alexander Graf , Alex Williamson On Thu, Feb 27, 2014 at 11:30:55PM +0100, Paolo Bonzini wrote: > Il 27/02/2014 22:41, Gabriel L. Somlo ha scritto: > >On Thu, Feb 27, 2014 at 07:05:49PM +0200, Michael S. Tsirkin wrote: > >>apic polarity in KVM does not work: too many things assume active high. > >>Let's not pretend it works, let's just ignore polarity flag. If we ever > >>want to emulate it exactly, this will need a feature flag anyway. > >> > >>Also report this to userspace: this makes it > >>possible to report the interrupt active-low > >>in ACPI, this way we are closer to real hardware. > >> > >>This patch fixes OSX running on KVM. > >> > >>Reported-by: "Gabriel L. Somlo" > >>Signed-off-by: Michael S. Tsirkin > >> > >>--- > > > >So, the way I understand this (and I'm writing this mainly for myself, > >to make sure I understand correctly, so please kick me if I got it > >wrong), ACPI tells the guest OS how to configure "physical" ioapic polarity. > > > >With ActiveHigh, "physical" == "logical", i.e. "high" == "asserted" > >and "low" == "deasserted". > > > >With ActiveLow, "physical" == !"logical", so the other way around. > > > >QEMU being hard-coded to ActiveHigh is the moral equivalent of always > >sending the kernel (KVM) "logical" line states, rather than "physical" > >ones. > > > >Assuming KVM's userland clients are all coded for ActiveHigh, we can > >(should, for sanity's sake) just assume line states from userland are > >logical, and stop paying attention the polarity bits. That way, > >misbehaving guests [*] can configure their ioapics as they please, and > >things will just work OK regardless. > > > >As you pointed out earlier, even KVM itself already kind-of assumes > >ActiveHigh (e.g. in __kvm_irq_line_state(), which should be coded > >differently if ActiveLow were a serious possibility, and, BTW, > >irq_states[irq] would probably have to be initialized to all-1's if > >ActiveLow wre used, etc, etc). > > This is a much better description. Can you turn it into a patch to > Documentation/virtual/kvm/api.txt and a more complete commit > message? Do you mean one patch to change both virt/kvm/ioapic.c and Documentation/virtual/kvm/api.txt ? Or a separate documentation patch ? (sorry for my ignorance, I'm new to being a KVM contributor :) ) > >With Fedora 20 Live x86_64: > > > >If all I do is 's/ActiveHigh/ActiveLow/' in hw/i386/[q36-]acpi-dsdt.dsl, > >but otherwise don't try to change how QEMU deals with "logical" vs. > >"physical" ioapic polarity, things work great. Printk's show polarity > >set to 1, but with the ignore-polarity patch things work fine. > > > >With normal (ActiveHigh) ACPI, printk reports polarity set to 0, and > >things *still* work exactly the same. > > Also, there is a problem in this: we definitely do not want to have > different ACPI tables for TCG vs. KVM. Have you guys tested what > happens with Linux guests + TCG if interrupts are declared > active-low? I think removing the polarity xor from KVM is about giving up on trying to add ActiveLow support to QEMU altogether. What I tested was what would happen if Linux (which pays attention to ACPI) were told to use ActiveLow, but thre rest of QEMU continued being hardcoded as ActiveHigh. Basically, another datapoint similar to what happens with OS X, which completely ignores ACPI and configures the ioapic as ActiveLow (even while running on ActiveHigh "hardware", i.e. QEMU). With KVM no longer paying attention to the polarity bit, things work fine, both with Linux-thinking-it's-ActiveLow, and with OS X. But, since QEMU will stay ActiveHigh, I don't think TCG will be impacted in any way by this change. (Hmmm, maybe this one of the reasons I never got OS X to boot without -enable-kvm... I should look at the QEMU hw/intc/ioapic*.c, and see if *it* cares about guest-configured polarity, and maybe get it to *stop* caring :) Thanks, --Gabriel > > QEMU likely has many other places that hard-code active-high. One > approach could be to add a QOM property to the ioapic that is a > bitmask of which GSIs are active-low. The ioapic can consult it > like this: > > if (vector >= 0 && vector < IOAPIC_NUM_PINS) { > uint32_t mask = 1 << vector; > uint64_t entry = s->ioredtbl[vector]; > > if (entry & (1 << IOAPIC_LVT_POLARITY_SHIFT)) { > level = !level; > } > + if (s->active_low_mask & (1 << vector)) { > + level = !level; > + } > if (((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1) == > IOAPIC_TRIGGER_LEVEL) { > /* level triggered */ > > etc. so that the two NOTs undo each other, making the input to > QEMU's ioapic also "logical" rather than "physical". > > Paolo