From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFA08C433FE for ; Sun, 26 Sep 2021 09:02:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BED8760FC2 for ; Sun, 26 Sep 2021 09:02:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229713AbhIZJEI (ORCPT ); Sun, 26 Sep 2021 05:04:08 -0400 Received: from mail.kernel.org ([198.145.29.99]:39872 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229551AbhIZJEH (ORCPT ); Sun, 26 Sep 2021 05:04:07 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6F4DC60FC1; Sun, 26 Sep 2021 09:02:31 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mUQ3J-00D1iV-6U; Sun, 26 Sep 2021 10:02:29 +0100 Date: Sun, 26 Sep 2021 10:02:28 +0100 Message-ID: <877df3btgb.wl-maz@kernel.org> From: Marc Zyngier To: Paolo Bonzini Cc: Sean Christopherson , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Christian Borntraeger , Janosch Frank , James Morse , Alexandru Elisei , Suzuki K Poulose , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, linux-kernel@vger.kernel.org, David Matlack , Jing Zhang Subject: Re: [PATCH 07/14] KVM: Don't block+unblock when halt-polling is successful In-Reply-To: <80d90ee6-0d43-3735-5c26-be8c3d72d493@redhat.com> References: <20210925005528.1145584-1-seanjc@google.com> <20210925005528.1145584-8-seanjc@google.com> <878rzlass2.wl-maz@kernel.org> <80d90ee6-0d43-3735-5c26-be8c3d72d493@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: pbonzini@redhat.com, seanjc@google.com, chenhuacai@kernel.org, aleksandar.qemu.devel@gmail.com, paulus@ozlabs.org, borntraeger@de.ibm.com, frankja@linux.ibm.com, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, david@redhat.com, cohuck@redhat.com, imbrenda@linux.ibm.com, vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com, joro@8bytes.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, linux-kernel@vger.kernel.org, dmatlack@google.com, jingzhangos@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Sun, 26 Sep 2021 07:27:28 +0100, Paolo Bonzini wrote: > > On 25/09/21 11:50, Marc Zyngier wrote: > >> there is no need for arm64 to put/load > >> the vGIC as KVM hasn't relinquished control of the vCPU in any way. > > > > This doesn't mean that there is no requirement for any state > > change. The put/load on GICv4 is crucial for performance, and the VMCR > > resync is a correctness requirement. > > I wouldn't even say it's crucial for performance: halt polling cannot > work and is a waste of time without (the current implementation of) > put/load. Not quite. A non-V{LPI,SGI} could still be used as the a wake-up from WFI (which is the only reason we end-up on this path). Only LPIs (and SGIs on GICv4.1) can be directly injected, meaning that SPIs and PPIs still follow the standard SW injection model. However, there is still the ICH_VMCR_EL2 requirement (to get the up-to-date priority mask and group enable bits) for SW-injected interrupt wake-up to work correctly, and I really don't want to save that one eagerly on each shallow exit. > However, is activating the doorbell necessary? If possible, polling > the VGIC directly for pending VLPIs without touching the ITS (for > example by emulating IAR reads) may make sense. IIUC that must be > done at EL2 though, so maybe it would even make sense to move all of > halt polling to EL2 for the nVHE case. It all depends on benchmark > results, of course. No, there is no architectural way to observe the VLPI state. EL2 cannot impersonate the guest an read ICV_IAR1_EL1 (because it conveniently has the same encoding as ICC_IAR1_EL1), and if it could, it would be *destructive* (not what you want). The equivalent of the LR that is used to hold the highest priority VLPI presented to the virtual CPU interface is not visible to SW at all. There are exactly two ways for the hypervisor to get a hint about the VLPI state (and that's only a hint, as everything can be spurious): - Make the vPE non resident and use GICR_VPENDBASER.PendingLast bit to find out whether there are pending VLPIs - Make the vPE non resident and get a doorbell interrupt See the common pattern? There is no polling mechanism, and the only way to flush the VLPI state to memory is to destroy the GIC view of the vPE, which is a bit counter-productive. It also only work on GICv4.1, and not GICv4 (which is why we don't support live migration on GICv4). > Sorry for the many stupid questions I'm asking lately, but I'm trying > to pay more attention to ARM and understand the VGIC and EL1/EL2 split > better. Feel free to ask any question. The more people understand how the architecture works, the better. M. -- Without deviation from the norm, progress is not possible.