From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6C7FC433EF for ; Mon, 25 Oct 2021 14:14:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B0D1560555 for ; Mon, 25 Oct 2021 14:14:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233395AbhJYOQa (ORCPT ); Mon, 25 Oct 2021 10:16:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29211 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233358AbhJYOQ2 (ORCPT ); Mon, 25 Oct 2021 10:16:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635171246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y7kdLh1gAibb40dMuQ3Jh5CoyoTAW5myWxAjnl8VLiA=; b=fS3gU7KRozuXjMcbBA6p467uoS6+3MVNEsYbZOq37YEQRKDssQe+AiqfHNjYzIfrSBsv7D 1OGeD1DNae6z2yDzomlBlL/0UC4mJbCtkk6xWv/TGd5lPaVlZ5lxcg1w4Pexe2W5tQ3axu 2os0Y3prpbV2CnOicjZHJvt19e1AYAo= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-527-U0L09rnUPYuw1blXHdxFiw-1; Mon, 25 Oct 2021 10:14:04 -0400 X-MC-Unique: U0L09rnUPYuw1blXHdxFiw-1 Received: by mail-ed1-f72.google.com with SMTP id f4-20020a50e084000000b003db585bc274so9985890edl.17 for ; Mon, 25 Oct 2021 07:14:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=Y7kdLh1gAibb40dMuQ3Jh5CoyoTAW5myWxAjnl8VLiA=; b=XAubtzVxAr68RXOY4H0QryIGHknTfH+hq7XZjo87GjBq4fueaU2JG0LLpFE6RnJ3je dRSN4kYNnI3BLhgzjg/KSjv3eRpIHxJpMVGiiz9Z1AnFVZaN22Z6f9vwZFPFDq8W3h7k MmZnFufJZnSjbAB59CHcVccC5ZTwVKdrBneAuy9p1wGO5rk8Jb7G7ruoP8WtP3MisJS7 tqR7FjkV1IsunRy7oRqXhgn+xhiCEpMNBjR5As0ez5JsXtkO3uXqFBrd28SJ5dsWnMrU TMUFp4Ov+Mc1HJzslBYYDKe+pxnbKcTJ8/z6PhtOs+cEs/IxRbc2cpr4WMEsI7o3+bLN eH1A== X-Gm-Message-State: AOAM533vN6cENz97yirH/mAw46Cjh5AyNYKg0zpJ0p0f9whFtKYMHbtH +JGSCLthh/GFXuXWVDAeXKKTl582y4y5ekV3xhjHYhDIno78HupH6xlF2HC5L74Sug4banTd7yh rHg7rYxjhzBJrP2sZKLshhsAA X-Received: by 2002:a17:906:e85:: with SMTP id p5mr23119737ejf.159.1635171242848; Mon, 25 Oct 2021 07:14:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyK5IfFyc+7Vmg+5R4lSOcrX0+AyAPjSbxoTCpVm6PUN+AjEhEtmloYpZcfK6N2LPBhJBDCSw== X-Received: by 2002:a17:906:e85:: with SMTP id p5mr23119652ejf.159.1635171242301; Mon, 25 Oct 2021 07:14:02 -0700 (PDT) Received: from ?IPV6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id n1sm2548649edf.45.2021.10.25.07.14.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 25 Oct 2021 07:14:01 -0700 (PDT) Message-ID: <614858dd-106c-64cc-04bc-f1887b2054d1@redhat.com> Date: Mon, 25 Oct 2021 16:13:59 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v2 00/43] KVM: Halt-polling and x86 APICv overhaul Content-Language: en-US To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Atish Patra , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, David Matlack , Oliver Upton , Jing Zhang References: <20211009021236.4122790-1-seanjc@google.com> From: Paolo Bonzini In-Reply-To: <20211009021236.4122790-1-seanjc@google.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/10/21 04:11, Sean Christopherson wrote: > This is basically two series smushed into one. The first "half" aims > to differentiate between "halt" and a more generic "block", where "halt" > aligns with x86's HLT instruction, the halt-polling mechanisms, and > associated stats, and "block" means any guest action that causes the vCPU > to block/wait. > > The second "half" overhauls x86's APIC virtualization code (Posted > Interrupts on Intel VMX, AVIC on AMD SVM) to do their updates in response > to vCPU (un)blocking in the vcpu_load/put() paths, keying off of the > vCPU's rcuwait status to determine when a blocking vCPU is being put and > reloaded. This idea comes from arm64's kvm_timer_vcpu_put(), which I > stumbled across when diving into the history of arm64's (un)blocking hooks. > > The x86 APICv overhaul allows for killing off several sets of hooks in > common KVM and in x86 KVM (to the vendor code). Moving everything to > vcpu_put/load() also realizes nice cleanups, especially for the Posted > Interrupt code, which required some impressive mental gymnastics to > understand how vCPU task migration interacted with vCPU blocking. > > Non-x86 folks, sorry for the noise. I'm hoping the common parts can get > applied without much fuss so that future versions can be x86-only. > > v2: > - Collect reviews. [Christian, David] > - Add patch to move arm64 WFI functionality out of hooks. [Marc] > - Add RISC-V to the fun. > - Add all the APICv fun. > > v1: https://lkml.kernel.org/r/20210925005528.1145584-1-seanjc@google.com > > Jing Zhang (1): > KVM: stats: Add stat to detect if vcpu is currently blocking > > Sean Christopherson (42): > KVM: VMX: Don't unblock vCPU w/ Posted IRQ if IRQs are disabled in > guest > KVM: SVM: Ensure target pCPU is read once when signalling AVIC > doorbell > KVM: s390: Ensure kvm_arch_no_poll() is read once when blocking vCPU > KVM: Force PPC to define its own rcuwait object > KVM: Update halt-polling stats if and only if halt-polling was > attempted > KVM: Refactor and document halt-polling stats update helper > KVM: Reconcile discrepancies in halt-polling stats > KVM: s390: Clear valid_wakeup in kvm_s390_handle_wait(), not in arch > hook > KVM: Drop obsolete kvm_arch_vcpu_block_finish() > KVM: arm64: Move vGIC v4 handling for WFI out arch callback hook > KVM: Don't block+unblock when halt-polling is successful > KVM: x86: Tweak halt emulation helper names to free up kvm_vcpu_halt() > KVM: Rename kvm_vcpu_block() => kvm_vcpu_halt() > KVM: Split out a kvm_vcpu_block() helper from kvm_vcpu_halt() > KVM: Don't redo ktime_get() when calculating halt-polling > stop/deadline > KVM: x86: Directly block (instead of "halting") UNINITIALIZED vCPUs > KVM: x86: Invoke kvm_vcpu_block() directly for non-HALTED wait states > KVM: Add helpers to wake/query blocking vCPU > KVM: VMX: Skip Posted Interrupt updates if APICv is hard disabled > KVM: VMX: Clean up PI pre/post-block WARNs > KVM: VMX: Drop unnecessary PI logic to handle impossible conditions > KVM: VMX: Use boolean returns for Posted Interrupt "test" helpers > KVM: VMX: Drop pointless PI.NDST update when blocking > KVM: VMX: Save/restore IRQs (instead of CLI/STI) during PI pre/post > block > KVM: VMX: Read Posted Interrupt "control" exactly once per loop > iteration > KVM: VMX: Move Posted Interrupt ndst computation out of write loop > KVM: VMX: Remove vCPU from PI wakeup list before updating PID.NV > KVM: VMX: Handle PI wakeup shenanigans during vcpu_put/load > KVM: Drop unused kvm_vcpu.pre_pcpu field > KVM: Move x86 VMX's posted interrupt list_head to vcpu_vmx > KVM: VMX: Move preemption timer <=> hrtimer dance to common x86 > KVM: x86: Unexport LAPIC's switch_to_{hv,sw}_timer() helpers > KVM: x86: Remove defunct pre_block/post_block kvm_x86_ops hooks > KVM: SVM: Signal AVIC doorbell iff vCPU is in guest mode > KVM: SVM: Don't bother checking for "running" AVIC when kicking for > IPIs > KVM: SVM: Unconditionally mark AVIC as running on vCPU load (with > APICv) > KVM: Drop defunct kvm_arch_vcpu_(un)blocking() hooks > KVM: VMX: Don't do full kick when triggering posted interrupt "fails" > KVM: VMX: Wake vCPU when delivering posted IRQ even if vCPU == this > vCPU > KVM: VMX: Pass desired vector instead of bool for triggering posted > IRQ > KVM: VMX: Fold fallback path into triggering posted IRQ helper > KVM: VMX: Don't do full kick when handling posted interrupt wakeup > > arch/arm64/include/asm/kvm_emulate.h | 2 + > arch/arm64/include/asm/kvm_host.h | 1 - > arch/arm64/kvm/arch_timer.c | 5 +- > arch/arm64/kvm/arm.c | 60 +++--- > arch/arm64/kvm/handle_exit.c | 5 +- > arch/arm64/kvm/psci.c | 2 +- > arch/mips/include/asm/kvm_host.h | 3 - > arch/mips/kvm/emulate.c | 2 +- > arch/powerpc/include/asm/kvm_host.h | 4 +- > arch/powerpc/kvm/book3s_pr.c | 2 +- > arch/powerpc/kvm/book3s_pr_papr.c | 2 +- > arch/powerpc/kvm/booke.c | 2 +- > arch/powerpc/kvm/powerpc.c | 5 +- > arch/riscv/include/asm/kvm_host.h | 1 - > arch/riscv/kvm/vcpu_exit.c | 2 +- > arch/s390/include/asm/kvm_host.h | 4 - > arch/s390/kvm/interrupt.c | 3 +- > arch/s390/kvm/kvm-s390.c | 7 +- > arch/x86/include/asm/kvm-x86-ops.h | 4 - > arch/x86/include/asm/kvm_host.h | 29 +-- > arch/x86/kvm/lapic.c | 4 +- > arch/x86/kvm/svm/avic.c | 95 ++++----- > arch/x86/kvm/svm/svm.c | 8 - > arch/x86/kvm/svm/svm.h | 14 -- > arch/x86/kvm/vmx/nested.c | 2 +- > arch/x86/kvm/vmx/posted_intr.c | 279 ++++++++++++--------------- > arch/x86/kvm/vmx/posted_intr.h | 14 +- > arch/x86/kvm/vmx/vmx.c | 63 +++--- > arch/x86/kvm/vmx/vmx.h | 3 + > arch/x86/kvm/x86.c | 55 ++++-- > include/linux/kvm_host.h | 27 ++- > include/linux/kvm_types.h | 1 + > virt/kvm/async_pf.c | 2 +- > virt/kvm/kvm_main.c | 138 +++++++------ > 34 files changed, 413 insertions(+), 437 deletions(-) > Queued 1-20 and 22-28. Initially I skipped 21 because I didn't receive it, but I have to think more about whether I agree with it. In reality the CMPXCHG loops can really fail just once, because they only race with the processor setting ON=1. But if the warnings were to trigger at all, it would mean that something iffy is happening in the pi_desc->control state machine, and having the check on every iteration is (very marginally) more effective. It's all theoretical, granted. Paolo From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EF77C433F5 for ; Mon, 25 Oct 2021 14:14:21 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 31C5D60555 for ; Mon, 25 Oct 2021 14:14:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 31C5D60555 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tdhkPxP8090Nf7eXr6JEQ1cPWvcqjzm6JPNEdFu42x0=; b=Wb3z20nVqEMuPx NX6Z9q0DDrb2LRiZ2GhwDFqwMqJ6mjOxly6zdrJ8FQC3fhzdpnfubh7JVT5hYavfDnvQO/MKRgmjV K21Kg/QrvFEhPlkuHkvF4cThqXh/B1JfunWX/fvh0QytF2elOa9Q3y9aGo8qz87ydI5G7Aa7Bzwg8 +q0rC9qIs4Uj8CUsJ191QFg19BjD1V109lF0M7Hp840reHuMKQfyzdQHrglg441eO7SauU4wy2J3R cvROxAc9AWFV64n2W3pqfp34O1FWGaj+HGfTRRlMV1wjWZkW0nMt5h/6tOVi47LJIuuiBSPqmvU0/ dh/5YEgLedWVjp2EP0aA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mf0js-00Ge8X-FX; Mon, 25 Oct 2021 14:14:12 +0000 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mf0jo-00Ge72-TF for linux-riscv@lists.infradead.org; Mon, 25 Oct 2021 14:14:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635171248; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y7kdLh1gAibb40dMuQ3Jh5CoyoTAW5myWxAjnl8VLiA=; b=iqVTCohoFvry8IkurRkFX+1xyJvJT5Oi2GzyMN2INY65aNKdJldSSYpvUsStuyHxyCB/bj u8goDLgZ0tpZ5v7RvUn/2VKGvs744wqD+cxV2x3H2pN81CLKPFr8tSyt+Q/DeeFaUEqLo4 Dxi10KBlxENCl8cCbzLqCCdQIEyr3nE= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-451-lGwnjRuDNBaSbjT36m4bAQ-1; Mon, 25 Oct 2021 10:14:04 -0400 X-MC-Unique: lGwnjRuDNBaSbjT36m4bAQ-1 Received: by mail-ed1-f72.google.com with SMTP id j17-20020aa7ca51000000b003dd395c0389so5196577edt.4 for ; Mon, 25 Oct 2021 07:14:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=Y7kdLh1gAibb40dMuQ3Jh5CoyoTAW5myWxAjnl8VLiA=; b=vlhv14nfBPTtzAxZ6GtzNlkfhWrX6GCtuhKy20Y2djlGx9pWUbGcnS3tbTgetcsXEw yWfD0VbkpVWh9HkTQfATHeNM556gKXnOG7GgPOJe59XDDmPGHmYhY8RWWYefMLRxaQh3 1JwChHWReeoggh8LTBCbk2+bbQsHSsp0bfg1Ld6bLEt3PLaHP3+1Vs6q1GBFS8u5fiJX dV4XaqJghQ0oVOmjRLL1dCZ/oshBnZT21jbR1aMzlklCEOATccjn+733uTRVlF2Z8g3/ HDaZU9HPnjwtGRFYF99qrlsjf0YoIzdUJliBTNVG5mXyMMC/v9of69OeyxUZc3FDw8VG 9cAA== X-Gm-Message-State: AOAM533wcSl4z2zRwjo++E0n0PVBxUfLcthDV4ksVQ0ny4uvm+w6kELL DFV9fVuGaQ+IOBLYauMEJ2NA6xab9r1KthTdNVKVjQCOmScBs7Nc+goI8H9rUjO04mXOT1BbqX0 +UX2+tOaX4Yf0oZdWkxxsYNXpXhaG X-Received: by 2002:a17:906:e85:: with SMTP id p5mr23119725ejf.159.1635171242851; Mon, 25 Oct 2021 07:14:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyK5IfFyc+7Vmg+5R4lSOcrX0+AyAPjSbxoTCpVm6PUN+AjEhEtmloYpZcfK6N2LPBhJBDCSw== X-Received: by 2002:a17:906:e85:: with SMTP id p5mr23119652ejf.159.1635171242301; Mon, 25 Oct 2021 07:14:02 -0700 (PDT) Received: from ?IPV6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id n1sm2548649edf.45.2021.10.25.07.14.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 25 Oct 2021 07:14:01 -0700 (PDT) Message-ID: <614858dd-106c-64cc-04bc-f1887b2054d1@redhat.com> Date: Mon, 25 Oct 2021 16:13:59 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v2 00/43] KVM: Halt-polling and x86 APICv overhaul To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Atish Patra , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, David Matlack , Oliver Upton , Jing Zhang References: <20211009021236.4122790-1-seanjc@google.com> From: Paolo Bonzini In-Reply-To: <20211009021236.4122790-1-seanjc@google.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211025_071409_250534_F4DC8E1E X-CRM114-Status: GOOD ( 22.50 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On 09/10/21 04:11, Sean Christopherson wrote: > This is basically two series smushed into one. The first "half" aims > to differentiate between "halt" and a more generic "block", where "halt" > aligns with x86's HLT instruction, the halt-polling mechanisms, and > associated stats, and "block" means any guest action that causes the vCPU > to block/wait. > > The second "half" overhauls x86's APIC virtualization code (Posted > Interrupts on Intel VMX, AVIC on AMD SVM) to do their updates in response > to vCPU (un)blocking in the vcpu_load/put() paths, keying off of the > vCPU's rcuwait status to determine when a blocking vCPU is being put and > reloaded. This idea comes from arm64's kvm_timer_vcpu_put(), which I > stumbled across when diving into the history of arm64's (un)blocking hooks. > > The x86 APICv overhaul allows for killing off several sets of hooks in > common KVM and in x86 KVM (to the vendor code). Moving everything to > vcpu_put/load() also realizes nice cleanups, especially for the Posted > Interrupt code, which required some impressive mental gymnastics to > understand how vCPU task migration interacted with vCPU blocking. > > Non-x86 folks, sorry for the noise. I'm hoping the common parts can get > applied without much fuss so that future versions can be x86-only. > > v2: > - Collect reviews. [Christian, David] > - Add patch to move arm64 WFI functionality out of hooks. [Marc] > - Add RISC-V to the fun. > - Add all the APICv fun. > > v1: https://lkml.kernel.org/r/20210925005528.1145584-1-seanjc@google.com > > Jing Zhang (1): > KVM: stats: Add stat to detect if vcpu is currently blocking > > Sean Christopherson (42): > KVM: VMX: Don't unblock vCPU w/ Posted IRQ if IRQs are disabled in > guest > KVM: SVM: Ensure target pCPU is read once when signalling AVIC > doorbell > KVM: s390: Ensure kvm_arch_no_poll() is read once when blocking vCPU > KVM: Force PPC to define its own rcuwait object > KVM: Update halt-polling stats if and only if halt-polling was > attempted > KVM: Refactor and document halt-polling stats update helper > KVM: Reconcile discrepancies in halt-polling stats > KVM: s390: Clear valid_wakeup in kvm_s390_handle_wait(), not in arch > hook > KVM: Drop obsolete kvm_arch_vcpu_block_finish() > KVM: arm64: Move vGIC v4 handling for WFI out arch callback hook > KVM: Don't block+unblock when halt-polling is successful > KVM: x86: Tweak halt emulation helper names to free up kvm_vcpu_halt() > KVM: Rename kvm_vcpu_block() => kvm_vcpu_halt() > KVM: Split out a kvm_vcpu_block() helper from kvm_vcpu_halt() > KVM: Don't redo ktime_get() when calculating halt-polling > stop/deadline > KVM: x86: Directly block (instead of "halting") UNINITIALIZED vCPUs > KVM: x86: Invoke kvm_vcpu_block() directly for non-HALTED wait states > KVM: Add helpers to wake/query blocking vCPU > KVM: VMX: Skip Posted Interrupt updates if APICv is hard disabled > KVM: VMX: Clean up PI pre/post-block WARNs > KVM: VMX: Drop unnecessary PI logic to handle impossible conditions > KVM: VMX: Use boolean returns for Posted Interrupt "test" helpers > KVM: VMX: Drop pointless PI.NDST update when blocking > KVM: VMX: Save/restore IRQs (instead of CLI/STI) during PI pre/post > block > KVM: VMX: Read Posted Interrupt "control" exactly once per loop > iteration > KVM: VMX: Move Posted Interrupt ndst computation out of write loop > KVM: VMX: Remove vCPU from PI wakeup list before updating PID.NV > KVM: VMX: Handle PI wakeup shenanigans during vcpu_put/load > KVM: Drop unused kvm_vcpu.pre_pcpu field > KVM: Move x86 VMX's posted interrupt list_head to vcpu_vmx > KVM: VMX: Move preemption timer <=> hrtimer dance to common x86 > KVM: x86: Unexport LAPIC's switch_to_{hv,sw}_timer() helpers > KVM: x86: Remove defunct pre_block/post_block kvm_x86_ops hooks > KVM: SVM: Signal AVIC doorbell iff vCPU is in guest mode > KVM: SVM: Don't bother checking for "running" AVIC when kicking for > IPIs > KVM: SVM: Unconditionally mark AVIC as running on vCPU load (with > APICv) > KVM: Drop defunct kvm_arch_vcpu_(un)blocking() hooks > KVM: VMX: Don't do full kick when triggering posted interrupt "fails" > KVM: VMX: Wake vCPU when delivering posted IRQ even if vCPU == this > vCPU > KVM: VMX: Pass desired vector instead of bool for triggering posted > IRQ > KVM: VMX: Fold fallback path into triggering posted IRQ helper > KVM: VMX: Don't do full kick when handling posted interrupt wakeup > > arch/arm64/include/asm/kvm_emulate.h | 2 + > arch/arm64/include/asm/kvm_host.h | 1 - > arch/arm64/kvm/arch_timer.c | 5 +- > arch/arm64/kvm/arm.c | 60 +++--- > arch/arm64/kvm/handle_exit.c | 5 +- > arch/arm64/kvm/psci.c | 2 +- > arch/mips/include/asm/kvm_host.h | 3 - > arch/mips/kvm/emulate.c | 2 +- > arch/powerpc/include/asm/kvm_host.h | 4 +- > arch/powerpc/kvm/book3s_pr.c | 2 +- > arch/powerpc/kvm/book3s_pr_papr.c | 2 +- > arch/powerpc/kvm/booke.c | 2 +- > arch/powerpc/kvm/powerpc.c | 5 +- > arch/riscv/include/asm/kvm_host.h | 1 - > arch/riscv/kvm/vcpu_exit.c | 2 +- > arch/s390/include/asm/kvm_host.h | 4 - > arch/s390/kvm/interrupt.c | 3 +- > arch/s390/kvm/kvm-s390.c | 7 +- > arch/x86/include/asm/kvm-x86-ops.h | 4 - > arch/x86/include/asm/kvm_host.h | 29 +-- > arch/x86/kvm/lapic.c | 4 +- > arch/x86/kvm/svm/avic.c | 95 ++++----- > arch/x86/kvm/svm/svm.c | 8 - > arch/x86/kvm/svm/svm.h | 14 -- > arch/x86/kvm/vmx/nested.c | 2 +- > arch/x86/kvm/vmx/posted_intr.c | 279 ++++++++++++--------------- > arch/x86/kvm/vmx/posted_intr.h | 14 +- > arch/x86/kvm/vmx/vmx.c | 63 +++--- > arch/x86/kvm/vmx/vmx.h | 3 + > arch/x86/kvm/x86.c | 55 ++++-- > include/linux/kvm_host.h | 27 ++- > include/linux/kvm_types.h | 1 + > virt/kvm/async_pf.c | 2 +- > virt/kvm/kvm_main.c | 138 +++++++------ > 34 files changed, 413 insertions(+), 437 deletions(-) > Queued 1-20 and 22-28. Initially I skipped 21 because I didn't receive it, but I have to think more about whether I agree with it. In reality the CMPXCHG loops can really fail just once, because they only race with the processor setting ON=1. But if the warnings were to trigger at all, it would mean that something iffy is happening in the pi_desc->control state machine, and having the check on every iteration is (very marginally) more effective. It's all theoretical, granted. Paolo _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD67CC4332F for ; Mon, 25 Oct 2021 14:14:14 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 41CF260F9B for ; Mon, 25 Oct 2021 14:14:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 41CF260F9B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id BC7104B0B3; Mon, 25 Oct 2021 10:14:13 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@redhat.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6R2+MBXAgMWD; Mon, 25 Oct 2021 10:14:12 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 229724B129; Mon, 25 Oct 2021 10:14:12 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 2A2B44B0B3 for ; Mon, 25 Oct 2021 10:14:10 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1wpH8C-2T4oG for ; Mon, 25 Oct 2021 10:14:08 -0400 (EDT) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 138E349FB7 for ; Mon, 25 Oct 2021 10:14:08 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635171247; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y7kdLh1gAibb40dMuQ3Jh5CoyoTAW5myWxAjnl8VLiA=; b=U17DKI+KTqABAjexjSHwS/O/pdf5LmsUXgreVvRRPw0XBHSV+Zri1Xnz9OE1eP31Wh4UUo eYt+6P8ie0L3u6CQt8NKUOUdUAe7it2Fz7vnTY6TB4C6pjAHpfVKR9Tj/oJ8X04ma0OPqB 2RMZT9bE/IeITXLvB8Hm0ZMTiFEvJVY= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-108-Of8v3bTVPG2dkfz9YFj0BQ-1; Mon, 25 Oct 2021 10:14:04 -0400 X-MC-Unique: Of8v3bTVPG2dkfz9YFj0BQ-1 Received: by mail-ed1-f72.google.com with SMTP id m16-20020a056402431000b003dd2005af01so8383049edc.5 for ; Mon, 25 Oct 2021 07:14:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=Y7kdLh1gAibb40dMuQ3Jh5CoyoTAW5myWxAjnl8VLiA=; b=DhFIzWp1aNvE5dKqnL/pSEBFmuBWfzpjDTUCqu3UrPeCF7K5F88FzizJfTtJVV5qcb 04yR1juKFDWVDP5SRH32PLuWpLcvEDz1niddqDa6OLapQlarTSX/UQXJiipGcIWFRKdo cHCU0B4NW2KJ197kMynOBTmNhOEDtOj6dAKtDOhIWvmvUSu8F+qRYfYNbu2k4xFF1HBg zyXJqiYJOY/k3tKrVJ8KO6AJVcCnMJjUJ/QDrjR2qWQZwpo5Rany6KFWLvRjU3i2uze5 RRWinW+oYg3eLyCqhdVkbvyLwNXUXtAxi0fHNmrgHvijLwmEno1t75JmbfKTWpwGCMid bPug== X-Gm-Message-State: AOAM530cZ+7+DDWOFV7PxLf15QBINHe61uhHHeHcd6i7yoHNTGKXeTwn EwAGQHWZpGxi59yyx5iIRFO2WvXvPTlG0KWw/HNkAnaA6KfHvGOb7Xj59tvkNHvobCfFGPcINKW NPYonx/5HeSPLh3XLACVC+C3R X-Received: by 2002:a17:906:e85:: with SMTP id p5mr23119720ejf.159.1635171242847; Mon, 25 Oct 2021 07:14:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyK5IfFyc+7Vmg+5R4lSOcrX0+AyAPjSbxoTCpVm6PUN+AjEhEtmloYpZcfK6N2LPBhJBDCSw== X-Received: by 2002:a17:906:e85:: with SMTP id p5mr23119652ejf.159.1635171242301; Mon, 25 Oct 2021 07:14:02 -0700 (PDT) Received: from ?IPV6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id n1sm2548649edf.45.2021.10.25.07.14.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 25 Oct 2021 07:14:01 -0700 (PDT) Message-ID: <614858dd-106c-64cc-04bc-f1887b2054d1@redhat.com> Date: Mon, 25 Oct 2021 16:13:59 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v2 00/43] KVM: Halt-polling and x86 APICv overhaul To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank References: <20211009021236.4122790-1-seanjc@google.com> From: Paolo Bonzini In-Reply-To: <20211009021236.4122790-1-seanjc@google.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: Wanpeng Li , kvm@vger.kernel.org, David Hildenbrand , linux-kernel@vger.kernel.org, Atish Patra , linux-riscv@lists.infradead.org, Claudio Imbrenda , kvmarm@lists.cs.columbia.edu, Joerg Roedel , kvm-ppc@vger.kernel.org, David Matlack , linux-arm-kernel@lists.infradead.org, Jim Mattson , Cornelia Huck , linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, Vitaly Kuznetsov X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On 09/10/21 04:11, Sean Christopherson wrote: > This is basically two series smushed into one. The first "half" aims > to differentiate between "halt" and a more generic "block", where "halt" > aligns with x86's HLT instruction, the halt-polling mechanisms, and > associated stats, and "block" means any guest action that causes the vCPU > to block/wait. > > The second "half" overhauls x86's APIC virtualization code (Posted > Interrupts on Intel VMX, AVIC on AMD SVM) to do their updates in response > to vCPU (un)blocking in the vcpu_load/put() paths, keying off of the > vCPU's rcuwait status to determine when a blocking vCPU is being put and > reloaded. This idea comes from arm64's kvm_timer_vcpu_put(), which I > stumbled across when diving into the history of arm64's (un)blocking hooks. > > The x86 APICv overhaul allows for killing off several sets of hooks in > common KVM and in x86 KVM (to the vendor code). Moving everything to > vcpu_put/load() also realizes nice cleanups, especially for the Posted > Interrupt code, which required some impressive mental gymnastics to > understand how vCPU task migration interacted with vCPU blocking. > > Non-x86 folks, sorry for the noise. I'm hoping the common parts can get > applied without much fuss so that future versions can be x86-only. > > v2: > - Collect reviews. [Christian, David] > - Add patch to move arm64 WFI functionality out of hooks. [Marc] > - Add RISC-V to the fun. > - Add all the APICv fun. > > v1: https://lkml.kernel.org/r/20210925005528.1145584-1-seanjc@google.com > > Jing Zhang (1): > KVM: stats: Add stat to detect if vcpu is currently blocking > > Sean Christopherson (42): > KVM: VMX: Don't unblock vCPU w/ Posted IRQ if IRQs are disabled in > guest > KVM: SVM: Ensure target pCPU is read once when signalling AVIC > doorbell > KVM: s390: Ensure kvm_arch_no_poll() is read once when blocking vCPU > KVM: Force PPC to define its own rcuwait object > KVM: Update halt-polling stats if and only if halt-polling was > attempted > KVM: Refactor and document halt-polling stats update helper > KVM: Reconcile discrepancies in halt-polling stats > KVM: s390: Clear valid_wakeup in kvm_s390_handle_wait(), not in arch > hook > KVM: Drop obsolete kvm_arch_vcpu_block_finish() > KVM: arm64: Move vGIC v4 handling for WFI out arch callback hook > KVM: Don't block+unblock when halt-polling is successful > KVM: x86: Tweak halt emulation helper names to free up kvm_vcpu_halt() > KVM: Rename kvm_vcpu_block() => kvm_vcpu_halt() > KVM: Split out a kvm_vcpu_block() helper from kvm_vcpu_halt() > KVM: Don't redo ktime_get() when calculating halt-polling > stop/deadline > KVM: x86: Directly block (instead of "halting") UNINITIALIZED vCPUs > KVM: x86: Invoke kvm_vcpu_block() directly for non-HALTED wait states > KVM: Add helpers to wake/query blocking vCPU > KVM: VMX: Skip Posted Interrupt updates if APICv is hard disabled > KVM: VMX: Clean up PI pre/post-block WARNs > KVM: VMX: Drop unnecessary PI logic to handle impossible conditions > KVM: VMX: Use boolean returns for Posted Interrupt "test" helpers > KVM: VMX: Drop pointless PI.NDST update when blocking > KVM: VMX: Save/restore IRQs (instead of CLI/STI) during PI pre/post > block > KVM: VMX: Read Posted Interrupt "control" exactly once per loop > iteration > KVM: VMX: Move Posted Interrupt ndst computation out of write loop > KVM: VMX: Remove vCPU from PI wakeup list before updating PID.NV > KVM: VMX: Handle PI wakeup shenanigans during vcpu_put/load > KVM: Drop unused kvm_vcpu.pre_pcpu field > KVM: Move x86 VMX's posted interrupt list_head to vcpu_vmx > KVM: VMX: Move preemption timer <=> hrtimer dance to common x86 > KVM: x86: Unexport LAPIC's switch_to_{hv,sw}_timer() helpers > KVM: x86: Remove defunct pre_block/post_block kvm_x86_ops hooks > KVM: SVM: Signal AVIC doorbell iff vCPU is in guest mode > KVM: SVM: Don't bother checking for "running" AVIC when kicking for > IPIs > KVM: SVM: Unconditionally mark AVIC as running on vCPU load (with > APICv) > KVM: Drop defunct kvm_arch_vcpu_(un)blocking() hooks > KVM: VMX: Don't do full kick when triggering posted interrupt "fails" > KVM: VMX: Wake vCPU when delivering posted IRQ even if vCPU == this > vCPU > KVM: VMX: Pass desired vector instead of bool for triggering posted > IRQ > KVM: VMX: Fold fallback path into triggering posted IRQ helper > KVM: VMX: Don't do full kick when handling posted interrupt wakeup > > arch/arm64/include/asm/kvm_emulate.h | 2 + > arch/arm64/include/asm/kvm_host.h | 1 - > arch/arm64/kvm/arch_timer.c | 5 +- > arch/arm64/kvm/arm.c | 60 +++--- > arch/arm64/kvm/handle_exit.c | 5 +- > arch/arm64/kvm/psci.c | 2 +- > arch/mips/include/asm/kvm_host.h | 3 - > arch/mips/kvm/emulate.c | 2 +- > arch/powerpc/include/asm/kvm_host.h | 4 +- > arch/powerpc/kvm/book3s_pr.c | 2 +- > arch/powerpc/kvm/book3s_pr_papr.c | 2 +- > arch/powerpc/kvm/booke.c | 2 +- > arch/powerpc/kvm/powerpc.c | 5 +- > arch/riscv/include/asm/kvm_host.h | 1 - > arch/riscv/kvm/vcpu_exit.c | 2 +- > arch/s390/include/asm/kvm_host.h | 4 - > arch/s390/kvm/interrupt.c | 3 +- > arch/s390/kvm/kvm-s390.c | 7 +- > arch/x86/include/asm/kvm-x86-ops.h | 4 - > arch/x86/include/asm/kvm_host.h | 29 +-- > arch/x86/kvm/lapic.c | 4 +- > arch/x86/kvm/svm/avic.c | 95 ++++----- > arch/x86/kvm/svm/svm.c | 8 - > arch/x86/kvm/svm/svm.h | 14 -- > arch/x86/kvm/vmx/nested.c | 2 +- > arch/x86/kvm/vmx/posted_intr.c | 279 ++++++++++++--------------- > arch/x86/kvm/vmx/posted_intr.h | 14 +- > arch/x86/kvm/vmx/vmx.c | 63 +++--- > arch/x86/kvm/vmx/vmx.h | 3 + > arch/x86/kvm/x86.c | 55 ++++-- > include/linux/kvm_host.h | 27 ++- > include/linux/kvm_types.h | 1 + > virt/kvm/async_pf.c | 2 +- > virt/kvm/kvm_main.c | 138 +++++++------ > 34 files changed, 413 insertions(+), 437 deletions(-) > Queued 1-20 and 22-28. Initially I skipped 21 because I didn't receive it, but I have to think more about whether I agree with it. In reality the CMPXCHG loops can really fail just once, because they only race with the processor setting ON=1. But if the warnings were to trigger at all, it would mean that something iffy is happening in the pi_desc->control state machine, and having the check on every iteration is (very marginally) more effective. It's all theoretical, granted. Paolo _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB6D9C433EF for ; Mon, 25 Oct 2021 14:15:31 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B12F860F9B for ; Mon, 25 Oct 2021 14:15:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B12F860F9B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=TqLzSHXDd8kHI4qH0hrF4xIC/QTTepc/YvckW8Gh2Yc=; b=RyfeiZDFeIiJfz i++i9X/v8ggo2xhFT81XdyjdbP8Q6tk5/r1rgntnwRxGTHot5ho+3m3Hlu8C3Jklh1koWHq2cebyr nF5paxpzDzdfYjdxdqJHlFaz/luL09DxYoigCrX63bK4ujV4rLxDhz9h5OJBrccDe9kvD9MKGGvXK 4seeHNrdT47n40lJC9K99bD1i9qaTCNqFhrglA2dMzLZw4NOw6MMq5Zprno/Pm5nq1Hws7lJLD8wO o4skBauvIzet3isDzpYJvFgY3O6icxk3/pvuREUvi9nVqvwHUM7QotiQmeMaz3uPolKACe269VGOl 80Rc1+zvH7Ek93bypUdg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mf0jv-00Ge99-4D; Mon, 25 Oct 2021 14:14:15 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mf0jo-00Ge74-W3 for linux-arm-kernel@lists.infradead.org; Mon, 25 Oct 2021 14:14:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635171248; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y7kdLh1gAibb40dMuQ3Jh5CoyoTAW5myWxAjnl8VLiA=; b=iqVTCohoFvry8IkurRkFX+1xyJvJT5Oi2GzyMN2INY65aNKdJldSSYpvUsStuyHxyCB/bj u8goDLgZ0tpZ5v7RvUn/2VKGvs744wqD+cxV2x3H2pN81CLKPFr8tSyt+Q/DeeFaUEqLo4 Dxi10KBlxENCl8cCbzLqCCdQIEyr3nE= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-100-af6lB3SDM2GGWXZu0xndLQ-1; Mon, 25 Oct 2021 10:14:04 -0400 X-MC-Unique: af6lB3SDM2GGWXZu0xndLQ-1 Received: by mail-ed1-f72.google.com with SMTP id y3-20020a056402358300b003dd490c775cso794632edc.22 for ; Mon, 25 Oct 2021 07:14:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=Y7kdLh1gAibb40dMuQ3Jh5CoyoTAW5myWxAjnl8VLiA=; b=l28mt+umSqQVxskXhiiMAD+HX9RkTHWeG+xeZzwIwwilafupAl9AyYQFeus5AHUCUl RCmmh+KikAZeXO+1DE0RHkKNfpEBQiQzBQFvOlJFbaqB9FYQpwK/jkGuT6N0e+X+ERJ8 u8XMQPKSyUWYKwviRzs/y0lddI3QuNWkylNaXezCv0p6wNQQOs07eGN5b0fyFw6ncHo3 istcM8l0oORGMAq/lhVags8CqIJ6i4HLZGHn2Za8DS6lvZ9+nhDgsN2wUMKcHteKfXbS Z/TC1yg/3qKlQ4pCrQC/RBGcfH+SUkYAacLYts35Eh9VkYWNXju17wiAJ7yt6CRzPcuH yR5A== X-Gm-Message-State: AOAM533gWtrvS3+YCL7ovUQFNUtomZJgIkcoZuCQuaGCr8lShLf0FO4F 5rKl7obTO1iOic0E2A1Rb64euf88dJyatJHsG/8mp8cSi8GplX2gDXZ7DsBIJvyjSDiQfUeB3Pv i8/3KdOMh7Vg+pEF/XrxcM/u5R+C2/EzaS0Y= X-Received: by 2002:a17:906:e85:: with SMTP id p5mr23119741ejf.159.1635171242850; Mon, 25 Oct 2021 07:14:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyK5IfFyc+7Vmg+5R4lSOcrX0+AyAPjSbxoTCpVm6PUN+AjEhEtmloYpZcfK6N2LPBhJBDCSw== X-Received: by 2002:a17:906:e85:: with SMTP id p5mr23119652ejf.159.1635171242301; Mon, 25 Oct 2021 07:14:02 -0700 (PDT) Received: from ?IPV6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id n1sm2548649edf.45.2021.10.25.07.14.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 25 Oct 2021 07:14:01 -0700 (PDT) Message-ID: <614858dd-106c-64cc-04bc-f1887b2054d1@redhat.com> Date: Mon, 25 Oct 2021 16:13:59 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v2 00/43] KVM: Halt-polling and x86 APICv overhaul To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Atish Patra , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, David Matlack , Oliver Upton , Jing Zhang References: <20211009021236.4122790-1-seanjc@google.com> From: Paolo Bonzini In-Reply-To: <20211009021236.4122790-1-seanjc@google.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211025_071409_250489_B6BE7D78 X-CRM114-Status: GOOD ( 23.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 09/10/21 04:11, Sean Christopherson wrote: > This is basically two series smushed into one. The first "half" aims > to differentiate between "halt" and a more generic "block", where "halt" > aligns with x86's HLT instruction, the halt-polling mechanisms, and > associated stats, and "block" means any guest action that causes the vCPU > to block/wait. > > The second "half" overhauls x86's APIC virtualization code (Posted > Interrupts on Intel VMX, AVIC on AMD SVM) to do their updates in response > to vCPU (un)blocking in the vcpu_load/put() paths, keying off of the > vCPU's rcuwait status to determine when a blocking vCPU is being put and > reloaded. This idea comes from arm64's kvm_timer_vcpu_put(), which I > stumbled across when diving into the history of arm64's (un)blocking hooks. > > The x86 APICv overhaul allows for killing off several sets of hooks in > common KVM and in x86 KVM (to the vendor code). Moving everything to > vcpu_put/load() also realizes nice cleanups, especially for the Posted > Interrupt code, which required some impressive mental gymnastics to > understand how vCPU task migration interacted with vCPU blocking. > > Non-x86 folks, sorry for the noise. I'm hoping the common parts can get > applied without much fuss so that future versions can be x86-only. > > v2: > - Collect reviews. [Christian, David] > - Add patch to move arm64 WFI functionality out of hooks. [Marc] > - Add RISC-V to the fun. > - Add all the APICv fun. > > v1: https://lkml.kernel.org/r/20210925005528.1145584-1-seanjc@google.com > > Jing Zhang (1): > KVM: stats: Add stat to detect if vcpu is currently blocking > > Sean Christopherson (42): > KVM: VMX: Don't unblock vCPU w/ Posted IRQ if IRQs are disabled in > guest > KVM: SVM: Ensure target pCPU is read once when signalling AVIC > doorbell > KVM: s390: Ensure kvm_arch_no_poll() is read once when blocking vCPU > KVM: Force PPC to define its own rcuwait object > KVM: Update halt-polling stats if and only if halt-polling was > attempted > KVM: Refactor and document halt-polling stats update helper > KVM: Reconcile discrepancies in halt-polling stats > KVM: s390: Clear valid_wakeup in kvm_s390_handle_wait(), not in arch > hook > KVM: Drop obsolete kvm_arch_vcpu_block_finish() > KVM: arm64: Move vGIC v4 handling for WFI out arch callback hook > KVM: Don't block+unblock when halt-polling is successful > KVM: x86: Tweak halt emulation helper names to free up kvm_vcpu_halt() > KVM: Rename kvm_vcpu_block() => kvm_vcpu_halt() > KVM: Split out a kvm_vcpu_block() helper from kvm_vcpu_halt() > KVM: Don't redo ktime_get() when calculating halt-polling > stop/deadline > KVM: x86: Directly block (instead of "halting") UNINITIALIZED vCPUs > KVM: x86: Invoke kvm_vcpu_block() directly for non-HALTED wait states > KVM: Add helpers to wake/query blocking vCPU > KVM: VMX: Skip Posted Interrupt updates if APICv is hard disabled > KVM: VMX: Clean up PI pre/post-block WARNs > KVM: VMX: Drop unnecessary PI logic to handle impossible conditions > KVM: VMX: Use boolean returns for Posted Interrupt "test" helpers > KVM: VMX: Drop pointless PI.NDST update when blocking > KVM: VMX: Save/restore IRQs (instead of CLI/STI) during PI pre/post > block > KVM: VMX: Read Posted Interrupt "control" exactly once per loop > iteration > KVM: VMX: Move Posted Interrupt ndst computation out of write loop > KVM: VMX: Remove vCPU from PI wakeup list before updating PID.NV > KVM: VMX: Handle PI wakeup shenanigans during vcpu_put/load > KVM: Drop unused kvm_vcpu.pre_pcpu field > KVM: Move x86 VMX's posted interrupt list_head to vcpu_vmx > KVM: VMX: Move preemption timer <=> hrtimer dance to common x86 > KVM: x86: Unexport LAPIC's switch_to_{hv,sw}_timer() helpers > KVM: x86: Remove defunct pre_block/post_block kvm_x86_ops hooks > KVM: SVM: Signal AVIC doorbell iff vCPU is in guest mode > KVM: SVM: Don't bother checking for "running" AVIC when kicking for > IPIs > KVM: SVM: Unconditionally mark AVIC as running on vCPU load (with > APICv) > KVM: Drop defunct kvm_arch_vcpu_(un)blocking() hooks > KVM: VMX: Don't do full kick when triggering posted interrupt "fails" > KVM: VMX: Wake vCPU when delivering posted IRQ even if vCPU == this > vCPU > KVM: VMX: Pass desired vector instead of bool for triggering posted > IRQ > KVM: VMX: Fold fallback path into triggering posted IRQ helper > KVM: VMX: Don't do full kick when handling posted interrupt wakeup > > arch/arm64/include/asm/kvm_emulate.h | 2 + > arch/arm64/include/asm/kvm_host.h | 1 - > arch/arm64/kvm/arch_timer.c | 5 +- > arch/arm64/kvm/arm.c | 60 +++--- > arch/arm64/kvm/handle_exit.c | 5 +- > arch/arm64/kvm/psci.c | 2 +- > arch/mips/include/asm/kvm_host.h | 3 - > arch/mips/kvm/emulate.c | 2 +- > arch/powerpc/include/asm/kvm_host.h | 4 +- > arch/powerpc/kvm/book3s_pr.c | 2 +- > arch/powerpc/kvm/book3s_pr_papr.c | 2 +- > arch/powerpc/kvm/booke.c | 2 +- > arch/powerpc/kvm/powerpc.c | 5 +- > arch/riscv/include/asm/kvm_host.h | 1 - > arch/riscv/kvm/vcpu_exit.c | 2 +- > arch/s390/include/asm/kvm_host.h | 4 - > arch/s390/kvm/interrupt.c | 3 +- > arch/s390/kvm/kvm-s390.c | 7 +- > arch/x86/include/asm/kvm-x86-ops.h | 4 - > arch/x86/include/asm/kvm_host.h | 29 +-- > arch/x86/kvm/lapic.c | 4 +- > arch/x86/kvm/svm/avic.c | 95 ++++----- > arch/x86/kvm/svm/svm.c | 8 - > arch/x86/kvm/svm/svm.h | 14 -- > arch/x86/kvm/vmx/nested.c | 2 +- > arch/x86/kvm/vmx/posted_intr.c | 279 ++++++++++++--------------- > arch/x86/kvm/vmx/posted_intr.h | 14 +- > arch/x86/kvm/vmx/vmx.c | 63 +++--- > arch/x86/kvm/vmx/vmx.h | 3 + > arch/x86/kvm/x86.c | 55 ++++-- > include/linux/kvm_host.h | 27 ++- > include/linux/kvm_types.h | 1 + > virt/kvm/async_pf.c | 2 +- > virt/kvm/kvm_main.c | 138 +++++++------ > 34 files changed, 413 insertions(+), 437 deletions(-) > Queued 1-20 and 22-28. Initially I skipped 21 because I didn't receive it, but I have to think more about whether I agree with it. In reality the CMPXCHG loops can really fail just once, because they only race with the processor setting ON=1. But if the warnings were to trigger at all, it would mean that something iffy is happening in the pi_desc->control state machine, and having the check on every iteration is (very marginally) more effective. It's all theoretical, granted. Paolo _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 From: Paolo Bonzini Date: Mon, 25 Oct 2021 14:13:59 +0000 Subject: Re: [PATCH v2 00/43] KVM: Halt-polling and x86 APICv overhaul Message-Id: <614858dd-106c-64cc-04bc-f1887b2054d1@redhat.com> List-Id: References: <20211009021236.4122790-1-seanjc@google.com> In-Reply-To: <20211009021236.4122790-1-seanjc@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Atish Patra , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, David Matlack , Oliver Upton , Jing Zhang On 09/10/21 04:11, Sean Christopherson wrote: > This is basically two series smushed into one. The first "half" aims > to differentiate between "halt" and a more generic "block", where "halt" > aligns with x86's HLT instruction, the halt-polling mechanisms, and > associated stats, and "block" means any guest action that causes the vCPU > to block/wait. > > The second "half" overhauls x86's APIC virtualization code (Posted > Interrupts on Intel VMX, AVIC on AMD SVM) to do their updates in response > to vCPU (un)blocking in the vcpu_load/put() paths, keying off of the > vCPU's rcuwait status to determine when a blocking vCPU is being put and > reloaded. This idea comes from arm64's kvm_timer_vcpu_put(), which I > stumbled across when diving into the history of arm64's (un)blocking hooks. > > The x86 APICv overhaul allows for killing off several sets of hooks in > common KVM and in x86 KVM (to the vendor code). Moving everything to > vcpu_put/load() also realizes nice cleanups, especially for the Posted > Interrupt code, which required some impressive mental gymnastics to > understand how vCPU task migration interacted with vCPU blocking. > > Non-x86 folks, sorry for the noise. I'm hoping the common parts can get > applied without much fuss so that future versions can be x86-only. > > v2: > - Collect reviews. [Christian, David] > - Add patch to move arm64 WFI functionality out of hooks. [Marc] > - Add RISC-V to the fun. > - Add all the APICv fun. > > v1: https://lkml.kernel.org/r/20210925005528.1145584-1-seanjc@google.com > > Jing Zhang (1): > KVM: stats: Add stat to detect if vcpu is currently blocking > > Sean Christopherson (42): > KVM: VMX: Don't unblock vCPU w/ Posted IRQ if IRQs are disabled in > guest > KVM: SVM: Ensure target pCPU is read once when signalling AVIC > doorbell > KVM: s390: Ensure kvm_arch_no_poll() is read once when blocking vCPU > KVM: Force PPC to define its own rcuwait object > KVM: Update halt-polling stats if and only if halt-polling was > attempted > KVM: Refactor and document halt-polling stats update helper > KVM: Reconcile discrepancies in halt-polling stats > KVM: s390: Clear valid_wakeup in kvm_s390_handle_wait(), not in arch > hook > KVM: Drop obsolete kvm_arch_vcpu_block_finish() > KVM: arm64: Move vGIC v4 handling for WFI out arch callback hook > KVM: Don't block+unblock when halt-polling is successful > KVM: x86: Tweak halt emulation helper names to free up kvm_vcpu_halt() > KVM: Rename kvm_vcpu_block() => kvm_vcpu_halt() > KVM: Split out a kvm_vcpu_block() helper from kvm_vcpu_halt() > KVM: Don't redo ktime_get() when calculating halt-polling > stop/deadline > KVM: x86: Directly block (instead of "halting") UNINITIALIZED vCPUs > KVM: x86: Invoke kvm_vcpu_block() directly for non-HALTED wait states > KVM: Add helpers to wake/query blocking vCPU > KVM: VMX: Skip Posted Interrupt updates if APICv is hard disabled > KVM: VMX: Clean up PI pre/post-block WARNs > KVM: VMX: Drop unnecessary PI logic to handle impossible conditions > KVM: VMX: Use boolean returns for Posted Interrupt "test" helpers > KVM: VMX: Drop pointless PI.NDST update when blocking > KVM: VMX: Save/restore IRQs (instead of CLI/STI) during PI pre/post > block > KVM: VMX: Read Posted Interrupt "control" exactly once per loop > iteration > KVM: VMX: Move Posted Interrupt ndst computation out of write loop > KVM: VMX: Remove vCPU from PI wakeup list before updating PID.NV > KVM: VMX: Handle PI wakeup shenanigans during vcpu_put/load > KVM: Drop unused kvm_vcpu.pre_pcpu field > KVM: Move x86 VMX's posted interrupt list_head to vcpu_vmx > KVM: VMX: Move preemption timer <=> hrtimer dance to common x86 > KVM: x86: Unexport LAPIC's switch_to_{hv,sw}_timer() helpers > KVM: x86: Remove defunct pre_block/post_block kvm_x86_ops hooks > KVM: SVM: Signal AVIC doorbell iff vCPU is in guest mode > KVM: SVM: Don't bother checking for "running" AVIC when kicking for > IPIs > KVM: SVM: Unconditionally mark AVIC as running on vCPU load (with > APICv) > KVM: Drop defunct kvm_arch_vcpu_(un)blocking() hooks > KVM: VMX: Don't do full kick when triggering posted interrupt "fails" > KVM: VMX: Wake vCPU when delivering posted IRQ even if vCPU = this > vCPU > KVM: VMX: Pass desired vector instead of bool for triggering posted > IRQ > KVM: VMX: Fold fallback path into triggering posted IRQ helper > KVM: VMX: Don't do full kick when handling posted interrupt wakeup > > arch/arm64/include/asm/kvm_emulate.h | 2 + > arch/arm64/include/asm/kvm_host.h | 1 - > arch/arm64/kvm/arch_timer.c | 5 +- > arch/arm64/kvm/arm.c | 60 +++--- > arch/arm64/kvm/handle_exit.c | 5 +- > arch/arm64/kvm/psci.c | 2 +- > arch/mips/include/asm/kvm_host.h | 3 - > arch/mips/kvm/emulate.c | 2 +- > arch/powerpc/include/asm/kvm_host.h | 4 +- > arch/powerpc/kvm/book3s_pr.c | 2 +- > arch/powerpc/kvm/book3s_pr_papr.c | 2 +- > arch/powerpc/kvm/booke.c | 2 +- > arch/powerpc/kvm/powerpc.c | 5 +- > arch/riscv/include/asm/kvm_host.h | 1 - > arch/riscv/kvm/vcpu_exit.c | 2 +- > arch/s390/include/asm/kvm_host.h | 4 - > arch/s390/kvm/interrupt.c | 3 +- > arch/s390/kvm/kvm-s390.c | 7 +- > arch/x86/include/asm/kvm-x86-ops.h | 4 - > arch/x86/include/asm/kvm_host.h | 29 +-- > arch/x86/kvm/lapic.c | 4 +- > arch/x86/kvm/svm/avic.c | 95 ++++----- > arch/x86/kvm/svm/svm.c | 8 - > arch/x86/kvm/svm/svm.h | 14 -- > arch/x86/kvm/vmx/nested.c | 2 +- > arch/x86/kvm/vmx/posted_intr.c | 279 ++++++++++++--------------- > arch/x86/kvm/vmx/posted_intr.h | 14 +- > arch/x86/kvm/vmx/vmx.c | 63 +++--- > arch/x86/kvm/vmx/vmx.h | 3 + > arch/x86/kvm/x86.c | 55 ++++-- > include/linux/kvm_host.h | 27 ++- > include/linux/kvm_types.h | 1 + > virt/kvm/async_pf.c | 2 +- > virt/kvm/kvm_main.c | 138 +++++++------ > 34 files changed, 413 insertions(+), 437 deletions(-) > Queued 1-20 and 22-28. Initially I skipped 21 because I didn't receive it, but I have to think more about whether I agree with it. In reality the CMPXCHG loops can really fail just once, because they only race with the processor setting ON=1. But if the warnings were to trigger at all, it would mean that something iffy is happening in the pi_desc->control state machine, and having the check on every iteration is (very marginally) more effective. It's all theoretical, granted. Paolo