From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4EE9C433FE for ; Mon, 25 Oct 2021 14:01:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B683261039 for ; Mon, 25 Oct 2021 14:01:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233187AbhJYOEM (ORCPT ); Mon, 25 Oct 2021 10:04:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46000 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233083AbhJYOEK (ORCPT ); Mon, 25 Oct 2021 10:04:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635170508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5GxuWusEjMHZ4JQn5qLWUyg1MyGQPS66ltUGy4HnHFU=; b=LTAJJXA6gfcvH8FfYagOIEjcLjtfU1pf6A1jqzc9+E74Nzq4xvgT/emRiNrwGpl7EeuaZG rxMUo8rjBjFRlw1ujqmRYrN8yaly/emG/PCs4jUIsivvieTF3mvRkJFgXU3CWw2VIHMg+V IBTgj3rYwZXMbdSyiBwwtz/OPhEVtfA= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-555-YAQ6iQXcOQmu_4oCsl9cEQ-1; Mon, 25 Oct 2021 10:01:46 -0400 X-MC-Unique: YAQ6iQXcOQmu_4oCsl9cEQ-1 Received: by mail-wm1-f69.google.com with SMTP id 128-20020a1c0486000000b0030dcd45476aso3731604wme.0 for ; Mon, 25 Oct 2021 07:01:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=5GxuWusEjMHZ4JQn5qLWUyg1MyGQPS66ltUGy4HnHFU=; b=hnEYZwIbcMv0B+pA+8vp4HZN82ZKIOaKdWNjMh4LXGOqFuoD0Ov/K6J5+/Oay6pGgQ zB0akNPIjlsN6UrCTdEaBbSwaY+m2PKOtbdgangmDne9tYoryXQVaHrVU8y4oq2Ljm2q ExZN25Yg9zsEsS7hyz+ySuVYAtqtPF788GO9NhZUPGU6VFoc80flZKq2acmT/nLrqJ+A fyPNYzo6/+jrTdjKvkivjTZesHmh6S04aWK77l5GUhDT2kq48LXXA7Z5hnHMjMjk1ebe 3uQ5zl3TFrfAN0kbxVaK+btMmj7JCKRjCC0bOAJZT1qxazpU420LNGa/6WB69OXwve/M Hilg== X-Gm-Message-State: AOAM533bR3DpJznHgnaNwe9ZR/pIOPZjbG7GguvyuJSl3FgoF1NWD02p 0+6X29Npy1OPchqLn7RnF2y1amgxmTr3oR42jGtZrU7P+Ka2koOLgMLdk4y5uWBok3PlZSzlKOY McZIa2sAgUmZWCVpsz7sua1Hn X-Received: by 2002:a5d:59ae:: with SMTP id p14mr23589898wrr.76.1635170505455; Mon, 25 Oct 2021 07:01:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwiqFlcSYWH7tg/r7nyk/acEU7XFm6PKMcaIfNQ0d2l1GPzyRzlNJCdOi1PnQIqdfTr2oUQaw== X-Received: by 2002:a5d:59ae:: with SMTP id p14mr23589820wrr.76.1635170505123; Mon, 25 Oct 2021 07:01:45 -0700 (PDT) Received: from ?IPV6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id x2sm5765371wmj.3.2021.10.25.07.01.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 25 Oct 2021 07:01:44 -0700 (PDT) Message-ID: <18e6a656-f583-0ad4-6770-9678be3f5cf4@redhat.com> Date: Mon, 25 Oct 2021 16:01:42 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v2 24/43] KVM: VMX: Drop pointless PI.NDST update when blocking Content-Language: en-US To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Atish Patra , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, David Matlack , Oliver Upton , Jing Zhang References: <20211009021236.4122790-1-seanjc@google.com> <20211009021236.4122790-25-seanjc@google.com> From: Paolo Bonzini In-Reply-To: <20211009021236.4122790-25-seanjc@google.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/10/21 04:12, Sean Christopherson wrote: > Don't update Posted Interrupt's NDST, a.k.a. the target pCPU, in the > pre-block path, as NDST is guaranteed to be up-to-date. The comment > about the vCPU being preempted during the update is simply wrong, as the > update path runs with IRQs disabled (from before snapshotting vcpu->cpu, > until after the update completes). Right, it didn't as of commit bf9f6ac8d74969690df1485b33b7c238ca9f2269 (when VT-d posted interrupts were introduced). The interrupt disable/enable pair was added in the same commit that motivated the introduction of the sanity checks: commit 8b306e2f3c41939ea528e6174c88cfbfff893ce1 Author: Paolo Bonzini Date: Tue Jun 6 12:57:05 2017 +0200 KVM: VMX: avoid double list add with VT-d posted interrupts In some cases, for example involving hot-unplug of assigned devices, pi_post_block can forget to remove the vCPU from the blocked_vcpu_list. When this happens, the next call to pi_pre_block corrupts the list. Fix this in two ways. First, check vcpu->pre_pcpu in pi_pre_block and WARN instead of adding the element twice in the list. Second, always do the list removal in pi_post_block if vcpu->pre_pcpu is set (not -1). The new code keeps interrupts disabled for the whole duration of pi_pre_block/pi_post_block. This is not strictly necessary, but easier to follow. For the same reason, PI.ON is checked only after the cmpxchg, and to handle it we just call the post-block code. This removes duplication of the list removal code. At the time, I didn't notice the now useless NDST update. Paolo > The vCPU can get preempted_before_ the update starts, but not during. > And if the vCPU is preempted before, vmx_vcpu_pi_load() is responsible > for updating NDST when the vCPU is scheduled back in. In that case, the > check against the wakeup vector in vmx_vcpu_pi_load() cannot be true as > that would require the notification vector to have been set to the wakeup > vector_before_ blocking. > > Opportunistically switch to using vcpu->cpu for the list/lock lookups, > which presumably used pre_pcpu only for some phantom preemption logic. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01A61C433FE for ; Mon, 25 Oct 2021 14:01:54 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 76AFC60F9B for ; Mon, 25 Oct 2021 14:01:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 76AFC60F9B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id F2ED14B15A; Mon, 25 Oct 2021 10:01:52 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@redhat.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IeCerDgM7S+W; Mon, 25 Oct 2021 10:01:51 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 9A27B4B0B9; Mon, 25 Oct 2021 10:01:51 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id DDE584B0B9 for ; Mon, 25 Oct 2021 10:01:49 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9DSKoi3mzO27 for ; Mon, 25 Oct 2021 10:01:48 -0400 (EDT) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 739B74B0B8 for ; Mon, 25 Oct 2021 10:01:48 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635170508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5GxuWusEjMHZ4JQn5qLWUyg1MyGQPS66ltUGy4HnHFU=; b=LTAJJXA6gfcvH8FfYagOIEjcLjtfU1pf6A1jqzc9+E74Nzq4xvgT/emRiNrwGpl7EeuaZG rxMUo8rjBjFRlw1ujqmRYrN8yaly/emG/PCs4jUIsivvieTF3mvRkJFgXU3CWw2VIHMg+V IBTgj3rYwZXMbdSyiBwwtz/OPhEVtfA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-31-LsAdj3JmP9G0zH6gEWKtNA-1; Mon, 25 Oct 2021 10:01:46 -0400 X-MC-Unique: LsAdj3JmP9G0zH6gEWKtNA-1 Received: by mail-wr1-f72.google.com with SMTP id v15-20020adfa1cf000000b00160940b17a2so3234172wrv.19 for ; Mon, 25 Oct 2021 07:01:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=5GxuWusEjMHZ4JQn5qLWUyg1MyGQPS66ltUGy4HnHFU=; b=LAqocajIt/+ysOrOn4LWmWPgEBMNC7vVK+8VhJeM66LnpmbxStfhciTQYkA8ZDQhLX JWnTSpOUMiPSdKw3IskvGmmrGQE4x+I6Rq1UrV0SdxSnv+XnAS5Df659nO6nX1mn43mW 76tcfhMRIrrzOPRCfrtx1HvzsAOgaJ1X04lr/2uK/D2fR1FknMMzYoKAhEW8wfkBiFO9 RqYjmQZUD4TkIlt/a+ipOfXMc+d+ziQP5m5ehtTsX24UW6pMDR6qInVyJyIZ4sIzyvFa 3giUoEQrwfZbsVO2oG58DK3pvvVVKqgVCewf9g6iSHMzDVykmAHQrq9UhTrrvLkHDmw0 epXg== X-Gm-Message-State: AOAM5308HZlgaDO83g8iIc7AaQ8dJ6qpEJqoPJxQ+jlptmg4qvmMDDHS +KUGS3wdQrC9i37Gy9yJRhFGwdTwk2ejKjIYRJ17sj8jWAzNgNFK5ca/LEXjE6Z43xm9NaV1HIH R4qf/YvaAet1wkT/lb2jI6axE X-Received: by 2002:a5d:59ae:: with SMTP id p14mr23589897wrr.76.1635170505454; Mon, 25 Oct 2021 07:01:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwiqFlcSYWH7tg/r7nyk/acEU7XFm6PKMcaIfNQ0d2l1GPzyRzlNJCdOi1PnQIqdfTr2oUQaw== X-Received: by 2002:a5d:59ae:: with SMTP id p14mr23589820wrr.76.1635170505123; Mon, 25 Oct 2021 07:01:45 -0700 (PDT) Received: from ?IPV6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id x2sm5765371wmj.3.2021.10.25.07.01.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 25 Oct 2021 07:01:44 -0700 (PDT) Message-ID: <18e6a656-f583-0ad4-6770-9678be3f5cf4@redhat.com> Date: Mon, 25 Oct 2021 16:01:42 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v2 24/43] KVM: VMX: Drop pointless PI.NDST update when blocking To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank References: <20211009021236.4122790-1-seanjc@google.com> <20211009021236.4122790-25-seanjc@google.com> From: Paolo Bonzini In-Reply-To: <20211009021236.4122790-25-seanjc@google.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: Wanpeng Li , kvm@vger.kernel.org, David Hildenbrand , linux-kernel@vger.kernel.org, Atish Patra , linux-riscv@lists.infradead.org, Claudio Imbrenda , kvmarm@lists.cs.columbia.edu, Joerg Roedel , kvm-ppc@vger.kernel.org, David Matlack , linux-arm-kernel@lists.infradead.org, Jim Mattson , Cornelia Huck , linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org, Vitaly Kuznetsov X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On 09/10/21 04:12, Sean Christopherson wrote: > Don't update Posted Interrupt's NDST, a.k.a. the target pCPU, in the > pre-block path, as NDST is guaranteed to be up-to-date. The comment > about the vCPU being preempted during the update is simply wrong, as the > update path runs with IRQs disabled (from before snapshotting vcpu->cpu, > until after the update completes). Right, it didn't as of commit bf9f6ac8d74969690df1485b33b7c238ca9f2269 (when VT-d posted interrupts were introduced). The interrupt disable/enable pair was added in the same commit that motivated the introduction of the sanity checks: commit 8b306e2f3c41939ea528e6174c88cfbfff893ce1 Author: Paolo Bonzini Date: Tue Jun 6 12:57:05 2017 +0200 KVM: VMX: avoid double list add with VT-d posted interrupts In some cases, for example involving hot-unplug of assigned devices, pi_post_block can forget to remove the vCPU from the blocked_vcpu_list. When this happens, the next call to pi_pre_block corrupts the list. Fix this in two ways. First, check vcpu->pre_pcpu in pi_pre_block and WARN instead of adding the element twice in the list. Second, always do the list removal in pi_post_block if vcpu->pre_pcpu is set (not -1). The new code keeps interrupts disabled for the whole duration of pi_pre_block/pi_post_block. This is not strictly necessary, but easier to follow. For the same reason, PI.ON is checked only after the cmpxchg, and to handle it we just call the post-block code. This removes duplication of the list removal code. At the time, I didn't notice the now useless NDST update. Paolo > The vCPU can get preempted_before_ the update starts, but not during. > And if the vCPU is preempted before, vmx_vcpu_pi_load() is responsible > for updating NDST when the vCPU is scheduled back in. In that case, the > check against the wakeup vector in vmx_vcpu_pi_load() cannot be true as > that would require the notification vector to have been set to the wakeup > vector_before_ blocking. > > Opportunistically switch to using vcpu->cpu for the list/lock lookups, > which presumably used pre_pcpu only for some phantom preemption logic. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3863C433EF for ; Mon, 25 Oct 2021 14:02:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A203F60E08 for ; Mon, 25 Oct 2021 14:02:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A203F60E08 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ihBKG46fIevwPVn9UT+9Gdosj+BApSluID2Xm7M2dcM=; b=423r2OD2JDoYs2 BVTd+B3DpzMIq2KcvZ9kyFMwmO+o/ZcdQx3UobildQ0I8IKU6OUsSIR9FKMUB6HtfuEz2ne2aNdsv 4HgKkG3r7wEAAdwRRJ0Dc/AUltidhsM6XYw8D6BMr/Z338graE2ZHjW8rcvejash9iqaD/SW/wGZK EpSszeOZKCss5rgKPSKioZfeEaL2zymzmb/dkioeY2Cc3xOju7JfLyJlMRmBVKU4ArICwQ4x68YQT y9UYIpzGYxDDlEHkh4cppKuBvMeb1DAUnF2/apX0V6VS+8JrgNt460YwSt0eusQ9XiDSuiRIbCiuP 8IgwLeWBNK3yFokZtxhw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mf0Y5-00GcBM-9r; Mon, 25 Oct 2021 14:02:01 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mf0Xt-00Gc8Q-BF for linux-riscv@lists.infradead.org; Mon, 25 Oct 2021 14:01:51 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635170508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5GxuWusEjMHZ4JQn5qLWUyg1MyGQPS66ltUGy4HnHFU=; b=LTAJJXA6gfcvH8FfYagOIEjcLjtfU1pf6A1jqzc9+E74Nzq4xvgT/emRiNrwGpl7EeuaZG rxMUo8rjBjFRlw1ujqmRYrN8yaly/emG/PCs4jUIsivvieTF3mvRkJFgXU3CWw2VIHMg+V IBTgj3rYwZXMbdSyiBwwtz/OPhEVtfA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-76-sBx1erdZOy2GvIaHisobJA-1; Mon, 25 Oct 2021 10:01:46 -0400 X-MC-Unique: sBx1erdZOy2GvIaHisobJA-1 Received: by mail-wr1-f72.google.com with SMTP id z8-20020adfec88000000b00168672580e0so1928567wrn.21 for ; Mon, 25 Oct 2021 07:01:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=5GxuWusEjMHZ4JQn5qLWUyg1MyGQPS66ltUGy4HnHFU=; b=K94Ef1AWlUQdp4DJ8bAGrIStb3+hYttJfpTEtv9kfnwTpHop2OqEpZW/DPsBsIFvn+ SJ1E07/xFQ1aUuePknO9ELEFlzu4N6rHU0kbDVLfilStQWKjFjQV3eZ//Sf5+kINu744 kB+SNHBvjiGMc21sr3lRQTEQRVmcFbV71BUwS+fvJ+14TwcZ91i4RyGlbSZm0tYxe2W3 n5HPmuQYPuvkqqPHfi1qOHAuwBvn8W+4eK29ndExazXKCiXiuRfjXOqHdv959BKQlqok UTrBESJfda1WgEwrw1fWP3QhVHm15YTso4nIO6s17Y9wVfe6Yude8Sg6P7HZMZQbip+8 0SHw== X-Gm-Message-State: AOAM530toHDIAv7FsLT9hZfy0nqmG8k3IoPriYPwxH3/n5/NGW2YmXJm 5JfdOfytKplL3Tc/mhCklHo2a+fLISCiKmOlIDmv5kQiCA8cGkLaHP0BPsJmV+gjjNYsJB7T4jb +r99mDkA5cgQNArJo4M5MX7RbMcpQ X-Received: by 2002:a5d:59ae:: with SMTP id p14mr23589890wrr.76.1635170505449; Mon, 25 Oct 2021 07:01:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwiqFlcSYWH7tg/r7nyk/acEU7XFm6PKMcaIfNQ0d2l1GPzyRzlNJCdOi1PnQIqdfTr2oUQaw== X-Received: by 2002:a5d:59ae:: with SMTP id p14mr23589820wrr.76.1635170505123; Mon, 25 Oct 2021 07:01:45 -0700 (PDT) Received: from ?IPV6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id x2sm5765371wmj.3.2021.10.25.07.01.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 25 Oct 2021 07:01:44 -0700 (PDT) Message-ID: <18e6a656-f583-0ad4-6770-9678be3f5cf4@redhat.com> Date: Mon, 25 Oct 2021 16:01:42 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v2 24/43] KVM: VMX: Drop pointless PI.NDST update when blocking To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Atish Patra , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, David Matlack , Oliver Upton , Jing Zhang References: <20211009021236.4122790-1-seanjc@google.com> <20211009021236.4122790-25-seanjc@google.com> From: Paolo Bonzini In-Reply-To: <20211009021236.4122790-25-seanjc@google.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211025_070149_475588_EF769DE2 X-CRM114-Status: GOOD ( 16.77 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On 09/10/21 04:12, Sean Christopherson wrote: > Don't update Posted Interrupt's NDST, a.k.a. the target pCPU, in the > pre-block path, as NDST is guaranteed to be up-to-date. The comment > about the vCPU being preempted during the update is simply wrong, as the > update path runs with IRQs disabled (from before snapshotting vcpu->cpu, > until after the update completes). Right, it didn't as of commit bf9f6ac8d74969690df1485b33b7c238ca9f2269 (when VT-d posted interrupts were introduced). The interrupt disable/enable pair was added in the same commit that motivated the introduction of the sanity checks: commit 8b306e2f3c41939ea528e6174c88cfbfff893ce1 Author: Paolo Bonzini Date: Tue Jun 6 12:57:05 2017 +0200 KVM: VMX: avoid double list add with VT-d posted interrupts In some cases, for example involving hot-unplug of assigned devices, pi_post_block can forget to remove the vCPU from the blocked_vcpu_list. When this happens, the next call to pi_pre_block corrupts the list. Fix this in two ways. First, check vcpu->pre_pcpu in pi_pre_block and WARN instead of adding the element twice in the list. Second, always do the list removal in pi_post_block if vcpu->pre_pcpu is set (not -1). The new code keeps interrupts disabled for the whole duration of pi_pre_block/pi_post_block. This is not strictly necessary, but easier to follow. For the same reason, PI.ON is checked only after the cmpxchg, and to handle it we just call the post-block code. This removes duplication of the list removal code. At the time, I didn't notice the now useless NDST update. Paolo > The vCPU can get preempted_before_ the update starts, but not during. > And if the vCPU is preempted before, vmx_vcpu_pi_load() is responsible > for updating NDST when the vCPU is scheduled back in. In that case, the > check against the wakeup vector in vmx_vcpu_pi_load() cannot be true as > that would require the notification vector to have been set to the wakeup > vector_before_ blocking. > > Opportunistically switch to using vcpu->cpu for the list/lock lookups, > which presumably used pre_pcpu only for some phantom preemption logic. _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F81CC433EF for ; Mon, 25 Oct 2021 14:03:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 253CE60F22 for ; Mon, 25 Oct 2021 14:03:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 253CE60F22 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=OJ7M38/qA+FBLVn+AlIceCGl7xmjsPPHAQuI2fE28Uk=; b=MG0fflcYEw46HK Nu5sN2H0eMRzGkwUlF/iUTRBAKczqmA2YtVDCF5kQVyaYfVtpjLe5s7j/7yi0n/pA2wx1D784CIQM Fl8qt4g7bUeIUxpx0Fz5DZzIi3UJbhDqOfOBxsh0Ewt202J78B19eIniANDVukdFvaCC3mN3cM2zk Q0lwGTbCgKIzqLQ37GGrKsLfQL1P/sriGTw0a8pXdIQNrfysa/g0n45p8cK8Ff4B24LjDyqY5jBgb 1n9Winxq6k5vx25+wUs4TqgrYP6f1+IW3ZmnMmnCppLa80nF5KDenFSpvqOo0lvh3ucjHKefuC4rm R8LFHkWzlHogbFZxot0Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mf0Xw-00Gc9N-4x; Mon, 25 Oct 2021 14:01:52 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mf0Xs-00Gc8P-Vm for linux-arm-kernel@lists.infradead.org; Mon, 25 Oct 2021 14:01:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635170508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5GxuWusEjMHZ4JQn5qLWUyg1MyGQPS66ltUGy4HnHFU=; b=LTAJJXA6gfcvH8FfYagOIEjcLjtfU1pf6A1jqzc9+E74Nzq4xvgT/emRiNrwGpl7EeuaZG rxMUo8rjBjFRlw1ujqmRYrN8yaly/emG/PCs4jUIsivvieTF3mvRkJFgXU3CWw2VIHMg+V IBTgj3rYwZXMbdSyiBwwtz/OPhEVtfA= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-246-hcCkvXwMNGOOr1MdIcuo-g-1; Mon, 25 Oct 2021 10:01:46 -0400 X-MC-Unique: hcCkvXwMNGOOr1MdIcuo-g-1 Received: by mail-wr1-f71.google.com with SMTP id g15-20020a5d64ef000000b0016a1331535eso504563wri.10 for ; Mon, 25 Oct 2021 07:01:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=5GxuWusEjMHZ4JQn5qLWUyg1MyGQPS66ltUGy4HnHFU=; b=sHyTWdjqcJ1bUVPmYMk6ZPwAoKpJ+egzPZj4MV+rSpHtmk1+Lygk0ep0a7zBJprs3V IYKX9M66ixNiMuxubOhkJFybKIB7qtm68cbbb5IDklUpsGKLJvlg6HFr9Jzslbjycrq2 zJAQPfItxz6eVjGySh+WQsfrwaAwoSu3OLQ5aDSEetJi246mKgdxgiN6Yc4/1hqm3p6Q iwWV+KIsUk3yBRVZ+LHyEFHxa/Y/YIkTXF55Y+duHecgpHpgrMsDlCk3cDpooacKOSEU FZxN2SEl2EyTTcJbt4d2RuI/pVlDI7+CoQG65LDAN7QGD9YG0YliJYPVQ/AlUbFWKMYh 7P3A== X-Gm-Message-State: AOAM530fqePhUUlbgAVx068xWx2dzQ6n/6vJagzj00wF5foqtXLke1qo tYxDXOH5YdTEBK7Ks838t2x+XdTPufIIhevKmyubw6Y52p0PAMHoCb4rcFlYVXOUkxkky1SFgNk 7SVUxvbOr0GMOt9qPpoynVhd11kcYLhilsy4= X-Received: by 2002:a5d:59ae:: with SMTP id p14mr23589894wrr.76.1635170505453; Mon, 25 Oct 2021 07:01:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwiqFlcSYWH7tg/r7nyk/acEU7XFm6PKMcaIfNQ0d2l1GPzyRzlNJCdOi1PnQIqdfTr2oUQaw== X-Received: by 2002:a5d:59ae:: with SMTP id p14mr23589820wrr.76.1635170505123; Mon, 25 Oct 2021 07:01:45 -0700 (PDT) Received: from ?IPV6:2001:b07:6468:f312:c8dd:75d4:99ab:290a? ([2001:b07:6468:f312:c8dd:75d4:99ab:290a]) by smtp.gmail.com with ESMTPSA id x2sm5765371wmj.3.2021.10.25.07.01.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 25 Oct 2021 07:01:44 -0700 (PDT) Message-ID: <18e6a656-f583-0ad4-6770-9678be3f5cf4@redhat.com> Date: Mon, 25 Oct 2021 16:01:42 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v2 24/43] KVM: VMX: Drop pointless PI.NDST update when blocking To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Atish Patra , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, David Matlack , Oliver Upton , Jing Zhang References: <20211009021236.4122790-1-seanjc@google.com> <20211009021236.4122790-25-seanjc@google.com> From: Paolo Bonzini In-Reply-To: <20211009021236.4122790-25-seanjc@google.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211025_070149_129754_52EC9910 X-CRM114-Status: GOOD ( 18.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 09/10/21 04:12, Sean Christopherson wrote: > Don't update Posted Interrupt's NDST, a.k.a. the target pCPU, in the > pre-block path, as NDST is guaranteed to be up-to-date. The comment > about the vCPU being preempted during the update is simply wrong, as the > update path runs with IRQs disabled (from before snapshotting vcpu->cpu, > until after the update completes). Right, it didn't as of commit bf9f6ac8d74969690df1485b33b7c238ca9f2269 (when VT-d posted interrupts were introduced). The interrupt disable/enable pair was added in the same commit that motivated the introduction of the sanity checks: commit 8b306e2f3c41939ea528e6174c88cfbfff893ce1 Author: Paolo Bonzini Date: Tue Jun 6 12:57:05 2017 +0200 KVM: VMX: avoid double list add with VT-d posted interrupts In some cases, for example involving hot-unplug of assigned devices, pi_post_block can forget to remove the vCPU from the blocked_vcpu_list. When this happens, the next call to pi_pre_block corrupts the list. Fix this in two ways. First, check vcpu->pre_pcpu in pi_pre_block and WARN instead of adding the element twice in the list. Second, always do the list removal in pi_post_block if vcpu->pre_pcpu is set (not -1). The new code keeps interrupts disabled for the whole duration of pi_pre_block/pi_post_block. This is not strictly necessary, but easier to follow. For the same reason, PI.ON is checked only after the cmpxchg, and to handle it we just call the post-block code. This removes duplication of the list removal code. At the time, I didn't notice the now useless NDST update. Paolo > The vCPU can get preempted_before_ the update starts, but not during. > And if the vCPU is preempted before, vmx_vcpu_pi_load() is responsible > for updating NDST when the vCPU is scheduled back in. In that case, the > check against the wakeup vector in vmx_vcpu_pi_load() cannot be true as > that would require the notification vector to have been set to the wakeup > vector_before_ blocking. > > Opportunistically switch to using vcpu->cpu for the list/lock lookups, > which presumably used pre_pcpu only for some phantom preemption logic. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 From: Paolo Bonzini Date: Mon, 25 Oct 2021 14:01:42 +0000 Subject: Re: [PATCH v2 24/43] KVM: VMX: Drop pointless PI.NDST update when blocking Message-Id: <18e6a656-f583-0ad4-6770-9678be3f5cf4@redhat.com> List-Id: References: <20211009021236.4122790-1-seanjc@google.com> <20211009021236.4122790-25-seanjc@google.com> In-Reply-To: <20211009021236.4122790-25-seanjc@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Sean Christopherson , Marc Zyngier , Huacai Chen , Aleksandar Markovic , Paul Mackerras , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank Cc: James Morse , Alexandru Elisei , Suzuki K Poulose , Atish Patra , David Hildenbrand , Cornelia Huck , Claudio Imbrenda , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, David Matlack , Oliver Upton , Jing Zhang On 09/10/21 04:12, Sean Christopherson wrote: > Don't update Posted Interrupt's NDST, a.k.a. the target pCPU, in the > pre-block path, as NDST is guaranteed to be up-to-date. The comment > about the vCPU being preempted during the update is simply wrong, as the > update path runs with IRQs disabled (from before snapshotting vcpu->cpu, > until after the update completes). Right, it didn't as of commit bf9f6ac8d74969690df1485b33b7c238ca9f2269 (when VT-d posted interrupts were introduced). The interrupt disable/enable pair was added in the same commit that motivated the introduction of the sanity checks: commit 8b306e2f3c41939ea528e6174c88cfbfff893ce1 Author: Paolo Bonzini Date: Tue Jun 6 12:57:05 2017 +0200 KVM: VMX: avoid double list add with VT-d posted interrupts In some cases, for example involving hot-unplug of assigned devices, pi_post_block can forget to remove the vCPU from the blocked_vcpu_list. When this happens, the next call to pi_pre_block corrupts the list. Fix this in two ways. First, check vcpu->pre_pcpu in pi_pre_block and WARN instead of adding the element twice in the list. Second, always do the list removal in pi_post_block if vcpu->pre_pcpu is set (not -1). The new code keeps interrupts disabled for the whole duration of pi_pre_block/pi_post_block. This is not strictly necessary, but easier to follow. For the same reason, PI.ON is checked only after the cmpxchg, and to handle it we just call the post-block code. This removes duplication of the list removal code. At the time, I didn't notice the now useless NDST update. Paolo > The vCPU can get preempted_before_ the update starts, but not during. > And if the vCPU is preempted before, vmx_vcpu_pi_load() is responsible > for updating NDST when the vCPU is scheduled back in. In that case, the > check against the wakeup vector in vmx_vcpu_pi_load() cannot be true as > that would require the notification vector to have been set to the wakeup > vector_before_ blocking. > > Opportunistically switch to using vcpu->cpu for the list/lock lookups, > which presumably used pre_pcpu only for some phantom preemption logic.