From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 586E5C433DB for ; Wed, 10 Feb 2021 01:26:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB60964DD6 for ; Wed, 10 Feb 2021 01:26:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233905AbhBJBZr (ORCPT ); Tue, 9 Feb 2021 20:25:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233894AbhBIW67 (ORCPT ); Tue, 9 Feb 2021 17:58:59 -0500 Received: from mail-qt1-x849.google.com (mail-qt1-x849.google.com [IPv6:2607:f8b0:4864:20::849]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF364C061794 for ; Tue, 9 Feb 2021 14:58:15 -0800 (PST) Received: by mail-qt1-x849.google.com with SMTP id t5so282858qti.5 for ; Tue, 09 Feb 2021 14:58:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=f+Sfzn1G8buhFeqBg3rOZ1CfjEptJKUCS3yLyjW3ukw=; b=fXjKnajjwqEth6ETF1zmiiLoxMzax7fdB27dVNR262PGThQHBnxbxMp2UGW3t2jeCH miWabgHcl2q1ZWnK9uGK9EoALG8Euee/lastlI5L2KdpJXgaIX0dqzTO9OuW0rc5cx6k C5m5C7QQ9kc9qJRskvxfK1MN46Xvp2TVVnw2BTXt9ulo9j+Gc5ZzXBz5Oe+It8EeP8JE 8/BrdaedgiCLsiyzN/pd/4asmxv6isVt3lEg34B0E6I+bqoVL13zXrNS8RwcNcvQQDnH nh6GuasHgotQuhqOcrCJMz7h5gyUoQluE6z2fn31ffj646DsMg1Ngnerf4JFEPNNtyRM ujgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=f+Sfzn1G8buhFeqBg3rOZ1CfjEptJKUCS3yLyjW3ukw=; b=tNgHgGc+JMKQOR+lnXsmlDtlOK8G/5mDlp/PKtXC49UDC/8Rl6t8W4jkIbnF4rmtVe H7CKZoShXr1p2a2ZHl8hFUgeSz2gW+PwwTRHr9hs1hecFlND3NRvlDsC/6lkn1q26Q4e vhGCzVoexabPcdTEJzIcPG08hL1ncrD6vttAADMWVK0g9bDMQKFb3P1NCUMB4jFJxxh1 9olJ31n6X0lWmM+ohetdv73wT8RDkBHW4oTTdVc2jqB/ObvTTYBr3u3oUE2mazZ0vVao quXd1iinK6WXr+hZphFiUxX0b4VrYLwSUfVLj0hEhn43jdbgvzZsDTOhABkF2OsOIeyG 1f2A== X-Gm-Message-State: AOAM531yc44nR8iX+mjH6otWfjFJFiBU4IZyw5fXQzWbh/io0NKbTHJs h/bVrgf8i5jFXkhaEM4OfIil7v2gCVn85A== X-Google-Smtp-Source: ABdhPJwVyVK2Hl7yFH//muAVUIkgbpJI5i2isxVp7CWbCELE91YgCxgjZ6Rx+Ko4UqOEcwLTPJzkL/RFmhbwPg== Sender: "jmattson via sendgmr" X-Received: from turtle.sea.corp.google.com ([2620:15c:100:202:9942:27af:a628:1d1b]) (user=jmattson job=sendgmr) by 2002:a0c:b59f:: with SMTP id g31mr446987qve.28.1612911494938; Tue, 09 Feb 2021 14:58:14 -0800 (PST) Date: Tue, 9 Feb 2021 14:56:53 -0800 Message-Id: <20210209225653.1393771-1-jmattson@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.30.0.478.g8a0d178c01-goog Subject: Trouble with perf_guest_get_msrs() and pebs_no_isolation From: Jim Mattson To: Peter Shier , Alexander Shishkin , Arnaldo Carvalho de Melo , Borislav Petkov , "H. Peter Anvin" , Ingo Molnar , Jiri Olsa , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Mark Rutland , Namhyung Kim , Paolo Bonzini , Peter Zijlstra , Sean Christopherson , Thomas Gleixner , Vitaly Kuznetsov , Wanpeng Li , x86@kernel.org Cc: Jim Mattson Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On a host that suffers from pebs_no_isolation, perf_guest_get_msrs() adds an entry to cpuc->guest_switch_msrs for MSR_IA32_PEBS_ENABLE. Kvm's atomic_switch_perf_msrs() is the only caller of perf_guest_get_msrs(). If atomic_switch_perf_msrs() finds an entry for MSR_IA32_PEBS_ENABLE in cpuc->guest_switch_msrs, it puts the "host" value of this entry on the VM-exit MSR-load list for the current VMCS. At the next VM-exit, that "host" value will be written to MSR_IA32_PEBS_ENABLE. The problem is that by the next VM-exit, that "host" value may be stale. Though maskable interrupts are blocked from well before atomic_switch_perf_msrs() to the next VM-entry, PMIs are delivered as NMIs. Moreover, due to PMI throttling, the PMI handler may clear bits in MSR_IA32_PEBS_ENABLE. See the comment to that effect in handle_pmi_common(). In fact, by the time that perf_guest_get_msrs() returns to its caller, the "host" value that it has recorded for MSR_IA32_PEBS_ENABLE could already be stale. What happens if a VM-exit sets a bit in MSR_IA32_PEBS_ENABLE that the perf subsystem thinks should be clear? In the short term, nothing happens at all. But note that this situation may not get better at the next VM-exit, because kvm won't add MSR_IA32_PEBS_ENABLE to the VM-exit MSR-load list if perf_guest_get_mrs() reports that the "host" value of the MSR is 0. So, if the new MSR_IA32_PEBS_ENABLE "host" value is 0, the stale bits can actually persist for a long time. If, at some point in the future, the perf subsystem programs a counter overflow interrupt on the same PMC for a PEBS-capable event, we are in for some nasty surprises. (Note that the perf subsystem never *intentionally* programs a PMC for both PEBS and counter overflow interrupts at the same time.) If a PEBS assist is triggered while in VMX non-root operation, the CPU will try to access the host's DS_AREA using the guest's page tables, and a page fault is likely (specifically on a read of the PEBS interrupt threshold at offset 0x38 in the DS_AREA). If the PEBS interrupt threshold is met while in VMX root operation, two separate PMIs are generated: one for the PEBS interrupt threshold and one for the counter overflow. This results in a message from unknown_nmi_error(): "Uhhuh. NMI received for unknown reason on CPU ."