From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6834FC77B7A for ; Wed, 24 May 2023 21:33:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230430AbjEXVdE (ORCPT ); Wed, 24 May 2023 17:33:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231736AbjEXVdA (ORCPT ); Wed, 24 May 2023 17:33:00 -0400 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3B5D19C for ; Wed, 24 May 2023 14:32:55 -0700 (PDT) Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1ae64580e9fso20655ad.1 for ; Wed, 24 May 2023 14:32:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684963975; x=1687555975; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=borTZX0aRFrFNvcfsYFdIey5QEoR1S37jxXMV2gp3uc=; b=LuLuiIEHZd/GedTVjAdtxMkvqVLd/EyPaWvXghGqApnmkJzWlUZceJ0hqxMVwghgsS KRFUBfZg56xxn57X7EtlK1CTtrqy52VXU8IItMU25QFE/+0EIQ0XYIhSEJpmxQ/fxDdj BcEHyR/jO3GktSY1/QUg/mJaCkY60HKae8c5wOuZSZKXH7RJRbK76Dt1IdkMvbPQ+MVm Ma9qNSPk4BE8dquB1+lAoeSmLvtCVJHZJUjbeJZxQd7GUSGZPsFmjb3CoHwb/0wnQc1/ cZyX13T8+eiLEz631aMj2/110MbWumMZvNqM1hcTtq4brNSWNs6QUMI62Qxi4jNqidKJ qxxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684963975; x=1687555975; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=borTZX0aRFrFNvcfsYFdIey5QEoR1S37jxXMV2gp3uc=; b=jchVKpdxLtH5t5B2Rbs1p1yFmLbSqWXdJLUST+TDrvl6WqODaOg4WwpcFP1qgBbuBY jcUX6afU6PtsRjoufrOOBek4+i/agpM7vrpzeC8A15O+HBX1ebhMDZ1Wke60mDQZpBS7 jMGrLOXrQCYNuBfroevRTofCW1C5i+0kkyo8eJGjCPwXYplK/JppDB9r9Oxi02pT4jhy 2sjCQvQEtwTMb43ky6YP6hQqAyjApFS3OMSMDm9LOzB5oTceUShWUO3dHQKNgONjj234 MVI4xYqkBU5HaMUNSI0tD2Ht0xqIlLtZ2OeYU6kDB3ch82FfYTFdf3EOO1GMbYjWfa7S kOKA== X-Gm-Message-State: AC+VfDx2fgD+CCIO3pSWyL/NBrhVrQ/lnSd4c3fzJRUxXmIXluWn6rYS z62ONAK1IDupGImrKcy60dGN6l4Oz6B6B6HAJgXBrA== X-Google-Smtp-Source: ACHHUZ56rjqoJRHN/O2Wiv8yc5HA0r/Qko53lMwd+9CO4xlJ/nwois7pexgsBdvq4jmOsdBvvzGsw8FlVldOgJGw9/o= X-Received: by 2002:a17:902:e801:b0:1af:90ce:5261 with SMTP id u1-20020a170902e80100b001af90ce5261mr68864plg.24.1684963975207; Wed, 24 May 2023 14:32:55 -0700 (PDT) MIME-Version: 1.0 References: <20230310105346.12302-1-likexu@tencent.com> <20230310105346.12302-6-likexu@tencent.com> <509b697f-4e60-94e5-f785-95f7f0a14006@gmail.com> <34b5dd08-edac-e32f-1884-c8f2b85f7971@gmail.com> <59ef9af0-9528-e220-625a-ff16e6971f23@amd.com> In-Reply-To: From: Jim Mattson Date: Wed, 24 May 2023 14:32:44 -0700 Message-ID: Subject: Re: [PATCH 5/5] KVM: x86/pmu: Hide guest counter updates from the VMRUN instruction To: Sean Christopherson Cc: Sandipan Das , Like Xu , Paolo Bonzini , Ravi Bangoria , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Santosh Shukla , "Tom Lendacky (AMD)" , Ananth Narayan Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, May 24, 2023 at 2:29=E2=80=AFPM Sean Christopherson wrote: > > On Wed, May 24, 2023, Jim Mattson wrote: > > On Wed, May 24, 2023 at 1:41=E2=80=AFPM Sean Christopherson wrote: > > > > > > On Wed, Apr 26, 2023, Sandipan Das wrote: > > > > Hi Sean, Like, > > > > > > > > On 4/19/2023 7:11 PM, Like Xu wrote: > > > > >> Heh, it's very much explicable, it's just not desirable, and you= and I would argue > > > > >> that it's also incorrect. > > > > > > > > > > This is completely inaccurate from the end guest pmu user's persp= ective. > > > > > > > > > > I have a toy that looks like virtio-pmu, through which guest user= s can get hypervisor performance data. > > > > > But the side effect of letting the guest see the VMRUN instructio= n by default is unacceptable, isn't it ? > > > > > > > > > >> > > > > >> AMD folks, are there plans to document this as an erratum?=C3=AF= =C2=BF=C2=BD I agree with Like that > > > > >> counting VMRUN as a taken branch in guest context is a CPU bug, = even if the behavior > > > > >> is known/expected. > > > > > > > > > > > > > This behaviour is architectural and an erratum will not be issued. = However, for clarity, a future > > > > release of the APM will include additional details like the followi= ng: > > > > > > > > 1) From the perspective of performance monitoring counters, VMRUN= s are considered as far control > > > > transfers and VMEXITs as exceptions. > > > > > > > > 2) When the performance monitoring counters are set up to count e= vents only in certain modes > > > > through the "OsUserMode" and "HostGuestOnly" bits, instruction= s and events that change the > > > > mode are counted in the target mode. For example, a SYSCALL fr= om CPL 3 to CPL 0 with a > > > > counter set to count retired instructions with USR=3D1 and OS= =3D0 will not cause an increment of > > > > the counter. However, the SYSRET back from CPL 0 to CPL 3 will= cause an increment of the > > > > counter and the total count will end up correct. Similarly, wh= en counting PMCx0C6 (retired > > > > far control transfers, including exceptions and interrupts) wi= th Guest=3D1 and Host=3D0, a VMRUN > > > > instruction will cause an increment of the counter. However, t= he subsequent VMEXIT that occurs, > > > > since the target is in the host, will not cause an increment o= f the counter and so the total > > > > count will end up correct. > > > > > > The count from the guest's perspective does not "end up correct". Un= like SYSCALL, > > > where _userspace_ deliberately and synchronously executes a branch in= struction, > > > VMEXIT and VMRUN are supposed to be transparent to the guest and can = be completely > > > asynchronous with respect to guest code execution, e.g. if the host i= s spamming > > > IRQs, the guest will see a potentially large number of bogus (from it= 's perspective) > > > branches retired. > > > > The reverse problem occurs when a PMC is configured to count "CPUID > > instructions retired." Since KVM intercepts CPUID and emulates it, the > > PMC will always read 0, even if the guest executes a tight loop of > > CPUID instructions. > > > > The PMU is not virtualizable on AMD CPUs without significant > > hypervisor corrections. I have to wonder if it's really worth the > > effort. > > Per our offlist chat, my understanding is that there are caveats with vPM= Us that > it's simply not feasible for a hypervisor to handle. I.e. virtualizing a= ny x86 > PMU with 100% accuracy isn't happening anytime soon. > > The way forward is likely to evaluate each caveat on a case-by-case basis= to > determine whether or not the cost of the fixup in KVM is worth the benefi= t to > the guest. E.g. emulating "CPUID instructions retired" seems like it wou= ld be > fairly straightforward. AFAICT, fixing up the VMRUN stuff is quite diffi= cult though. Yeah. The problem with fixing up "CPUID instructions retired" is tracking what the event encoding is for every F/M/S out there. It's not worth it.