From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEA36C433FE for ; Mon, 13 Dec 2021 06:38:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230446AbhLMGiI (ORCPT ); Mon, 13 Dec 2021 01:38:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229716AbhLMGiH (ORCPT ); Mon, 13 Dec 2021 01:38:07 -0500 Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com [IPv6:2607:f8b0:4864:20::22d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C203C06173F for ; Sun, 12 Dec 2021 22:38:06 -0800 (PST) Received: by mail-oi1-x22d.google.com with SMTP id 7so21883335oip.12 for ; Sun, 12 Dec 2021 22:38:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8lP8OOorDp4RjZXoLRJhvTaaFlxLo7QMGtZf6Mua7X0=; b=VvqbD7i/phDRw4qlo/3D8JSYLBuK9N9Idj+2AhKszf22QRc3mns9rbQIAGYmQR9Cre JJpX/99WXVnfLUPMIEYw2BTEp9BUVjizx52QBoujIaooCUu8OPiqPGCw82L8McPAGRwT casmlHZ73tCkjgVEfcjpBt9xLgXUc90nWtz8/VKTj7VYJ2Vx6cTxdr6bSrsa3jKd0HNR 3gdzjbK/ZXnJy/ZFb1NuP0MOixRinsKK6eqEnnAUNPJu02kCTdEWdJrxMnwSwJ+xOHak k7h4HU4Yiic7u4xbqqGgQzrRaf3PyGTYV4OAhvomtwbga8lCFria43ju8aUxFnAS32XI lhtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8lP8OOorDp4RjZXoLRJhvTaaFlxLo7QMGtZf6Mua7X0=; b=hmNu72jQ+qpT/ne/OgK4NWH1XqbVZrbryi20pP3wQCIqBAqnx3WoXi2Pwo11DzKsey wSaWeyYvVfmfoyrCnN7ugVe+Pj+XjnzqLwUBDyIqG1Q0MEyHUvAoT9xNgAPFAJ0z6V/P BIK9OMdpUk+tyNMaSt/pEO2HIo39SjiNp5PEJW9OugUKg5jvlFfLgcDW1cMeGi8/sjQD xOMfaZcOjh0JDWvYCWRHac4FdkT03/TLWYFt6weoDBHIK+Q9NL6zOglzvX76lolfqoUv XsJJ24OeRPqiuRDQA8skwmK7hZXOyjGxvWWd6zCPzsgP2tmMMxQ20YPhUkVfRObzYz0c 7yxA== X-Gm-Message-State: AOAM533JjfJS/+3aHzfwE2LfyRf9wt5XBJVrbU2oZU4Bvbx+ylidwn17 ohfiJEeXCJGy9GzjZLjhLPh1pg1BptBnGDP42ROVqQ== X-Google-Smtp-Source: ABdhPJxrvfANGAnK2tE5T2zE6CDhSVPk9jE0d3+LQ+mW7YRbOoSb0FlzP9qgVi00dKe8M4+tNVTTAP024Bdvq936/zY= X-Received: by 2002:aca:674a:: with SMTP id b10mr26934304oiy.66.1639377485743; Sun, 12 Dec 2021 22:38:05 -0800 (PST) MIME-Version: 1.0 References: <20211130074221.93635-1-likexu@tencent.com> <20211130074221.93635-5-likexu@tencent.com> <0ca44f61-f7f1-0440-e1e1-8d5e8aa9b540@gmail.com> In-Reply-To: From: Jim Mattson Date: Sun, 12 Dec 2021 22:37:54 -0800 Message-ID: Subject: Re: [PATCH v2 4/6] KVM: x86/pmu: Add pmc->intr to refactor kvm_perf_overflow{_intr}() To: Paolo Bonzini Cc: Like Xu , Andi Kleen , Kim Phillips , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Like Xu Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, Dec 11, 2021 at 8:56 PM Jim Mattson wrote: > > On Fri, Dec 10, 2021 at 3:31 PM Jim Mattson wrote: > > > > On Fri, Dec 10, 2021 at 2:59 PM Paolo Bonzini wrote: > > > > > > On 12/10/21 23:55, Jim Mattson wrote: > > > >> > > > >> Even for tracing the SDM says "Like the value returned by RDTSC, TSC > > > >> packets will include these adjustments, but other timing packets (such > > > >> as MTC, CYC, and CBR) are not impacted". Considering that "stand-alone > > > >> TSC packets are typically generated only when generation of other timing > > > >> packets (MTCs and CYCs) has ceased for a period of time", I'm not even > > > >> sure it's a good thing that the values in TSC packets are scaled and offset. > > > >> > > > >> Back to the PMU, for non-architectural counters it's not really possible > > > >> to know if they count in cycles or not. So it may not be a good idea to > > > >> special case the architectural counters. > > > > > > > > In that case, what we're doing with the guest PMU is not > > > > virtualization. I don't know what it is, but it's not virtualization. > > > > > > It is virtualization even if it is incompatible with live migration to a > > > different SKU (where, as you point out below, multiple TSC frequencies > > > might also count as multiple SKUs). But yeah, it's virtualization with > > > more caveats than usual. > > > > It's not virtualization if the counters don't count at the rate the > > guest expects them to count. > > Per the SDM, unhalted reference cycles count at "a fixed frequency." > If the frequency changes on migration, then the value of this event is > questionable at best. For unhalted core cycles, on the other hand, the > SDM says, "The performance counter for this event counts across > performance state transitions using different core clock frequencies." > That does seem to permit frequency changes on migration, but I suspect > that software expects the event to count at a fixed frequency if > INVARIANT_TSC is set. Actually, I now realize that unhalted reference cycles is independent of the host or guest TSC, so it is not affected by TSC scaling. However, we still have to decide on a specific fixed frequency to virtualize so that the frequency doesn't change on migration. As a practical matter, it may be the case that the reference cycles frequency is the same on all processors in a migration pool, and we don't have to do anything. > I'm not sure that I buy your argument regarding consistency. In > general, I would expect the hypervisor to exclude non-architected > events from the allow-list for any VM instances running in a > heterogeneous migration pool. Certainly, those events could be allowed > in a heterogeneous migration pool consisting of multiple SKUs of the > same microarchitecture running at different clock frequencies, but > that seems like a niche case. > > > > > > Exposing non-architectural events is questionable with live migration, > > > > and TSC scaling is unnecessary without live migration. I suppose you > > > > could have a migration pool with different SKUs of the same generation > > > > with 'seemingly compatible' PMU events but different TSC frequencies, > > > > in which case it might be reasonable to expose non-architectural > > > > events, but I would argue that any of those 'seemingly compatible' > > > > events are actually not compatible if they count in cycles. > > > I agree. Support for marshaling/unmarshaling PMU state exists but it's > > > more useful for intra-host updates than for actual live migration, since > > > these days most live migration will use TSC scaling on the destination. > > > > > > Paolo > > > > > > > > > > > Unless, of course, Like is right, and the PMU counters do count fractionally. > > > > > > >