From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86126C19F2A for ; Thu, 11 Aug 2022 12:56:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235498AbiHKM42 (ORCPT ); Thu, 11 Aug 2022 08:56:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234179AbiHKM41 (ORCPT ); Thu, 11 Aug 2022 08:56:27 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CBF13F314 for ; Thu, 11 Aug 2022 05:56:26 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id DF9B7B820BE for ; Thu, 11 Aug 2022 12:56:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A5D9CC433C1; Thu, 11 Aug 2022 12:56:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1660222583; bh=BG0EIiB0G/2fnhVV42nQWkp6CJVQTQC8ZIJ8vLyaPYs=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=UZFsNSvUvlt4QQphm8bTQrlb6gUnTbmaewplUS9QqX7KcPa0EGGYIKFX5gFzwCOdt G+zzEeuqsjTr6MEkjg0aK0hAG+nEp6wmyT6NXwwBPjnOCmsjgQMG1L6B7MBL+lS2FF SJoS5mN7edBxOClxqjO2eL3maQOuG7rwofgGJG5PG1IJCHU8PFWIxvf5csDLtqjF6C SmZkq1dxGUIrTq0yJAhV27gLiQBAFheuryrt3oX7ZnqVlT7Ew/dS6EuDD931wPIAjQ B44EkWf4x+sCMKIZUSf6KuLmrai3mLCmjzqT9AmwnsDwZKsj1c8/dc77Kc2Qn/y8h3 EREDXpWk1fUwQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oM7jZ-002LKU-Kz; Thu, 11 Aug 2022 13:56:21 +0100 Date: Thu, 11 Aug 2022 13:56:21 +0100 Message-ID: <87lervuefe.wl-maz@kernel.org> From: Marc Zyngier To: Ricardo Koller Cc: Oliver Upton , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, James Morse , Suzuki K Poulose , Alexandru Elisei , kernel-team@android.com Subject: Re: [PATCH 0/9] KVM: arm64: PMU: Fixing chained events, and PMUv3p5 support In-Reply-To: References: <20220805135813.2102034-1-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: ricarkol@google.com, oliver.upton@linux.dev, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Wed, 10 Aug 2022 22:55:03 +0100, Ricardo Koller wrote: >=20 > On Wed, Aug 10, 2022 at 02:33:53PM -0500, Oliver Upton wrote: > > Hi Ricardo, > >=20 > > On Wed, Aug 10, 2022 at 11:46:22AM -0700, Ricardo Koller wrote: > > > On Fri, Aug 05, 2022 at 02:58:04PM +0100, Marc Zyngier wrote: > > > > Ricardo recently reported[1] that our PMU emulation was busted when= it > > > > comes to chained events, as we cannot expose the overflow on a 32bit > > > > boundary (which the architecture requires). > > > >=20 > > > > This series aims at fixing this (by deleting a lot of code), and as= a > > > > bonus adds support for PMUv3p5, as this requires us to fix a few mo= re > > > > things. > > > >=20 > > > > Tested on A53 (PMUv3) and FVP (PMUv3p5). > > > >=20 > > > > [1] https://lore.kernel.org/r/20220805004139.990531-1-ricarkol@goog= le.com > > > >=20 > > > > Marc Zyngier (9): > > > > KVM: arm64: PMU: Align chained counter implementation with > > > > architecture pseudocode > > > > KVM: arm64: PMU: Distinguish between 64bit counter and 64bit over= flow > > > > KVM: arm64: PMU: Only narrow counters that are not 64bit wide > > > > KVM: arm64: PMU: Add counter_index_to_*reg() helpers > > > > KVM: arm64: PMU: Simplify setting a counter to a specific value > > > > KVM: arm64: PMU: Move the ID_AA64DFR0_EL1.PMUver limit to VM crea= tion > > > > KVM: arm64: PMU: Aleven ID_AA64DFR0_EL1.PMUver to be set from use= rspace > > > > KVM: arm64: PMU: Implement PMUv3p5 long counter support > > > > KVM: arm64: PMU: Aleven PMUv3p5 to be exposed to the guest > > > >=20 > > > > arch/arm64/include/asm/kvm_host.h | 1 + > > > > arch/arm64/kvm/arm.c | 6 + > > > > arch/arm64/kvm/pmu-emul.c | 372 ++++++++++----------------= ---- > > > > arch/arm64/kvm/sys_regs.c | 65 +++++- > > > > include/kvm/arm_pmu.h | 16 +- > > > > 5 files changed, 208 insertions(+), 252 deletions(-) > > > >=20 > > > > --=20 > > > > 2.34.1 > > > >=20 > > >=20 > > > Hi Marc, > > >=20 > > > There is one extra potential issue with exposing PMUv3p5. I see this > > > weird behavior when doing passthrough ("bare metal") on the fast-model > > > configured to emulate PMUv3p5: the [63:32] half of the counters > > > overflowing at 32-bits is still incremented. > > >=20 > > > Fast model - ARMv8.5: > > > =20 > > > Assuming the initial state is even=3D0xFFFFFFFF and odd=3D0x0, > > > incrementing the even counter leads to: > > >=20 > > > 0x00000001_00000000 0x00000000_00000001 0x1 > > > even counter odd counter PMOVSET > > >=20 > > > Assuming the initial state is even=3D0xFFFFFFFF and odd=3D0xFFFFFFFF, > > > incrementing the even counter leads to: > > >=20 > > > 0x00000001_00000000 0x00000001_00000000 0x3 > > > even counter odd counter PMOVSET > >=20 > > This is to be expected, actually. PMUv8p5 counters are always 64 bit, > > regardless of the configured overflow. > >=20 > > DDI 0487H D8.3 Behavior on overflow > >=20 > > If FEAT_PMUv3p5 is implemented, 64-bit event counters are implemented, > > HDCR.HPMN is not 0, and either n is in the range [0 .. (HDCR.HPMN-1)] > > or EL2 is not implemented, then event counter overflow is configured > > by PMCR.LP: > >=20 > > =E2=80=94 When PMCR.LP is set to 0, if incrementing PMEVCNTR cause= s an unsigned > > overflow of bits [31:0] of the event counter, the PE sets PMOVSCLR[= n] to 1. > > =E2=80=94 When PMCR.LP is set to 1, if incrementing PMEVCNTR cause= s an unsigned > > overflow of bits [63:0] of the event counter, the PE sets PMOVSCLR[= n] to 1. > >=20 > > [...] > >=20 > > For all 64-bit counters, incrementing the counter is the same whether= an > > unsigned overflow occurs at [31:0] or [63:0]. If the counter incremen= ts > > for an event, bits [63:0] are always incremented. > >=20 > > Do you see this same (expected) failure w/ Marc's series? >=20 > I don't know, I'm hitting another bug it seems. >=20 > Just realized that KVM does not offer PMUv3p5 (with this series applied) > when the real hardware is only Armv8.2 (the setup I originally tried). > So, tried these other two setups on the fast model: >=20 > has_arm_v8-5=3D1 >=20 > # ./lkvm-static run --nodefaults --pmu pmu.flat -p pmu-chained-sw-incr > # lkvm run -k pmu.flat -m 704 -c 8 --name guest-135 >=20 > INFO: PMU version: 0x6 > ^^^ > PMUv3 for Armv8.5 > INFO: PMU implementer/ID code: 0x41("A")/0 > INFO: Implements 8 event counters > FAIL: pmu: pmu-chained-sw-incr: overflow and chain counter incremented a= fter 100 SW_INCR/CHAIN > INFO: pmu: pmu-chained-sw-incr: overflow=3D0x0, #0=3D4294967380 #1=3D0 > ^^^ > no overflows > FAIL: pmu: pmu-chained-sw-incr: expected overflows and values after 100 = SW_INCR/CHAIN > INFO: pmu: pmu-chained-sw-incr: overflow=3D0x0, #0=3D84 #1=3D-1 > INFO: pmu: pmu-chained-sw-incr: overflow=3D0x0, #0=3D4294967380 #1=3D429= 4967295 > SUMMARY: 2 tests, 2 unexpected failures Hmm. I think I see what's wrong. In kvm_pmu_create_perf_event(), we have this: if (kvm_pmu_idx_is_64bit(vcpu, select_idx)) attr.config1 |=3D 1; counter =3D kvm_pmu_get_counter_value(vcpu, select_idx); /* The initial sample period (overflow count) of an event. */ if (kvm_pmu_idx_has_64bit_overflow(vcpu, select_idx)) attr.sample_period =3D (-counter) & GENMASK(63, 0); else attr.sample_period =3D (-counter) & GENMASK(31, 0); but the initial sampling period shouldn't be based on the *guest* counter overflow. It really is about the getting to an overflow on the *host*, so the initial code was correct, and only the width of the counter matters here. /me goes back to running the FVP... Thanks, M. --=20 Without deviation from the norm, progress is not possible.