From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47C1FECAAA1 for ; Mon, 24 Oct 2022 18:06:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+Xhgd1I8QEiP6zQJXmqpmw8sWzLu5eLB8Y3Zuw/M644=; b=tpksUpvI1QhVPN +71yG2lJctELoCeHymsvn4a9GTHExn1UvzaQAqQpb9A+b2m6EURkYx1W6VewMrHPudmPgSzLbDHOb N39E9p7cVCAKJZyvLM5j2zjFGNI13G1aCu37etX+vPfQttpob0U5IMbP0g0HajqhOkZTmyxvOgUgR Eb5ArP+W3cl9j7LxQ/PxvBQWCKPsl4+z7755DUrtj/UT+JEK6+Nl1LpMdDNjFSRyZ+Dv9LFdujh+E 3G4tw56QzuIx02DxTEjo75kPw+jNTq6mAah1c9dVDR1RipK2x7f70g5CkNfE5+6ffis9ExoMjMyQT scXhd9ePlylJGItVy9Tg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1on1pF-002bYf-SA; Mon, 24 Oct 2022 18:05:26 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1on1p8-002bWL-1r for linux-arm-kernel@lists.infradead.org; Mon, 24 Oct 2022 18:05:22 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 827E7B8114A; Mon, 24 Oct 2022 18:05:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 31E20C433C1; Mon, 24 Oct 2022 18:05:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666634715; bh=B+c2N65DjkDMDGVo0eOlt+EOtgjk3znwuz//Nw5771g=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=lhR3syt4caWU8SMSSHvNdWWxn7AX82aUmCSuRKonW2VOVrFD5aAJ3iE3fnC3DDi60 MUoYqEX2Yd3oiacBtT8LDeWRgpVw4KgQkMXUkVh7Osw77Ae8AEkmCJ7ehnKasd1m2X hPBJkNavDj4C0WAp/vhdil05E+vR9AnY690OmJJgKwFaLszcaYH5K/KLvfpFkg8rp8 5x4vAZSqwU2JeanQnX7y0hG+M1HLrgS7QsgFktUsYZ7B0GXC61iSTZzSKnXQ+32pXN 04JPceDn41Zpws9O5EAlnUPNnRe59TlEtiQkKkJGdtc1RakEu+6+X5eSAEKZAqq00K 4RgJ2AsQ81vkw== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1on1p2-001IHZ-Rb; Mon, 24 Oct 2022 19:05:13 +0100 Date: Mon, 24 Oct 2022 19:05:12 +0100 Message-ID: <86y1t5m71j.wl-maz@kernel.org> From: Marc Zyngier To: Ricardo Koller Cc: Oliver Upton , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, James Morse , Suzuki K Poulose , Alexandru Elisei , kernel-team@android.com Subject: Re: [PATCH 0/9] KVM: arm64: PMU: Fixing chained events, and PMUv3p5 support In-Reply-To: References: <20220805135813.2102034-1-maz@kernel.org> <87lervuefe.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: ricarkol@google.com, oliver.upton@linux.dev, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221024_110518_432632_AEE8E5D2 X-CRM114-Status: GOOD ( 32.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Ricardo, On Fri, 12 Aug 2022 23:53:44 +0100, Ricardo Koller wrote: > > On Thu, Aug 11, 2022 at 01:56:21PM +0100, Marc Zyngier wrote: > > On Wed, 10 Aug 2022 22:55:03 +0100, > > Ricardo Koller wrote: > > > > > > Just realized that KVM does not offer PMUv3p5 (with this series applied) > > > when the real hardware is only Armv8.2 (the setup I originally tried). > > > So, tried these other two setups on the fast model: > > > > > > has_arm_v8-5=1 > > > > > > # ./lkvm-static run --nodefaults --pmu pmu.flat -p pmu-chained-sw-incr > > > # lkvm run -k pmu.flat -m 704 -c 8 --name guest-135 > > > > > > INFO: PMU version: 0x6 > > > ^^^ > > > PMUv3 for Armv8.5 > > > INFO: PMU implementer/ID code: 0x41("A")/0 > > > INFO: Implements 8 event counters > > > FAIL: pmu: pmu-chained-sw-incr: overflow and chain counter incremented after 100 SW_INCR/CHAIN > > > INFO: pmu: pmu-chained-sw-incr: overflow=0x0, #0=4294967380 #1=0 > > > ^^^ > > > no overflows > > > FAIL: pmu: pmu-chained-sw-incr: expected overflows and values after 100 SW_INCR/CHAIN > > > INFO: pmu: pmu-chained-sw-incr: overflow=0x0, #0=84 #1=-1 > > > INFO: pmu: pmu-chained-sw-incr: overflow=0x0, #0=4294967380 #1=4294967295 > > > SUMMARY: 2 tests, 2 unexpected failures > > > > Hmm. I think I see what's wrong. In kvm_pmu_create_perf_event(), we > > have this: > > > > if (kvm_pmu_idx_is_64bit(vcpu, select_idx)) > > attr.config1 |= 1; > > > > counter = kvm_pmu_get_counter_value(vcpu, select_idx); > > > > /* The initial sample period (overflow count) of an event. */ > > if (kvm_pmu_idx_has_64bit_overflow(vcpu, select_idx)) > > attr.sample_period = (-counter) & GENMASK(63, 0); > > else > > attr.sample_period = (-counter) & GENMASK(31, 0); > > > > but the initial sampling period shouldn't be based on the *guest* > > counter overflow. It really is about the getting to an overflow on the > > *host*, so the initial code was correct, and only the width of the > > counter matters here. > > Right, I think this requires bringing back some of the chained related > code (like update_pmc_chained() and pmc_is_chained()), because > > attr.sample_period = (-counter) & GENMASK(31, 0); > > should also be used when the counter is chained. Almost, but not quite. I came up with the following hack (not everything is relevant, but you'll get my drift): diff --git a/arch/arm64/kvm/pmu-emul.c b/arch/arm64/kvm/pmu-emul.c index 9f29212e8fcd..6470a42e981d 100644 --- a/arch/arm64/kvm/pmu-emul.c +++ b/arch/arm64/kvm/pmu-emul.c @@ -450,6 +450,9 @@ static void kvm_pmu_counter_increment(struct kvm_vcpu *vcpu, reg = lower_32_bits(reg); __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + i) = reg; + if (!kvm_pmu_idx_has_64bit_overflow(vcpu, i)) + reg = lower_32_bits(reg); + if (reg) /* No overflow? move on */ continue; @@ -483,7 +486,7 @@ static void kvm_pmu_perf_overflow(struct perf_event *perf_event, */ period = -(local64_read(&perf_event->count)); - if (!kvm_pmu_idx_has_64bit_overflow(vcpu, pmc->idx)) + if (!kvm_pmu_idx_is_64bit(vcpu, pmc->idx)) period &= GENMASK(31, 0); local64_set(&perf_event->hw.period_left, 0); @@ -605,17 +608,24 @@ static void kvm_pmu_create_perf_event(struct kvm_vcpu *vcpu, u64 select_idx) attr.exclude_host = 1; /* Don't count host events */ attr.config = eventsel; - /* If counting a 64bit event, advertise it to the perf code */ - if (kvm_pmu_idx_is_64bit(vcpu, select_idx)) - attr.config1 |= 1; - counter = kvm_pmu_get_counter_value(vcpu, select_idx); - /* The initial sample period (overflow count) of an event. */ - if (kvm_pmu_idx_has_64bit_overflow(vcpu, select_idx)) - attr.sample_period = (-counter) & GENMASK(63, 0); - else + /* + * If counting with a 64bit counter, advertise it to the perf + * code, carefully dealing with the initial sample period + * which also depends on the overflow. + */ + if (kvm_pmu_idx_is_64bit(vcpu, select_idx)) { + attr.config1 |= 1; + + if (!kvm_pmu_idx_has_64bit_overflow(vcpu, select_idx)) { + attr.sample_period = -(counter & GENMASK(31, 0)); + } else { + attr.sample_period = (-counter) & GENMASK(63, 0); + } + } else { attr.sample_period = (-counter) & GENMASK(31, 0); + } event = perf_event_create_kernel_counter(&attr, -1, current, kvm_pmu_perf_overflow, pmc); With this, I'm back in business (in QEMU, as I *still* cannot get ARM to give me a model that runs natively on arm64...): root@debian:~/kvm-unit-tests# ../kvmtool/lkvm run --nodefaults --pmu --firmware arm/pmu.flat -p pmu-chained-sw-incr # lkvm run --firmware arm/pmu.flat -m 448 -c 4 --name guest-400 Info: Removed ghost socket file "/root/.lkvm//guest-400.sock". WARNING: early print support may not work. Found uart at 0x1000000, but early base is 0x9000000. INFO: PMU version: 0x6 INFO: PMU implementer/ID code: 0x41("A")/0x1 INFO: Implements 6 event counters FAIL: pmu: pmu-chained-sw-incr: no overflow and chain counter incremented after 100 SW_INCR/CHAIN INFO: pmu: pmu-chained-sw-incr: overflow=0x1, #0=4294967380 #1=1 FAIL: pmu: pmu-chained-sw-incr: overflow on chain counter and expected values after 100 SW_INCR/CHAIN INFO: pmu: pmu-chained-sw-incr: overflow=0x3, #0=4294967380 #1=4294967296 SUMMARY: 2 tests, 2 unexpected failures The tests themselves need some extra love to account for the fact that the counters are always 64bit irrespective of the overflow, but at least I'm now correctly seeing the odd counter incrementing. I'll try to continue addressing the comments tomorrow. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel