From: Leo Yan <leo.yan@linaro.org> To: Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, Arnaldo Carvalho de Melo <acme@kernel.org>, Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Thomas Gleixner <tglx@linutronix.de>, Paul Cercueil <paul@crapouillou.net>, "Ben Dooks (Codethink)" <ben.dooks@codethink.co.uk>, "Ahmed S. Darwish" <a.darwish@linutronix.de>, Adrian Hunter <adrian.hunter@intel.com>, Kan Liang <kan.liang@linux.intel.com>, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/6] arm64: perf: Implement correct cap_user_time Date: Wed, 15 Jul 2020 10:05:08 +0800 [thread overview] Message-ID: <20200715020512.20991-3-leo.yan@linaro.org> (raw) In-Reply-To: <20200715020512.20991-1-leo.yan@linaro.org> From: Peter Zijlstra <peterz@infradead.org> As reported by Leo; the existing implementation is broken when the clock and counter don't intersect at 0. Use the sched_clock's struct clock_read_data information to correctly implement cap_user_time and cap_user_time_zero. Note that the ARM64 counter is architecturally only guaranteed to be 56bit wide (implementations are allowed to be wider) and the existing perf ABI cannot deal with wrap-around. This implementation should also be faster than the old; seeing how we don't need to recompute mult and shift all the time. [leoyan: Use quot/rem to convert cyc to ns to avoid overflow] Reported-by: Leo Yan <leo.yan@linaro.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> --- arch/arm64/kernel/perf_event.c | 40 ++++++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c index 4d7879484cec..35c2c737d4af 100644 --- a/arch/arm64/kernel/perf_event.c +++ b/arch/arm64/kernel/perf_event.c @@ -19,6 +19,7 @@ #include <linux/of.h> #include <linux/perf/arm_pmu.h> #include <linux/platform_device.h> +#include <linux/sched_clock.h> #include <linux/smp.h> /* ARMv8 Cortex-A53 specific event types. */ @@ -1165,28 +1166,49 @@ device_initcall(armv8_pmu_driver_init) void arch_perf_update_userpage(struct perf_event *event, struct perf_event_mmap_page *userpg, u64 now) { - u32 freq; - u32 shift; + struct clock_read_data *rd; + unsigned int seq; + u64 quot, rem, ns; /* * Internal timekeeping for enabled/running/stopped times * is always computed with the sched_clock. */ - freq = arch_timer_get_rate(); userpg->cap_user_time = 1; + userpg->cap_user_time_zero = 1; + + do { + rd = sched_clock_read_begin(&seq); + + userpg->time_mult = rd->mult; + userpg->time_shift = rd->shift; + userpg->time_zero = rd->epoch_ns; + + /* + * This isn't strictly correct, the ARM64 counter can be + * 'short' and then we get funnies when it wraps. The correct + * thing would be to extend the perf ABI with a cycle and mask + * value, but because wrapping on ARM64 is very rare in + * practise this 'works'. + */ + quot = rd->epoch_cyc >> rd->shift; + rem = rd->epoch_cyc & (((u64)1 << rd->shift) - 1); + ns = quot * rd->mult + ((rem * rd->mult) >> rd->shift); + userpg->time_zero -= ns; + + } while (sched_clock_read_retry(seq)); + + userpg->time_offset = userpg->time_zero - now; - clocks_calc_mult_shift(&userpg->time_mult, &shift, freq, - NSEC_PER_SEC, 0); /* * time_shift is not expected to be greater than 31 due to * the original published conversion algorithm shifting a * 32-bit value (now specifies a 64-bit value) - refer * perf_event_mmap_page documentation in perf_event.h. */ - if (shift == 32) { - shift = 31; + if (userpg->time_shift == 32) { + userpg->time_shift = 31; userpg->time_mult >>= 1; } - userpg->time_shift = (u16)shift; - userpg->time_offset = -now; + } -- 2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: Leo Yan <leo.yan@linaro.org> To: Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>, Arnaldo Carvalho de Melo <acme@kernel.org>, Alexander Shishkin <alexander.shishkin@linux.intel.com>, Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Thomas Gleixner <tglx@linutronix.de>, Paul Cercueil <paul@crapouillou.net>, "Ben Dooks (Codethink)" <ben.dooks@codethink.co.uk>, "Ahmed S. Darwish" <a.darwish@linutronix.de>, Adrian Hunter <adrian.hunter@intel.com>, Kan Liang <kan.liang@linux.intel.com>, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/6] arm64: perf: Implement correct cap_user_time Date: Wed, 15 Jul 2020 10:05:08 +0800 [thread overview] Message-ID: <20200715020512.20991-3-leo.yan@linaro.org> (raw) In-Reply-To: <20200715020512.20991-1-leo.yan@linaro.org> From: Peter Zijlstra <peterz@infradead.org> As reported by Leo; the existing implementation is broken when the clock and counter don't intersect at 0. Use the sched_clock's struct clock_read_data information to correctly implement cap_user_time and cap_user_time_zero. Note that the ARM64 counter is architecturally only guaranteed to be 56bit wide (implementations are allowed to be wider) and the existing perf ABI cannot deal with wrap-around. This implementation should also be faster than the old; seeing how we don't need to recompute mult and shift all the time. [leoyan: Use quot/rem to convert cyc to ns to avoid overflow] Reported-by: Leo Yan <leo.yan@linaro.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> --- arch/arm64/kernel/perf_event.c | 40 ++++++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c index 4d7879484cec..35c2c737d4af 100644 --- a/arch/arm64/kernel/perf_event.c +++ b/arch/arm64/kernel/perf_event.c @@ -19,6 +19,7 @@ #include <linux/of.h> #include <linux/perf/arm_pmu.h> #include <linux/platform_device.h> +#include <linux/sched_clock.h> #include <linux/smp.h> /* ARMv8 Cortex-A53 specific event types. */ @@ -1165,28 +1166,49 @@ device_initcall(armv8_pmu_driver_init) void arch_perf_update_userpage(struct perf_event *event, struct perf_event_mmap_page *userpg, u64 now) { - u32 freq; - u32 shift; + struct clock_read_data *rd; + unsigned int seq; + u64 quot, rem, ns; /* * Internal timekeeping for enabled/running/stopped times * is always computed with the sched_clock. */ - freq = arch_timer_get_rate(); userpg->cap_user_time = 1; + userpg->cap_user_time_zero = 1; + + do { + rd = sched_clock_read_begin(&seq); + + userpg->time_mult = rd->mult; + userpg->time_shift = rd->shift; + userpg->time_zero = rd->epoch_ns; + + /* + * This isn't strictly correct, the ARM64 counter can be + * 'short' and then we get funnies when it wraps. The correct + * thing would be to extend the perf ABI with a cycle and mask + * value, but because wrapping on ARM64 is very rare in + * practise this 'works'. + */ + quot = rd->epoch_cyc >> rd->shift; + rem = rd->epoch_cyc & (((u64)1 << rd->shift) - 1); + ns = quot * rd->mult + ((rem * rd->mult) >> rd->shift); + userpg->time_zero -= ns; + + } while (sched_clock_read_retry(seq)); + + userpg->time_offset = userpg->time_zero - now; - clocks_calc_mult_shift(&userpg->time_mult, &shift, freq, - NSEC_PER_SEC, 0); /* * time_shift is not expected to be greater than 31 due to * the original published conversion algorithm shifting a * 32-bit value (now specifies a 64-bit value) - refer * perf_event_mmap_page documentation in perf_event.h. */ - if (shift == 32) { - shift = 31; + if (userpg->time_shift == 32) { + userpg->time_shift = 31; userpg->time_mult >>= 1; } - userpg->time_shift = (u16)shift; - userpg->time_offset = -now; + } -- 2.17.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-07-15 2:05 UTC|newest] Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-07-15 2:05 [PATCH v2 0/6] arm64: perf: Proper cap_user_time* support Leo Yan 2020-07-15 2:05 ` Leo Yan 2020-07-15 2:05 ` [PATCH v2 1/6] sched_clock: Expose struct clock_read_data Leo Yan 2020-07-15 2:05 ` Leo Yan 2020-07-15 5:56 ` Ahmed S. Darwish 2020-07-15 5:56 ` Ahmed S. Darwish 2020-07-15 6:54 ` Leo Yan 2020-07-15 6:54 ` Leo Yan 2020-07-15 7:21 ` Ahmed S. Darwish 2020-07-15 7:21 ` Ahmed S. Darwish 2020-07-15 8:12 ` Peter Zijlstra 2020-07-15 8:12 ` Peter Zijlstra 2020-07-15 8:14 ` peterz 2020-07-15 8:14 ` peterz 2020-07-15 9:23 ` Ahmed S. Darwish 2020-07-15 9:23 ` Ahmed S. Darwish 2020-07-15 9:52 ` Peter Zijlstra 2020-07-15 9:52 ` Peter Zijlstra 2020-07-15 11:59 ` [PATCH] time/sched_clock: Use raw_read_seqcount_latch() Ahmed S. Darwish 2020-07-15 15:29 ` Leo Yan 2020-07-15 15:58 ` Peter Zijlstra 2020-07-16 5:22 ` Leo Yan 2020-09-10 15:08 ` [tip: locking/core] time/sched_clock: Use raw_read_seqcount_latch() during suspend tip-bot2 for Ahmed S. Darwish 2020-07-15 2:05 ` Leo Yan [this message] 2020-07-15 2:05 ` [PATCH v2 2/6] arm64: perf: Implement correct cap_user_time Leo Yan 2020-07-15 8:38 ` Peter Zijlstra 2020-07-15 8:38 ` Peter Zijlstra 2020-07-15 15:39 ` Leo Yan 2020-07-15 15:39 ` Leo Yan 2020-07-15 2:05 ` [PATCH v2 3/6] arm64: perf: Only advertise cap_user_time for arch_timer Leo Yan 2020-07-15 2:05 ` Leo Yan 2020-07-15 2:05 ` [PATCH v2 4/6] perf: Add perf_event_mmap_page::cap_user_time_short ABI Leo Yan 2020-07-15 2:05 ` Leo Yan 2020-07-15 2:05 ` [PATCH v2 5/6] arm64: perf: Add cap_user_time_short Leo Yan 2020-07-15 2:05 ` Leo Yan 2020-07-15 2:05 ` [PATCH v2 6/6] tools headers UAPI: Update tools's copy of linux/perf_event.h Leo Yan 2020-07-15 2:05 ` Leo Yan 2020-07-15 5:17 ` [PATCH v2 0/6] arm64: perf: Proper cap_user_time* support Ahmed S. Darwish 2020-07-15 5:17 ` Ahmed S. Darwish 2020-07-15 6:29 ` Leo Yan 2020-07-15 6:29 ` Leo Yan
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200715020512.20991-3-leo.yan@linaro.org \ --to=leo.yan@linaro.org \ --cc=a.darwish@linutronix.de \ --cc=acme@kernel.org \ --cc=adrian.hunter@intel.com \ --cc=alexander.shishkin@linux.intel.com \ --cc=ben.dooks@codethink.co.uk \ --cc=catalin.marinas@arm.com \ --cc=jolsa@redhat.com \ --cc=kan.liang@linux.intel.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mark.rutland@arm.com \ --cc=mingo@redhat.com \ --cc=namhyung@kernel.org \ --cc=paul@crapouillou.net \ --cc=peterz@infradead.org \ --cc=tglx@linutronix.de \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.