From: Marc Zyngier <maz@kernel.org>
To: linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org,
Daniel Lezcano <daniel.lezcano@linaro.org>,
Will Deacon <will@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Peter Shier <pshier@google.com>,
Raghavendra Rao Ananta <rananta@google.com>,
Ricardo Koller <ricarkol@google.com>,
Oliver Upton <oupton@google.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Linus Walleij <linus.walleij@linaro.org>,
kernel-team@android.com
Subject: [PATCH v4 15/17] arm64: Add CNT{P,V}CTSS_EL0 alternatives to cnt{p,v}ct_el0
Date: Sun, 17 Oct 2021 13:42:23 +0100 [thread overview]
Message-ID: <20211017124225.3018098-16-maz@kernel.org> (raw)
In-Reply-To: <20211017124225.3018098-1-maz@kernel.org>
CNTPCTSS_EL0 and CNTVCTSS_EL0 are alternatives to the usual
CNTPCT_EL0 and CNTVCT_EL0 that do not require a previous ISB
to be synchronised (SS stands for Self-Synchronising).
Use the ARM64_HAS_ECV capability to control alternative sequences
that switch to these low(er)-cost primitives. Note that the
counter access in the VDSO is for now left alone until we decide
whether we want to allow this.
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/arch_timer.h | 32 +++++++++++++++++++++--------
arch/arm64/include/asm/sysreg.h | 3 +++
2 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 519ac1f7f859..af1fafbe7e1d 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -64,14 +64,26 @@ DECLARE_PER_CPU(const struct arch_timer_erratum_workaround *,
static inline notrace u64 arch_timer_read_cntpct_el0(void)
{
- isb();
- return read_sysreg(cntpct_el0);
+ u64 cnt;
+
+ asm volatile(ALTERNATIVE("isb\n mrs %0, cntpct_el0",
+ "nop\n" __mrs_s("%0", SYS_CNTPCTSS_EL0),
+ ARM64_HAS_ECV)
+ : "=r" (cnt));
+
+ return cnt;
}
static inline notrace u64 arch_timer_read_cntvct_el0(void)
{
- isb();
- return read_sysreg(cntvct_el0);
+ u64 cnt;
+
+ asm volatile(ALTERNATIVE("isb\n mrs %0, cntvct_el0",
+ "nop\n" __mrs_s("%0", SYS_CNTVCTSS_EL0),
+ ARM64_HAS_ECV)
+ : "=r" (cnt));
+
+ return cnt;
}
#define arch_timer_reg_read_stable(reg) \
@@ -174,8 +186,10 @@ static __always_inline u64 __arch_counter_get_cntpct(void)
{
u64 cnt;
- isb();
- cnt = read_sysreg(cntpct_el0);
+ asm volatile(ALTERNATIVE("isb\n mrs %0, cntpct_el0",
+ "nop\n" __mrs_s("%0", SYS_CNTPCTSS_EL0),
+ ARM64_HAS_ECV)
+ : "=r" (cnt));
arch_counter_enforce_ordering(cnt);
return cnt;
}
@@ -193,8 +207,10 @@ static __always_inline u64 __arch_counter_get_cntvct(void)
{
u64 cnt;
- isb();
- cnt = read_sysreg(cntvct_el0);
+ asm volatile(ALTERNATIVE("isb\n mrs %0, cntvct_el0",
+ "nop\n" __mrs_s("%0", SYS_CNTVCTSS_EL0),
+ ARM64_HAS_ECV)
+ : "=r" (cnt));
arch_counter_enforce_ordering(cnt);
return cnt;
}
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index b268082d67ed..5ce70c034d37 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -507,6 +507,9 @@
#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0)
+#define SYS_CNTPCTSS_EL0 sys_reg(3, 3, 14, 0, 5)
+#define SYS_CNTVCTSS_EL0 sys_reg(3, 3, 14, 0, 6)
+
#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)
#define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1)
#define SYS_CNTP_CVAL_EL0 sys_reg(3, 3, 14, 2, 2)
--
2.30.2
next prev parent reply other threads:[~2021-10-17 12:46 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-17 12:42 [PATCH v4 00/17] clocksource/arm_arch_timer: Add basic ARMv8.6 support Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 01/17] clocksource/arm_arch_timer: Add build-time guards for unhandled register accesses Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 02/17] clocksource/arm_arch_timer: Drop CNT*_TVAL read accessors Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 03/17] clocksource/arm_arch_timer: Extend write side of timer register accessors to u64 Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 04/17] clocksource/arm_arch_timer: Move system register timer programming over to CVAL Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 05/17] clocksource/arm_arch_timer: Move drop _tval from erratum function names Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 06/17] clocksource/arm_arch_timer: Fix MMIO base address vs callback ordering issue Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 07/17] clocksource/arm_arch_timer: Move MMIO timer programming over to CVAL Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 08/17] clocksource/arm_arch_timer: Advertise 56bit timer to the core code Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 09/17] clocksource/arm_arch_timer: Work around broken CVAL implementations Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 10/17] clocksource/arm_arch_timer: Remove any trace of the TVAL programming interface Marc Zyngier
2021-10-24 15:40 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 11/17] clocksource/arm_arch_timer: Drop unnecessary ISB on CVAL programming Marc Zyngier
2021-10-24 15:39 ` [tip: timers/core] clocksource/drivers/arm_arch_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 12/17] clocksource/arm_arch_timer: Fix masking for high freq counters Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 13/17] clocksource/arch_arm_timer: Move workaround synchronisation around Marc Zyngier
2021-10-24 15:39 ` [tip: timers/core] clocksource/drivers/arch_arm_timer: " tip-bot2 for Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 14/17] arm64: Add a capability for FEAT_ECV Marc Zyngier
2021-10-17 12:42 ` Marc Zyngier [this message]
2021-10-17 12:42 ` [PATCH v4 16/17] arm64: Add handling of CNTVCTSS traps Marc Zyngier
2021-10-17 12:42 ` [PATCH v4 17/17] arm64: Add HWCAP for self-synchronising virtual counter Marc Zyngier
2021-10-19 12:20 ` [PATCH v4 00/17] clocksource/arm_arch_timer: Add basic ARMv8.6 support Will Deacon
2021-10-24 15:40 ` [tip: timers/core] Merge branch 'timers/drivers/armv8.6_arch_timer' into timers/drivers/next tip-bot2 for Daniel Lezcano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211017124225.3018098-16-maz@kernel.org \
--to=maz@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=daniel.lezcano@linaro.org \
--cc=kernel-team@android.com \
--cc=linus.walleij@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=oupton@google.com \
--cc=pshier@google.com \
--cc=rananta@google.com \
--cc=ricarkol@google.com \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).