[PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[Marc Zyngier]

It has recently become obvious that a number of arm64 systems have
been blessed with a set of timers that are slightly less than perfect,
and require a bit of hand-holding. We already have a bunch of
errata-specific code to deal with this, but as we're adding more
potential detection methods (DT, ACPI, capability), things are getting
a bit out of hands.

Instead of adding more ad-hoc fixes to an already difficult code base,
let's give ourselves a bit of an infrastructure that can deal with
this and hide most of the uggliness behind frendly accessors.

The series is structured as such:

- The first half of the series rework the existing workarounds,
  allowing errata to be matched using a given detection method

- Another patch allows a workaround to affect a subset of the CPUs,
  and not the whole system

- Another set of patches allow the virtual counter to be trapped when
  accessed from userspace (something that affects the current set of
  broken platform, and that is not worked around yet)

- We then work around a Cortex-A73 erratum, whose counter can return a
  wrong value if read while crossing a 32bit boundary

- Finally, we add some ACPI-specific workarounds for HiSilicon
  platforms that have the HISILICON_ERRATUM_161010101 defect.

Note that so far, we only deal with arm64. Once the infrastructure is
agreed upon, we can look at generalizing it (to some extent) to 32bit
ARM (typical use case would be a 32bit guest running on an affected
host).

Thanks,

	M.

Marc Zyngier (17):
  arm64: arch_timer: Add infrastructure for multiple erratum detection
    methods
  arm64: arch_timer: Add erratum handler for globally defined capability
  arm64: Allow checking of a CPU-local erratum
  arm64: arch_timer: Add erratum handler for CPU-specific capability
  arm64: arch_timer: Move arch_timer_reg_read/write around
  arm64: arch_timer: Get rid of erratum_workaround_set_sne
  arm64: arch_timer: Rework the set_next_event workarounds
  arm64: arch_timer: Make workaround methods optional
  arm64: arch_timer: Allows a CPU-specific erratum to only affect a
    subset of CPUs
  arm64: Add CNTVCT_EL0 trap handler
  arm64: arch_timer: Move clocksource_counter and co around
  arm64: arch_timer: Enable CNTVCT_EL0 trap if workaround is enabled
  arm64: cpu_errata: Allow an erratum to be match for all revisions of a
    core
  arm64: Define Cortex-A73 MIDR
  arm64: arch_timer: Workaround for Cortex-A73 erratum 858921
  arm64: arch_timer: Allow erratum matching with ACPI OEM information
  arm64: arch_timer: Add HISILICON_ERRATUM_161010101 ACPI matching data

 Documentation/arm64/silicon-errata.txt |   1 +
 arch/arm64/include/asm/arch_timer.h    |  44 ++-
 arch/arm64/include/asm/cpucaps.h       |   3 +-
 arch/arm64/include/asm/cputype.h       |   2 +
 arch/arm64/include/asm/esr.h           |   2 +
 arch/arm64/kernel/cpu_errata.c         |  15 +
 arch/arm64/kernel/cpufeature.c         |  13 +-
 arch/arm64/kernel/traps.c              |  14 +
 drivers/clocksource/Kconfig            |  11 +
 drivers/clocksource/arm_arch_timer.c   | 529 +++++++++++++++++++++++----------
 10 files changed, 465 insertions(+), 169 deletions(-)

-- 
2.11.0

[PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[Marc Zyngier]

It has recently become obvious that a number of arm64 systems have
been blessed with a set of timers that are slightly less than perfect,
and require a bit of hand-holding. We already have a bunch of
errata-specific code to deal with this, but as we're adding more
potential detection methods (DT, ACPI, capability), things are getting
a bit out of hands.

Instead of adding more ad-hoc fixes to an already difficult code base,
let's give ourselves a bit of an infrastructure that can deal with
this and hide most of the uggliness behind frendly accessors.

The series is structured as such:

- The first half of the series rework the existing workarounds,
  allowing errata to be matched using a given detection method

- Another patch allows a workaround to affect a subset of the CPUs,
  and not the whole system

- Another set of patches allow the virtual counter to be trapped when
  accessed from userspace (something that affects the current set of
  broken platform, and that is not worked around yet)

- We then work around a Cortex-A73 erratum, whose counter can return a
  wrong value if read while crossing a 32bit boundary

- Finally, we add some ACPI-specific workarounds for HiSilicon
  platforms that have the HISILICON_ERRATUM_161010101 defect.

Note that so far, we only deal with arm64. Once the infrastructure is
agreed upon, we can look at generalizing it (to some extent) to 32bit
ARM (typical use case would be a 32bit guest running on an affected
host).

Thanks,

	M.

Marc Zyngier (17):
  arm64: arch_timer: Add infrastructure for multiple erratum detection
    methods
  arm64: arch_timer: Add erratum handler for globally defined capability
  arm64: Allow checking of a CPU-local erratum
  arm64: arch_timer: Add erratum handler for CPU-specific capability
  arm64: arch_timer: Move arch_timer_reg_read/write around
  arm64: arch_timer: Get rid of erratum_workaround_set_sne
  arm64: arch_timer: Rework the set_next_event workarounds
  arm64: arch_timer: Make workaround methods optional
  arm64: arch_timer: Allows a CPU-specific erratum to only affect a
    subset of CPUs
  arm64: Add CNTVCT_EL0 trap handler
  arm64: arch_timer: Move clocksource_counter and co around
  arm64: arch_timer: Enable CNTVCT_EL0 trap if workaround is enabled
  arm64: cpu_errata: Allow an erratum to be match for all revisions of a
    core
  arm64: Define Cortex-A73 MIDR
  arm64: arch_timer: Workaround for Cortex-A73 erratum 858921
  arm64: arch_timer: Allow erratum matching with ACPI OEM information
  arm64: arch_timer: Add HISILICON_ERRATUM_161010101 ACPI matching data

 Documentation/arm64/silicon-errata.txt |   1 +
 arch/arm64/include/asm/arch_timer.h    |  44 ++-
 arch/arm64/include/asm/cpucaps.h       |   3 +-
 arch/arm64/include/asm/cputype.h       |   2 +
 arch/arm64/include/asm/esr.h           |   2 +
 arch/arm64/kernel/cpu_errata.c         |  15 +
 arch/arm64/kernel/cpufeature.c         |  13 +-
 arch/arm64/kernel/traps.c              |  14 +
 drivers/clocksource/Kconfig            |  11 +
 drivers/clocksource/arm_arch_timer.c   | 529 +++++++++++++++++++++++----------
 10 files changed, 465 insertions(+), 169 deletions(-)

-- 
2.11.0

[PATCH 01/17] arm64: arch_timer: Add infrastructure for multiple erratum detection methods

[Marc Zyngier]

We're currently stuck with DT when it comes to handling errata, which
is pretty restrictive. In order to make things more flexible, let's
introduce an infrastructure that could support alternative discovery
methods. No change in functionnality.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  7 +++-
 drivers/clocksource/arm_arch_timer.c | 80 +++++++++++++++++++++++++++++++-----
 2 files changed, 75 insertions(+), 12 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index b4b34004a21e..1c92d52619a6 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -37,9 +37,14 @@ extern struct static_key_false arch_timer_read_ool_enabled;
 #define needs_unstable_timer_counter_workaround()  false
 #endif
 
+enum arch_timer_erratum_match_type {
+	ate_match_dt,
+};
 
 struct arch_timer_erratum_workaround {
-	const char *id;		/* Indicate the Erratum ID */
+	enum arch_timer_erratum_match_type match_type;
+	const void *id;		/* Indicate the Erratum ID */
+	const char *desc_str;
 	u32 (*read_cntp_tval_el0)(void);
 	u32 (*read_cntv_tval_el0)(void);
 	u64 (*read_cntvct_el0)(void);
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 7a8a4117f123..36980cdd5c0a 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -182,7 +182,9 @@ EXPORT_SYMBOL_GPL(arch_timer_read_ool_enabled);
 static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 #ifdef CONFIG_FSL_ERRATUM_A008585
 	{
+		.match_type = ate_match_dt,
 		.id = "fsl,erratum-a008585",
+		.desc_str = "Freescale erratum a005858",
 		.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,
 		.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,
 		.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,
@@ -190,13 +192,78 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 #endif
 #ifdef CONFIG_HISILICON_ERRATUM_161010101
 	{
+		.match_type = ate_match_dt,
 		.id = "hisilicon,erratum-161010101",
+		.desc_str = "HiSilicon erratum 161010101",
 		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
 		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
 		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
 	},
 #endif
 };
+
+typedef bool (*ate_match_fn_t)(const struct arch_timer_erratum_workaround *,
+			       const void *);
+
+static
+bool arch_timer_check_dt_erratum(const struct arch_timer_erratum_workaround *wa,
+				 const void *arg)
+{
+	const struct device_node *np = arg;
+
+	return of_property_read_bool(np, wa->id);
+}
+
+static const struct arch_timer_erratum_workaround *
+arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
+			  ate_match_fn_t match_fn,
+			  void *arg)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(ool_workarounds); i++) {
+		if (ool_workarounds[i].match_type != type)
+			continue;
+
+		if (match_fn(&ool_workarounds[i], arg))
+			return &ool_workarounds[i];
+	}
+
+	return NULL;
+}
+
+static
+void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa)
+{
+	timer_unstable_counter_workaround = wa;
+	static_branch_enable(&arch_timer_read_ool_enabled);
+}
+
+static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type type,
+					    void *arg)
+{
+	const struct arch_timer_erratum_workaround *wa;
+	ate_match_fn_t match_fn = NULL;
+
+	if (static_branch_unlikely(&arch_timer_read_ool_enabled))
+		return;
+
+	switch (type) {
+	case ate_match_dt:
+		match_fn = arch_timer_check_dt_erratum;
+		break;
+	}
+
+	wa = arch_timer_iterate_errata(type, match_fn, arg);
+	if (!wa)
+		return;
+
+	arch_timer_enable_workaround(wa);
+	pr_info("Enabling global workaround for %s\n", wa->desc_str);
+}
+
+#else
+#define arch_timer_check_ool_workaround(t,a)		do { } while(0)
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
 static __always_inline
@@ -960,17 +1027,8 @@ static int __init arch_timer_of_init(struct device_node *np)
 
 	arch_timer_c3stop = !of_property_read_bool(np, "always-on");
 
-#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-	for (i = 0; i < ARRAY_SIZE(ool_workarounds); i++) {
-		if (of_property_read_bool(np, ool_workarounds[i].id)) {
-			timer_unstable_counter_workaround = &ool_workarounds[i];
-			static_branch_enable(&arch_timer_read_ool_enabled);
-			pr_info("arch_timer: Enabling workaround for %s\n",
-				timer_unstable_counter_workaround->id);
-			break;
-		}
-	}
-#endif
+	/* Check for globally applicable workarounds */
+	arch_timer_check_ool_workaround(ate_match_dt, np);
 
 	/*
 	 * If we cannot rely on firmware initializing the timer registers then
-- 
2.11.0

[PATCH 01/17] arm64: arch_timer: Add infrastructure for multiple erratum detection methods

[Marc Zyngier]

We're currently stuck with DT when it comes to handling errata, which
is pretty restrictive. In order to make things more flexible, let's
introduce an infrastructure that could support alternative discovery
methods. No change in functionnality.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  7 +++-
 drivers/clocksource/arm_arch_timer.c | 80 +++++++++++++++++++++++++++++++-----
 2 files changed, 75 insertions(+), 12 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index b4b34004a21e..1c92d52619a6 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -37,9 +37,14 @@ extern struct static_key_false arch_timer_read_ool_enabled;
 #define needs_unstable_timer_counter_workaround()  false
 #endif
 
+enum arch_timer_erratum_match_type {
+	ate_match_dt,
+};
 
 struct arch_timer_erratum_workaround {
-	const char *id;		/* Indicate the Erratum ID */
+	enum arch_timer_erratum_match_type match_type;
+	const void *id;		/* Indicate the Erratum ID */
+	const char *desc_str;
 	u32 (*read_cntp_tval_el0)(void);
 	u32 (*read_cntv_tval_el0)(void);
 	u64 (*read_cntvct_el0)(void);
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 7a8a4117f123..36980cdd5c0a 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -182,7 +182,9 @@ EXPORT_SYMBOL_GPL(arch_timer_read_ool_enabled);
 static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 #ifdef CONFIG_FSL_ERRATUM_A008585
 	{
+		.match_type = ate_match_dt,
 		.id = "fsl,erratum-a008585",
+		.desc_str = "Freescale erratum a005858",
 		.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,
 		.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,
 		.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,
@@ -190,13 +192,78 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 #endif
 #ifdef CONFIG_HISILICON_ERRATUM_161010101
 	{
+		.match_type = ate_match_dt,
 		.id = "hisilicon,erratum-161010101",
+		.desc_str = "HiSilicon erratum 161010101",
 		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
 		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
 		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
 	},
 #endif
 };
+
+typedef bool (*ate_match_fn_t)(const struct arch_timer_erratum_workaround *,
+			       const void *);
+
+static
+bool arch_timer_check_dt_erratum(const struct arch_timer_erratum_workaround *wa,
+				 const void *arg)
+{
+	const struct device_node *np = arg;
+
+	return of_property_read_bool(np, wa->id);
+}
+
+static const struct arch_timer_erratum_workaround *
+arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
+			  ate_match_fn_t match_fn,
+			  void *arg)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(ool_workarounds); i++) {
+		if (ool_workarounds[i].match_type != type)
+			continue;
+
+		if (match_fn(&ool_workarounds[i], arg))
+			return &ool_workarounds[i];
+	}
+
+	return NULL;
+}
+
+static
+void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa)
+{
+	timer_unstable_counter_workaround = wa;
+	static_branch_enable(&arch_timer_read_ool_enabled);
+}
+
+static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type type,
+					    void *arg)
+{
+	const struct arch_timer_erratum_workaround *wa;
+	ate_match_fn_t match_fn = NULL;
+
+	if (static_branch_unlikely(&arch_timer_read_ool_enabled))
+		return;
+
+	switch (type) {
+	case ate_match_dt:
+		match_fn = arch_timer_check_dt_erratum;
+		break;
+	}
+
+	wa = arch_timer_iterate_errata(type, match_fn, arg);
+	if (!wa)
+		return;
+
+	arch_timer_enable_workaround(wa);
+	pr_info("Enabling global workaround for %s\n", wa->desc_str);
+}
+
+#else
+#define arch_timer_check_ool_workaround(t,a)		do { } while(0)
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
 static __always_inline
@@ -960,17 +1027,8 @@ static int __init arch_timer_of_init(struct device_node *np)
 
 	arch_timer_c3stop = !of_property_read_bool(np, "always-on");
 
-#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-	for (i = 0; i < ARRAY_SIZE(ool_workarounds); i++) {
-		if (of_property_read_bool(np, ool_workarounds[i].id)) {
-			timer_unstable_counter_workaround = &ool_workarounds[i];
-			static_branch_enable(&arch_timer_read_ool_enabled);
-			pr_info("arch_timer: Enabling workaround for %s\n",
-				timer_unstable_counter_workaround->id);
-			break;
-		}
-	}
-#endif
+	/* Check for globally applicable workarounds */
+	arch_timer_check_ool_workaround(ate_match_dt, np);
 
 	/*
 	 * If we cannot rely on firmware initializing the timer registers then
-- 
2.11.0

[PATCH 02/17] arm64: arch_timer: Add erratum handler for globally defined capability

[Marc Zyngier]

Should we ever have a workaround for an erratum that is detected using
a capability (and affecting the whole system), it'd be nice to have
a way to probe them directly.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  1 +
 drivers/clocksource/arm_arch_timer.c | 14 ++++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 1c92d52619a6..8b8beab20bb6 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -39,6 +39,7 @@ extern struct static_key_false arch_timer_read_ool_enabled;
 
 enum arch_timer_erratum_match_type {
 	ate_match_dt,
+	ate_match_global_cap_id,
 };
 
 struct arch_timer_erratum_workaround {
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 36980cdd5c0a..0470b07af6fe 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -214,6 +214,13 @@ bool arch_timer_check_dt_erratum(const struct arch_timer_erratum_workaround *wa,
 	return of_property_read_bool(np, wa->id);
 }
 
+static
+bool arch_timer_check_global_cap_erratum(const struct arch_timer_erratum_workaround *wa,
+					 const void *arg)
+{
+	return cpus_have_cap((uintptr_t)wa->id);
+}
+
 static const struct arch_timer_erratum_workaround *
 arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
 			  ate_match_fn_t match_fn,
@@ -252,6 +259,9 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 	case ate_match_dt:
 		match_fn = arch_timer_check_dt_erratum;
 		break;
+	case ate_match_global_cap_id:
+		match_fn = arch_timer_check_global_cap_erratum;
+		break;
 	}
 
 	wa = arch_timer_iterate_errata(type, match_fn, arg);
@@ -1029,6 +1039,7 @@ static int __init arch_timer_of_init(struct device_node *np)
 
 	/* Check for globally applicable workarounds */
 	arch_timer_check_ool_workaround(ate_match_dt, np);
+	arch_timer_check_ool_workaround(ate_match_global_cap_id, NULL);
 
 	/*
 	 * If we cannot rely on firmware initializing the timer registers then
@@ -1185,6 +1196,9 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
 	/* Always-on capability */
 	arch_timer_c3stop = !(gtdt->non_secure_el1_flags & ACPI_GTDT_ALWAYS_ON);
 
+	/* Check for globally applicable workarounds */
+	arch_timer_check_ool_workaround(ate_match_global_cap_id, NULL);
+
 	arch_timer_init();
 	return 0;
 }
-- 
2.11.0

[PATCH 02/17] arm64: arch_timer: Add erratum handler for globally defined capability

[Marc Zyngier]

Should we ever have a workaround for an erratum that is detected using
a capability (and affecting the whole system), it'd be nice to have
a way to probe them directly.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  1 +
 drivers/clocksource/arm_arch_timer.c | 14 ++++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 1c92d52619a6..8b8beab20bb6 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -39,6 +39,7 @@ extern struct static_key_false arch_timer_read_ool_enabled;
 
 enum arch_timer_erratum_match_type {
 	ate_match_dt,
+	ate_match_global_cap_id,
 };
 
 struct arch_timer_erratum_workaround {
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 36980cdd5c0a..0470b07af6fe 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -214,6 +214,13 @@ bool arch_timer_check_dt_erratum(const struct arch_timer_erratum_workaround *wa,
 	return of_property_read_bool(np, wa->id);
 }
 
+static
+bool arch_timer_check_global_cap_erratum(const struct arch_timer_erratum_workaround *wa,
+					 const void *arg)
+{
+	return cpus_have_cap((uintptr_t)wa->id);
+}
+
 static const struct arch_timer_erratum_workaround *
 arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
 			  ate_match_fn_t match_fn,
@@ -252,6 +259,9 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 	case ate_match_dt:
 		match_fn = arch_timer_check_dt_erratum;
 		break;
+	case ate_match_global_cap_id:
+		match_fn = arch_timer_check_global_cap_erratum;
+		break;
 	}
 
 	wa = arch_timer_iterate_errata(type, match_fn, arg);
@@ -1029,6 +1039,7 @@ static int __init arch_timer_of_init(struct device_node *np)
 
 	/* Check for globally applicable workarounds */
 	arch_timer_check_ool_workaround(ate_match_dt, np);
+	arch_timer_check_ool_workaround(ate_match_global_cap_id, NULL);
 
 	/*
 	 * If we cannot rely on firmware initializing the timer registers then
@@ -1185,6 +1196,9 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
 	/* Always-on capability */
 	arch_timer_c3stop = !(gtdt->non_secure_el1_flags & ACPI_GTDT_ALWAYS_ON);
 
+	/* Check for globally applicable workarounds */
+	arch_timer_check_ool_workaround(ate_match_global_cap_id, NULL);
+
 	arch_timer_init();
 	return 0;
 }
-- 
2.11.0

[PATCH 03/17] arm64: Allow checking of a CPU-local erratum

[Marc Zyngier]

this_cpu_has_cap() only checks the feature array, and not the errata
one. In order to be able to check for a CPU-local erratum, allow it
to inspect the latter as well.

This is consistent with cpus_have_cap()'s behaviour, which includes
errata already.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/cpufeature.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index abda8e861865..6eb77ae99b79 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1090,20 +1090,29 @@ static void __init setup_feature_capabilities(void)
  * Check if the current CPU has a given feature capability.
  * Should be called from non-preemptible context.
  */
-bool this_cpu_has_cap(unsigned int cap)
+static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
+			       unsigned int cap)
 {
 	const struct arm64_cpu_capabilities *caps;
 
 	if (WARN_ON(preemptible()))
 		return false;
 
-	for (caps = arm64_features; caps->desc; caps++)
+	for (caps = cap_array; caps->desc; caps++)
 		if (caps->capability == cap && caps->matches)
 			return caps->matches(caps, SCOPE_LOCAL_CPU);
 
 	return false;
 }
 
+extern const struct arm64_cpu_capabilities arm64_errata[];
+
+bool this_cpu_has_cap(unsigned int cap)
+{
+	return (__this_cpu_has_cap(arm64_features, cap) ||
+		__this_cpu_has_cap(arm64_errata, cap));
+}
+
 void __init setup_cpu_features(void)
 {
 	u32 cwg;
-- 
2.11.0

[PATCH 03/17] arm64: Allow checking of a CPU-local erratum

[Marc Zyngier]

this_cpu_has_cap() only checks the feature array, and not the errata
one. In order to be able to check for a CPU-local erratum, allow it
to inspect the latter as well.

This is consistent with cpus_have_cap()'s behaviour, which includes
errata already.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/cpufeature.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index abda8e861865..6eb77ae99b79 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1090,20 +1090,29 @@ static void __init setup_feature_capabilities(void)
  * Check if the current CPU has a given feature capability.
  * Should be called from non-preemptible context.
  */
-bool this_cpu_has_cap(unsigned int cap)
+static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
+			       unsigned int cap)
 {
 	const struct arm64_cpu_capabilities *caps;
 
 	if (WARN_ON(preemptible()))
 		return false;
 
-	for (caps = arm64_features; caps->desc; caps++)
+	for (caps = cap_array; caps->desc; caps++)
 		if (caps->capability == cap && caps->matches)
 			return caps->matches(caps, SCOPE_LOCAL_CPU);
 
 	return false;
 }
 
+extern const struct arm64_cpu_capabilities arm64_errata[];
+
+bool this_cpu_has_cap(unsigned int cap)
+{
+	return (__this_cpu_has_cap(arm64_features, cap) ||
+		__this_cpu_has_cap(arm64_errata, cap));
+}
+
 void __init setup_cpu_features(void)
 {
 	u32 cwg;
-- 
2.11.0

[PATCH 04/17] arm64: arch_timer: Add erratum handler for CPU-specific capability

[Marc Zyngier]

Should we ever have a workaround for an erratum that is detected using
a capability and affecting a particular CPU, it'd be nice to have
a way to probe them directly.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  1 +
 drivers/clocksource/arm_arch_timer.c | 28 ++++++++++++++++++++++++----
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 8b8beab20bb6..f841e08a0dfc 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -40,6 +40,7 @@ extern struct static_key_false arch_timer_read_ool_enabled;
 enum arch_timer_erratum_match_type {
 	ate_match_dt,
 	ate_match_global_cap_id,
+	ate_match_local_cap_id,
 };
 
 struct arch_timer_erratum_workaround {
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 0470b07af6fe..3018eeeee7fa 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -221,6 +221,13 @@ bool arch_timer_check_global_cap_erratum(const struct arch_timer_erratum_workaro
 	return cpus_have_cap((uintptr_t)wa->id);
 }
 
+static
+bool arch_timer_check_local_cap_erratum(const struct arch_timer_erratum_workaround *wa,
+					const void *arg)
+{
+	return this_cpu_has_cap((uintptr_t)wa->id);
+}
+
 static const struct arch_timer_erratum_workaround *
 arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
 			  ate_match_fn_t match_fn,
@@ -251,9 +258,7 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 {
 	const struct arch_timer_erratum_workaround *wa;
 	ate_match_fn_t match_fn = NULL;
-
-	if (static_branch_unlikely(&arch_timer_read_ool_enabled))
-		return;
+	bool local = false;
 
 	switch (type) {
 	case ate_match_dt:
@@ -262,14 +267,27 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 	case ate_match_global_cap_id:
 		match_fn = arch_timer_check_global_cap_erratum;
 		break;
+	case ate_match_local_cap_id:
+		match_fn = arch_timer_check_local_cap_erratum;
+		local = true;
+		break;
 	}
 
 	wa = arch_timer_iterate_errata(type, match_fn, arg);
 	if (!wa)
 		return;
 
+	if (static_branch_unlikely(&arch_timer_read_ool_enabled)) {
+		if (wa != timer_unstable_counter_workaround)
+			pr_warn("Can't enable workaround for %s (clashes with %s\n)",
+				wa->desc_str,
+				timer_unstable_counter_workaround->desc_str);
+		return;
+	}
+
 	arch_timer_enable_workaround(wa);
-	pr_info("Enabling global workaround for %s\n", wa->desc_str);
+	pr_info("Enabling %s workaround for %s\n",
+		local ? "local" : "global", wa->desc_str);
 }
 
 #else
@@ -529,6 +547,8 @@ static void __arch_timer_setup(unsigned type,
 			BUG();
 		}
 
+		arch_timer_check_ool_workaround(ate_match_local_cap_id, NULL);
+
 		erratum_workaround_set_sne(clk);
 	} else {
 		clk->features |= CLOCK_EVT_FEAT_DYNIRQ;
-- 
2.11.0

[PATCH 04/17] arm64: arch_timer: Add erratum handler for CPU-specific capability

[Marc Zyngier]

Should we ever have a workaround for an erratum that is detected using
a capability and affecting a particular CPU, it'd be nice to have
a way to probe them directly.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  1 +
 drivers/clocksource/arm_arch_timer.c | 28 ++++++++++++++++++++++++----
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 8b8beab20bb6..f841e08a0dfc 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -40,6 +40,7 @@ extern struct static_key_false arch_timer_read_ool_enabled;
 enum arch_timer_erratum_match_type {
 	ate_match_dt,
 	ate_match_global_cap_id,
+	ate_match_local_cap_id,
 };
 
 struct arch_timer_erratum_workaround {
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 0470b07af6fe..3018eeeee7fa 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -221,6 +221,13 @@ bool arch_timer_check_global_cap_erratum(const struct arch_timer_erratum_workaro
 	return cpus_have_cap((uintptr_t)wa->id);
 }
 
+static
+bool arch_timer_check_local_cap_erratum(const struct arch_timer_erratum_workaround *wa,
+					const void *arg)
+{
+	return this_cpu_has_cap((uintptr_t)wa->id);
+}
+
 static const struct arch_timer_erratum_workaround *
 arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
 			  ate_match_fn_t match_fn,
@@ -251,9 +258,7 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 {
 	const struct arch_timer_erratum_workaround *wa;
 	ate_match_fn_t match_fn = NULL;
-
-	if (static_branch_unlikely(&arch_timer_read_ool_enabled))
-		return;
+	bool local = false;
 
 	switch (type) {
 	case ate_match_dt:
@@ -262,14 +267,27 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 	case ate_match_global_cap_id:
 		match_fn = arch_timer_check_global_cap_erratum;
 		break;
+	case ate_match_local_cap_id:
+		match_fn = arch_timer_check_local_cap_erratum;
+		local = true;
+		break;
 	}
 
 	wa = arch_timer_iterate_errata(type, match_fn, arg);
 	if (!wa)
 		return;
 
+	if (static_branch_unlikely(&arch_timer_read_ool_enabled)) {
+		if (wa != timer_unstable_counter_workaround)
+			pr_warn("Can't enable workaround for %s (clashes with %s\n)",
+				wa->desc_str,
+				timer_unstable_counter_workaround->desc_str);
+		return;
+	}
+
 	arch_timer_enable_workaround(wa);
-	pr_info("Enabling global workaround for %s\n", wa->desc_str);
+	pr_info("Enabling %s workaround for %s\n",
+		local ? "local" : "global", wa->desc_str);
 }
 
 #else
@@ -529,6 +547,8 @@ static void __arch_timer_setup(unsigned type,
 			BUG();
 		}
 
+		arch_timer_check_ool_workaround(ate_match_local_cap_id, NULL);
+
 		erratum_workaround_set_sne(clk);
 	} else {
 		clk->features |= CLOCK_EVT_FEAT_DYNIRQ;
-- 
2.11.0

[PATCH 05/17] arm64: arch_timer: Move arch_timer_reg_read/write around

[Marc Zyngier]

As we're about to move things around, let's start with the low
level read/write functions. This allows us to use these functions
in the errata handling code without having to use forward declaration
of static functions.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 124 +++++++++++++++++------------------
 1 file changed, 62 insertions(+), 62 deletions(-)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 3018eeeee7fa..53cb862eb6df 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -96,6 +96,68 @@ early_param("clocksource.arm_arch_timer.evtstrm", early_evtstrm_cfg);
  * Architected system timer support.
  */
 
+static __always_inline
+void arch_timer_reg_write(int access, enum arch_timer_reg reg, u32 val,
+			  struct clock_event_device *clk)
+{
+	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
+		struct arch_timer *timer = to_arch_timer(clk);
+		switch (reg) {
+		case ARCH_TIMER_REG_CTRL:
+			writel_relaxed(val, timer->base + CNTP_CTL);
+			break;
+		case ARCH_TIMER_REG_TVAL:
+			writel_relaxed(val, timer->base + CNTP_TVAL);
+			break;
+		}
+	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
+		struct arch_timer *timer = to_arch_timer(clk);
+		switch (reg) {
+		case ARCH_TIMER_REG_CTRL:
+			writel_relaxed(val, timer->base + CNTV_CTL);
+			break;
+		case ARCH_TIMER_REG_TVAL:
+			writel_relaxed(val, timer->base + CNTV_TVAL);
+			break;
+		}
+	} else {
+		arch_timer_reg_write_cp15(access, reg, val);
+	}
+}
+
+static __always_inline
+u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
+			struct clock_event_device *clk)
+{
+	u32 val;
+
+	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
+		struct arch_timer *timer = to_arch_timer(clk);
+		switch (reg) {
+		case ARCH_TIMER_REG_CTRL:
+			val = readl_relaxed(timer->base + CNTP_CTL);
+			break;
+		case ARCH_TIMER_REG_TVAL:
+			val = readl_relaxed(timer->base + CNTP_TVAL);
+			break;
+		}
+	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
+		struct arch_timer *timer = to_arch_timer(clk);
+		switch (reg) {
+		case ARCH_TIMER_REG_CTRL:
+			val = readl_relaxed(timer->base + CNTV_CTL);
+			break;
+		case ARCH_TIMER_REG_TVAL:
+			val = readl_relaxed(timer->base + CNTV_TVAL);
+			break;
+		}
+	} else {
+		val = arch_timer_reg_read_cp15(access, reg);
+	}
+
+	return val;
+}
+
 #ifdef CONFIG_FSL_ERRATUM_A008585
 /*
  * The number of retries is an arbitrary value well beyond the highest number
@@ -294,68 +356,6 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
-static __always_inline
-void arch_timer_reg_write(int access, enum arch_timer_reg reg, u32 val,
-			  struct clock_event_device *clk)
-{
-	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
-		struct arch_timer *timer = to_arch_timer(clk);
-		switch (reg) {
-		case ARCH_TIMER_REG_CTRL:
-			writel_relaxed(val, timer->base + CNTP_CTL);
-			break;
-		case ARCH_TIMER_REG_TVAL:
-			writel_relaxed(val, timer->base + CNTP_TVAL);
-			break;
-		}
-	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
-		struct arch_timer *timer = to_arch_timer(clk);
-		switch (reg) {
-		case ARCH_TIMER_REG_CTRL:
-			writel_relaxed(val, timer->base + CNTV_CTL);
-			break;
-		case ARCH_TIMER_REG_TVAL:
-			writel_relaxed(val, timer->base + CNTV_TVAL);
-			break;
-		}
-	} else {
-		arch_timer_reg_write_cp15(access, reg, val);
-	}
-}
-
-static __always_inline
-u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
-			struct clock_event_device *clk)
-{
-	u32 val;
-
-	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
-		struct arch_timer *timer = to_arch_timer(clk);
-		switch (reg) {
-		case ARCH_TIMER_REG_CTRL:
-			val = readl_relaxed(timer->base + CNTP_CTL);
-			break;
-		case ARCH_TIMER_REG_TVAL:
-			val = readl_relaxed(timer->base + CNTP_TVAL);
-			break;
-		}
-	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
-		struct arch_timer *timer = to_arch_timer(clk);
-		switch (reg) {
-		case ARCH_TIMER_REG_CTRL:
-			val = readl_relaxed(timer->base + CNTV_CTL);
-			break;
-		case ARCH_TIMER_REG_TVAL:
-			val = readl_relaxed(timer->base + CNTV_TVAL);
-			break;
-		}
-	} else {
-		val = arch_timer_reg_read_cp15(access, reg);
-	}
-
-	return val;
-}
-
 static __always_inline irqreturn_t timer_handler(const int access,
 					struct clock_event_device *evt)
 {
-- 
2.11.0

[PATCH 05/17] arm64: arch_timer: Move arch_timer_reg_read/write around

[Marc Zyngier]

As we're about to move things around, let's start with the low
level read/write functions. This allows us to use these functions
in the errata handling code without having to use forward declaration
of static functions.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 124 +++++++++++++++++------------------
 1 file changed, 62 insertions(+), 62 deletions(-)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 3018eeeee7fa..53cb862eb6df 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -96,6 +96,68 @@ early_param("clocksource.arm_arch_timer.evtstrm", early_evtstrm_cfg);
  * Architected system timer support.
  */
 
+static __always_inline
+void arch_timer_reg_write(int access, enum arch_timer_reg reg, u32 val,
+			  struct clock_event_device *clk)
+{
+	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
+		struct arch_timer *timer = to_arch_timer(clk);
+		switch (reg) {
+		case ARCH_TIMER_REG_CTRL:
+			writel_relaxed(val, timer->base + CNTP_CTL);
+			break;
+		case ARCH_TIMER_REG_TVAL:
+			writel_relaxed(val, timer->base + CNTP_TVAL);
+			break;
+		}
+	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
+		struct arch_timer *timer = to_arch_timer(clk);
+		switch (reg) {
+		case ARCH_TIMER_REG_CTRL:
+			writel_relaxed(val, timer->base + CNTV_CTL);
+			break;
+		case ARCH_TIMER_REG_TVAL:
+			writel_relaxed(val, timer->base + CNTV_TVAL);
+			break;
+		}
+	} else {
+		arch_timer_reg_write_cp15(access, reg, val);
+	}
+}
+
+static __always_inline
+u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
+			struct clock_event_device *clk)
+{
+	u32 val;
+
+	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
+		struct arch_timer *timer = to_arch_timer(clk);
+		switch (reg) {
+		case ARCH_TIMER_REG_CTRL:
+			val = readl_relaxed(timer->base + CNTP_CTL);
+			break;
+		case ARCH_TIMER_REG_TVAL:
+			val = readl_relaxed(timer->base + CNTP_TVAL);
+			break;
+		}
+	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
+		struct arch_timer *timer = to_arch_timer(clk);
+		switch (reg) {
+		case ARCH_TIMER_REG_CTRL:
+			val = readl_relaxed(timer->base + CNTV_CTL);
+			break;
+		case ARCH_TIMER_REG_TVAL:
+			val = readl_relaxed(timer->base + CNTV_TVAL);
+			break;
+		}
+	} else {
+		val = arch_timer_reg_read_cp15(access, reg);
+	}
+
+	return val;
+}
+
 #ifdef CONFIG_FSL_ERRATUM_A008585
 /*
  * The number of retries is an arbitrary value well beyond the highest number
@@ -294,68 +356,6 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
-static __always_inline
-void arch_timer_reg_write(int access, enum arch_timer_reg reg, u32 val,
-			  struct clock_event_device *clk)
-{
-	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
-		struct arch_timer *timer = to_arch_timer(clk);
-		switch (reg) {
-		case ARCH_TIMER_REG_CTRL:
-			writel_relaxed(val, timer->base + CNTP_CTL);
-			break;
-		case ARCH_TIMER_REG_TVAL:
-			writel_relaxed(val, timer->base + CNTP_TVAL);
-			break;
-		}
-	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
-		struct arch_timer *timer = to_arch_timer(clk);
-		switch (reg) {
-		case ARCH_TIMER_REG_CTRL:
-			writel_relaxed(val, timer->base + CNTV_CTL);
-			break;
-		case ARCH_TIMER_REG_TVAL:
-			writel_relaxed(val, timer->base + CNTV_TVAL);
-			break;
-		}
-	} else {
-		arch_timer_reg_write_cp15(access, reg, val);
-	}
-}
-
-static __always_inline
-u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
-			struct clock_event_device *clk)
-{
-	u32 val;
-
-	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
-		struct arch_timer *timer = to_arch_timer(clk);
-		switch (reg) {
-		case ARCH_TIMER_REG_CTRL:
-			val = readl_relaxed(timer->base + CNTP_CTL);
-			break;
-		case ARCH_TIMER_REG_TVAL:
-			val = readl_relaxed(timer->base + CNTP_TVAL);
-			break;
-		}
-	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
-		struct arch_timer *timer = to_arch_timer(clk);
-		switch (reg) {
-		case ARCH_TIMER_REG_CTRL:
-			val = readl_relaxed(timer->base + CNTV_CTL);
-			break;
-		case ARCH_TIMER_REG_TVAL:
-			val = readl_relaxed(timer->base + CNTV_TVAL);
-			break;
-		}
-	} else {
-		val = arch_timer_reg_read_cp15(access, reg);
-	}
-
-	return val;
-}
-
 static __always_inline irqreturn_t timer_handler(const int access,
 					struct clock_event_device *evt)
 {
-- 
2.11.0

[PATCH 06/17] arm64: arch_timer: Get rid of erratum_workaround_set_sne

[Marc Zyngier]

Let's move the handling of workarounds affecting set_next_event
to the affected function, instead of overwriding the pointers
as an afterthough. Yes, this is an extra indirection on the
erratum handling path, but the HW is busted anyway.

This will allow for some more flexibility later.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 89 ++++++++++++++++--------------------
 1 file changed, 40 insertions(+), 49 deletions(-)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 53cb862eb6df..3e5f9539c0c0 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -241,6 +241,38 @@ EXPORT_SYMBOL_GPL(timer_unstable_counter_workaround);
 DEFINE_STATIC_KEY_FALSE(arch_timer_read_ool_enabled);
 EXPORT_SYMBOL_GPL(arch_timer_read_ool_enabled);
 
+static void erratum_set_next_event_tval_generic(const int access, unsigned long evt,
+						struct clock_event_device *clk)
+{
+	unsigned long ctrl;
+	u64 cval = evt + arch_counter_get_cntvct();
+
+	ctrl = arch_timer_reg_read(access, ARCH_TIMER_REG_CTRL, clk);
+	ctrl |= ARCH_TIMER_CTRL_ENABLE;
+	ctrl &= ~ARCH_TIMER_CTRL_IT_MASK;
+
+	if (access == ARCH_TIMER_PHYS_ACCESS)
+		write_sysreg(cval, cntp_cval_el0);
+	else
+		write_sysreg(cval, cntv_cval_el0);
+
+	arch_timer_reg_write(access, ARCH_TIMER_REG_CTRL, ctrl, clk);
+}
+
+static int erratum_set_next_event_tval_virt(unsigned long evt,
+					    struct clock_event_device *clk)
+{
+	erratum_set_next_event_tval_generic(ARCH_TIMER_VIRT_ACCESS, evt, clk);
+	return 0;
+}
+
+static int erratum_set_next_event_tval_phys(unsigned long evt,
+					    struct clock_event_device *clk)
+{
+	erratum_set_next_event_tval_generic(ARCH_TIMER_PHYS_ACCESS, evt, clk);
+	return 0;
+}
+
 static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 #ifdef CONFIG_FSL_ERRATUM_A008585
 	{
@@ -354,6 +386,8 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 
 #else
 #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
+#define erratum_set_next_event_tval_virt(...)		({BUG_ON(1); 0;})
+#define erratum_set_next_event_tval_phys(...)		({BUG_ON(1); 0;})
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
 static __always_inline irqreturn_t timer_handler(const int access,
@@ -443,43 +477,12 @@ static __always_inline void set_next_event(const int access, unsigned long evt,
 	arch_timer_reg_write(access, ARCH_TIMER_REG_CTRL, ctrl, clk);
 }
 
-#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-static __always_inline void erratum_set_next_event_generic(const int access,
-		unsigned long evt, struct clock_event_device *clk)
-{
-	unsigned long ctrl;
-	u64 cval = evt + arch_counter_get_cntvct();
-
-	ctrl = arch_timer_reg_read(access, ARCH_TIMER_REG_CTRL, clk);
-	ctrl |= ARCH_TIMER_CTRL_ENABLE;
-	ctrl &= ~ARCH_TIMER_CTRL_IT_MASK;
-
-	if (access == ARCH_TIMER_PHYS_ACCESS)
-		write_sysreg(cval, cntp_cval_el0);
-	else if (access == ARCH_TIMER_VIRT_ACCESS)
-		write_sysreg(cval, cntv_cval_el0);
-
-	arch_timer_reg_write(access, ARCH_TIMER_REG_CTRL, ctrl, clk);
-}
-
-static int erratum_set_next_event_virt(unsigned long evt,
-					   struct clock_event_device *clk)
-{
-	erratum_set_next_event_generic(ARCH_TIMER_VIRT_ACCESS, evt, clk);
-	return 0;
-}
-
-static int erratum_set_next_event_phys(unsigned long evt,
-					   struct clock_event_device *clk)
-{
-	erratum_set_next_event_generic(ARCH_TIMER_PHYS_ACCESS, evt, clk);
-	return 0;
-}
-#endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
-
 static int arch_timer_set_next_event_virt(unsigned long evt,
 					  struct clock_event_device *clk)
 {
+	if (needs_unstable_timer_counter_workaround())
+		return erratum_set_next_event_tval_virt(evt, clk);
+
 	set_next_event(ARCH_TIMER_VIRT_ACCESS, evt, clk);
 	return 0;
 }
@@ -487,6 +490,9 @@ static int arch_timer_set_next_event_virt(unsigned long evt,
 static int arch_timer_set_next_event_phys(unsigned long evt,
 					  struct clock_event_device *clk)
 {
+	if (needs_unstable_timer_counter_workaround())
+		return erratum_set_next_event_tval_phys(evt, clk);
+
 	set_next_event(ARCH_TIMER_PHYS_ACCESS, evt, clk);
 	return 0;
 }
@@ -505,19 +511,6 @@ static int arch_timer_set_next_event_phys_mem(unsigned long evt,
 	return 0;
 }
 
-static void erratum_workaround_set_sne(struct clock_event_device *clk)
-{
-#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-	if (!static_branch_unlikely(&arch_timer_read_ool_enabled))
-		return;
-
-	if (arch_timer_uses_ppi == VIRT_PPI)
-		clk->set_next_event = erratum_set_next_event_virt;
-	else
-		clk->set_next_event = erratum_set_next_event_phys;
-#endif
-}
-
 static void __arch_timer_setup(unsigned type,
 			       struct clock_event_device *clk)
 {
@@ -548,8 +541,6 @@ static void __arch_timer_setup(unsigned type,
 		}
 
 		arch_timer_check_ool_workaround(ate_match_local_cap_id, NULL);
-
-		erratum_workaround_set_sne(clk);
 	} else {
 		clk->features |= CLOCK_EVT_FEAT_DYNIRQ;
 		clk->name = "arch_mem_timer";
-- 
2.11.0

[PATCH 06/17] arm64: arch_timer: Get rid of erratum_workaround_set_sne

[Marc Zyngier]

Let's move the handling of workarounds affecting set_next_event
to the affected function, instead of overwriding the pointers
as an afterthough. Yes, this is an extra indirection on the
erratum handling path, but the HW is busted anyway.

This will allow for some more flexibility later.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 89 ++++++++++++++++--------------------
 1 file changed, 40 insertions(+), 49 deletions(-)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 53cb862eb6df..3e5f9539c0c0 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -241,6 +241,38 @@ EXPORT_SYMBOL_GPL(timer_unstable_counter_workaround);
 DEFINE_STATIC_KEY_FALSE(arch_timer_read_ool_enabled);
 EXPORT_SYMBOL_GPL(arch_timer_read_ool_enabled);
 
+static void erratum_set_next_event_tval_generic(const int access, unsigned long evt,
+						struct clock_event_device *clk)
+{
+	unsigned long ctrl;
+	u64 cval = evt + arch_counter_get_cntvct();
+
+	ctrl = arch_timer_reg_read(access, ARCH_TIMER_REG_CTRL, clk);
+	ctrl |= ARCH_TIMER_CTRL_ENABLE;
+	ctrl &= ~ARCH_TIMER_CTRL_IT_MASK;
+
+	if (access == ARCH_TIMER_PHYS_ACCESS)
+		write_sysreg(cval, cntp_cval_el0);
+	else
+		write_sysreg(cval, cntv_cval_el0);
+
+	arch_timer_reg_write(access, ARCH_TIMER_REG_CTRL, ctrl, clk);
+}
+
+static int erratum_set_next_event_tval_virt(unsigned long evt,
+					    struct clock_event_device *clk)
+{
+	erratum_set_next_event_tval_generic(ARCH_TIMER_VIRT_ACCESS, evt, clk);
+	return 0;
+}
+
+static int erratum_set_next_event_tval_phys(unsigned long evt,
+					    struct clock_event_device *clk)
+{
+	erratum_set_next_event_tval_generic(ARCH_TIMER_PHYS_ACCESS, evt, clk);
+	return 0;
+}
+
 static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 #ifdef CONFIG_FSL_ERRATUM_A008585
 	{
@@ -354,6 +386,8 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 
 #else
 #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
+#define erratum_set_next_event_tval_virt(...)		({BUG_ON(1); 0;})
+#define erratum_set_next_event_tval_phys(...)		({BUG_ON(1); 0;})
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
 static __always_inline irqreturn_t timer_handler(const int access,
@@ -443,43 +477,12 @@ static __always_inline void set_next_event(const int access, unsigned long evt,
 	arch_timer_reg_write(access, ARCH_TIMER_REG_CTRL, ctrl, clk);
 }
 
-#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-static __always_inline void erratum_set_next_event_generic(const int access,
-		unsigned long evt, struct clock_event_device *clk)
-{
-	unsigned long ctrl;
-	u64 cval = evt + arch_counter_get_cntvct();
-
-	ctrl = arch_timer_reg_read(access, ARCH_TIMER_REG_CTRL, clk);
-	ctrl |= ARCH_TIMER_CTRL_ENABLE;
-	ctrl &= ~ARCH_TIMER_CTRL_IT_MASK;
-
-	if (access == ARCH_TIMER_PHYS_ACCESS)
-		write_sysreg(cval, cntp_cval_el0);
-	else if (access == ARCH_TIMER_VIRT_ACCESS)
-		write_sysreg(cval, cntv_cval_el0);
-
-	arch_timer_reg_write(access, ARCH_TIMER_REG_CTRL, ctrl, clk);
-}
-
-static int erratum_set_next_event_virt(unsigned long evt,
-					   struct clock_event_device *clk)
-{
-	erratum_set_next_event_generic(ARCH_TIMER_VIRT_ACCESS, evt, clk);
-	return 0;
-}
-
-static int erratum_set_next_event_phys(unsigned long evt,
-					   struct clock_event_device *clk)
-{
-	erratum_set_next_event_generic(ARCH_TIMER_PHYS_ACCESS, evt, clk);
-	return 0;
-}
-#endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
-
 static int arch_timer_set_next_event_virt(unsigned long evt,
 					  struct clock_event_device *clk)
 {
+	if (needs_unstable_timer_counter_workaround())
+		return erratum_set_next_event_tval_virt(evt, clk);
+
 	set_next_event(ARCH_TIMER_VIRT_ACCESS, evt, clk);
 	return 0;
 }
@@ -487,6 +490,9 @@ static int arch_timer_set_next_event_virt(unsigned long evt,
 static int arch_timer_set_next_event_phys(unsigned long evt,
 					  struct clock_event_device *clk)
 {
+	if (needs_unstable_timer_counter_workaround())
+		return erratum_set_next_event_tval_phys(evt, clk);
+
 	set_next_event(ARCH_TIMER_PHYS_ACCESS, evt, clk);
 	return 0;
 }
@@ -505,19 +511,6 @@ static int arch_timer_set_next_event_phys_mem(unsigned long evt,
 	return 0;
 }
 
-static void erratum_workaround_set_sne(struct clock_event_device *clk)
-{
-#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-	if (!static_branch_unlikely(&arch_timer_read_ool_enabled))
-		return;
-
-	if (arch_timer_uses_ppi == VIRT_PPI)
-		clk->set_next_event = erratum_set_next_event_virt;
-	else
-		clk->set_next_event = erratum_set_next_event_phys;
-#endif
-}
-
 static void __arch_timer_setup(unsigned type,
 			       struct clock_event_device *clk)
 {
@@ -548,8 +541,6 @@ static void __arch_timer_setup(unsigned type,
 		}
 
 		arch_timer_check_ool_workaround(ate_match_local_cap_id, NULL);
-
-		erratum_workaround_set_sne(clk);
 	} else {
 		clk->features |= CLOCK_EVT_FEAT_DYNIRQ;
 		clk->name = "arch_mem_timer";
-- 
2.11.0

[PATCH 07/17] arm64: arch_timer: Rework the set_next_event workarounds

[Marc Zyngier]

The way we work around errata affecting set_next_event is not very
nice, at it imposes this workaround on errata that do not need it.

Add new workaround hooks and let the existing workarounds use them.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  4 ++++
 drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++------
 2 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index f841e08a0dfc..a5ed4cd2e972 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -43,6 +43,8 @@ enum arch_timer_erratum_match_type {
 	ate_match_local_cap_id,
 };
 
+struct clock_event_device;
+
 struct arch_timer_erratum_workaround {
 	enum arch_timer_erratum_match_type match_type;
 	const void *id;		/* Indicate the Erratum ID */
@@ -50,6 +52,8 @@ struct arch_timer_erratum_workaround {
 	u32 (*read_cntp_tval_el0)(void);
 	u32 (*read_cntv_tval_el0)(void);
 	u64 (*read_cntvct_el0)(void);
+	int (*set_next_event_phys)(unsigned long, struct clock_event_device *);
+	int (*set_next_event_virt)(unsigned long, struct clock_event_device *);
 };
 
 extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround;
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 3e5f9539c0c0..ac0ddb380d3a 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -282,6 +282,8 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 		.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,
 		.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,
 		.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,
+		.set_next_event_phys = erratum_set_next_event_tval_phys,
+		.set_next_event_virt = erratum_set_next_event_tval_virt,
 	},
 #endif
 #ifdef CONFIG_HISILICON_ERRATUM_161010101
@@ -292,6 +294,8 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
 		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
 		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
+		.set_next_event_phys = erratum_set_next_event_tval_phys,
+		.set_next_event_virt = erratum_set_next_event_tval_virt,
 	},
 #endif
 };
@@ -384,10 +388,22 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 		local ? "local" : "global", wa->desc_str);
 }
 
+#define erratum_handler(fn, r, ...)					\
+({									\
+  	bool __val;							\
+	if (needs_unstable_timer_counter_workaround() &&		\
+	    timer_unstable_counter_workaround->fn) {			\
+		r = timer_unstable_counter_workaround->fn(__VA_ARGS__);	\
+		__val = true;						\
+	} else {							\
+		__val = false;						\
+	}								\
+	__val;								\
+})
+
 #else
 #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
-#define erratum_set_next_event_tval_virt(...)		({BUG_ON(1); 0;})
-#define erratum_set_next_event_tval_phys(...)		({BUG_ON(1); 0;})
+#define erratum_handler(fn, r, ...)			({false;})
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
 static __always_inline irqreturn_t timer_handler(const int access,
@@ -480,8 +496,10 @@ static __always_inline void set_next_event(const int access, unsigned long evt,
 static int arch_timer_set_next_event_virt(unsigned long evt,
 					  struct clock_event_device *clk)
 {
-	if (needs_unstable_timer_counter_workaround())
-		return erratum_set_next_event_tval_virt(evt, clk);
+	int ret;
+
+	if (erratum_handler(set_next_event_virt, ret, evt, clk))
+		return ret;
 
 	set_next_event(ARCH_TIMER_VIRT_ACCESS, evt, clk);
 	return 0;
@@ -490,8 +508,10 @@ static int arch_timer_set_next_event_virt(unsigned long evt,
 static int arch_timer_set_next_event_phys(unsigned long evt,
 					  struct clock_event_device *clk)
 {
-	if (needs_unstable_timer_counter_workaround())
-		return erratum_set_next_event_tval_phys(evt, clk);
+	int ret;
+
+	if (erratum_handler(set_next_event_phys, ret, evt, clk))
+		return ret;
 
 	set_next_event(ARCH_TIMER_PHYS_ACCESS, evt, clk);
 	return 0;
-- 
2.11.0

[PATCH 07/17] arm64: arch_timer: Rework the set_next_event workarounds

[Marc Zyngier]

The way we work around errata affecting set_next_event is not very
nice, at it imposes this workaround on errata that do not need it.

Add new workaround hooks and let the existing workarounds use them.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  4 ++++
 drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++------
 2 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index f841e08a0dfc..a5ed4cd2e972 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -43,6 +43,8 @@ enum arch_timer_erratum_match_type {
 	ate_match_local_cap_id,
 };
 
+struct clock_event_device;
+
 struct arch_timer_erratum_workaround {
 	enum arch_timer_erratum_match_type match_type;
 	const void *id;		/* Indicate the Erratum ID */
@@ -50,6 +52,8 @@ struct arch_timer_erratum_workaround {
 	u32 (*read_cntp_tval_el0)(void);
 	u32 (*read_cntv_tval_el0)(void);
 	u64 (*read_cntvct_el0)(void);
+	int (*set_next_event_phys)(unsigned long, struct clock_event_device *);
+	int (*set_next_event_virt)(unsigned long, struct clock_event_device *);
 };
 
 extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround;
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 3e5f9539c0c0..ac0ddb380d3a 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -282,6 +282,8 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 		.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,
 		.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,
 		.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,
+		.set_next_event_phys = erratum_set_next_event_tval_phys,
+		.set_next_event_virt = erratum_set_next_event_tval_virt,
 	},
 #endif
 #ifdef CONFIG_HISILICON_ERRATUM_161010101
@@ -292,6 +294,8 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
 		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
 		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
+		.set_next_event_phys = erratum_set_next_event_tval_phys,
+		.set_next_event_virt = erratum_set_next_event_tval_virt,
 	},
 #endif
 };
@@ -384,10 +388,22 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 		local ? "local" : "global", wa->desc_str);
 }
 
+#define erratum_handler(fn, r, ...)					\
+({									\
+  	bool __val;							\
+	if (needs_unstable_timer_counter_workaround() &&		\
+	    timer_unstable_counter_workaround->fn) {			\
+		r = timer_unstable_counter_workaround->fn(__VA_ARGS__);	\
+		__val = true;						\
+	} else {							\
+		__val = false;						\
+	}								\
+	__val;								\
+})
+
 #else
 #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
-#define erratum_set_next_event_tval_virt(...)		({BUG_ON(1); 0;})
-#define erratum_set_next_event_tval_phys(...)		({BUG_ON(1); 0;})
+#define erratum_handler(fn, r, ...)			({false;})
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
 static __always_inline irqreturn_t timer_handler(const int access,
@@ -480,8 +496,10 @@ static __always_inline void set_next_event(const int access, unsigned long evt,
 static int arch_timer_set_next_event_virt(unsigned long evt,
 					  struct clock_event_device *clk)
 {
-	if (needs_unstable_timer_counter_workaround())
-		return erratum_set_next_event_tval_virt(evt, clk);
+	int ret;
+
+	if (erratum_handler(set_next_event_virt, ret, evt, clk))
+		return ret;
 
 	set_next_event(ARCH_TIMER_VIRT_ACCESS, evt, clk);
 	return 0;
@@ -490,8 +508,10 @@ static int arch_timer_set_next_event_virt(unsigned long evt,
 static int arch_timer_set_next_event_phys(unsigned long evt,
 					  struct clock_event_device *clk)
 {
-	if (needs_unstable_timer_counter_workaround())
-		return erratum_set_next_event_tval_phys(evt, clk);
+	int ret;
+
+	if (erratum_handler(set_next_event_phys, ret, evt, clk))
+		return ret;
 
 	set_next_event(ARCH_TIMER_PHYS_ACCESS, evt, clk);
 	return 0;
-- 
2.11.0

[PATCH 08/17] arm64: arch_timer: Make workaround methods optional

[Marc Zyngier]

Not all errata need to workaround all access types. Allow them to
be optional.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index a5ed4cd2e972..7fbb55ded0c9 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -61,8 +61,9 @@ extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workar
 #define arch_timer_reg_read_stable(reg) 		\
 ({							\
 	u64 _val;					\
-	if (needs_unstable_timer_counter_workaround())		\
-		_val = timer_unstable_counter_workaround->read_##reg();\
+	if (needs_unstable_timer_counter_workaround() &&		\
+	    timer_unstable_counter_workaround->read_##reg)		\
+		_val = timer_unstable_counter_workaround->read_##reg();	\
 	else						\
 		_val = read_sysreg(reg);		\
 	_val;						\
-- 
2.11.0

[PATCH 08/17] arm64: arch_timer: Make workaround methods optional

[Marc Zyngier]

Not all errata need to workaround all access types. Allow them to
be optional.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index a5ed4cd2e972..7fbb55ded0c9 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -61,8 +61,9 @@ extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workar
 #define arch_timer_reg_read_stable(reg) 		\
 ({							\
 	u64 _val;					\
-	if (needs_unstable_timer_counter_workaround())		\
-		_val = timer_unstable_counter_workaround->read_##reg();\
+	if (needs_unstable_timer_counter_workaround() &&		\
+	    timer_unstable_counter_workaround->read_##reg)		\
+		_val = timer_unstable_counter_workaround->read_##reg();	\
 	else						\
 		_val = read_sysreg(reg);		\
 	_val;						\
-- 
2.11.0

[PATCH 09/17] arm64: arch_timer: Allows a CPU-specific erratum to only affect a subset of CPUs

[Marc Zyngier]

Instead of applying a CPU-specific workaround to all CPUs in the system,
allow it to only affect a subset of them (typical big-little case).

This is done by turning the erratum pointer into a per-CPU variable.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  | 31 ++++++++++++++++----------
 drivers/clocksource/arm_arch_timer.c | 42 +++++++++++++++++++++++++-----------
 2 files changed, 50 insertions(+), 23 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 7fbb55ded0c9..159513479f6e 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -25,6 +25,7 @@
 #include <linux/bug.h>
 #include <linux/init.h>
 #include <linux/jump_label.h>
+#include <linux/smp.h>
 #include <linux/types.h>
 
 #include <clocksource/arm_arch_timer.h>
@@ -56,17 +57,25 @@ struct arch_timer_erratum_workaround {
 	int (*set_next_event_virt)(unsigned long, struct clock_event_device *);
 };
 
-extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround;
-
-#define arch_timer_reg_read_stable(reg) 		\
-({							\
-	u64 _val;					\
-	if (needs_unstable_timer_counter_workaround() &&		\
-	    timer_unstable_counter_workaround->read_##reg)		\
-		_val = timer_unstable_counter_workaround->read_##reg();	\
-	else						\
-		_val = read_sysreg(reg);		\
-	_val;						\
+DECLARE_PER_CPU(const struct arch_timer_erratum_workaround *,
+		timer_unstable_counter_workaround);
+
+#define arch_timer_reg_read_stable(reg)					\
+({									\
+	u64 _val;							\
+	if (needs_unstable_timer_counter_workaround()) {		\
+		const struct arch_timer_erratum_workaround *wa;		\
+		preempt_disable();					\
+		wa = __this_cpu_read(timer_unstable_counter_workaround); \
+		if (wa && wa->read_##reg)				\
+			_val = wa->read_##reg();			\
+		else							\
+			_val = read_sysreg(reg);			\
+		preempt_enable();					\
+	} else {							\
+		_val = read_sysreg(reg);				\
+	}								\
+	_val;								\
 })
 
 /*
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index ac0ddb380d3a..2bbe0b9c3c24 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -235,7 +235,8 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
 #endif
 
 #ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround = NULL;
+DEFINE_PER_CPU(const struct arch_timer_erratum_workaround *,
+	       timer_unstable_counter_workaround);
 EXPORT_SYMBOL_GPL(timer_unstable_counter_workaround);
 
 DEFINE_STATIC_KEY_FALSE(arch_timer_read_ool_enabled);
@@ -345,9 +346,18 @@ arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
 }
 
 static
-void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa)
+void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa,
+				  bool local)
 {
-	timer_unstable_counter_workaround = wa;
+	int i;
+
+	if (local) {
+		__this_cpu_write(timer_unstable_counter_workaround, wa);
+	} else {
+		for_each_possible_cpu(i)
+			per_cpu(timer_unstable_counter_workaround, i) = wa;
+	}
+
 	static_branch_enable(&arch_timer_read_ool_enabled);
 }
 
@@ -376,14 +386,17 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 		return;
 
 	if (static_branch_unlikely(&arch_timer_read_ool_enabled)) {
-		if (wa != timer_unstable_counter_workaround)
+		const struct arch_timer_erratum_workaround *__wa;
+		__wa = __this_cpu_read(timer_unstable_counter_workaround);
+		if (__wa && wa != __wa)
 			pr_warn("Can't enable workaround for %s (clashes with %s\n)",
-				wa->desc_str,
-				timer_unstable_counter_workaround->desc_str);
-		return;
+				wa->desc_str, __wa->desc_str);
+
+		if (__wa)
+			return;
 	}
 
-	arch_timer_enable_workaround(wa);
+	arch_timer_enable_workaround(wa, local);
 	pr_info("Enabling %s workaround for %s\n",
 		local ? "local" : "global", wa->desc_str);
 }
@@ -391,10 +404,15 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 #define erratum_handler(fn, r, ...)					\
 ({									\
   	bool __val;							\
-	if (needs_unstable_timer_counter_workaround() &&		\
-	    timer_unstable_counter_workaround->fn) {			\
-		r = timer_unstable_counter_workaround->fn(__VA_ARGS__);	\
-		__val = true;						\
+	if (needs_unstable_timer_counter_workaround()) {		\
+		const struct arch_timer_erratum_workaround *__wa;	\
+		__wa = __this_cpu_read(timer_unstable_counter_workaround); \
+		if (__wa && __wa->fn) {					\
+			r = __wa->fn(__VA_ARGS__);			\
+			__val = true;					\
+		} else {						\
+			__val = false;					\
+		}							\
 	} else {							\
 		__val = false;						\
 	}								\
-- 
2.11.0

[PATCH 09/17] arm64: arch_timer: Allows a CPU-specific erratum to only affect a subset of CPUs

[Marc Zyngier]

Instead of applying a CPU-specific workaround to all CPUs in the system,
allow it to only affect a subset of them (typical big-little case).

This is done by turning the erratum pointer into a per-CPU variable.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  | 31 ++++++++++++++++----------
 drivers/clocksource/arm_arch_timer.c | 42 +++++++++++++++++++++++++-----------
 2 files changed, 50 insertions(+), 23 deletions(-)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 7fbb55ded0c9..159513479f6e 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -25,6 +25,7 @@
 #include <linux/bug.h>
 #include <linux/init.h>
 #include <linux/jump_label.h>
+#include <linux/smp.h>
 #include <linux/types.h>
 
 #include <clocksource/arm_arch_timer.h>
@@ -56,17 +57,25 @@ struct arch_timer_erratum_workaround {
 	int (*set_next_event_virt)(unsigned long, struct clock_event_device *);
 };
 
-extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround;
-
-#define arch_timer_reg_read_stable(reg) 		\
-({							\
-	u64 _val;					\
-	if (needs_unstable_timer_counter_workaround() &&		\
-	    timer_unstable_counter_workaround->read_##reg)		\
-		_val = timer_unstable_counter_workaround->read_##reg();	\
-	else						\
-		_val = read_sysreg(reg);		\
-	_val;						\
+DECLARE_PER_CPU(const struct arch_timer_erratum_workaround *,
+		timer_unstable_counter_workaround);
+
+#define arch_timer_reg_read_stable(reg)					\
+({									\
+	u64 _val;							\
+	if (needs_unstable_timer_counter_workaround()) {		\
+		const struct arch_timer_erratum_workaround *wa;		\
+		preempt_disable();					\
+		wa = __this_cpu_read(timer_unstable_counter_workaround); \
+		if (wa && wa->read_##reg)				\
+			_val = wa->read_##reg();			\
+		else							\
+			_val = read_sysreg(reg);			\
+		preempt_enable();					\
+	} else {							\
+		_val = read_sysreg(reg);				\
+	}								\
+	_val;								\
 })
 
 /*
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index ac0ddb380d3a..2bbe0b9c3c24 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -235,7 +235,8 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
 #endif
 
 #ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround = NULL;
+DEFINE_PER_CPU(const struct arch_timer_erratum_workaround *,
+	       timer_unstable_counter_workaround);
 EXPORT_SYMBOL_GPL(timer_unstable_counter_workaround);
 
 DEFINE_STATIC_KEY_FALSE(arch_timer_read_ool_enabled);
@@ -345,9 +346,18 @@ arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
 }
 
 static
-void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa)
+void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa,
+				  bool local)
 {
-	timer_unstable_counter_workaround = wa;
+	int i;
+
+	if (local) {
+		__this_cpu_write(timer_unstable_counter_workaround, wa);
+	} else {
+		for_each_possible_cpu(i)
+			per_cpu(timer_unstable_counter_workaround, i) = wa;
+	}
+
 	static_branch_enable(&arch_timer_read_ool_enabled);
 }
 
@@ -376,14 +386,17 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 		return;
 
 	if (static_branch_unlikely(&arch_timer_read_ool_enabled)) {
-		if (wa != timer_unstable_counter_workaround)
+		const struct arch_timer_erratum_workaround *__wa;
+		__wa = __this_cpu_read(timer_unstable_counter_workaround);
+		if (__wa && wa != __wa)
 			pr_warn("Can't enable workaround for %s (clashes with %s\n)",
-				wa->desc_str,
-				timer_unstable_counter_workaround->desc_str);
-		return;
+				wa->desc_str, __wa->desc_str);
+
+		if (__wa)
+			return;
 	}
 
-	arch_timer_enable_workaround(wa);
+	arch_timer_enable_workaround(wa, local);
 	pr_info("Enabling %s workaround for %s\n",
 		local ? "local" : "global", wa->desc_str);
 }
@@ -391,10 +404,15 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 #define erratum_handler(fn, r, ...)					\
 ({									\
   	bool __val;							\
-	if (needs_unstable_timer_counter_workaround() &&		\
-	    timer_unstable_counter_workaround->fn) {			\
-		r = timer_unstable_counter_workaround->fn(__VA_ARGS__);	\
-		__val = true;						\
+	if (needs_unstable_timer_counter_workaround()) {		\
+		const struct arch_timer_erratum_workaround *__wa;	\
+		__wa = __this_cpu_read(timer_unstable_counter_workaround); \
+		if (__wa && __wa->fn) {					\
+			r = __wa->fn(__VA_ARGS__);			\
+			__val = true;					\
+		} else {						\
+			__val = false;					\
+		}							\
 	} else {							\
 		__val = false;						\
 	}								\
-- 
2.11.0

[PATCH 10/17] arm64: Add CNTVCT_EL0 trap handler

[Marc Zyngier]

Since people seem to make a point in breaking the userspace visible
counter, we have no choice but to trap the access. Add the required
handler.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/esr.h |  2 ++
 arch/arm64/kernel/traps.c    | 14 ++++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index d14c478976d0..ad42e79a5d4d 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -175,6 +175,8 @@
 #define ESR_ELx_SYS64_ISS_SYS_CTR_READ	(ESR_ELx_SYS64_ISS_SYS_CTR | \
 					 ESR_ELx_SYS64_ISS_DIR_READ)
 
+#define ESR_ELx_SYS64_ISS_SYS_CNTVCT	(ESR_ELx_SYS64_ISS_SYS_VAL(3, 3, 2, 14, 0) | \
+					 ESR_ELx_SYS64_ISS_DIR_READ)
 #ifndef __ASSEMBLY__
 #include <asm/types.h>
 
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index e52be6aa44ee..1de444e6c669 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -505,6 +505,14 @@ static void ctr_read_handler(unsigned int esr, struct pt_regs *regs)
 	regs->pc += 4;
 }
 
+static void cntvct_read_handler(unsigned int esr, struct pt_regs *regs)
+{
+	int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
+
+	pt_regs_write_reg(regs, rt, arch_counter_get_cntvct());
+	regs->pc += 4;
+}
+
 struct sys64_hook {
 	unsigned int esr_mask;
 	unsigned int esr_val;
@@ -523,6 +531,12 @@ static struct sys64_hook sys64_hooks[] = {
 		.esr_val = ESR_ELx_SYS64_ISS_SYS_CTR_READ,
 		.handler = ctr_read_handler,
 	},
+	{
+		/* Trap read access to CNTVCT_EL0 */
+		.esr_mask = ESR_ELx_SYS64_ISS_SYS_OP_MASK,
+		.esr_val = ESR_ELx_SYS64_ISS_SYS_CNTVCT,
+		.handler = cntvct_read_handler,
+	},
 	{},
 };
 
-- 
2.11.0

[PATCH 10/17] arm64: Add CNTVCT_EL0 trap handler

[Marc Zyngier]

Since people seem to make a point in breaking the userspace visible
counter, we have no choice but to trap the access. Add the required
handler.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/esr.h |  2 ++
 arch/arm64/kernel/traps.c    | 14 ++++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index d14c478976d0..ad42e79a5d4d 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -175,6 +175,8 @@
 #define ESR_ELx_SYS64_ISS_SYS_CTR_READ	(ESR_ELx_SYS64_ISS_SYS_CTR | \
 					 ESR_ELx_SYS64_ISS_DIR_READ)
 
+#define ESR_ELx_SYS64_ISS_SYS_CNTVCT	(ESR_ELx_SYS64_ISS_SYS_VAL(3, 3, 2, 14, 0) | \
+					 ESR_ELx_SYS64_ISS_DIR_READ)
 #ifndef __ASSEMBLY__
 #include <asm/types.h>
 
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index e52be6aa44ee..1de444e6c669 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -505,6 +505,14 @@ static void ctr_read_handler(unsigned int esr, struct pt_regs *regs)
 	regs->pc += 4;
 }
 
+static void cntvct_read_handler(unsigned int esr, struct pt_regs *regs)
+{
+	int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
+
+	pt_regs_write_reg(regs, rt, arch_counter_get_cntvct());
+	regs->pc += 4;
+}
+
 struct sys64_hook {
 	unsigned int esr_mask;
 	unsigned int esr_val;
@@ -523,6 +531,12 @@ static struct sys64_hook sys64_hooks[] = {
 		.esr_val = ESR_ELx_SYS64_ISS_SYS_CTR_READ,
 		.handler = ctr_read_handler,
 	},
+	{
+		/* Trap read access to CNTVCT_EL0 */
+		.esr_mask = ESR_ELx_SYS64_ISS_SYS_OP_MASK,
+		.esr_val = ESR_ELx_SYS64_ISS_SYS_CNTVCT,
+		.handler = cntvct_read_handler,
+	},
 	{},
 };
 
-- 
2.11.0

[PATCH 11/17] arm64: arch_timer: Move clocksource_counter and co around

[Marc Zyngier]

In order to access clocksource_counter from the errata handling code,
move it (together with the related structures and functions) towards
the top of the file.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 62 ++++++++++++++++++------------------
 1 file changed, 31 insertions(+), 31 deletions(-)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 2bbe0b9c3c24..a5a88db2ac90 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -158,6 +158,37 @@ u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
 	return val;
 }
 
+/*
+ * Default to cp15 based access because arm64 uses this function for
+ * sched_clock() before DT is probed and the cp15 method is guaranteed
+ * to exist on arm64. arm doesn't use this before DT is probed so even
+ * if we don't have the cp15 accessors we won't have a problem.
+ */
+u64 (*arch_timer_read_counter)(void) = arch_counter_get_cntvct;
+
+static u64 arch_counter_read(struct clocksource *cs)
+{
+	return arch_timer_read_counter();
+}
+
+static u64 arch_counter_read_cc(const struct cyclecounter *cc)
+{
+	return arch_timer_read_counter();
+}
+
+static struct clocksource clocksource_counter = {
+	.name	= "arch_sys_counter",
+	.rating	= 400,
+	.read	= arch_counter_read,
+	.mask	= CLOCKSOURCE_MASK(56),
+	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
+};
+
+static struct cyclecounter cyclecounter __ro_after_init = {
+	.read	= arch_counter_read_cc,
+	.mask	= CLOCKSOURCE_MASK(56),
+};
+
 #ifdef CONFIG_FSL_ERRATUM_A008585
 /*
  * The number of retries is an arbitrary value well beyond the highest number
@@ -747,37 +778,6 @@ static u64 arch_counter_get_cntvct_mem(void)
 	return ((u64) vct_hi << 32) | vct_lo;
 }
 
-/*
- * Default to cp15 based access because arm64 uses this function for
- * sched_clock() before DT is probed and the cp15 method is guaranteed
- * to exist on arm64. arm doesn't use this before DT is probed so even
- * if we don't have the cp15 accessors we won't have a problem.
- */
-u64 (*arch_timer_read_counter)(void) = arch_counter_get_cntvct;
-
-static u64 arch_counter_read(struct clocksource *cs)
-{
-	return arch_timer_read_counter();
-}
-
-static u64 arch_counter_read_cc(const struct cyclecounter *cc)
-{
-	return arch_timer_read_counter();
-}
-
-static struct clocksource clocksource_counter = {
-	.name	= "arch_sys_counter",
-	.rating	= 400,
-	.read	= arch_counter_read,
-	.mask	= CLOCKSOURCE_MASK(56),
-	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
-};
-
-static struct cyclecounter cyclecounter __ro_after_init = {
-	.read	= arch_counter_read_cc,
-	.mask	= CLOCKSOURCE_MASK(56),
-};
-
 static struct arch_timer_kvm_info arch_timer_kvm_info;
 
 struct arch_timer_kvm_info *arch_timer_get_kvm_info(void)
-- 
2.11.0

[PATCH 11/17] arm64: arch_timer: Move clocksource_counter and co around

[Marc Zyngier]

In order to access clocksource_counter from the errata handling code,
move it (together with the related structures and functions) towards
the top of the file.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 62 ++++++++++++++++++------------------
 1 file changed, 31 insertions(+), 31 deletions(-)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 2bbe0b9c3c24..a5a88db2ac90 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -158,6 +158,37 @@ u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
 	return val;
 }
 
+/*
+ * Default to cp15 based access because arm64 uses this function for
+ * sched_clock() before DT is probed and the cp15 method is guaranteed
+ * to exist on arm64. arm doesn't use this before DT is probed so even
+ * if we don't have the cp15 accessors we won't have a problem.
+ */
+u64 (*arch_timer_read_counter)(void) = arch_counter_get_cntvct;
+
+static u64 arch_counter_read(struct clocksource *cs)
+{
+	return arch_timer_read_counter();
+}
+
+static u64 arch_counter_read_cc(const struct cyclecounter *cc)
+{
+	return arch_timer_read_counter();
+}
+
+static struct clocksource clocksource_counter = {
+	.name	= "arch_sys_counter",
+	.rating	= 400,
+	.read	= arch_counter_read,
+	.mask	= CLOCKSOURCE_MASK(56),
+	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
+};
+
+static struct cyclecounter cyclecounter __ro_after_init = {
+	.read	= arch_counter_read_cc,
+	.mask	= CLOCKSOURCE_MASK(56),
+};
+
 #ifdef CONFIG_FSL_ERRATUM_A008585
 /*
  * The number of retries is an arbitrary value well beyond the highest number
@@ -747,37 +778,6 @@ static u64 arch_counter_get_cntvct_mem(void)
 	return ((u64) vct_hi << 32) | vct_lo;
 }
 
-/*
- * Default to cp15 based access because arm64 uses this function for
- * sched_clock() before DT is probed and the cp15 method is guaranteed
- * to exist on arm64. arm doesn't use this before DT is probed so even
- * if we don't have the cp15 accessors we won't have a problem.
- */
-u64 (*arch_timer_read_counter)(void) = arch_counter_get_cntvct;
-
-static u64 arch_counter_read(struct clocksource *cs)
-{
-	return arch_timer_read_counter();
-}
-
-static u64 arch_counter_read_cc(const struct cyclecounter *cc)
-{
-	return arch_timer_read_counter();
-}
-
-static struct clocksource clocksource_counter = {
-	.name	= "arch_sys_counter",
-	.rating	= 400,
-	.read	= arch_counter_read,
-	.mask	= CLOCKSOURCE_MASK(56),
-	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
-};
-
-static struct cyclecounter cyclecounter __ro_after_init = {
-	.read	= arch_counter_read_cc,
-	.mask	= CLOCKSOURCE_MASK(56),
-};
-
 static struct arch_timer_kvm_info arch_timer_kvm_info;
 
 struct arch_timer_kvm_info *arch_timer_get_kvm_info(void)
-- 
2.11.0

[PATCH 12/17] arm64: arch_timer: Enable CNTVCT_EL0 trap if workaround is enabled

[Marc Zyngier]

Userspace being allowed to use read CNTVCT_EL0 anytime (and not
only in the VDSO), we need to enable trapping whenever a cntvct
workaround is enabled on a given CPU.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 42 +++++++++++++++++++++++++-----------
 1 file changed, 30 insertions(+), 12 deletions(-)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index a5a88db2ac90..8e3638397347 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -83,6 +83,7 @@ static enum ppi_nr arch_timer_uses_ppi = VIRT_PPI;
 static bool arch_timer_c3stop;
 static bool arch_timer_mem_use_virtual;
 static bool arch_counter_suspend_stop;
+static bool vdso_default = true;
 
 static bool evtstrm_enable = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM);
 
@@ -390,6 +391,17 @@ void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa
 	}
 
 	static_branch_enable(&arch_timer_read_ool_enabled);
+
+	/*
+	 * Don't use the vdso fastpath if errata require using the
+	 * out-of-line counter accessor. We may change our mind pretty
+	 * late in the game (with a per-CPU erratum, for example), so
+	 * change both the default value and the vdso itself.
+	 */
+	if (wa->read_cntvct_el0) {
+		clocksource_counter.archdata.vdso_direct = false;
+		vdso_default = false;
+	}
 }
 
 static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type type,
@@ -450,9 +462,17 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 	__val;								\
 })
 
+static bool arch_timer_this_cpu_has_cntvct_wa(void)
+{
+	const struct arch_timer_erratum_workaround *wa;
+
+	wa = __this_cpu_read(timer_unstable_counter_workaround);
+	return wa && wa->read_cntvct_el0;
+}
 #else
 #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
 #define erratum_handler(fn, r, ...)			({false;})
+#define arch_timer_this_cpu_has_cntvct_wa()		({false;})
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
 static __always_inline irqreturn_t timer_handler(const int access,
@@ -672,8 +692,15 @@ static void arch_counter_set_user_access(void)
 			| ARCH_TIMER_VIRT_EVT_EN
 			| ARCH_TIMER_USR_PCT_ACCESS_EN);
 
-	/* Enable user access to the virtual counter */
-	cntkctl |= ARCH_TIMER_USR_VCT_ACCESS_EN;
+	/*
+	 * Enable user access to the virtual counter if it doesn't
+	 * need to be workaround. The vdso may have been already
+	 * disabled though.
+	 */
+	if (arch_timer_this_cpu_has_cntvct_wa())
+		pr_info("CPU%d: Trapping CNTVCT access\n", smp_processor_id());
+	else
+		cntkctl |= ARCH_TIMER_USR_VCT_ACCESS_EN;
 
 	arch_timer_set_cntkctl(cntkctl);
 }
@@ -796,16 +823,7 @@ static void __init arch_counter_register(unsigned type)
 		else
 			arch_timer_read_counter = arch_counter_get_cntpct;
 
-		clocksource_counter.archdata.vdso_direct = true;
-
-#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-		/*
-		 * Don't use the vdso fastpath if errata require using
-		 * the out-of-line counter accessor.
-		 */
-		if (static_branch_unlikely(&arch_timer_read_ool_enabled))
-			clocksource_counter.archdata.vdso_direct = false;
-#endif
+		clocksource_counter.archdata.vdso_direct = vdso_default;
 	} else {
 		arch_timer_read_counter = arch_counter_get_cntvct_mem;
 	}
-- 
2.11.0

[PATCH 12/17] arm64: arch_timer: Enable CNTVCT_EL0 trap if workaround is enabled

[Marc Zyngier]

Userspace being allowed to use read CNTVCT_EL0 anytime (and not
only in the VDSO), we need to enable trapping whenever a cntvct
workaround is enabled on a given CPU.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 42 +++++++++++++++++++++++++-----------
 1 file changed, 30 insertions(+), 12 deletions(-)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index a5a88db2ac90..8e3638397347 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -83,6 +83,7 @@ static enum ppi_nr arch_timer_uses_ppi = VIRT_PPI;
 static bool arch_timer_c3stop;
 static bool arch_timer_mem_use_virtual;
 static bool arch_counter_suspend_stop;
+static bool vdso_default = true;
 
 static bool evtstrm_enable = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM);
 
@@ -390,6 +391,17 @@ void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa
 	}
 
 	static_branch_enable(&arch_timer_read_ool_enabled);
+
+	/*
+	 * Don't use the vdso fastpath if errata require using the
+	 * out-of-line counter accessor. We may change our mind pretty
+	 * late in the game (with a per-CPU erratum, for example), so
+	 * change both the default value and the vdso itself.
+	 */
+	if (wa->read_cntvct_el0) {
+		clocksource_counter.archdata.vdso_direct = false;
+		vdso_default = false;
+	}
 }
 
 static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type type,
@@ -450,9 +462,17 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 	__val;								\
 })
 
+static bool arch_timer_this_cpu_has_cntvct_wa(void)
+{
+	const struct arch_timer_erratum_workaround *wa;
+
+	wa = __this_cpu_read(timer_unstable_counter_workaround);
+	return wa && wa->read_cntvct_el0;
+}
 #else
 #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
 #define erratum_handler(fn, r, ...)			({false;})
+#define arch_timer_this_cpu_has_cntvct_wa()		({false;})
 #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
 
 static __always_inline irqreturn_t timer_handler(const int access,
@@ -672,8 +692,15 @@ static void arch_counter_set_user_access(void)
 			| ARCH_TIMER_VIRT_EVT_EN
 			| ARCH_TIMER_USR_PCT_ACCESS_EN);
 
-	/* Enable user access to the virtual counter */
-	cntkctl |= ARCH_TIMER_USR_VCT_ACCESS_EN;
+	/*
+	 * Enable user access to the virtual counter if it doesn't
+	 * need to be workaround. The vdso may have been already
+	 * disabled though.
+	 */
+	if (arch_timer_this_cpu_has_cntvct_wa())
+		pr_info("CPU%d: Trapping CNTVCT access\n", smp_processor_id());
+	else
+		cntkctl |= ARCH_TIMER_USR_VCT_ACCESS_EN;
 
 	arch_timer_set_cntkctl(cntkctl);
 }
@@ -796,16 +823,7 @@ static void __init arch_counter_register(unsigned type)
 		else
 			arch_timer_read_counter = arch_counter_get_cntpct;
 
-		clocksource_counter.archdata.vdso_direct = true;
-
-#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
-		/*
-		 * Don't use the vdso fastpath if errata require using
-		 * the out-of-line counter accessor.
-		 */
-		if (static_branch_unlikely(&arch_timer_read_ool_enabled))
-			clocksource_counter.archdata.vdso_direct = false;
-#endif
+		clocksource_counter.archdata.vdso_direct = vdso_default;
 	} else {
 		arch_timer_read_counter = arch_counter_get_cntvct_mem;
 	}
-- 
2.11.0

[PATCH 13/17] arm64: cpu_errata: Allow an erratum to be match for all revisions of a core

[Marc Zyngier]

Some minor erratum may not be fixed in further revisions of a core,
leading to a situation where the workaround needs to be updated each
time an updated core is released.

Introduce a MIDR_ALL_VERSIONS match helper that will work for all
versions of that MIDR, once and for all.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/cpu_errata.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index f6cc67e7626e..2be1d1c05303 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -53,6 +53,13 @@ static int cpu_enable_trap_ctr_access(void *__unused)
 	.midr_range_min = min, \
 	.midr_range_max = max
 
+#define MIDR_ALL_VERSIONS(model) \
+	.def_scope = SCOPE_LOCAL_CPU, \
+	.matches = is_affected_midr_range, \
+	.midr_model = model, \
+	.midr_range_min = 0, \
+	.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
+
 const struct arm64_cpu_capabilities arm64_errata[] = {
 #if	defined(CONFIG_ARM64_ERRATUM_826319) || \
 	defined(CONFIG_ARM64_ERRATUM_827319) || \
-- 
2.11.0

[PATCH 13/17] arm64: cpu_errata: Allow an erratum to be match for all revisions of a core

[Marc Zyngier]

Some minor erratum may not be fixed in further revisions of a core,
leading to a situation where the workaround needs to be updated each
time an updated core is released.

Introduce a MIDR_ALL_VERSIONS match helper that will work for all
versions of that MIDR, once and for all.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/kernel/cpu_errata.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index f6cc67e7626e..2be1d1c05303 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -53,6 +53,13 @@ static int cpu_enable_trap_ctr_access(void *__unused)
 	.midr_range_min = min, \
 	.midr_range_max = max
 
+#define MIDR_ALL_VERSIONS(model) \
+	.def_scope = SCOPE_LOCAL_CPU, \
+	.matches = is_affected_midr_range, \
+	.midr_model = model, \
+	.midr_range_min = 0, \
+	.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
+
 const struct arm64_cpu_capabilities arm64_errata[] = {
 #if	defined(CONFIG_ARM64_ERRATUM_826319) || \
 	defined(CONFIG_ARM64_ERRATUM_827319) || \
-- 
2.11.0

[PATCH 14/17] arm64: Define Cortex-A73 MIDR

[Marc Zyngier]

As we're about to introduce a new workaround that is specific to
Cortex-A73, let's define the coresponding MIDR.

Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/cputype.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index fc502713ab37..0984d1b3a8f2 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -80,6 +80,7 @@
 #define ARM_CPU_PART_FOUNDATION		0xD00
 #define ARM_CPU_PART_CORTEX_A57		0xD07
 #define ARM_CPU_PART_CORTEX_A53		0xD03
+#define ARM_CPU_PART_CORTEX_A73		0xD09
 
 #define APM_CPU_PART_POTENZA		0x000
 
@@ -92,6 +93,7 @@
 
 #define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
 #define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
+#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73)
 #define MIDR_THUNDERX	MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
 #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
 #define MIDR_QCOM_FALKOR_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_FALKOR_V1)
-- 
2.11.0

[PATCH 14/17] arm64: Define Cortex-A73 MIDR

[Marc Zyngier]

As we're about to introduce a new workaround that is specific to
Cortex-A73, let's define the coresponding MIDR.

Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/cputype.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index fc502713ab37..0984d1b3a8f2 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -80,6 +80,7 @@
 #define ARM_CPU_PART_FOUNDATION		0xD00
 #define ARM_CPU_PART_CORTEX_A57		0xD07
 #define ARM_CPU_PART_CORTEX_A53		0xD03
+#define ARM_CPU_PART_CORTEX_A73		0xD09
 
 #define APM_CPU_PART_POTENZA		0x000
 
@@ -92,6 +93,7 @@
 
 #define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53)
 #define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57)
+#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73)
 #define MIDR_THUNDERX	MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
 #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
 #define MIDR_QCOM_FALKOR_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_FALKOR_V1)
-- 
2.11.0

[PATCH 15/17] arm64: arch_timer: Workaround for Cortex-A73 erratum 858921

[Marc Zyngier]

Cortex-A73 (all versions) counter read can return a wrong value
when the counter crosses a 32bit boundary.

The workaround involves performing the read twice, and to return
one or the other depending on whether a transition has taken place.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/arm64/silicon-errata.txt |  1 +
 arch/arm64/include/asm/cpucaps.h       |  3 ++-
 arch/arm64/kernel/cpu_errata.c         |  8 ++++++++
 drivers/clocksource/Kconfig            | 11 +++++++++++
 drivers/clocksource/arm_arch_timer.c   | 19 +++++++++++++++++++
 5 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt
index a71b8095dbd8..e0b51bdef445 100644
--- a/Documentation/arm64/silicon-errata.txt
+++ b/Documentation/arm64/silicon-errata.txt
@@ -54,6 +54,7 @@ stable kernels.
 | ARM            | Cortex-A57      | #852523         | N/A                         |
 | ARM            | Cortex-A57      | #834220         | ARM64_ERRATUM_834220        |
 | ARM            | Cortex-A72      | #853709         | N/A                         |
+| ARM            | Cortex-A73      | #858921         | ARM64_ERRATUM_858921        |
 | ARM            | MMU-500         | #841119,#826419 | N/A                         |
 |                |                 |                 |                             |
 | Cavium         | ThunderX ITS    | #22375, #24313  | CAVIUM_ERRATUM_22375        |
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index fb78a5d3b60b..b3aab8a17868 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -37,7 +37,8 @@
 #define ARM64_HAS_NO_FPSIMD			16
 #define ARM64_WORKAROUND_REPEAT_TLBI		17
 #define ARM64_WORKAROUND_QCOM_FALKOR_E1003	18
+#define ARM64_WORKAROUND_858921			19
 
-#define ARM64_NCAPS				19
+#define ARM64_NCAPS				20
 
 #endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 2be1d1c05303..2ed2a7657711 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -158,6 +158,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 			   MIDR_CPU_VAR_REV(0, 0)),
 	},
 #endif
+#ifdef CONFIG_ARM64_ERRATUM_858921
+	{
+	/* Cortex-A73 all versions */
+		.desc = "ARM erratum 858921",
+		.capability = ARM64_WORKAROUND_858921,
+		MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+	},
+#endif
 	{
 	}
 };
diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig
index 3356ab821624..af80a7c44faf 100644
--- a/drivers/clocksource/Kconfig
+++ b/drivers/clocksource/Kconfig
@@ -366,6 +366,17 @@ config HISILICON_ERRATUM_161010101
 	  161010101. The workaround will be active if the hisilicon,erratum-161010101
 	  property is found in the timer node.
 
+config ARM64_ERRATUM_858921
+	bool "Workaround for Cortex-A73 erratum 858921"
+	default y
+	select ARM_ARCH_TIMER_OOL_WORKAROUND
+	depends on ARM_ARCH_TIMER && ARM64
+	help
+	  This option enables a workaround applicable to Cortex-A73
+	  (all versions), whose counter may return incorrect values.
+	  The workaround will be dynamically enabled when an affected
+	  core is detected.
+
 config ARM_GLOBAL_TIMER
 	bool "Support for the ARM global timer" if COMPILE_TEST
 	select CLKSRC_OF if OF
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 8e3638397347..4eb1109da330 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -266,6 +266,17 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
 }
 #endif
 
+#ifdef CONFIG_ARM64_ERRATUM_858921
+static u64 notrace arm64_858921_read_cntvct_el0(void)
+{
+	u64 _old, _new;
+
+	_old = read_sysreg(cntvct_el0);
+	_new = read_sysreg(cntvct_el0);
+	return (((_old ^ _new) >> 32) & 1) ? _old : _new;
+}
+#endif
+
 #ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
 DEFINE_PER_CPU(const struct arch_timer_erratum_workaround *,
 	       timer_unstable_counter_workaround);
@@ -331,6 +342,14 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 		.set_next_event_virt = erratum_set_next_event_tval_virt,
 	},
 #endif
+#ifdef CONFIG_ARM64_ERRATUM_858921
+	{
+		.match_type = ate_match_local_cap_id,
+		.id = (void *)ARM64_WORKAROUND_858921,
+		.desc_str = "ARM erratum 858921",
+		.read_cntvct_el0 = arm64_858921_read_cntvct_el0,
+	},
+#endif
 };
 
 typedef bool (*ate_match_fn_t)(const struct arch_timer_erratum_workaround *,
-- 
2.11.0

[PATCH 15/17] arm64: arch_timer: Workaround for Cortex-A73 erratum 858921

[Marc Zyngier]

Cortex-A73 (all versions) counter read can return a wrong value
when the counter crosses a 32bit boundary.

The workaround involves performing the read twice, and to return
one or the other depending on whether a transition has taken place.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/arm64/silicon-errata.txt |  1 +
 arch/arm64/include/asm/cpucaps.h       |  3 ++-
 arch/arm64/kernel/cpu_errata.c         |  8 ++++++++
 drivers/clocksource/Kconfig            | 11 +++++++++++
 drivers/clocksource/arm_arch_timer.c   | 19 +++++++++++++++++++
 5 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt
index a71b8095dbd8..e0b51bdef445 100644
--- a/Documentation/arm64/silicon-errata.txt
+++ b/Documentation/arm64/silicon-errata.txt
@@ -54,6 +54,7 @@ stable kernels.
 | ARM            | Cortex-A57      | #852523         | N/A                         |
 | ARM            | Cortex-A57      | #834220         | ARM64_ERRATUM_834220        |
 | ARM            | Cortex-A72      | #853709         | N/A                         |
+| ARM            | Cortex-A73      | #858921         | ARM64_ERRATUM_858921        |
 | ARM            | MMU-500         | #841119,#826419 | N/A                         |
 |                |                 |                 |                             |
 | Cavium         | ThunderX ITS    | #22375, #24313  | CAVIUM_ERRATUM_22375        |
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index fb78a5d3b60b..b3aab8a17868 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -37,7 +37,8 @@
 #define ARM64_HAS_NO_FPSIMD			16
 #define ARM64_WORKAROUND_REPEAT_TLBI		17
 #define ARM64_WORKAROUND_QCOM_FALKOR_E1003	18
+#define ARM64_WORKAROUND_858921			19
 
-#define ARM64_NCAPS				19
+#define ARM64_NCAPS				20
 
 #endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 2be1d1c05303..2ed2a7657711 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -158,6 +158,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
 			   MIDR_CPU_VAR_REV(0, 0)),
 	},
 #endif
+#ifdef CONFIG_ARM64_ERRATUM_858921
+	{
+	/* Cortex-A73 all versions */
+		.desc = "ARM erratum 858921",
+		.capability = ARM64_WORKAROUND_858921,
+		MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+	},
+#endif
 	{
 	}
 };
diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig
index 3356ab821624..af80a7c44faf 100644
--- a/drivers/clocksource/Kconfig
+++ b/drivers/clocksource/Kconfig
@@ -366,6 +366,17 @@ config HISILICON_ERRATUM_161010101
 	  161010101. The workaround will be active if the hisilicon,erratum-161010101
 	  property is found in the timer node.
 
+config ARM64_ERRATUM_858921
+	bool "Workaround for Cortex-A73 erratum 858921"
+	default y
+	select ARM_ARCH_TIMER_OOL_WORKAROUND
+	depends on ARM_ARCH_TIMER && ARM64
+	help
+	  This option enables a workaround applicable to Cortex-A73
+	  (all versions), whose counter may return incorrect values.
+	  The workaround will be dynamically enabled when an affected
+	  core is detected.
+
 config ARM_GLOBAL_TIMER
 	bool "Support for the ARM global timer" if COMPILE_TEST
 	select CLKSRC_OF if OF
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 8e3638397347..4eb1109da330 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -266,6 +266,17 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
 }
 #endif
 
+#ifdef CONFIG_ARM64_ERRATUM_858921
+static u64 notrace arm64_858921_read_cntvct_el0(void)
+{
+	u64 _old, _new;
+
+	_old = read_sysreg(cntvct_el0);
+	_new = read_sysreg(cntvct_el0);
+	return (((_old ^ _new) >> 32) & 1) ? _old : _new;
+}
+#endif
+
 #ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
 DEFINE_PER_CPU(const struct arch_timer_erratum_workaround *,
 	       timer_unstable_counter_workaround);
@@ -331,6 +342,14 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 		.set_next_event_virt = erratum_set_next_event_tval_virt,
 	},
 #endif
+#ifdef CONFIG_ARM64_ERRATUM_858921
+	{
+		.match_type = ate_match_local_cap_id,
+		.id = (void *)ARM64_WORKAROUND_858921,
+		.desc_str = "ARM erratum 858921",
+		.read_cntvct_el0 = arm64_858921_read_cntvct_el0,
+	},
+#endif
 };
 
 typedef bool (*ate_match_fn_t)(const struct arch_timer_erratum_workaround *,
-- 
2.11.0

[PATCH 16/17] arm64: arch_timer: Allow erratum matching with ACPI OEM information

[Marc Zyngier]

Just as we're able to identify a broken platform using some DT
information, let's enable a way to spot the offenders with ACPI.

The difference is that we can only match on some OEM info instead
of implementation-specific properties. So in order to avoid the
insane multiplication of errata structures, we allow an array
of OEM descriptions to be attached to an erratum structure.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  1 +
 drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 159513479f6e..2e635deba776 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -42,6 +42,7 @@ enum arch_timer_erratum_match_type {
 	ate_match_dt,
 	ate_match_global_cap_id,
 	ate_match_local_cap_id,
+	ate_match_acpi_oem_info,
 };
 
 struct clock_event_device;
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 4eb1109da330..6182871af4eb 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -190,6 +190,12 @@ static struct cyclecounter cyclecounter __ro_after_init = {
 	.mask	= CLOCKSOURCE_MASK(56),
 };
 
+struct ate_acpi_oem_info {
+	char oem_id[ACPI_OEM_ID_SIZE + 1];
+	char oem_table_id[ACPI_OEM_TABLE_ID_SIZE + 1];
+	u32 oem_revision;
+};
+
 #ifdef CONFIG_FSL_ERRATUM_A008585
 /*
  * The number of retries is an arbitrary value well beyond the highest number
@@ -378,6 +384,28 @@ bool arch_timer_check_local_cap_erratum(const struct arch_timer_erratum_workarou
 	return this_cpu_has_cap((uintptr_t)wa->id);
 }
 
+
+static
+bool arch_timer_check_acpi_oem_erratum(const struct arch_timer_erratum_workaround *wa,
+				       const void *arg)
+{
+	static const struct ate_acpi_oem_info empty_oem_info = {};
+	const struct ate_acpi_oem_info *info = wa->id;
+	const struct acpi_table_header *table = arg;
+
+	/* Iterate over the ACPI EOM info array, looking for a match */
+	while (memcmp(info, &empty_oem_info, sizeof(*info))) {
+		if (!memcmp(info->oem_id, table->oem_id, ACPI_OEM_ID_SIZE) &&
+		    !memcmp(info->oem_table_id, table->oem_table_id, ACPI_OEM_TABLE_ID_SIZE) &&
+		    info->oem_revision == table->oem_revision)
+			return true;
+
+		info++;
+	}
+
+	return false;
+}
+
 static const struct arch_timer_erratum_workaround *
 arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
 			  ate_match_fn_t match_fn,
@@ -441,6 +469,9 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 		match_fn = arch_timer_check_local_cap_erratum;
 		local = true;
 		break;
+	case ate_match_acpi_oem_info:
+		match_fn = arch_timer_check_acpi_oem_erratum;
+		break;
 	}
 
 	wa = arch_timer_iterate_errata(type, match_fn, arg);
@@ -1284,6 +1315,7 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
 
 	/* Check for globally applicable workarounds */
 	arch_timer_check_ool_workaround(ate_match_global_cap_id, NULL);
+	arch_timer_check_ool_workaround(ate_match_acpi_oem_info, table);
 
 	arch_timer_init();
 	return 0;
-- 
2.11.0

[PATCH 16/17] arm64: arch_timer: Allow erratum matching with ACPI OEM information

[Marc Zyngier]

Just as we're able to identify a broken platform using some DT
information, let's enable a way to spot the offenders with ACPI.

The difference is that we can only match on some OEM info instead
of implementation-specific properties. So in order to avoid the
insane multiplication of errata structures, we allow an array
of OEM descriptions to be attached to an erratum structure.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm64/include/asm/arch_timer.h  |  1 +
 drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
index 159513479f6e..2e635deba776 100644
--- a/arch/arm64/include/asm/arch_timer.h
+++ b/arch/arm64/include/asm/arch_timer.h
@@ -42,6 +42,7 @@ enum arch_timer_erratum_match_type {
 	ate_match_dt,
 	ate_match_global_cap_id,
 	ate_match_local_cap_id,
+	ate_match_acpi_oem_info,
 };
 
 struct clock_event_device;
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 4eb1109da330..6182871af4eb 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -190,6 +190,12 @@ static struct cyclecounter cyclecounter __ro_after_init = {
 	.mask	= CLOCKSOURCE_MASK(56),
 };
 
+struct ate_acpi_oem_info {
+	char oem_id[ACPI_OEM_ID_SIZE + 1];
+	char oem_table_id[ACPI_OEM_TABLE_ID_SIZE + 1];
+	u32 oem_revision;
+};
+
 #ifdef CONFIG_FSL_ERRATUM_A008585
 /*
  * The number of retries is an arbitrary value well beyond the highest number
@@ -378,6 +384,28 @@ bool arch_timer_check_local_cap_erratum(const struct arch_timer_erratum_workarou
 	return this_cpu_has_cap((uintptr_t)wa->id);
 }
 
+
+static
+bool arch_timer_check_acpi_oem_erratum(const struct arch_timer_erratum_workaround *wa,
+				       const void *arg)
+{
+	static const struct ate_acpi_oem_info empty_oem_info = {};
+	const struct ate_acpi_oem_info *info = wa->id;
+	const struct acpi_table_header *table = arg;
+
+	/* Iterate over the ACPI EOM info array, looking for a match */
+	while (memcmp(info, &empty_oem_info, sizeof(*info))) {
+		if (!memcmp(info->oem_id, table->oem_id, ACPI_OEM_ID_SIZE) &&
+		    !memcmp(info->oem_table_id, table->oem_table_id, ACPI_OEM_TABLE_ID_SIZE) &&
+		    info->oem_revision == table->oem_revision)
+			return true;
+
+		info++;
+	}
+
+	return false;
+}
+
 static const struct arch_timer_erratum_workaround *
 arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
 			  ate_match_fn_t match_fn,
@@ -441,6 +469,9 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
 		match_fn = arch_timer_check_local_cap_erratum;
 		local = true;
 		break;
+	case ate_match_acpi_oem_info:
+		match_fn = arch_timer_check_acpi_oem_erratum;
+		break;
 	}
 
 	wa = arch_timer_iterate_errata(type, match_fn, arg);
@@ -1284,6 +1315,7 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
 
 	/* Check for globally applicable workarounds */
 	arch_timer_check_ool_workaround(ate_match_global_cap_id, NULL);
+	arch_timer_check_ool_workaround(ate_match_acpi_oem_info, table);
 
 	arch_timer_init();
 	return 0;
-- 
2.11.0

[PATCH 17/17] arm64: arch_timer: Add HISILICON_ERRATUM_161010101 ACPI matching data

[Marc Zyngier]

In order to deal with ACPI enabled platforms suffering from the
HISILICON_ERRATUM_161010101, let's add the required OEM data that
allow the workaround to be enabled.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 6182871af4eb..b46584f058b6 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -270,6 +270,25 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
 {
 	return __hisi_161010101_read_reg(cntvct_el0);
 }
+
+static struct ate_acpi_oem_info hisi_161010101_oem_info[] = {
+	{
+		.oem_id		= "HISI  ",
+		.oem_table_id	= "HIP05   ",
+		.oem_revision	= 0,
+	},
+	{
+		.oem_id		= "HISI  ",
+		.oem_table_id	= "HIP06   ",
+		.oem_revision	= 0,
+	},
+	{
+		.oem_id		= "HISI  ",
+		.oem_table_id	= "HIP07   ",
+		.oem_revision	= 0,
+	},
+	{ },
+};
 #endif
 
 #ifdef CONFIG_ARM64_ERRATUM_858921
@@ -347,6 +366,16 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 		.set_next_event_phys = erratum_set_next_event_tval_phys,
 		.set_next_event_virt = erratum_set_next_event_tval_virt,
 	},
+	{
+		.match_type = ate_match_acpi_oem_info,
+		.id = hisi_161010101_oem_info,
+		.desc_str = "HiSilicon erratum 161010101",
+		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
+		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
+		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
+		.set_next_event_phys = erratum_set_next_event_tval_phys,
+		.set_next_event_virt = erratum_set_next_event_tval_virt,
+	},
 #endif
 #ifdef CONFIG_ARM64_ERRATUM_858921
 	{
-- 
2.11.0

[PATCH 17/17] arm64: arch_timer: Add HISILICON_ERRATUM_161010101 ACPI matching data

[Marc Zyngier]

In order to deal with ACPI enabled platforms suffering from the
HISILICON_ERRATUM_161010101, let's add the required OEM data that
allow the workaround to be enabled.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/clocksource/arm_arch_timer.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 6182871af4eb..b46584f058b6 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -270,6 +270,25 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
 {
 	return __hisi_161010101_read_reg(cntvct_el0);
 }
+
+static struct ate_acpi_oem_info hisi_161010101_oem_info[] = {
+	{
+		.oem_id		= "HISI  ",
+		.oem_table_id	= "HIP05   ",
+		.oem_revision	= 0,
+	},
+	{
+		.oem_id		= "HISI  ",
+		.oem_table_id	= "HIP06   ",
+		.oem_revision	= 0,
+	},
+	{
+		.oem_id		= "HISI  ",
+		.oem_table_id	= "HIP07   ",
+		.oem_revision	= 0,
+	},
+	{ },
+};
 #endif
 
 #ifdef CONFIG_ARM64_ERRATUM_858921
@@ -347,6 +366,16 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
 		.set_next_event_phys = erratum_set_next_event_tval_phys,
 		.set_next_event_virt = erratum_set_next_event_tval_virt,
 	},
+	{
+		.match_type = ate_match_acpi_oem_info,
+		.id = hisi_161010101_oem_info,
+		.desc_str = "HiSilicon erratum 161010101",
+		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
+		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
+		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
+		.set_next_event_phys = erratum_set_next_event_tval_phys,
+		.set_next_event_virt = erratum_set_next_event_tval_virt,
+	},
 #endif
 #ifdef CONFIG_ARM64_ERRATUM_858921
 	{
-- 
2.11.0

Re: [PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[dann frazier]

On Mon, Mar 6, 2017 at 4:26 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> It has recently become obvious that a number of arm64 systems have
> been blessed with a set of timers that are slightly less than perfect,
> and require a bit of hand-holding. We already have a bunch of
> errata-specific code to deal with this, but as we're adding more
> potential detection methods (DT, ACPI, capability), things are getting
> a bit out of hands.
>
> Instead of adding more ad-hoc fixes to an already difficult code base,
> let's give ourselves a bit of an infrastructure that can deal with
> this and hide most of the uggliness behind frendly accessors.
>
> The series is structured as such:
>
> - The first half of the series rework the existing workarounds,
>   allowing errata to be matched using a given detection method
>
> - Another patch allows a workaround to affect a subset of the CPUs,
>   and not the whole system
>
> - Another set of patches allow the virtual counter to be trapped when
>   accessed from userspace (something that affects the current set of
>   broken platform, and that is not worked around yet)
>
> - We then work around a Cortex-A73 erratum, whose counter can return a
>   wrong value if read while crossing a 32bit boundary
>
> - Finally, we add some ACPI-specific workarounds for HiSilicon
>   platforms that have the HISILICON_ERRATUM_161010101 defect.
>
> Note that so far, we only deal with arm64. Once the infrastructure is
> agreed upon, we can look at generalizing it (to some extent) to 32bit
> ARM (typical use case would be a 32bit guest running on an affected
> host).

Thanks Marc. Worked on our HiSilicon board:

[    0.000000] arm_arch_timer: Enabling global workaround for
HiSilicon erratum 161010101
[    0.000000] arm_arch_timer: CPU0: Trapping CNTVCT access
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at
50.00MHz (phys).
[    0.266571] arm_arch_timer: CPU1: Trapping CNTVCT access
[    0.270108] arm_arch_timer: CPU2: Trapping CNTVCT access
[...]

Tested-by: dann frazier <dann.frazier@canonical.com>

  -dann

[PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[dann frazier]

On Mon, Mar 6, 2017 at 4:26 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> It has recently become obvious that a number of arm64 systems have
> been blessed with a set of timers that are slightly less than perfect,
> and require a bit of hand-holding. We already have a bunch of
> errata-specific code to deal with this, but as we're adding more
> potential detection methods (DT, ACPI, capability), things are getting
> a bit out of hands.
>
> Instead of adding more ad-hoc fixes to an already difficult code base,
> let's give ourselves a bit of an infrastructure that can deal with
> this and hide most of the uggliness behind frendly accessors.
>
> The series is structured as such:
>
> - The first half of the series rework the existing workarounds,
>   allowing errata to be matched using a given detection method
>
> - Another patch allows a workaround to affect a subset of the CPUs,
>   and not the whole system
>
> - Another set of patches allow the virtual counter to be trapped when
>   accessed from userspace (something that affects the current set of
>   broken platform, and that is not worked around yet)
>
> - We then work around a Cortex-A73 erratum, whose counter can return a
>   wrong value if read while crossing a 32bit boundary
>
> - Finally, we add some ACPI-specific workarounds for HiSilicon
>   platforms that have the HISILICON_ERRATUM_161010101 defect.
>
> Note that so far, we only deal with arm64. Once the infrastructure is
> agreed upon, we can look at generalizing it (to some extent) to 32bit
> ARM (typical use case would be a 32bit guest running on an affected
> host).

Thanks Marc. Worked on our HiSilicon board:

[    0.000000] arm_arch_timer: Enabling global workaround for
HiSilicon erratum 161010101
[    0.000000] arm_arch_timer: CPU0: Trapping CNTVCT access
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at
50.00MHz (phys).
[    0.266571] arm_arch_timer: CPU1: Trapping CNTVCT access
[    0.270108] arm_arch_timer: CPU2: Trapping CNTVCT access
[...]

Tested-by: dann frazier <dann.frazier@canonical.com>

  -dann

Re: [PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[Hanjun Guo]

Hi Marc,

On 2017/3/6 19:26, Marc Zyngier wrote:
> It has recently become obvious that a number of arm64 systems have
> been blessed with a set of timers that are slightly less than perfect,
> and require a bit of hand-holding. We already have a bunch of
> errata-specific code to deal with this, but as we're adding more
> potential detection methods (DT, ACPI, capability), things are getting
> a bit out of hands.
>
> Instead of adding more ad-hoc fixes to an already difficult code base,
> let's give ourselves a bit of an infrastructure that can deal with
> this and hide most of the uggliness behind frendly accessors.
>
> The series is structured as such:
>
> - The first half of the series rework the existing workarounds,
>   allowing errata to be matched using a given detection method
>
> - Another patch allows a workaround to affect a subset of the CPUs,
>   and not the whole system
>
> - Another set of patches allow the virtual counter to be trapped when
>   accessed from userspace (something that affects the current set of
>   broken platform, and that is not worked around yet)
>
> - We then work around a Cortex-A73 erratum, whose counter can return a
>   wrong value if read while crossing a 32bit boundary
>
> - Finally, we add some ACPI-specific workarounds for HiSilicon
>   platforms that have the HISILICON_ERRATUM_161010101 defect.

Thanks for doing this, I tested this patch set on D03, and it boots
OK with log:

[    0.000000] arm_arch_timer: Enabling global workaround for HiSilicon 
erratum 161010101
[    0.000000] arm_arch_timer: CPU0: Trapping CNTVCT access
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 
50.00MHz (phys).

With patches other than Cortex-A73 erratum,

Tested-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

[PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[Hanjun Guo]

Hi Marc,

On 2017/3/6 19:26, Marc Zyngier wrote:
> It has recently become obvious that a number of arm64 systems have
> been blessed with a set of timers that are slightly less than perfect,
> and require a bit of hand-holding. We already have a bunch of
> errata-specific code to deal with this, but as we're adding more
> potential detection methods (DT, ACPI, capability), things are getting
> a bit out of hands.
>
> Instead of adding more ad-hoc fixes to an already difficult code base,
> let's give ourselves a bit of an infrastructure that can deal with
> this and hide most of the uggliness behind frendly accessors.
>
> The series is structured as such:
>
> - The first half of the series rework the existing workarounds,
>   allowing errata to be matched using a given detection method
>
> - Another patch allows a workaround to affect a subset of the CPUs,
>   and not the whole system
>
> - Another set of patches allow the virtual counter to be trapped when
>   accessed from userspace (something that affects the current set of
>   broken platform, and that is not worked around yet)
>
> - We then work around a Cortex-A73 erratum, whose counter can return a
>   wrong value if read while crossing a 32bit boundary
>
> - Finally, we add some ACPI-specific workarounds for HiSilicon
>   platforms that have the HISILICON_ERRATUM_161010101 defect.

Thanks for doing this, I tested this patch set on D03, and it boots
OK with log:

[    0.000000] arm_arch_timer: Enabling global workaround for HiSilicon 
erratum 161010101
[    0.000000] arm_arch_timer: CPU0: Trapping CNTVCT access
[    0.000000] arm_arch_timer: Architected cp15 timer(s) running at 
50.00MHz (phys).

With patches other than Cortex-A73 erratum,

Tested-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

Re: [PATCH 01/17] arm64: arch_timer: Add infrastructure for multiple erratum detection methods

[Hanjun Guo]

On 2017/3/6 19:26, Marc Zyngier wrote:
> We're currently stuck with DT when it comes to handling errata, which
> is pretty restrictive. In order to make things more flexible, let's
> introduce an infrastructure that could support alternative discovery
> methods. No change in functionnality.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/arch_timer.h  |  7 +++-
>  drivers/clocksource/arm_arch_timer.c | 80 +++++++++++++++++++++++++++++++-----
>  2 files changed, 75 insertions(+), 12 deletions(-)
>
> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
> index b4b34004a21e..1c92d52619a6 100644
> --- a/arch/arm64/include/asm/arch_timer.h
> +++ b/arch/arm64/include/asm/arch_timer.h
> @@ -37,9 +37,14 @@ extern struct static_key_false arch_timer_read_ool_enabled;
>  #define needs_unstable_timer_counter_workaround()  false
>  #endif
>
> +enum arch_timer_erratum_match_type {
> +	ate_match_dt,
> +};
>
>  struct arch_timer_erratum_workaround {
> -	const char *id;		/* Indicate the Erratum ID */
> +	enum arch_timer_erratum_match_type match_type;
> +	const void *id;		/* Indicate the Erratum ID */
> +	const char *desc_str;
>  	u32 (*read_cntp_tval_el0)(void);
>  	u32 (*read_cntv_tval_el0)(void);
>  	u64 (*read_cntvct_el0)(void);
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 7a8a4117f123..36980cdd5c0a 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -182,7 +182,9 @@ EXPORT_SYMBOL_GPL(arch_timer_read_ool_enabled);
>  static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  #ifdef CONFIG_FSL_ERRATUM_A008585
>  	{
> +		.match_type = ate_match_dt,
>  		.id = "fsl,erratum-a008585",
> +		.desc_str = "Freescale erratum a005858",
>  		.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,
>  		.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,
>  		.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,
> @@ -190,13 +192,78 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  #endif
>  #ifdef CONFIG_HISILICON_ERRATUM_161010101
>  	{
> +		.match_type = ate_match_dt,
>  		.id = "hisilicon,erratum-161010101",
> +		.desc_str = "HiSilicon erratum 161010101",
>  		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
>  		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
>  		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
>  	},
>  #endif
>  };
> +
> +typedef bool (*ate_match_fn_t)(const struct arch_timer_erratum_workaround *,
> +			       const void *);
> +
> +static
> +bool arch_timer_check_dt_erratum(const struct arch_timer_erratum_workaround *wa,
> +				 const void *arg)
> +{
> +	const struct device_node *np = arg;
> +
> +	return of_property_read_bool(np, wa->id);
> +}
> +
> +static const struct arch_timer_erratum_workaround *
> +arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
> +			  ate_match_fn_t match_fn,
> +			  void *arg)
> +{
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(ool_workarounds); i++) {
> +		if (ool_workarounds[i].match_type != type)
> +			continue;
> +
> +		if (match_fn(&ool_workarounds[i], arg))
> +			return &ool_workarounds[i];
> +	}
> +
> +	return NULL;
> +}
> +
> +static
> +void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa)
> +{
> +	timer_unstable_counter_workaround = wa;
> +	static_branch_enable(&arch_timer_read_ool_enabled);
> +}
> +
> +static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type type,
> +					    void *arg)
> +{
> +	const struct arch_timer_erratum_workaround *wa;
> +	ate_match_fn_t match_fn = NULL;
> +
> +	if (static_branch_unlikely(&arch_timer_read_ool_enabled))
> +		return;
> +
> +	switch (type) {
> +	case ate_match_dt:
> +		match_fn = arch_timer_check_dt_erratum;
> +		break;
> +	}
> +
> +	wa = arch_timer_iterate_errata(type, match_fn, arg);
> +	if (!wa)
> +		return;
> +
> +	arch_timer_enable_workaround(wa);
> +	pr_info("Enabling global workaround for %s\n", wa->desc_str);
> +}
> +
> +#else
> +#define arch_timer_check_ool_workaround(t,a)		do { } while(0)
>  #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
>
>  static __always_inline
> @@ -960,17 +1027,8 @@ static int __init arch_timer_of_init(struct device_node *np)
>
>  	arch_timer_c3stop = !of_property_read_bool(np, "always-on");
>
> -#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
> -	for (i = 0; i < ARRAY_SIZE(ool_workarounds); i++) {
> -		if (of_property_read_bool(np, ool_workarounds[i].id)) {
> -			timer_unstable_counter_workaround = &ool_workarounds[i];
> -			static_branch_enable(&arch_timer_read_ool_enabled);
> -			pr_info("arch_timer: Enabling workaround for %s\n",
> -				timer_unstable_counter_workaround->id);
> -			break;
> -		}
> -	}
> -#endif
> +	/* Check for globally applicable workarounds */
> +	arch_timer_check_ool_workaround(ate_match_dt, np);
>
>  	/*
>  	 * If we cannot rely on firmware initializing the timer registers then
>

It's cool to me,

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

[PATCH 01/17] arm64: arch_timer: Add infrastructure for multiple erratum detection methods

[Hanjun Guo]

On 2017/3/6 19:26, Marc Zyngier wrote:
> We're currently stuck with DT when it comes to handling errata, which
> is pretty restrictive. In order to make things more flexible, let's
> introduce an infrastructure that could support alternative discovery
> methods. No change in functionnality.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/arch_timer.h  |  7 +++-
>  drivers/clocksource/arm_arch_timer.c | 80 +++++++++++++++++++++++++++++++-----
>  2 files changed, 75 insertions(+), 12 deletions(-)
>
> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
> index b4b34004a21e..1c92d52619a6 100644
> --- a/arch/arm64/include/asm/arch_timer.h
> +++ b/arch/arm64/include/asm/arch_timer.h
> @@ -37,9 +37,14 @@ extern struct static_key_false arch_timer_read_ool_enabled;
>  #define needs_unstable_timer_counter_workaround()  false
>  #endif
>
> +enum arch_timer_erratum_match_type {
> +	ate_match_dt,
> +};
>
>  struct arch_timer_erratum_workaround {
> -	const char *id;		/* Indicate the Erratum ID */
> +	enum arch_timer_erratum_match_type match_type;
> +	const void *id;		/* Indicate the Erratum ID */
> +	const char *desc_str;
>  	u32 (*read_cntp_tval_el0)(void);
>  	u32 (*read_cntv_tval_el0)(void);
>  	u64 (*read_cntvct_el0)(void);
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 7a8a4117f123..36980cdd5c0a 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -182,7 +182,9 @@ EXPORT_SYMBOL_GPL(arch_timer_read_ool_enabled);
>  static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  #ifdef CONFIG_FSL_ERRATUM_A008585
>  	{
> +		.match_type = ate_match_dt,
>  		.id = "fsl,erratum-a008585",
> +		.desc_str = "Freescale erratum a005858",
>  		.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,
>  		.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,
>  		.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,
> @@ -190,13 +192,78 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  #endif
>  #ifdef CONFIG_HISILICON_ERRATUM_161010101
>  	{
> +		.match_type = ate_match_dt,
>  		.id = "hisilicon,erratum-161010101",
> +		.desc_str = "HiSilicon erratum 161010101",
>  		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
>  		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
>  		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
>  	},
>  #endif
>  };
> +
> +typedef bool (*ate_match_fn_t)(const struct arch_timer_erratum_workaround *,
> +			       const void *);
> +
> +static
> +bool arch_timer_check_dt_erratum(const struct arch_timer_erratum_workaround *wa,
> +				 const void *arg)
> +{
> +	const struct device_node *np = arg;
> +
> +	return of_property_read_bool(np, wa->id);
> +}
> +
> +static const struct arch_timer_erratum_workaround *
> +arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
> +			  ate_match_fn_t match_fn,
> +			  void *arg)
> +{
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(ool_workarounds); i++) {
> +		if (ool_workarounds[i].match_type != type)
> +			continue;
> +
> +		if (match_fn(&ool_workarounds[i], arg))
> +			return &ool_workarounds[i];
> +	}
> +
> +	return NULL;
> +}
> +
> +static
> +void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa)
> +{
> +	timer_unstable_counter_workaround = wa;
> +	static_branch_enable(&arch_timer_read_ool_enabled);
> +}
> +
> +static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type type,
> +					    void *arg)
> +{
> +	const struct arch_timer_erratum_workaround *wa;
> +	ate_match_fn_t match_fn = NULL;
> +
> +	if (static_branch_unlikely(&arch_timer_read_ool_enabled))
> +		return;
> +
> +	switch (type) {
> +	case ate_match_dt:
> +		match_fn = arch_timer_check_dt_erratum;
> +		break;
> +	}
> +
> +	wa = arch_timer_iterate_errata(type, match_fn, arg);
> +	if (!wa)
> +		return;
> +
> +	arch_timer_enable_workaround(wa);
> +	pr_info("Enabling global workaround for %s\n", wa->desc_str);
> +}
> +
> +#else
> +#define arch_timer_check_ool_workaround(t,a)		do { } while(0)
>  #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
>
>  static __always_inline
> @@ -960,17 +1027,8 @@ static int __init arch_timer_of_init(struct device_node *np)
>
>  	arch_timer_c3stop = !of_property_read_bool(np, "always-on");
>
> -#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND
> -	for (i = 0; i < ARRAY_SIZE(ool_workarounds); i++) {
> -		if (of_property_read_bool(np, ool_workarounds[i].id)) {
> -			timer_unstable_counter_workaround = &ool_workarounds[i];
> -			static_branch_enable(&arch_timer_read_ool_enabled);
> -			pr_info("arch_timer: Enabling workaround for %s\n",
> -				timer_unstable_counter_workaround->id);
> -			break;
> -		}
> -	}
> -#endif
> +	/* Check for globally applicable workarounds */
> +	arch_timer_check_ool_workaround(ate_match_dt, np);
>
>  	/*
>  	 * If we cannot rely on firmware initializing the timer registers then
>

It's cool to me,

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

Re: [PATCH 16/17] arm64: arch_timer: Allow erratum matching with ACPI OEM information

[Hanjun Guo]

On 2017/3/6 19:26, Marc Zyngier wrote:
> Just as we're able to identify a broken platform using some DT
> information, let's enable a way to spot the offenders with ACPI.
>
> The difference is that we can only match on some OEM info instead
> of implementation-specific properties. So in order to avoid the
> insane multiplication of errata structures, we allow an array
> of OEM descriptions to be attached to an erratum structure.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/arch_timer.h  |  1 +
>  drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++++++++
>  2 files changed, 33 insertions(+)
>
> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
> index 159513479f6e..2e635deba776 100644
> --- a/arch/arm64/include/asm/arch_timer.h
> +++ b/arch/arm64/include/asm/arch_timer.h
> @@ -42,6 +42,7 @@ enum arch_timer_erratum_match_type {
>  	ate_match_dt,
>  	ate_match_global_cap_id,
>  	ate_match_local_cap_id,
> +	ate_match_acpi_oem_info,
>  };
>
>  struct clock_event_device;
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 4eb1109da330..6182871af4eb 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -190,6 +190,12 @@ static struct cyclecounter cyclecounter __ro_after_init = {
>  	.mask	= CLOCKSOURCE_MASK(56),
>  };
>
> +struct ate_acpi_oem_info {
> +	char oem_id[ACPI_OEM_ID_SIZE + 1];
> +	char oem_table_id[ACPI_OEM_TABLE_ID_SIZE + 1];
> +	u32 oem_revision;
> +};
> +
>  #ifdef CONFIG_FSL_ERRATUM_A008585
>  /*
>   * The number of retries is an arbitrary value well beyond the highest number
> @@ -378,6 +384,28 @@ bool arch_timer_check_local_cap_erratum(const struct arch_timer_erratum_workarou
>  	return this_cpu_has_cap((uintptr_t)wa->id);
>  }
>
> +
> +static
> +bool arch_timer_check_acpi_oem_erratum(const struct arch_timer_erratum_workaround *wa,
> +				       const void *arg)
> +{
> +	static const struct ate_acpi_oem_info empty_oem_info = {};
> +	const struct ate_acpi_oem_info *info = wa->id;
> +	const struct acpi_table_header *table = arg;
> +
> +	/* Iterate over the ACPI EOM info array, looking for a match */

typo, 'OEM'.

> +	while (memcmp(info, &empty_oem_info, sizeof(*info))) {
> +		if (!memcmp(info->oem_id, table->oem_id, ACPI_OEM_ID_SIZE) &&
> +		    !memcmp(info->oem_table_id, table->oem_table_id, ACPI_OEM_TABLE_ID_SIZE) &&
> +		    info->oem_revision == table->oem_revision)
> +			return true;
> +
> +		info++;
> +	}
> +
> +	return false;
> +}
> +
>  static const struct arch_timer_erratum_workaround *
>  arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
>  			  ate_match_fn_t match_fn,
> @@ -441,6 +469,9 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
>  		match_fn = arch_timer_check_local_cap_erratum;
>  		local = true;
>  		break;
> +	case ate_match_acpi_oem_info:
> +		match_fn = arch_timer_check_acpi_oem_erratum;
> +		break;
>  	}
>
>  	wa = arch_timer_iterate_errata(type, match_fn, arg);
> @@ -1284,6 +1315,7 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
>
>  	/* Check for globally applicable workarounds */
>  	arch_timer_check_ool_workaround(ate_match_global_cap_id, NULL);
> +	arch_timer_check_ool_workaround(ate_match_acpi_oem_info, table);
>
>  	arch_timer_init();
>  	return 0;

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

[PATCH 16/17] arm64: arch_timer: Allow erratum matching with ACPI OEM information

[Hanjun Guo]

On 2017/3/6 19:26, Marc Zyngier wrote:
> Just as we're able to identify a broken platform using some DT
> information, let's enable a way to spot the offenders with ACPI.
>
> The difference is that we can only match on some OEM info instead
> of implementation-specific properties. So in order to avoid the
> insane multiplication of errata structures, we allow an array
> of OEM descriptions to be attached to an erratum structure.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/arch_timer.h  |  1 +
>  drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++++++++
>  2 files changed, 33 insertions(+)
>
> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
> index 159513479f6e..2e635deba776 100644
> --- a/arch/arm64/include/asm/arch_timer.h
> +++ b/arch/arm64/include/asm/arch_timer.h
> @@ -42,6 +42,7 @@ enum arch_timer_erratum_match_type {
>  	ate_match_dt,
>  	ate_match_global_cap_id,
>  	ate_match_local_cap_id,
> +	ate_match_acpi_oem_info,
>  };
>
>  struct clock_event_device;
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 4eb1109da330..6182871af4eb 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -190,6 +190,12 @@ static struct cyclecounter cyclecounter __ro_after_init = {
>  	.mask	= CLOCKSOURCE_MASK(56),
>  };
>
> +struct ate_acpi_oem_info {
> +	char oem_id[ACPI_OEM_ID_SIZE + 1];
> +	char oem_table_id[ACPI_OEM_TABLE_ID_SIZE + 1];
> +	u32 oem_revision;
> +};
> +
>  #ifdef CONFIG_FSL_ERRATUM_A008585
>  /*
>   * The number of retries is an arbitrary value well beyond the highest number
> @@ -378,6 +384,28 @@ bool arch_timer_check_local_cap_erratum(const struct arch_timer_erratum_workarou
>  	return this_cpu_has_cap((uintptr_t)wa->id);
>  }
>
> +
> +static
> +bool arch_timer_check_acpi_oem_erratum(const struct arch_timer_erratum_workaround *wa,
> +				       const void *arg)
> +{
> +	static const struct ate_acpi_oem_info empty_oem_info = {};
> +	const struct ate_acpi_oem_info *info = wa->id;
> +	const struct acpi_table_header *table = arg;
> +
> +	/* Iterate over the ACPI EOM info array, looking for a match */

typo, 'OEM'.

> +	while (memcmp(info, &empty_oem_info, sizeof(*info))) {
> +		if (!memcmp(info->oem_id, table->oem_id, ACPI_OEM_ID_SIZE) &&
> +		    !memcmp(info->oem_table_id, table->oem_table_id, ACPI_OEM_TABLE_ID_SIZE) &&
> +		    info->oem_revision == table->oem_revision)
> +			return true;
> +
> +		info++;
> +	}
> +
> +	return false;
> +}
> +
>  static const struct arch_timer_erratum_workaround *
>  arch_timer_iterate_errata(enum arch_timer_erratum_match_type type,
>  			  ate_match_fn_t match_fn,
> @@ -441,6 +469,9 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
>  		match_fn = arch_timer_check_local_cap_erratum;
>  		local = true;
>  		break;
> +	case ate_match_acpi_oem_info:
> +		match_fn = arch_timer_check_acpi_oem_erratum;
> +		break;
>  	}
>
>  	wa = arch_timer_iterate_errata(type, match_fn, arg);
> @@ -1284,6 +1315,7 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
>
>  	/* Check for globally applicable workarounds */
>  	arch_timer_check_ool_workaround(ate_match_global_cap_id, NULL);
> +	arch_timer_check_ool_workaround(ate_match_acpi_oem_info, table);
>
>  	arch_timer_init();
>  	return 0;

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

Re: [PATCH 17/17] arm64: arch_timer: Add HISILICON_ERRATUM_161010101 ACPI matching data

[Hanjun Guo]

On 2017/3/6 19:26, Marc Zyngier wrote:
> In order to deal with ACPI enabled platforms suffering from the
> HISILICON_ERRATUM_161010101, let's add the required OEM data that
> allow the workaround to be enabled.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/clocksource/arm_arch_timer.c | 29 +++++++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
>
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 6182871af4eb..b46584f058b6 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -270,6 +270,25 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
>  {
>  	return __hisi_161010101_read_reg(cntvct_el0);
>  }
> +
> +static struct ate_acpi_oem_info hisi_161010101_oem_info[] = {
> +	{
> +		.oem_id		= "HISI  ",
> +		.oem_table_id	= "HIP05   ",
> +		.oem_revision	= 0,
> +	},
> +	{
> +		.oem_id		= "HISI  ",
> +		.oem_table_id	= "HIP06   ",
> +		.oem_revision	= 0,
> +	},
> +	{
> +		.oem_id		= "HISI  ",
> +		.oem_table_id	= "HIP07   ",
> +		.oem_revision	= 0,
> +	},
> +	{ },

Maybe it's better to add a comments that to say "It's needed to match
the end of oem info", but it's up to you :)

> +};
>  #endif
>
>  #ifdef CONFIG_ARM64_ERRATUM_858921
> @@ -347,6 +366,16 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  		.set_next_event_phys = erratum_set_next_event_tval_phys,
>  		.set_next_event_virt = erratum_set_next_event_tval_virt,
>  	},
> +	{
> +		.match_type = ate_match_acpi_oem_info,
> +		.id = hisi_161010101_oem_info,
> +		.desc_str = "HiSilicon erratum 161010101",
> +		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
> +		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
> +		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
> +		.set_next_event_phys = erratum_set_next_event_tval_phys,
> +		.set_next_event_virt = erratum_set_next_event_tval_virt,
> +	},
>  #endif
>  #ifdef CONFIG_ARM64_ERRATUM_858921
>  	{
>

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

[PATCH 17/17] arm64: arch_timer: Add HISILICON_ERRATUM_161010101 ACPI matching data

[Hanjun Guo]

On 2017/3/6 19:26, Marc Zyngier wrote:
> In order to deal with ACPI enabled platforms suffering from the
> HISILICON_ERRATUM_161010101, let's add the required OEM data that
> allow the workaround to be enabled.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/clocksource/arm_arch_timer.c | 29 +++++++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
>
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 6182871af4eb..b46584f058b6 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -270,6 +270,25 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
>  {
>  	return __hisi_161010101_read_reg(cntvct_el0);
>  }
> +
> +static struct ate_acpi_oem_info hisi_161010101_oem_info[] = {
> +	{
> +		.oem_id		= "HISI  ",
> +		.oem_table_id	= "HIP05   ",
> +		.oem_revision	= 0,
> +	},
> +	{
> +		.oem_id		= "HISI  ",
> +		.oem_table_id	= "HIP06   ",
> +		.oem_revision	= 0,
> +	},
> +	{
> +		.oem_id		= "HISI  ",
> +		.oem_table_id	= "HIP07   ",
> +		.oem_revision	= 0,
> +	},
> +	{ },

Maybe it's better to add a comments that to say "It's needed to match
the end of oem info", but it's up to you :)

> +};
>  #endif
>
>  #ifdef CONFIG_ARM64_ERRATUM_858921
> @@ -347,6 +366,16 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  		.set_next_event_phys = erratum_set_next_event_tval_phys,
>  		.set_next_event_virt = erratum_set_next_event_tval_virt,
>  	},
> +	{
> +		.match_type = ate_match_acpi_oem_info,
> +		.id = hisi_161010101_oem_info,
> +		.desc_str = "HiSilicon erratum 161010101",
> +		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
> +		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
> +		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
> +		.set_next_event_phys = erratum_set_next_event_tval_phys,
> +		.set_next_event_virt = erratum_set_next_event_tval_virt,
> +	},
>  #endif
>  #ifdef CONFIG_ARM64_ERRATUM_858921
>  	{
>

Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>

Thanks
Hanjun

Re: [PATCH 07/17] arm64: arch_timer: Rework the set_next_event workarounds

[Hanjun Guo]

On 2017/3/6 19:26, Marc Zyngier wrote:
> The way we work around errata affecting set_next_event is not very
> nice, at it imposes this workaround on errata that do not need it.
>
> Add new workaround hooks and let the existing workarounds use them.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/arch_timer.h  |  4 ++++
>  drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++------
>  2 files changed, 30 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
> index f841e08a0dfc..a5ed4cd2e972 100644
> --- a/arch/arm64/include/asm/arch_timer.h
> +++ b/arch/arm64/include/asm/arch_timer.h
> @@ -43,6 +43,8 @@ enum arch_timer_erratum_match_type {
>  	ate_match_local_cap_id,
>  };
>
> +struct clock_event_device;
> +
>  struct arch_timer_erratum_workaround {
>  	enum arch_timer_erratum_match_type match_type;
>  	const void *id;		/* Indicate the Erratum ID */
> @@ -50,6 +52,8 @@ struct arch_timer_erratum_workaround {
>  	u32 (*read_cntp_tval_el0)(void);
>  	u32 (*read_cntv_tval_el0)(void);
>  	u64 (*read_cntvct_el0)(void);
> +	int (*set_next_event_phys)(unsigned long, struct clock_event_device *);
> +	int (*set_next_event_virt)(unsigned long, struct clock_event_device *);
>  };
>
>  extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround;
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 3e5f9539c0c0..ac0ddb380d3a 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -282,6 +282,8 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  		.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,
>  		.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,
>  		.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,
> +		.set_next_event_phys = erratum_set_next_event_tval_phys,
> +		.set_next_event_virt = erratum_set_next_event_tval_virt,
>  	},
>  #endif
>  #ifdef CONFIG_HISILICON_ERRATUM_161010101
> @@ -292,6 +294,8 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
>  		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
>  		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
> +		.set_next_event_phys = erratum_set_next_event_tval_phys,
> +		.set_next_event_virt = erratum_set_next_event_tval_virt,
>  	},
>  #endif
>  };
> @@ -384,10 +388,22 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
>  		local ? "local" : "global", wa->desc_str);
>  }
>
> +#define erratum_handler(fn, r, ...)					\
> +({									\
> +  	bool __val;		

Just noticed spaces before bool.

Thanks
Hanjun

[PATCH 07/17] arm64: arch_timer: Rework the set_next_event workarounds

[Hanjun Guo]

On 2017/3/6 19:26, Marc Zyngier wrote:
> The way we work around errata affecting set_next_event is not very
> nice, at it imposes this workaround on errata that do not need it.
>
> Add new workaround hooks and let the existing workarounds use them.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/arch_timer.h  |  4 ++++
>  drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++------
>  2 files changed, 30 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
> index f841e08a0dfc..a5ed4cd2e972 100644
> --- a/arch/arm64/include/asm/arch_timer.h
> +++ b/arch/arm64/include/asm/arch_timer.h
> @@ -43,6 +43,8 @@ enum arch_timer_erratum_match_type {
>  	ate_match_local_cap_id,
>  };
>
> +struct clock_event_device;
> +
>  struct arch_timer_erratum_workaround {
>  	enum arch_timer_erratum_match_type match_type;
>  	const void *id;		/* Indicate the Erratum ID */
> @@ -50,6 +52,8 @@ struct arch_timer_erratum_workaround {
>  	u32 (*read_cntp_tval_el0)(void);
>  	u32 (*read_cntv_tval_el0)(void);
>  	u64 (*read_cntvct_el0)(void);
> +	int (*set_next_event_phys)(unsigned long, struct clock_event_device *);
> +	int (*set_next_event_virt)(unsigned long, struct clock_event_device *);
>  };
>
>  extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround;
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 3e5f9539c0c0..ac0ddb380d3a 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -282,6 +282,8 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  		.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,
>  		.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,
>  		.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,
> +		.set_next_event_phys = erratum_set_next_event_tval_phys,
> +		.set_next_event_virt = erratum_set_next_event_tval_virt,
>  	},
>  #endif
>  #ifdef CONFIG_HISILICON_ERRATUM_161010101
> @@ -292,6 +294,8 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {
>  		.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,
>  		.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,
>  		.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,
> +		.set_next_event_phys = erratum_set_next_event_tval_phys,
> +		.set_next_event_virt = erratum_set_next_event_tval_virt,
>  	},
>  #endif
>  };
> @@ -384,10 +388,22 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
>  		local ? "local" : "global", wa->desc_str);
>  }
>
> +#define erratum_handler(fn, r, ...)					\
> +({									\
> +  	bool __val;		

Just noticed spaces before bool.

Thanks
Hanjun

Re: [PATCH 01/17] arm64: arch_timer: Add infrastructure for multiple erratum detection methods

[Mark Rutland]

Hi,

On Mon, Mar 06, 2017 at 11:26:06AM +0000, Marc Zyngier wrote:
> We're currently stuck with DT when it comes to handling errata, which
> is pretty restrictive. In order to make things more flexible, let's
> introduce an infrastructure that could support alternative discovery
> methods. No change in functionnality.

Nit: functionality

> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/arch_timer.h  |  7 +++-
>  drivers/clocksource/arm_arch_timer.c | 80 +++++++++++++++++++++++++++++++-----
>  2 files changed, 75 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
> index b4b34004a21e..1c92d52619a6 100644
> --- a/arch/arm64/include/asm/arch_timer.h
> +++ b/arch/arm64/include/asm/arch_timer.h
> @@ -37,9 +37,14 @@ extern struct static_key_false arch_timer_read_ool_enabled;
>  #define needs_unstable_timer_counter_workaround()  false
>  #endif
>  
> +enum arch_timer_erratum_match_type {
> +	ate_match_dt,
> +};
>  
>  struct arch_timer_erratum_workaround {
> -	const char *id;		/* Indicate the Erratum ID */
> +	enum arch_timer_erratum_match_type match_type;
> +	const void *id;		/* Indicate the Erratum ID */

This comment isn't much help. Can we drop it?

Either that, or expand on what it is in each case, e.g.

	/*
	 * Data specific to the match_type.
	 *
	 * For ate_match_dt, this is a DT property name to look for.
	 */
	const void *id;

> +	const char *desc_str;

Elesewhere we just the name desc for strings like this.

Can we s/desc_str/desc/, please?

Thanks,
Mark.

[PATCH 01/17] arm64: arch_timer: Add infrastructure for multiple erratum detection methods

[Mark Rutland]

Hi,

On Mon, Mar 06, 2017 at 11:26:06AM +0000, Marc Zyngier wrote:
> We're currently stuck with DT when it comes to handling errata, which
> is pretty restrictive. In order to make things more flexible, let's
> introduce an infrastructure that could support alternative discovery
> methods. No change in functionnality.

Nit: functionality

> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/arch_timer.h  |  7 +++-
>  drivers/clocksource/arm_arch_timer.c | 80 +++++++++++++++++++++++++++++++-----
>  2 files changed, 75 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h
> index b4b34004a21e..1c92d52619a6 100644
> --- a/arch/arm64/include/asm/arch_timer.h
> +++ b/arch/arm64/include/asm/arch_timer.h
> @@ -37,9 +37,14 @@ extern struct static_key_false arch_timer_read_ool_enabled;
>  #define needs_unstable_timer_counter_workaround()  false
>  #endif
>  
> +enum arch_timer_erratum_match_type {
> +	ate_match_dt,
> +};
>  
>  struct arch_timer_erratum_workaround {
> -	const char *id;		/* Indicate the Erratum ID */
> +	enum arch_timer_erratum_match_type match_type;
> +	const void *id;		/* Indicate the Erratum ID */

This comment isn't much help. Can we drop it?

Either that, or expand on what it is in each case, e.g.

	/*
	 * Data specific to the match_type.
	 *
	 * For ate_match_dt, this is a DT property name to look for.
	 */
	const void *id;

> +	const char *desc_str;

Elesewhere we just the name desc for strings like this.

Can we s/desc_str/desc/, please?

Thanks,
Mark.

Re: [PATCH 03/17] arm64: Allow checking of a CPU-local erratum

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:08AM +0000, Marc Zyngier wrote:
> this_cpu_has_cap() only checks the feature array, and not the errata
> one. In order to be able to check for a CPU-local erratum, allow it
> to inspect the latter as well.
> 
> This is consistent with cpus_have_cap()'s behaviour, which includes
> errata already.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

This sounds sensible to me.

Suzuki, any comments?

Thanks,
Mark.

> ---
>  arch/arm64/kernel/cpufeature.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index abda8e861865..6eb77ae99b79 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -1090,20 +1090,29 @@ static void __init setup_feature_capabilities(void)
>   * Check if the current CPU has a given feature capability.
>   * Should be called from non-preemptible context.
>   */
> -bool this_cpu_has_cap(unsigned int cap)
> +static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
> +			       unsigned int cap)
>  {
>  	const struct arm64_cpu_capabilities *caps;
>  
>  	if (WARN_ON(preemptible()))
>  		return false;
>  
> -	for (caps = arm64_features; caps->desc; caps++)
> +	for (caps = cap_array; caps->desc; caps++)
>  		if (caps->capability == cap && caps->matches)
>  			return caps->matches(caps, SCOPE_LOCAL_CPU);
>  
>  	return false;
>  }
>  
> +extern const struct arm64_cpu_capabilities arm64_errata[];
> +
> +bool this_cpu_has_cap(unsigned int cap)
> +{
> +	return (__this_cpu_has_cap(arm64_features, cap) ||
> +		__this_cpu_has_cap(arm64_errata, cap));
> +}
> +
>  void __init setup_cpu_features(void)
>  {
>  	u32 cwg;
> -- 
> 2.11.0
> 

[PATCH 03/17] arm64: Allow checking of a CPU-local erratum

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:08AM +0000, Marc Zyngier wrote:
> this_cpu_has_cap() only checks the feature array, and not the errata
> one. In order to be able to check for a CPU-local erratum, allow it
> to inspect the latter as well.
> 
> This is consistent with cpus_have_cap()'s behaviour, which includes
> errata already.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

This sounds sensible to me.

Suzuki, any comments?

Thanks,
Mark.

> ---
>  arch/arm64/kernel/cpufeature.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index abda8e861865..6eb77ae99b79 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -1090,20 +1090,29 @@ static void __init setup_feature_capabilities(void)
>   * Check if the current CPU has a given feature capability.
>   * Should be called from non-preemptible context.
>   */
> -bool this_cpu_has_cap(unsigned int cap)
> +static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
> +			       unsigned int cap)
>  {
>  	const struct arm64_cpu_capabilities *caps;
>  
>  	if (WARN_ON(preemptible()))
>  		return false;
>  
> -	for (caps = arm64_features; caps->desc; caps++)
> +	for (caps = cap_array; caps->desc; caps++)
>  		if (caps->capability == cap && caps->matches)
>  			return caps->matches(caps, SCOPE_LOCAL_CPU);
>  
>  	return false;
>  }
>  
> +extern const struct arm64_cpu_capabilities arm64_errata[];
> +
> +bool this_cpu_has_cap(unsigned int cap)
> +{
> +	return (__this_cpu_has_cap(arm64_features, cap) ||
> +		__this_cpu_has_cap(arm64_errata, cap));
> +}
> +
>  void __init setup_cpu_features(void)
>  {
>  	u32 cwg;
> -- 
> 2.11.0
> 

Re: [PATCH 05/17] arm64: arch_timer: Move arch_timer_reg_read/write around

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:10AM +0000, Marc Zyngier wrote:
> As we're about to move things around, let's start with the low
> level read/write functions. This allows us to use these functions
> in the errata handling code without having to use forward declaration
> of static functions.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

Looks sensible to me.

Acked-by: Mark Rutland <mark.rutland@arm.com>

Mark.

> ---
>  drivers/clocksource/arm_arch_timer.c | 124 +++++++++++++++++------------------
>  1 file changed, 62 insertions(+), 62 deletions(-)
> 
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 3018eeeee7fa..53cb862eb6df 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -96,6 +96,68 @@ early_param("clocksource.arm_arch_timer.evtstrm", early_evtstrm_cfg);
>   * Architected system timer support.
>   */
>  
> +static __always_inline
> +void arch_timer_reg_write(int access, enum arch_timer_reg reg, u32 val,
> +			  struct clock_event_device *clk)
> +{
> +	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
> +		struct arch_timer *timer = to_arch_timer(clk);
> +		switch (reg) {
> +		case ARCH_TIMER_REG_CTRL:
> +			writel_relaxed(val, timer->base + CNTP_CTL);
> +			break;
> +		case ARCH_TIMER_REG_TVAL:
> +			writel_relaxed(val, timer->base + CNTP_TVAL);
> +			break;
> +		}
> +	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
> +		struct arch_timer *timer = to_arch_timer(clk);
> +		switch (reg) {
> +		case ARCH_TIMER_REG_CTRL:
> +			writel_relaxed(val, timer->base + CNTV_CTL);
> +			break;
> +		case ARCH_TIMER_REG_TVAL:
> +			writel_relaxed(val, timer->base + CNTV_TVAL);
> +			break;
> +		}
> +	} else {
> +		arch_timer_reg_write_cp15(access, reg, val);
> +	}
> +}
> +
> +static __always_inline
> +u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
> +			struct clock_event_device *clk)
> +{
> +	u32 val;
> +
> +	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
> +		struct arch_timer *timer = to_arch_timer(clk);
> +		switch (reg) {
> +		case ARCH_TIMER_REG_CTRL:
> +			val = readl_relaxed(timer->base + CNTP_CTL);
> +			break;
> +		case ARCH_TIMER_REG_TVAL:
> +			val = readl_relaxed(timer->base + CNTP_TVAL);
> +			break;
> +		}
> +	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
> +		struct arch_timer *timer = to_arch_timer(clk);
> +		switch (reg) {
> +		case ARCH_TIMER_REG_CTRL:
> +			val = readl_relaxed(timer->base + CNTV_CTL);
> +			break;
> +		case ARCH_TIMER_REG_TVAL:
> +			val = readl_relaxed(timer->base + CNTV_TVAL);
> +			break;
> +		}
> +	} else {
> +		val = arch_timer_reg_read_cp15(access, reg);
> +	}
> +
> +	return val;
> +}
> +
>  #ifdef CONFIG_FSL_ERRATUM_A008585
>  /*
>   * The number of retries is an arbitrary value well beyond the highest number
> @@ -294,68 +356,6 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
>  #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
>  #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
>  
> -static __always_inline
> -void arch_timer_reg_write(int access, enum arch_timer_reg reg, u32 val,
> -			  struct clock_event_device *clk)
> -{
> -	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
> -		struct arch_timer *timer = to_arch_timer(clk);
> -		switch (reg) {
> -		case ARCH_TIMER_REG_CTRL:
> -			writel_relaxed(val, timer->base + CNTP_CTL);
> -			break;
> -		case ARCH_TIMER_REG_TVAL:
> -			writel_relaxed(val, timer->base + CNTP_TVAL);
> -			break;
> -		}
> -	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
> -		struct arch_timer *timer = to_arch_timer(clk);
> -		switch (reg) {
> -		case ARCH_TIMER_REG_CTRL:
> -			writel_relaxed(val, timer->base + CNTV_CTL);
> -			break;
> -		case ARCH_TIMER_REG_TVAL:
> -			writel_relaxed(val, timer->base + CNTV_TVAL);
> -			break;
> -		}
> -	} else {
> -		arch_timer_reg_write_cp15(access, reg, val);
> -	}
> -}
> -
> -static __always_inline
> -u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
> -			struct clock_event_device *clk)
> -{
> -	u32 val;
> -
> -	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
> -		struct arch_timer *timer = to_arch_timer(clk);
> -		switch (reg) {
> -		case ARCH_TIMER_REG_CTRL:
> -			val = readl_relaxed(timer->base + CNTP_CTL);
> -			break;
> -		case ARCH_TIMER_REG_TVAL:
> -			val = readl_relaxed(timer->base + CNTP_TVAL);
> -			break;
> -		}
> -	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
> -		struct arch_timer *timer = to_arch_timer(clk);
> -		switch (reg) {
> -		case ARCH_TIMER_REG_CTRL:
> -			val = readl_relaxed(timer->base + CNTV_CTL);
> -			break;
> -		case ARCH_TIMER_REG_TVAL:
> -			val = readl_relaxed(timer->base + CNTV_TVAL);
> -			break;
> -		}
> -	} else {
> -		val = arch_timer_reg_read_cp15(access, reg);
> -	}
> -
> -	return val;
> -}
> -
>  static __always_inline irqreturn_t timer_handler(const int access,
>  					struct clock_event_device *evt)
>  {
> -- 
> 2.11.0
> 

[PATCH 05/17] arm64: arch_timer: Move arch_timer_reg_read/write around

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:10AM +0000, Marc Zyngier wrote:
> As we're about to move things around, let's start with the low
> level read/write functions. This allows us to use these functions
> in the errata handling code without having to use forward declaration
> of static functions.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

Looks sensible to me.

Acked-by: Mark Rutland <mark.rutland@arm.com>

Mark.

> ---
>  drivers/clocksource/arm_arch_timer.c | 124 +++++++++++++++++------------------
>  1 file changed, 62 insertions(+), 62 deletions(-)
> 
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index 3018eeeee7fa..53cb862eb6df 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -96,6 +96,68 @@ early_param("clocksource.arm_arch_timer.evtstrm", early_evtstrm_cfg);
>   * Architected system timer support.
>   */
>  
> +static __always_inline
> +void arch_timer_reg_write(int access, enum arch_timer_reg reg, u32 val,
> +			  struct clock_event_device *clk)
> +{
> +	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
> +		struct arch_timer *timer = to_arch_timer(clk);
> +		switch (reg) {
> +		case ARCH_TIMER_REG_CTRL:
> +			writel_relaxed(val, timer->base + CNTP_CTL);
> +			break;
> +		case ARCH_TIMER_REG_TVAL:
> +			writel_relaxed(val, timer->base + CNTP_TVAL);
> +			break;
> +		}
> +	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
> +		struct arch_timer *timer = to_arch_timer(clk);
> +		switch (reg) {
> +		case ARCH_TIMER_REG_CTRL:
> +			writel_relaxed(val, timer->base + CNTV_CTL);
> +			break;
> +		case ARCH_TIMER_REG_TVAL:
> +			writel_relaxed(val, timer->base + CNTV_TVAL);
> +			break;
> +		}
> +	} else {
> +		arch_timer_reg_write_cp15(access, reg, val);
> +	}
> +}
> +
> +static __always_inline
> +u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
> +			struct clock_event_device *clk)
> +{
> +	u32 val;
> +
> +	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
> +		struct arch_timer *timer = to_arch_timer(clk);
> +		switch (reg) {
> +		case ARCH_TIMER_REG_CTRL:
> +			val = readl_relaxed(timer->base + CNTP_CTL);
> +			break;
> +		case ARCH_TIMER_REG_TVAL:
> +			val = readl_relaxed(timer->base + CNTP_TVAL);
> +			break;
> +		}
> +	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
> +		struct arch_timer *timer = to_arch_timer(clk);
> +		switch (reg) {
> +		case ARCH_TIMER_REG_CTRL:
> +			val = readl_relaxed(timer->base + CNTV_CTL);
> +			break;
> +		case ARCH_TIMER_REG_TVAL:
> +			val = readl_relaxed(timer->base + CNTV_TVAL);
> +			break;
> +		}
> +	} else {
> +		val = arch_timer_reg_read_cp15(access, reg);
> +	}
> +
> +	return val;
> +}
> +
>  #ifdef CONFIG_FSL_ERRATUM_A008585
>  /*
>   * The number of retries is an arbitrary value well beyond the highest number
> @@ -294,68 +356,6 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t
>  #define arch_timer_check_ool_workaround(t,a)		do { } while(0)
>  #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */
>  
> -static __always_inline
> -void arch_timer_reg_write(int access, enum arch_timer_reg reg, u32 val,
> -			  struct clock_event_device *clk)
> -{
> -	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
> -		struct arch_timer *timer = to_arch_timer(clk);
> -		switch (reg) {
> -		case ARCH_TIMER_REG_CTRL:
> -			writel_relaxed(val, timer->base + CNTP_CTL);
> -			break;
> -		case ARCH_TIMER_REG_TVAL:
> -			writel_relaxed(val, timer->base + CNTP_TVAL);
> -			break;
> -		}
> -	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
> -		struct arch_timer *timer = to_arch_timer(clk);
> -		switch (reg) {
> -		case ARCH_TIMER_REG_CTRL:
> -			writel_relaxed(val, timer->base + CNTV_CTL);
> -			break;
> -		case ARCH_TIMER_REG_TVAL:
> -			writel_relaxed(val, timer->base + CNTV_TVAL);
> -			break;
> -		}
> -	} else {
> -		arch_timer_reg_write_cp15(access, reg, val);
> -	}
> -}
> -
> -static __always_inline
> -u32 arch_timer_reg_read(int access, enum arch_timer_reg reg,
> -			struct clock_event_device *clk)
> -{
> -	u32 val;
> -
> -	if (access == ARCH_TIMER_MEM_PHYS_ACCESS) {
> -		struct arch_timer *timer = to_arch_timer(clk);
> -		switch (reg) {
> -		case ARCH_TIMER_REG_CTRL:
> -			val = readl_relaxed(timer->base + CNTP_CTL);
> -			break;
> -		case ARCH_TIMER_REG_TVAL:
> -			val = readl_relaxed(timer->base + CNTP_TVAL);
> -			break;
> -		}
> -	} else if (access == ARCH_TIMER_MEM_VIRT_ACCESS) {
> -		struct arch_timer *timer = to_arch_timer(clk);
> -		switch (reg) {
> -		case ARCH_TIMER_REG_CTRL:
> -			val = readl_relaxed(timer->base + CNTV_CTL);
> -			break;
> -		case ARCH_TIMER_REG_TVAL:
> -			val = readl_relaxed(timer->base + CNTV_TVAL);
> -			break;
> -		}
> -	} else {
> -		val = arch_timer_reg_read_cp15(access, reg);
> -	}
> -
> -	return val;
> -}
> -
>  static __always_inline irqreturn_t timer_handler(const int access,
>  					struct clock_event_device *evt)
>  {
> -- 
> 2.11.0
> 

Re: [PATCH 06/17] arm64: arch_timer: Get rid of erratum_workaround_set_sne

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:11AM +0000, Marc Zyngier wrote:
> Let's move the handling of workarounds affecting set_next_event
> to the affected function, instead of overwriding the pointers

Nit: overriding

> as an afterthough. Yes, this is an extra indirection on the
> erratum handling path, but the HW is busted anyway.

[...]

> +#define erratum_set_next_event_tval_virt(...)		({BUG_ON(1); 0;})
> +#define erratum_set_next_event_tval_phys(...)		({BUG_ON(1); 0;})

Nit: BUG()

[...]

>  {
> +	if (needs_unstable_timer_counter_workaround())
> +		return erratum_set_next_event_tval_virt(evt, clk);
> +
>  	set_next_event(ARCH_TIMER_VIRT_ACCESS, evt, clk);
>  	return 0;
>  }
> @@ -487,6 +490,9 @@ static int arch_timer_set_next_event_virt(unsigned long evt,
>  static int arch_timer_set_next_event_phys(unsigned long evt,
>  					  struct clock_event_device *clk)
>  {
> +	if (needs_unstable_timer_counter_workaround())
> +		return erratum_set_next_event_tval_phys(evt, clk);
> +
>  	set_next_event(ARCH_TIMER_PHYS_ACCESS, evt, clk);
>  	return 0;
>  }

I take it that the new conditionals are NOPs in the !erratum case, so
this doesn't adversely affect the usual case.

Thanks,
Mark.

[PATCH 06/17] arm64: arch_timer: Get rid of erratum_workaround_set_sne

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:11AM +0000, Marc Zyngier wrote:
> Let's move the handling of workarounds affecting set_next_event
> to the affected function, instead of overwriding the pointers

Nit: overriding

> as an afterthough. Yes, this is an extra indirection on the
> erratum handling path, but the HW is busted anyway.

[...]

> +#define erratum_set_next_event_tval_virt(...)		({BUG_ON(1); 0;})
> +#define erratum_set_next_event_tval_phys(...)		({BUG_ON(1); 0;})

Nit: BUG()

[...]

>  {
> +	if (needs_unstable_timer_counter_workaround())
> +		return erratum_set_next_event_tval_virt(evt, clk);
> +
>  	set_next_event(ARCH_TIMER_VIRT_ACCESS, evt, clk);
>  	return 0;
>  }
> @@ -487,6 +490,9 @@ static int arch_timer_set_next_event_virt(unsigned long evt,
>  static int arch_timer_set_next_event_phys(unsigned long evt,
>  					  struct clock_event_device *clk)
>  {
> +	if (needs_unstable_timer_counter_workaround())
> +		return erratum_set_next_event_tval_phys(evt, clk);
> +
>  	set_next_event(ARCH_TIMER_PHYS_ACCESS, evt, clk);
>  	return 0;
>  }

I take it that the new conditionals are NOPs in the !erratum case, so
this doesn't adversely affect the usual case.

Thanks,
Mark.

Re: [PATCH 03/17] arm64: Allow checking of a CPU-local erratum

[Suzuki K Poulose]

On 20/03/17 13:56, Mark Rutland wrote:
> On Mon, Mar 06, 2017 at 11:26:08AM +0000, Marc Zyngier wrote:
>> this_cpu_has_cap() only checks the feature array, and not the errata
>> one. In order to be able to check for a CPU-local erratum, allow it
>> to inspect the latter as well.
>>
>> This is consistent with cpus_have_cap()'s behaviour, which includes
>> errata already.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>
> This sounds sensible to me.
>
> Suzuki, any comments?

Yes, this looks fine to me.

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

>
> Thanks,
> Mark.
>
>> ---
>>  arch/arm64/kernel/cpufeature.c | 13 +++++++++++--
>>  1 file changed, 11 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index abda8e861865..6eb77ae99b79 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -1090,20 +1090,29 @@ static void __init setup_feature_capabilities(void)
>>   * Check if the current CPU has a given feature capability.
>>   * Should be called from non-preemptible context.
>>   */
>> -bool this_cpu_has_cap(unsigned int cap)
>> +static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
>> +			       unsigned int cap)
>>  {
>>  	const struct arm64_cpu_capabilities *caps;
>>
>>  	if (WARN_ON(preemptible()))
>>  		return false;
>>
>> -	for (caps = arm64_features; caps->desc; caps++)
>> +	for (caps = cap_array; caps->desc; caps++)
>>  		if (caps->capability == cap && caps->matches)
>>  			return caps->matches(caps, SCOPE_LOCAL_CPU);
>>
>>  	return false;
>>  }
>>
>> +extern const struct arm64_cpu_capabilities arm64_errata[];
>> +
>> +bool this_cpu_has_cap(unsigned int cap)
>> +{
>> +	return (__this_cpu_has_cap(arm64_features, cap) ||
>> +		__this_cpu_has_cap(arm64_errata, cap));
>> +}
>> +
>>  void __init setup_cpu_features(void)
>>  {
>>  	u32 cwg;
>> --
>> 2.11.0
>>

[PATCH 03/17] arm64: Allow checking of a CPU-local erratum

[Suzuki K Poulose]

On 20/03/17 13:56, Mark Rutland wrote:
> On Mon, Mar 06, 2017 at 11:26:08AM +0000, Marc Zyngier wrote:
>> this_cpu_has_cap() only checks the feature array, and not the errata
>> one. In order to be able to check for a CPU-local erratum, allow it
>> to inspect the latter as well.
>>
>> This is consistent with cpus_have_cap()'s behaviour, which includes
>> errata already.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>
> This sounds sensible to me.
>
> Suzuki, any comments?

Yes, this looks fine to me.

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

>
> Thanks,
> Mark.
>
>> ---
>>  arch/arm64/kernel/cpufeature.c | 13 +++++++++++--
>>  1 file changed, 11 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index abda8e861865..6eb77ae99b79 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -1090,20 +1090,29 @@ static void __init setup_feature_capabilities(void)
>>   * Check if the current CPU has a given feature capability.
>>   * Should be called from non-preemptible context.
>>   */
>> -bool this_cpu_has_cap(unsigned int cap)
>> +static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
>> +			       unsigned int cap)
>>  {
>>  	const struct arm64_cpu_capabilities *caps;
>>
>>  	if (WARN_ON(preemptible()))
>>  		return false;
>>
>> -	for (caps = arm64_features; caps->desc; caps++)
>> +	for (caps = cap_array; caps->desc; caps++)
>>  		if (caps->capability == cap && caps->matches)
>>  			return caps->matches(caps, SCOPE_LOCAL_CPU);
>>
>>  	return false;
>>  }
>>
>> +extern const struct arm64_cpu_capabilities arm64_errata[];
>> +
>> +bool this_cpu_has_cap(unsigned int cap)
>> +{
>> +	return (__this_cpu_has_cap(arm64_features, cap) ||
>> +		__this_cpu_has_cap(arm64_errata, cap));
>> +}
>> +
>>  void __init setup_cpu_features(void)
>>  {
>>  	u32 cwg;
>> --
>> 2.11.0
>>

Re: [PATCH 10/17] arm64: Add CNTVCT_EL0 trap handler

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:15AM +0000, Marc Zyngier wrote:
> Since people seem to make a point in breaking the userspace visible
> counter, we have no choice but to trap the access. Add the required
> handler.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/esr.h |  2 ++
>  arch/arm64/kernel/traps.c    | 14 ++++++++++++++
>  2 files changed, 16 insertions(+)
> 
> diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
> index d14c478976d0..ad42e79a5d4d 100644
> --- a/arch/arm64/include/asm/esr.h
> +++ b/arch/arm64/include/asm/esr.h
> @@ -175,6 +175,8 @@
>  #define ESR_ELx_SYS64_ISS_SYS_CTR_READ	(ESR_ELx_SYS64_ISS_SYS_CTR | \
>  					 ESR_ELx_SYS64_ISS_DIR_READ)
>  
> +#define ESR_ELx_SYS64_ISS_SYS_CNTVCT	(ESR_ELx_SYS64_ISS_SYS_VAL(3, 3, 2, 14, 0) | \
> +					 ESR_ELx_SYS64_ISS_DIR_READ)
>  #ifndef __ASSEMBLY__
>  #include <asm/types.h>
>  
> diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> index e52be6aa44ee..1de444e6c669 100644
> --- a/arch/arm64/kernel/traps.c
> +++ b/arch/arm64/kernel/traps.c
> @@ -505,6 +505,14 @@ static void ctr_read_handler(unsigned int esr, struct pt_regs *regs)
>  	regs->pc += 4;
>  }
>  
> +static void cntvct_read_handler(unsigned int esr, struct pt_regs *regs)
> +{
> +	int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
> +
> +	pt_regs_write_reg(regs, rt, arch_counter_get_cntvct());
> +	regs->pc += 4;
> +}

I was going to suggest it'd be worth using AARCH64_INSN_SIZE, but I see
most of the other handlers in this file don't, so I guess that's not a
big deal.

Otherwise, this looks fine to me. FWIW:

Acked-by: Mark Rutland <mark.rutland@arm.com>

Mark.

> +
>  struct sys64_hook {
>  	unsigned int esr_mask;
>  	unsigned int esr_val;
> @@ -523,6 +531,12 @@ static struct sys64_hook sys64_hooks[] = {
>  		.esr_val = ESR_ELx_SYS64_ISS_SYS_CTR_READ,
>  		.handler = ctr_read_handler,
>  	},
> +	{
> +		/* Trap read access to CNTVCT_EL0 */
> +		.esr_mask = ESR_ELx_SYS64_ISS_SYS_OP_MASK,
> +		.esr_val = ESR_ELx_SYS64_ISS_SYS_CNTVCT,
> +		.handler = cntvct_read_handler,
> +	},
>  	{},
>  };
>  
> -- 
> 2.11.0
> 

[PATCH 10/17] arm64: Add CNTVCT_EL0 trap handler

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:15AM +0000, Marc Zyngier wrote:
> Since people seem to make a point in breaking the userspace visible
> counter, we have no choice but to trap the access. Add the required
> handler.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  arch/arm64/include/asm/esr.h |  2 ++
>  arch/arm64/kernel/traps.c    | 14 ++++++++++++++
>  2 files changed, 16 insertions(+)
> 
> diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
> index d14c478976d0..ad42e79a5d4d 100644
> --- a/arch/arm64/include/asm/esr.h
> +++ b/arch/arm64/include/asm/esr.h
> @@ -175,6 +175,8 @@
>  #define ESR_ELx_SYS64_ISS_SYS_CTR_READ	(ESR_ELx_SYS64_ISS_SYS_CTR | \
>  					 ESR_ELx_SYS64_ISS_DIR_READ)
>  
> +#define ESR_ELx_SYS64_ISS_SYS_CNTVCT	(ESR_ELx_SYS64_ISS_SYS_VAL(3, 3, 2, 14, 0) | \
> +					 ESR_ELx_SYS64_ISS_DIR_READ)
>  #ifndef __ASSEMBLY__
>  #include <asm/types.h>
>  
> diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> index e52be6aa44ee..1de444e6c669 100644
> --- a/arch/arm64/kernel/traps.c
> +++ b/arch/arm64/kernel/traps.c
> @@ -505,6 +505,14 @@ static void ctr_read_handler(unsigned int esr, struct pt_regs *regs)
>  	regs->pc += 4;
>  }
>  
> +static void cntvct_read_handler(unsigned int esr, struct pt_regs *regs)
> +{
> +	int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
> +
> +	pt_regs_write_reg(regs, rt, arch_counter_get_cntvct());
> +	regs->pc += 4;
> +}

I was going to suggest it'd be worth using AARCH64_INSN_SIZE, but I see
most of the other handlers in this file don't, so I guess that's not a
big deal.

Otherwise, this looks fine to me. FWIW:

Acked-by: Mark Rutland <mark.rutland@arm.com>

Mark.

> +
>  struct sys64_hook {
>  	unsigned int esr_mask;
>  	unsigned int esr_val;
> @@ -523,6 +531,12 @@ static struct sys64_hook sys64_hooks[] = {
>  		.esr_val = ESR_ELx_SYS64_ISS_SYS_CTR_READ,
>  		.handler = ctr_read_handler,
>  	},
> +	{
> +		/* Trap read access to CNTVCT_EL0 */
> +		.esr_mask = ESR_ELx_SYS64_ISS_SYS_OP_MASK,
> +		.esr_val = ESR_ELx_SYS64_ISS_SYS_CNTVCT,
> +		.handler = cntvct_read_handler,
> +	},
>  	{},
>  };
>  
> -- 
> 2.11.0
> 

Re: [PATCH 13/17] arm64: cpu_errata: Allow an erratum to be match for all revisions of a core

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:18AM +0000, Marc Zyngier wrote:
> Some minor erratum may not be fixed in further revisions of a core,
> leading to a situation where the workaround needs to be updated each
> time an updated core is released.

That is somewhat unfortunate. :(

> Introduce a MIDR_ALL_VERSIONS match helper that will work for all
> versions of that MIDR, once and for all.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

This looks sane to me. FWIW:

Acked-by: Mark Rutland <mark.rutland@arm.com>

Suzuki, any thoughts?

Mark.

> ---
>  arch/arm64/kernel/cpu_errata.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> index f6cc67e7626e..2be1d1c05303 100644
> --- a/arch/arm64/kernel/cpu_errata.c
> +++ b/arch/arm64/kernel/cpu_errata.c
> @@ -53,6 +53,13 @@ static int cpu_enable_trap_ctr_access(void *__unused)
>  	.midr_range_min = min, \
>  	.midr_range_max = max
>  
> +#define MIDR_ALL_VERSIONS(model) \
> +	.def_scope = SCOPE_LOCAL_CPU, \
> +	.matches = is_affected_midr_range, \
> +	.midr_model = model, \
> +	.midr_range_min = 0, \
> +	.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
> +
>  const struct arm64_cpu_capabilities arm64_errata[] = {
>  #if	defined(CONFIG_ARM64_ERRATUM_826319) || \
>  	defined(CONFIG_ARM64_ERRATUM_827319) || \
> -- 
> 2.11.0
> 

[PATCH 13/17] arm64: cpu_errata: Allow an erratum to be match for all revisions of a core

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:18AM +0000, Marc Zyngier wrote:
> Some minor erratum may not be fixed in further revisions of a core,
> leading to a situation where the workaround needs to be updated each
> time an updated core is released.

That is somewhat unfortunate. :(

> Introduce a MIDR_ALL_VERSIONS match helper that will work for all
> versions of that MIDR, once and for all.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

This looks sane to me. FWIW:

Acked-by: Mark Rutland <mark.rutland@arm.com>

Suzuki, any thoughts?

Mark.

> ---
>  arch/arm64/kernel/cpu_errata.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> index f6cc67e7626e..2be1d1c05303 100644
> --- a/arch/arm64/kernel/cpu_errata.c
> +++ b/arch/arm64/kernel/cpu_errata.c
> @@ -53,6 +53,13 @@ static int cpu_enable_trap_ctr_access(void *__unused)
>  	.midr_range_min = min, \
>  	.midr_range_max = max
>  
> +#define MIDR_ALL_VERSIONS(model) \
> +	.def_scope = SCOPE_LOCAL_CPU, \
> +	.matches = is_affected_midr_range, \
> +	.midr_model = model, \
> +	.midr_range_min = 0, \
> +	.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
> +
>  const struct arm64_cpu_capabilities arm64_errata[] = {
>  #if	defined(CONFIG_ARM64_ERRATUM_826319) || \
>  	defined(CONFIG_ARM64_ERRATUM_827319) || \
> -- 
> 2.11.0
> 

Re: [PATCH 15/17] arm64: arch_timer: Workaround for Cortex-A73 erratum 858921

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:20AM +0000, Marc Zyngier wrote:
> Cortex-A73 (all versions) counter read can return a wrong value
> when the counter crosses a 32bit boundary.
> 
> The workaround involves performing the read twice, and to return
> one or the other depending on whether a transition has taken place.
 
> +#ifdef CONFIG_ARM64_ERRATUM_858921
> +static u64 notrace arm64_858921_read_cntvct_el0(void)
> +{
> +	u64 _old, _new;
> +
> +	_old = read_sysreg(cntvct_el0);
> +	_new = read_sysreg(cntvct_el0);
> +	return (((_old ^ _new) >> 32) & 1) ? _old : _new;
> +}
> +#endif

Given this is a static inline, I think we can drop the underscores here.

Thanks,
Mark.

[PATCH 15/17] arm64: arch_timer: Workaround for Cortex-A73 erratum 858921

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:20AM +0000, Marc Zyngier wrote:
> Cortex-A73 (all versions) counter read can return a wrong value
> when the counter crosses a 32bit boundary.
> 
> The workaround involves performing the read twice, and to return
> one or the other depending on whether a transition has taken place.
 
> +#ifdef CONFIG_ARM64_ERRATUM_858921
> +static u64 notrace arm64_858921_read_cntvct_el0(void)
> +{
> +	u64 _old, _new;
> +
> +	_old = read_sysreg(cntvct_el0);
> +	_new = read_sysreg(cntvct_el0);
> +	return (((_old ^ _new) >> 32) & 1) ? _old : _new;
> +}
> +#endif

Given this is a static inline, I think we can drop the underscores here.

Thanks,
Mark.

Re: [PATCH 17/17] arm64: arch_timer: Add HISILICON_ERRATUM_161010101 ACPI matching data

[Mark Rutland]

On Tue, Mar 07, 2017 at 09:19:21PM +0800, Hanjun Guo wrote:
> On 2017/3/6 19:26, Marc Zyngier wrote:
> >In order to deal with ACPI enabled platforms suffering from the
> >HISILICON_ERRATUM_161010101, let's add the required OEM data that
> >allow the workaround to be enabled.
> >
> >Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >---
> > drivers/clocksource/arm_arch_timer.c | 29 +++++++++++++++++++++++++++++
> > 1 file changed, 29 insertions(+)
> >
> >diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> >index 6182871af4eb..b46584f058b6 100644
> >--- a/drivers/clocksource/arm_arch_timer.c
> >+++ b/drivers/clocksource/arm_arch_timer.c
> >@@ -270,6 +270,25 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
> > {
> > 	return __hisi_161010101_read_reg(cntvct_el0);
> > }
> >+
> >+static struct ate_acpi_oem_info hisi_161010101_oem_info[] = {
> >+	{
> >+		.oem_id		= "HISI  ",
> >+		.oem_table_id	= "HIP05   ",
> >+		.oem_revision	= 0,
> >+	},
> >+	{
> >+		.oem_id		= "HISI  ",
> >+		.oem_table_id	= "HIP06   ",
> >+		.oem_revision	= 0,
> >+	},
> >+	{
> >+		.oem_id		= "HISI  ",
> >+		.oem_table_id	= "HIP07   ",
> >+		.oem_revision	= 0,
> >+	},
> >+	{ },
> 
> Maybe it's better to add a comments that to say "It's needed to match
> the end of oem info", but it's up to you :)

Indeed. A comment as to why the sapces are necessary would be very
helpful.

Thanks,
Mark.

[PATCH 17/17] arm64: arch_timer: Add HISILICON_ERRATUM_161010101 ACPI matching data

[Mark Rutland]

On Tue, Mar 07, 2017 at 09:19:21PM +0800, Hanjun Guo wrote:
> On 2017/3/6 19:26, Marc Zyngier wrote:
> >In order to deal with ACPI enabled platforms suffering from the
> >HISILICON_ERRATUM_161010101, let's add the required OEM data that
> >allow the workaround to be enabled.
> >
> >Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >---
> > drivers/clocksource/arm_arch_timer.c | 29 +++++++++++++++++++++++++++++
> > 1 file changed, 29 insertions(+)
> >
> >diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> >index 6182871af4eb..b46584f058b6 100644
> >--- a/drivers/clocksource/arm_arch_timer.c
> >+++ b/drivers/clocksource/arm_arch_timer.c
> >@@ -270,6 +270,25 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void)
> > {
> > 	return __hisi_161010101_read_reg(cntvct_el0);
> > }
> >+
> >+static struct ate_acpi_oem_info hisi_161010101_oem_info[] = {
> >+	{
> >+		.oem_id		= "HISI  ",
> >+		.oem_table_id	= "HIP05   ",
> >+		.oem_revision	= 0,
> >+	},
> >+	{
> >+		.oem_id		= "HISI  ",
> >+		.oem_table_id	= "HIP06   ",
> >+		.oem_revision	= 0,
> >+	},
> >+	{
> >+		.oem_id		= "HISI  ",
> >+		.oem_table_id	= "HIP07   ",
> >+		.oem_revision	= 0,
> >+	},
> >+	{ },
> 
> Maybe it's better to add a comments that to say "It's needed to match
> the end of oem info", but it's up to you :)

Indeed. A comment as to why the sapces are necessary would be very
helpful.

Thanks,
Mark.

Re: [PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:05AM +0000, Marc Zyngier wrote:
> It has recently become obvious that a number of arm64 systems have
> been blessed with a set of timers that are slightly less than perfect,
> and require a bit of hand-holding. We already have a bunch of
> errata-specific code to deal with this, but as we're adding more
> potential detection methods (DT, ACPI, capability), things are getting
> a bit out of hands.
> 
> Instead of adding more ad-hoc fixes to an already difficult code base,
> let's give ourselves a bit of an infrastructure that can deal with
> this and hide most of the uggliness behind frendly accessors.

>  Documentation/arm64/silicon-errata.txt |   1 +
>  arch/arm64/include/asm/arch_timer.h    |  44 ++-
>  arch/arm64/include/asm/cpucaps.h       |   3 +-
>  arch/arm64/include/asm/cputype.h       |   2 +
>  arch/arm64/include/asm/esr.h           |   2 +
>  arch/arm64/kernel/cpu_errata.c         |  15 +
>  arch/arm64/kernel/cpufeature.c         |  13 +-
>  arch/arm64/kernel/traps.c              |  14 +
>  drivers/clocksource/Kconfig            |  11 +
>  drivers/clocksource/arm_arch_timer.c   | 529 +++++++++++++++++++++++----------
>  10 files changed, 465 insertions(+), 169 deletions(-)

Largely this looks ok.

What's the plan for merging this?

If you can respin this, I can see about collating this with the ACPI
GTDT patches, but I'm not sure what to do about the arch/arm64/ parts.

Thanks,
Mark.

[PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[Mark Rutland]

On Mon, Mar 06, 2017 at 11:26:05AM +0000, Marc Zyngier wrote:
> It has recently become obvious that a number of arm64 systems have
> been blessed with a set of timers that are slightly less than perfect,
> and require a bit of hand-holding. We already have a bunch of
> errata-specific code to deal with this, but as we're adding more
> potential detection methods (DT, ACPI, capability), things are getting
> a bit out of hands.
> 
> Instead of adding more ad-hoc fixes to an already difficult code base,
> let's give ourselves a bit of an infrastructure that can deal with
> this and hide most of the uggliness behind frendly accessors.

>  Documentation/arm64/silicon-errata.txt |   1 +
>  arch/arm64/include/asm/arch_timer.h    |  44 ++-
>  arch/arm64/include/asm/cpucaps.h       |   3 +-
>  arch/arm64/include/asm/cputype.h       |   2 +
>  arch/arm64/include/asm/esr.h           |   2 +
>  arch/arm64/kernel/cpu_errata.c         |  15 +
>  arch/arm64/kernel/cpufeature.c         |  13 +-
>  arch/arm64/kernel/traps.c              |  14 +
>  drivers/clocksource/Kconfig            |  11 +
>  drivers/clocksource/arm_arch_timer.c   | 529 +++++++++++++++++++++++----------
>  10 files changed, 465 insertions(+), 169 deletions(-)

Largely this looks ok.

What's the plan for merging this?

If you can respin this, I can see about collating this with the ACPI
GTDT patches, but I'm not sure what to do about the arch/arm64/ parts.

Thanks,
Mark.

Re: [PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[Marc Zyngier]

On 20/03/17 15:07, Mark Rutland wrote:
> On Mon, Mar 06, 2017 at 11:26:05AM +0000, Marc Zyngier wrote:
>> It has recently become obvious that a number of arm64 systems have
>> been blessed with a set of timers that are slightly less than perfect,
>> and require a bit of hand-holding. We already have a bunch of
>> errata-specific code to deal with this, but as we're adding more
>> potential detection methods (DT, ACPI, capability), things are getting
>> a bit out of hands.
>>
>> Instead of adding more ad-hoc fixes to an already difficult code base,
>> let's give ourselves a bit of an infrastructure that can deal with
>> this and hide most of the uggliness behind frendly accessors.
> 
>>  Documentation/arm64/silicon-errata.txt |   1 +
>>  arch/arm64/include/asm/arch_timer.h    |  44 ++-
>>  arch/arm64/include/asm/cpucaps.h       |   3 +-
>>  arch/arm64/include/asm/cputype.h       |   2 +
>>  arch/arm64/include/asm/esr.h           |   2 +
>>  arch/arm64/kernel/cpu_errata.c         |  15 +
>>  arch/arm64/kernel/cpufeature.c         |  13 +-
>>  arch/arm64/kernel/traps.c              |  14 +
>>  drivers/clocksource/Kconfig            |  11 +
>>  drivers/clocksource/arm_arch_timer.c   | 529 +++++++++++++++++++++++----------
>>  10 files changed, 465 insertions(+), 169 deletions(-)
> 
> Largely this looks ok.
> 
> What's the plan for merging this?
> 
> If you can respin this, I can see about collating this with the ACPI
> GTDT patches, but I'm not sure what to do about the arch/arm64/ parts.

If Will/Catalin agree with the arm64-specific patches (which I can
probably move to the beginning of the series), we can have a shared
branch that gets merged via both the arm64 and clocksource trees.

Or we get everything merged via arm64, which wouldn't be that bad either
given that this code (and the GTDT patches) only targets arm64. We'd
need Daniel's Ack though.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

[PATCH 00/17] clocksource/arch_timer: Errara workaround infrastructure rework

[Marc Zyngier]

On 20/03/17 15:07, Mark Rutland wrote:
> On Mon, Mar 06, 2017 at 11:26:05AM +0000, Marc Zyngier wrote:
>> It has recently become obvious that a number of arm64 systems have
>> been blessed with a set of timers that are slightly less than perfect,
>> and require a bit of hand-holding. We already have a bunch of
>> errata-specific code to deal with this, but as we're adding more
>> potential detection methods (DT, ACPI, capability), things are getting
>> a bit out of hands.
>>
>> Instead of adding more ad-hoc fixes to an already difficult code base,
>> let's give ourselves a bit of an infrastructure that can deal with
>> this and hide most of the uggliness behind frendly accessors.
> 
>>  Documentation/arm64/silicon-errata.txt |   1 +
>>  arch/arm64/include/asm/arch_timer.h    |  44 ++-
>>  arch/arm64/include/asm/cpucaps.h       |   3 +-
>>  arch/arm64/include/asm/cputype.h       |   2 +
>>  arch/arm64/include/asm/esr.h           |   2 +
>>  arch/arm64/kernel/cpu_errata.c         |  15 +
>>  arch/arm64/kernel/cpufeature.c         |  13 +-
>>  arch/arm64/kernel/traps.c              |  14 +
>>  drivers/clocksource/Kconfig            |  11 +
>>  drivers/clocksource/arm_arch_timer.c   | 529 +++++++++++++++++++++++----------
>>  10 files changed, 465 insertions(+), 169 deletions(-)
> 
> Largely this looks ok.
> 
> What's the plan for merging this?
> 
> If you can respin this, I can see about collating this with the ACPI
> GTDT patches, but I'm not sure what to do about the arch/arm64/ parts.

If Will/Catalin agree with the arm64-specific patches (which I can
probably move to the beginning of the series), we can have a shared
branch that gets merged via both the arm64 and clocksource trees.

Or we get everything merged via arm64, which wouldn't be that bad either
given that this code (and the GTDT patches) only targets arm64. We'd
need Daniel's Ack though.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH 13/17] arm64: cpu_errata: Allow an erratum to be match for all revisions of a core

[Suzuki K Poulose]

On 20/03/17 14:56, Mark Rutland wrote:
> On Mon, Mar 06, 2017 at 11:26:18AM +0000, Marc Zyngier wrote:
>> Some minor erratum may not be fixed in further revisions of a core,
>> leading to a situation where the workaround needs to be updated each
>> time an updated core is released.
>
> That is somewhat unfortunate. :(
>
>> Introduce a MIDR_ALL_VERSIONS match helper that will work for all
>> versions of that MIDR, once and for all.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>
> This looks sane to me. FWIW:
>
> Acked-by: Mark Rutland <mark.rutland@arm.com>
>
> Suzuki, any thoughts?

Fine by me.

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>


>
> Mark.
>
>> ---
>>  arch/arm64/kernel/cpu_errata.c | 7 +++++++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
>> index f6cc67e7626e..2be1d1c05303 100644
>> --- a/arch/arm64/kernel/cpu_errata.c
>> +++ b/arch/arm64/kernel/cpu_errata.c
>> @@ -53,6 +53,13 @@ static int cpu_enable_trap_ctr_access(void *__unused)
>>  	.midr_range_min = min, \
>>  	.midr_range_max = max
>>
>> +#define MIDR_ALL_VERSIONS(model) \
>> +	.def_scope = SCOPE_LOCAL_CPU, \
>> +	.matches = is_affected_midr_range, \
>> +	.midr_model = model, \
>> +	.midr_range_min = 0, \
>> +	.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
>> +
>>  const struct arm64_cpu_capabilities arm64_errata[] = {
>>  #if	defined(CONFIG_ARM64_ERRATUM_826319) || \
>>  	defined(CONFIG_ARM64_ERRATUM_827319) || \
>> --
>> 2.11.0
>>

[PATCH 13/17] arm64: cpu_errata: Allow an erratum to be match for all revisions of a core

[Suzuki K Poulose]

On 20/03/17 14:56, Mark Rutland wrote:
> On Mon, Mar 06, 2017 at 11:26:18AM +0000, Marc Zyngier wrote:
>> Some minor erratum may not be fixed in further revisions of a core,
>> leading to a situation where the workaround needs to be updated each
>> time an updated core is released.
>
> That is somewhat unfortunate. :(
>
>> Introduce a MIDR_ALL_VERSIONS match helper that will work for all
>> versions of that MIDR, once and for all.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>
> This looks sane to me. FWIW:
>
> Acked-by: Mark Rutland <mark.rutland@arm.com>
>
> Suzuki, any thoughts?

Fine by me.

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>


>
> Mark.
>
>> ---
>>  arch/arm64/kernel/cpu_errata.c | 7 +++++++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
>> index f6cc67e7626e..2be1d1c05303 100644
>> --- a/arch/arm64/kernel/cpu_errata.c
>> +++ b/arch/arm64/kernel/cpu_errata.c
>> @@ -53,6 +53,13 @@ static int cpu_enable_trap_ctr_access(void *__unused)
>>  	.midr_range_min = min, \
>>  	.midr_range_max = max
>>
>> +#define MIDR_ALL_VERSIONS(model) \
>> +	.def_scope = SCOPE_LOCAL_CPU, \
>> +	.matches = is_affected_midr_range, \
>> +	.midr_model = model, \
>> +	.midr_range_min = 0, \
>> +	.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
>> +
>>  const struct arm64_cpu_capabilities arm64_errata[] = {
>>  #if	defined(CONFIG_ARM64_ERRATUM_826319) || \
>>  	defined(CONFIG_ARM64_ERRATUM_827319) || \
>> --
>> 2.11.0
>>

Re: [PATCH 06/17] arm64: arch_timer: Get rid of erratum_workaround_set_sne

[Marc Zyngier]

On 20/03/17 14:06, Mark Rutland wrote:
> On Mon, Mar 06, 2017 at 11:26:11AM +0000, Marc Zyngier wrote:
>> Let's move the handling of workarounds affecting set_next_event
>> to the affected function, instead of overwriding the pointers
> 
> Nit: overriding
> 
>> as an afterthough. Yes, this is an extra indirection on the
>> erratum handling path, but the HW is busted anyway.
> 
> [...]
> 
>> +#define erratum_set_next_event_tval_virt(...)		({BUG_ON(1); 0;})
>> +#define erratum_set_next_event_tval_phys(...)		({BUG_ON(1); 0;})
> 
> Nit: BUG()
> 
> [...]
> 
>>  {
>> +	if (needs_unstable_timer_counter_workaround())
>> +		return erratum_set_next_event_tval_virt(evt, clk);
>> +
>>  	set_next_event(ARCH_TIMER_VIRT_ACCESS, evt, clk);
>>  	return 0;
>>  }
>> @@ -487,6 +490,9 @@ static int arch_timer_set_next_event_virt(unsigned long evt,
>>  static int arch_timer_set_next_event_phys(unsigned long evt,
>>  					  struct clock_event_device *clk)
>>  {
>> +	if (needs_unstable_timer_counter_workaround())
>> +		return erratum_set_next_event_tval_phys(evt, clk);
>> +
>>  	set_next_event(ARCH_TIMER_PHYS_ACCESS, evt, clk);
>>  	return 0;
>>  }
> 
> I take it that the new conditionals are NOPs in the !erratum case, so
> this doesn't adversely affect the usual case.

Indeed, we only get an extra NOP on the normal path. The erratum path
triggers the insertion of a jump targeting the end of the function,
where the workaround gets handled.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

[PATCH 06/17] arm64: arch_timer: Get rid of erratum_workaround_set_sne

[Marc Zyngier]

On 20/03/17 14:06, Mark Rutland wrote:
> On Mon, Mar 06, 2017 at 11:26:11AM +0000, Marc Zyngier wrote:
>> Let's move the handling of workarounds affecting set_next_event
>> to the affected function, instead of overwriding the pointers
> 
> Nit: overriding
> 
>> as an afterthough. Yes, this is an extra indirection on the
>> erratum handling path, but the HW is busted anyway.
> 
> [...]
> 
>> +#define erratum_set_next_event_tval_virt(...)		({BUG_ON(1); 0;})
>> +#define erratum_set_next_event_tval_phys(...)		({BUG_ON(1); 0;})
> 
> Nit: BUG()
> 
> [...]
> 
>>  {
>> +	if (needs_unstable_timer_counter_workaround())
>> +		return erratum_set_next_event_tval_virt(evt, clk);
>> +
>>  	set_next_event(ARCH_TIMER_VIRT_ACCESS, evt, clk);
>>  	return 0;
>>  }
>> @@ -487,6 +490,9 @@ static int arch_timer_set_next_event_virt(unsigned long evt,
>>  static int arch_timer_set_next_event_phys(unsigned long evt,
>>  					  struct clock_event_device *clk)
>>  {
>> +	if (needs_unstable_timer_counter_workaround())
>> +		return erratum_set_next_event_tval_phys(evt, clk);
>> +
>>  	set_next_event(ARCH_TIMER_PHYS_ACCESS, evt, clk);
>>  	return 0;
>>  }
> 
> I take it that the new conditionals are NOPs in the !erratum case, so
> this doesn't adversely affect the usual case.

Indeed, we only get an extra NOP on the normal path. The erratum path
triggers the insertion of a jump targeting the end of the function,
where the workaround gets handled.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...