[PATCH v2 0/3] Add L1 and L2 error detection for A53 and A57

[PATCH v2 0/3] Add L1 and L2 error detection for A53 and A57

[Sascha Hauer]

This driver is based on an earlier version from York Sun which can
be found here: https://lkml.org/lkml/2018/3/14/1203.

At that time the conclusion was that this driver is not suitable for
mainline as it used IMPLEMENTATION DEFINED CPU registers and also
NXP specific SMC calls. All this was used for the error injection only,
for error reporting it is not needed.

This is another try to get this driver to mainline. All error injection
code has been removed (though it has initially been used to test this
driver on an i.MX8 SoC), what's left is unfortunately not testable, but
also doesn't contain none of the doubtful code anymore.

Changes since v1:
- Split dt-binding into separate patch
- Sort local function variables in reverse-xmas tree order
- drop unnecessary comparison and make variable bool

Sascha Hauer (2):
  dt-bindings: edac: Add binding for L1/L2 error detection for Cortex
    A53/57
  drivers/edac: Add L1 and L2 error detection for A53 and A57

York Sun (1):
  arm64: dts: ls104x: Add L1/L2 cache edac node

 .../bindings/edac/arm,cortex-a5x-edac.yaml    |  32 +++
 .../arm64/boot/dts/freescale/fsl-ls1043a.dtsi |   5 +
 .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi |   5 +
 drivers/edac/Kconfig                          |   6 +
 drivers/edac/Makefile                         |   1 +
 drivers/edac/cortex_arm64_l1_l2.c             | 208 ++++++++++++++++++
 6 files changed, 257 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml
 create mode 100644 drivers/edac/cortex_arm64_l1_l2.c

-- 
2.28.0

[PATCH 1/3] dt-bindings: edac: Add binding for L1/L2 error detection for Cortex A53/57

[Sascha Hauer]

The ARM Cortex-A53 and A57 CPUs support error detection for the L1/L2
caches. This patch adds a binding for the corresponding driver.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 .../bindings/edac/arm,cortex-a5x-edac.yaml    | 32 +++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml

diff --git a/Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml b/Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml
new file mode 100644
index 000000000000..de9325b688a0
--- /dev/null
+++ b/Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml
@@ -0,0 +1,32 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/edac/arm,cortex-a5x-edac.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: ARM cortex A53/A57 EDAC bindings
+
+description: |+
+  This contains the binding to support error detection for the L1 and L2 caches
+  on ARM Cortex A53 and A57 cores.
+
+properties:
+  compatible:
+    items:
+      - const: arm,cortex-a53-edac
+      - const: arm,cortex-a57-edac
+
+  cpus:
+    minItems: 1
+    description: phandles to the cpu nodes this device handles
+
+required:
+  - compatible
+  - cpus
+
+examples:
+  - |
+    edac-a53 {
+        compatible = "arm,cortex-a53-edac";
+        cpus = <&cpu0 &cpu1 &cpu2 &cpu3>;
+    };
-- 
2.28.0

[PATCH 2/3] drivers/edac: Add L1 and L2 error detection for A53 and A57

[Sascha Hauer]

The Cortex A53 and A57 cores have error detection capabilities for the
L1/L2 Caches, this patch adds a driver for them.

Unfortunately there is no robust way to inject errors into the caches,
so this driver doesn't contain any code to actually test it. It has
been tested though with code taken from an older version of this driver
found here: https://lkml.org/lkml/2018/3/14/1203. For reasons stated
in this thread the error injection code is not suitable for mainline,
so it is removed from the driver.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 drivers/edac/Kconfig              |   6 +
 drivers/edac/Makefile             |   1 +
 drivers/edac/cortex_arm64_l1_l2.c | 208 ++++++++++++++++++++++++++++++
 3 files changed, 215 insertions(+)
 create mode 100644 drivers/edac/cortex_arm64_l1_l2.c

diff --git a/drivers/edac/Kconfig b/drivers/edac/Kconfig
index 7b6ec3014ba2..2914156f263f 100644
--- a/drivers/edac/Kconfig
+++ b/drivers/edac/Kconfig
@@ -530,4 +530,10 @@ config EDAC_DMC520
 	  Support for error detection and correction on the
 	  SoCs with ARM DMC-520 DRAM controller.
 
+config EDAC_CORTEX_ARM64_L1_L2
+	tristate "ARM Cortex A57/A53"
+	depends on ARM64
+	help
+	  Support for L1/L2 cache error detection on ARM Cortex A57 and A53.
+
 endif # EDAC
diff --git a/drivers/edac/Makefile b/drivers/edac/Makefile
index 269e15118cea..3edba6bea350 100644
--- a/drivers/edac/Makefile
+++ b/drivers/edac/Makefile
@@ -88,3 +88,4 @@ obj-$(CONFIG_EDAC_QCOM)			+= qcom_edac.o
 obj-$(CONFIG_EDAC_ASPEED)		+= aspeed_edac.o
 obj-$(CONFIG_EDAC_BLUEFIELD)		+= bluefield_edac.o
 obj-$(CONFIG_EDAC_DMC520)		+= dmc520_edac.o
+obj-$(CONFIG_EDAC_CORTEX_ARM64_L1_L2)	+= cortex_arm64_l1_l2.o
diff --git a/drivers/edac/cortex_arm64_l1_l2.c b/drivers/edac/cortex_arm64_l1_l2.c
new file mode 100644
index 000000000000..fb8386eb40ac
--- /dev/null
+++ b/drivers/edac/cortex_arm64_l1_l2.c
@@ -0,0 +1,208 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Cortex A57 and A53 EDAC L1 and L2 cache error detection
+ *
+ * Copyright (c) 2020 Pengutronix, Sascha Hauer <s.hauer@pengutronix.de>
+ *
+ * Based on Code from:
+ * Copyright (c) 2018, NXP Semiconductor
+ * Author: York Sun <york.sun@nxp.com>
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/of_platform.h>
+#include <linux/of_device.h>
+#include <linux/bitfield.h>
+#include <asm/smp_plat.h>
+
+#include "edac_module.h"
+
+#define EDAC_MOD_STR			"cortex_edac_l1_l2"
+
+#define CPUMERRSR_EL1_RAMID		GENMASK(30, 24)
+
+#define CPUMERRSR_EL1_VALID		BIT(31)
+#define CPUMERRSR_EL1_FATAL		BIT(63)
+
+#define L1_I_TAG_RAM			0x00
+#define L1_I_DATA_RAM			0x01
+#define L1_D_TAG_RAM			0x08
+#define L1_D_DATA_RAM			0x09
+#define L1_D_DIRTY_RAM			0x14
+#define TLB_RAM				0x18
+
+#define L2MERRSR_EL1_VALID		BIT(31)
+#define L2MERRSR_EL1_FATAL		BIT(63)
+
+struct arm64_pvt {
+	call_single_data_t __percpu *csd_check;
+	struct cpumask compat_mask;
+};
+
+#define MESSAGE_SIZE 64
+
+static void read_errors(void *data)
+{
+	struct edac_device_ctl_info *edac_ctl = data;
+	int cpu = smp_processor_id();
+	char msg[MESSAGE_SIZE];
+	u64 cpumerr, l2merr;
+
+	/* cpumerrsr_el1 */
+	asm volatile("mrs %0, s3_1_c15_c2_2" : "=r" (cpumerr));
+	asm volatile("msr s3_1_c15_c2_2, %0" :: "r" (0));
+
+	if (cpumerr & CPUMERRSR_EL1_VALID) {
+		const char *str;
+		bool fatal = cpumerr & CPUMERRSR_EL1_FATAL;
+
+		switch (FIELD_GET(CPUMERRSR_EL1_RAMID, cpumerr)) {
+		case L1_I_TAG_RAM:
+			str = "L1-I Tag RAM";
+			break;
+		case L1_I_DATA_RAM:
+			str = "L1-I Data RAM";
+			break;
+		case L1_D_TAG_RAM:
+			str = "L1-D Tag RAM";
+			break;
+		case L1_D_DATA_RAM:
+			str = "L1-D Data RAM";
+			break;
+		case L1_D_DIRTY_RAM:
+			str = "L1 Dirty RAM";
+			break;
+		case TLB_RAM:
+			str = "TLB RAM";
+			break;
+		default:
+			str = "unknown";
+			break;
+		}
+
+		snprintf(msg, MESSAGE_SIZE, "%s %s error(s) on CPU %d",
+			 str, fatal ? "fatal" : "correctable", cpu);
+
+		if (fatal)
+			edac_device_handle_ue(edac_ctl, 0, 0, msg);
+		else
+			edac_device_handle_ce(edac_ctl, 0, 0, msg);
+	}
+
+	/* l2merrsr_el1 */
+	asm volatile("mrs %0, s3_1_c15_c2_3" : "=r" (l2merr));
+	asm volatile("msr s3_1_c15_c2_3, %0" :: "r" (0));
+
+	if (l2merr & L2MERRSR_EL1_VALID) {
+		bool fatal = l2merr & L2MERRSR_EL1_FATAL;
+
+		snprintf(msg, MESSAGE_SIZE, "L2 %s error(s) on CPU %d",
+			 fatal ? "fatal" : "correctable", cpu);
+		if (fatal)
+			edac_device_handle_ue(edac_ctl, 0, 1, msg);
+		else
+			edac_device_handle_ce(edac_ctl, 0, 1, msg);
+	}
+}
+
+static void cortex_arm64_edac_check(struct edac_device_ctl_info *edac_ctl)
+{
+	struct arm64_pvt *pvt = edac_ctl->pvt_info;
+	call_single_data_t *csd;
+	int cpu;
+
+	get_online_cpus();
+	for_each_cpu_and(cpu, cpu_online_mask, &pvt->compat_mask) {
+		csd = per_cpu_ptr(pvt->csd_check, cpu);
+		csd->func = read_errors;
+		csd->info = edac_ctl;
+		csd->flags = 0;
+		/* Read CPU L1/L2 errors */
+		smp_call_function_single_async(cpu, csd);
+		/* Wait until flags cleared */
+		smp_cond_load_acquire(&csd->flags, !VAL);
+	}
+	put_online_cpus();
+}
+
+static int cortex_arm64_edac_probe(struct platform_device *pdev)
+{
+	struct device_node *np, *dn = pdev->dev.of_node;
+	struct edac_device_ctl_info *edac_ctl;
+	struct device *dev = &pdev->dev;
+	struct of_phandle_iterator it;
+	struct arm64_pvt *pvt;
+	int rc, cpu;
+
+	edac_ctl = edac_device_alloc_ctl_info(sizeof(*pvt), "cpu_cache",
+					      1, "L", 2, 1, NULL, 0,
+					      edac_device_alloc_index());
+	if (!edac_ctl)
+		return -ENOMEM;
+
+	pvt = edac_ctl->pvt_info;
+	pvt->csd_check = devm_alloc_percpu(dev, call_single_data_t);
+	if (!pvt->csd_check) {
+		rc = -ENOMEM;
+		goto out_dev;
+	}
+
+	of_for_each_phandle(&it, rc, dn, "cpus", NULL, 0) {
+		np = it.node;
+		cpu = of_cpu_node_to_id(np);
+		if (cpu < 0)
+			continue;
+
+		cpumask_set_cpu(cpu, &pvt->compat_mask);
+	}
+
+	edac_ctl->edac_check = cortex_arm64_edac_check;
+	edac_ctl->dev = dev;
+	edac_ctl->mod_name = dev_name(dev);
+	edac_ctl->dev_name = dev_name(dev);
+	edac_ctl->ctl_name = EDAC_MOD_STR;
+	dev_set_drvdata(dev, edac_ctl);
+
+	rc = edac_device_add_device(edac_ctl);
+	if (rc)
+		goto out_dev;
+
+	return 0;
+
+out_dev:
+	edac_device_free_ctl_info(edac_ctl);
+
+	return rc;
+}
+
+static int cortex_arm64_edac_remove(struct platform_device *pdev)
+{
+	struct edac_device_ctl_info *edac_ctl = dev_get_drvdata(&pdev->dev);
+
+	edac_device_del_device(edac_ctl->dev);
+	edac_device_free_ctl_info(edac_ctl);
+
+	return 0;
+}
+
+static const struct of_device_id cortex_arm64_edac_of_match[] = {
+	{ .compatible = "arm,cortex-a53-edac" },
+	{ .compatible = "arm,cortex-a57-edac" },
+	{}
+};
+MODULE_DEVICE_TABLE(of, cortex_arm64_edac_of_match);
+
+static struct platform_driver cortex_arm64_edac_driver = {
+	.probe = cortex_arm64_edac_probe,
+	.remove = cortex_arm64_edac_remove,
+	.driver = {
+		.name = EDAC_MOD_STR,
+		.of_match_table = cortex_arm64_edac_of_match,
+	},
+};
+module_platform_driver(cortex_arm64_edac_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Sascha Hauer <s.hauer@pengutronix.de>");
+MODULE_DESCRIPTION("Cortex A57 and A53 L1 and L2 cache EDAC driver");
-- 
2.28.0

[PATCH 3/3] arm64: dts: ls104x: Add L1/L2 cache edac node

[Sascha Hauer]

From: York Sun <york.sun@nxp.com>

The Cortex A53/A57 cores on the Layerscape LS104x SoCs support EDAC
for the L1/L2 caches. Add the corresponding nodes for it.

Signed-off-by: York Sun <york.sun@nxp.com>
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi | 5 +++++
 arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
index 5c2e370f6316..76cc62b02494 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi
@@ -935,6 +935,11 @@ optee {
 		};
 	};
 
+	edac-a53 {
+		compatible = "arm,cortex-a53-edac";
+		cpus = <&cpu0>, <&cpu1>, <&cpu2>, <&cpu3>;
+	};
+
 };
 
 #include "qoriq-qman-portals.dtsi"
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
index 0246d975a206..ed35211c9b35 100644
--- a/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
+++ b/arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi
@@ -895,6 +895,11 @@ optee {
 			method = "smc";
 		};
 	};
+
+	edac-a57 {
+		compatible = "arm,cortex-a57-edac";
+		cpus = <&cpu0>, <&cpu1>, <&cpu2>, <&cpu3>;
+	};
 };
 
 #include "qoriq-qman-portals.dtsi"
-- 
2.28.0

Re: [PATCH v2 0/3] Add L1 and L2 error detection for A53 and A57

[Rob Herring]

On Tue, Oct 13, 2020 at 02:50:30PM +0200, Sascha Hauer wrote:
> This driver is based on an earlier version from York Sun which can
> be found here: https://lkml.org/lkml/2018/3/14/1203.
> 
> At that time the conclusion was that this driver is not suitable for
> mainline as it used IMPLEMENTATION DEFINED CPU registers and also
> NXP specific SMC calls. All this was used for the error injection only,
> for error reporting it is not needed.

Have you looked at Amazon's version:
http://lore.kernel.org/r/20200510151310.17372-2-hhhawa@amazon.com

Which is an A57 EDAC driver. Looks like it never got upstream though, 
but it's not clear why.

You'll note that it doesn't have a virtual DT node either.

> This is another try to get this driver to mainline. All error injection
> code has been removed (though it has initially been used to test this
> driver on an i.MX8 SoC), what's left is unfortunately not testable, but
> also doesn't contain none of the doubtful code anymore.
> 
> Changes since v1:
> - Split dt-binding into separate patch
> - Sort local function variables in reverse-xmas tree order
> - drop unnecessary comparison and make variable bool
> 
> Sascha Hauer (2):
>   dt-bindings: edac: Add binding for L1/L2 error detection for Cortex
>     A53/57
>   drivers/edac: Add L1 and L2 error detection for A53 and A57
> 
> York Sun (1):
>   arm64: dts: ls104x: Add L1/L2 cache edac node
> 
>  .../bindings/edac/arm,cortex-a5x-edac.yaml    |  32 +++
>  .../arm64/boot/dts/freescale/fsl-ls1043a.dtsi |   5 +
>  .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi |   5 +
>  drivers/edac/Kconfig                          |   6 +
>  drivers/edac/Makefile                         |   1 +
>  drivers/edac/cortex_arm64_l1_l2.c             | 208 ++++++++++++++++++
>  6 files changed, 257 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml
>  create mode 100644 drivers/edac/cortex_arm64_l1_l2.c
> 
> -- 
> 2.28.0
> 

Re: [PATCH 1/3] dt-bindings: edac: Add binding for L1/L2 error detection for Cortex A53/57

[Rob Herring]

On Tue, 13 Oct 2020 14:50:31 +0200, Sascha Hauer wrote:
> The ARM Cortex-A53 and A57 CPUs support error detection for the L1/L2
> caches. This patch adds a binding for the corresponding driver.
> 
> Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
> ---
>  .../bindings/edac/arm,cortex-a5x-edac.yaml    | 32 +++++++++++++++++++
>  1 file changed, 32 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml
> 


My bot found errors running 'make dt_binding_check' on your patch:

/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml: 'maintainers' is a required property
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml: ignoring, error in schema: 
warning: no schema found in file: ./Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml


See https://patchwork.ozlabs.org/patch/1381567

If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure dt-schema is up to date:

pip3 install git+https://github.com/devicetree-org/dt-schema.git@master --upgrade

Please check and re-submit.

Re: [PATCH v2 0/3] Add L1 and L2 error detection for A53 and A57

[Sascha Hauer]

On Wed, Oct 14, 2020 at 08:25:11AM -0500, Rob Herring wrote:
> On Tue, Oct 13, 2020 at 02:50:30PM +0200, Sascha Hauer wrote:
> > This driver is based on an earlier version from York Sun which can
> > be found here: https://lkml.org/lkml/2018/3/14/1203.
> > 
> > At that time the conclusion was that this driver is not suitable for
> > mainline as it used IMPLEMENTATION DEFINED CPU registers and also
> > NXP specific SMC calls. All this was used for the error injection only,
> > for error reporting it is not needed.
> 
> Have you looked at Amazon's version:
> http://lore.kernel.org/r/20200510151310.17372-2-hhhawa@amazon.com

No, I was not aware of that driver. It's basically the same driver, but
limited to a single SoC. Looks like at least some things are better in
that driver, read_sysreg_s(ARM_CA57_L2MERRSR_EL1) reads better than my
open coded variant.

> 
> Which is an A57 EDAC driver. Looks like it never got upstream though, 
> but it's not clear why.
> 
> You'll note that it doesn't have a virtual DT node either.

Testing the SoC type in an initcall looks odd to me. Wouldn't a
dedicated node be preferred?

Sascha

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH v2 0/3] Add L1 and L2 error detection for A53 and A57

[Rob Herring]

On Wed, Oct 14, 2020 at 9:04 AM Sascha Hauer <s.hauer@pengutronix.de> wrote:
>
> On Wed, Oct 14, 2020 at 08:25:11AM -0500, Rob Herring wrote:
> > On Tue, Oct 13, 2020 at 02:50:30PM +0200, Sascha Hauer wrote:
> > > This driver is based on an earlier version from York Sun which can
> > > be found here: https://lkml.org/lkml/2018/3/14/1203.
> > >
> > > At that time the conclusion was that this driver is not suitable for
> > > mainline as it used IMPLEMENTATION DEFINED CPU registers and also
> > > NXP specific SMC calls. All this was used for the error injection only,
> > > for error reporting it is not needed.
> >
> > Have you looked at Amazon's version:
> > http://lore.kernel.org/r/20200510151310.17372-2-hhhawa@amazon.com
>
> No, I was not aware of that driver. It's basically the same driver, but
> limited to a single SoC. Looks like at least some things are better in
> that driver, read_sysreg_s(ARM_CA57_L2MERRSR_EL1) reads better than my
> open coded variant.
>
> >
> > Which is an A57 EDAC driver. Looks like it never got upstream though,
> > but it's not clear why.
> >
> > You'll note that it doesn't have a virtual DT node either.
>
> Testing the SoC type in an initcall looks odd to me. Wouldn't a
> dedicated node be preferred?

Yes, the one with "arm,cortex-a57". But no, a virtual node isn't
preferred. We discussed this at length on Amazon's version IIRC. I
could perhaps be convinced to add a property in the cpu nodes that ECC
is present/enabled. Then you'd just match on cpu compatible(s) and
check for the property. You're still creating the device yourself, but
that's the kernel's problem which shouldn't dictate how you design
your bindings.

Rob

Re: [PATCH 2/3] drivers/edac: Add L1 and L2 error detection for A53 and A57

[James Morse]

Hi Sascha,

On 13/10/2020 13:50, Sascha Hauer wrote:
> The Cortex A53 and A57 cores have error detection capabilities for the
> L1/L2 Caches, this patch adds a driver for them.
> 
> Unfortunately there is no robust way to inject errors into the caches,
> so this driver doesn't contain any code to actually test it. It has
> been tested though with code taken from an older version of this driver
> found here: https://lkml.org/lkml/2018/3/14/1203.

> For reasons stated
> in this thread the error injection code is not suitable for mainline,
> so it is removed from the driver.


> diff --git a/drivers/edac/cortex_arm64_l1_l2.c b/drivers/edac/cortex_arm64_l1_l2.c
> new file mode 100644
> index 000000000000..fb8386eb40ac
> --- /dev/null
> +++ b/drivers/edac/cortex_arm64_l1_l2.c
> @@ -0,0 +1,208 @@

> +static void read_errors(void *data)
> +{
> +	struct edac_device_ctl_info *edac_ctl = data;
> +	int cpu = smp_processor_id();
> +	char msg[MESSAGE_SIZE];
> +	u64 cpumerr, l2merr;
> +
> +	/* cpumerrsr_el1 */
> +	asm volatile("mrs %0, s3_1_c15_c2_2" : "=r" (cpumerr));
> +	asm volatile("msr s3_1_c15_c2_2, %0" :: "r" (0));

I think you've seen earlier comments on using the sys_reg macros for this. There were
versions of binutils out there that choke on this.

[...]

> +}
> +
> +static void cortex_arm64_edac_check(struct edac_device_ctl_info *edac_ctl)
> +{
> +	struct arm64_pvt *pvt = edac_ctl->pvt_info;
> +	call_single_data_t *csd;
> +	int cpu;
> +
> +	get_online_cpus();
> +	for_each_cpu_and(cpu, cpu_online_mask, &pvt->compat_mask) {
> +		csd = per_cpu_ptr(pvt->csd_check, cpu);
> +		csd->func = read_errors;
> +		csd->info = edac_ctl;
> +		csd->flags = 0;

> +		/* Read CPU L1/L2 errors */
> +		smp_call_function_single_async(cpu, csd);
> +		/* Wait until flags cleared */
> +		smp_cond_load_acquire(&csd->flags, !VAL);

Hmm. We end up waiting for each CPU to schedule something else. I can't see any reason we
can't sleep here.

Can't we use smp_call_function_many() here? It already considers cpu_online_mask, you'd
just need to deal with read_errors() being called in parallel with itself.

(concurrent calls into edac are one problem, but two CPUs read/writing the same L2
register could lead to double counting)


> +	}
> +	put_online_cpus();
> +}


> +static int cortex_arm64_edac_probe(struct platform_device *pdev)
> +{
> +	struct device_node *np, *dn = pdev->dev.of_node;
> +	struct edac_device_ctl_info *edac_ctl;
> +	struct device *dev = &pdev->dev;
> +	struct of_phandle_iterator it;
> +	struct arm64_pvt *pvt;
> +	int rc, cpu;
> +
> +	edac_ctl = edac_device_alloc_ctl_info(sizeof(*pvt), "cpu_cache",
> +					      1, "L", 2, 1, NULL, 0,
> +					      edac_device_alloc_index());

I used this series to test on Juno to poke the user-space interface:
This chokes on a big-little system as it can't register "cpu_cache" a second time.

I think we should try to make the topology look like the one in edac_device.h. This means
calling it 'cpu', and registering all of them up front.
On a big/little system the second probe() call would need to be careful.

I can have a go at this if you don't have a platform to hand.


(The 'L2-cache' thing in edac_device.h turns out to be impossible and the 'Lx' you've done
here is the most popular option. I'll post a patch to change the documentation to what
people are doing)


[...]

> +}


Thanks,

James