* [PATCH v5 0/2] Add L1 and L2 error detection for A53 and A57 @ 2021-04-01 11:06 Sascha Hauer 2021-04-01 11:06 ` [PATCH 1/2] drivers/edac: " Sascha Hauer 2021-04-01 11:06 ` [PATCH 2/2] dt-bindings: arm: cpus: Add edac-enabled property Sascha Hauer 0 siblings, 2 replies; 11+ messages in thread From: Sascha Hauer @ 2021-04-01 11:06 UTC (permalink / raw) To: linux-edac Cc: Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Rob Herring, Mark Rutland, Marc Zyngier, Sascha Hauer Hi, Resending this mainly because Marc Zyngier and Mark Rutland raised concerns about using implementation defined registers and I forgot to Cc them with the last version. This version, like v4 already, should fix these concerns. Looking forward to feedback. Sascha Changes since v4: - Rebase on v5.12-rc5 Changes since v3: - Add edac-enabled property to make EDAC support optional Changes since v2: - drop usage of virtual dt node (Robh) - use read_sysreg_s instead of open coded variant (James Morse) - separate error retrieving from error reporting - use smp_call_function_single rather than smp_call_function_single_async - make driver single instance and register all 'cpu' hierarchy up front once Changes since v1: - Split dt-binding into separate patch - Sort local function variables in reverse-xmas tree order - drop unnecessary comparison and make variable bool Sascha Hauer (2): drivers/edac: Add L1 and L2 error detection for A53 and A57 dt-bindings: arm: cpus: Add edac-enabled property .../devicetree/bindings/arm/cpus.yaml | 6 + drivers/edac/Kconfig | 6 + drivers/edac/Makefile | 1 + drivers/edac/cortex_arm64_l1_l2.c | 221 ++++++++++++++++++ 4 files changed, 234 insertions(+) create mode 100644 drivers/edac/cortex_arm64_l1_l2.c -- 2.29.2 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 1/2] drivers/edac: Add L1 and L2 error detection for A53 and A57 2021-04-01 11:06 [PATCH v5 0/2] Add L1 and L2 error detection for A53 and A57 Sascha Hauer @ 2021-04-01 11:06 ` Sascha Hauer 2021-04-02 10:06 ` Marc Zyngier 2021-04-01 11:06 ` [PATCH 2/2] dt-bindings: arm: cpus: Add edac-enabled property Sascha Hauer 1 sibling, 1 reply; 11+ messages in thread From: Sascha Hauer @ 2021-04-01 11:06 UTC (permalink / raw) To: linux-edac Cc: Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Rob Herring, Mark Rutland, Marc Zyngier, Sascha Hauer The Cortex A53 and A57 cores have error detection capabilities for the L1/L2 Caches, this patch adds a driver for them. Unfortunately there is no robust way to inject errors into the caches, so this driver doesn't contain any code to actually test it. It has been tested though with code taken from an older version of this driver found here: https://lkml.org/lkml/2018/3/14/1203. For reasons stated in this thread the error injection code is not suitable for mainline, so it is removed from the driver. Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> --- drivers/edac/Kconfig | 6 + drivers/edac/Makefile | 1 + drivers/edac/cortex_arm64_l1_l2.c | 218 ++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+) create mode 100644 drivers/edac/cortex_arm64_l1_l2.c diff --git a/drivers/edac/Kconfig b/drivers/edac/Kconfig index 27d0c4cdc58d..b038aed35e93 100644 --- a/drivers/edac/Kconfig +++ b/drivers/edac/Kconfig @@ -538,4 +538,10 @@ config EDAC_DMC520 Support for error detection and correction on the SoCs with ARM DMC-520 DRAM controller. +config EDAC_CORTEX_ARM64_L1_L2 + tristate "ARM Cortex A57/A53" + depends on ARM64 + help + Support for L1/L2 cache error detection on ARM Cortex A57 and A53. + endif # EDAC diff --git a/drivers/edac/Makefile b/drivers/edac/Makefile index 2d1641a27a28..5849d8bb32ae 100644 --- a/drivers/edac/Makefile +++ b/drivers/edac/Makefile @@ -84,3 +84,4 @@ obj-$(CONFIG_EDAC_QCOM) += qcom_edac.o obj-$(CONFIG_EDAC_ASPEED) += aspeed_edac.o obj-$(CONFIG_EDAC_BLUEFIELD) += bluefield_edac.o obj-$(CONFIG_EDAC_DMC520) += dmc520_edac.o +obj-$(CONFIG_EDAC_CORTEX_ARM64_L1_L2) += cortex_arm64_l1_l2.o diff --git a/drivers/edac/cortex_arm64_l1_l2.c b/drivers/edac/cortex_arm64_l1_l2.c new file mode 100644 index 000000000000..3b1e2f3ccab6 --- /dev/null +++ b/drivers/edac/cortex_arm64_l1_l2.c @@ -0,0 +1,218 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Cortex A57 and A53 EDAC L1 and L2 cache error detection + * + * Copyright (c) 2020 Pengutronix, Sascha Hauer <s.hauer@pengutronix.de> + * + * Based on Code from: + * Copyright (c) 2018, NXP Semiconductor + * Author: York Sun <york.sun@nxp.com> + * + */ + +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/of_device.h> +#include <linux/bitfield.h> +#include <asm/smp_plat.h> + +#include "edac_module.h" + +#define DRVNAME "cortex-arm64-edac" + +#define CPUMERRSR_EL1_RAMID GENMASK(30, 24) + +#define CPUMERRSR_EL1_VALID BIT(31) +#define CPUMERRSR_EL1_FATAL BIT(63) + +#define L1_I_TAG_RAM 0x00 +#define L1_I_DATA_RAM 0x01 +#define L1_D_TAG_RAM 0x08 +#define L1_D_DATA_RAM 0x09 +#define L1_D_DIRTY_RAM 0x14 +#define TLB_RAM 0x18 + +#define L2MERRSR_EL1_VALID BIT(31) +#define L2MERRSR_EL1_FATAL BIT(63) + +struct merrsr { + u64 cpumerr; + u64 l2merr; +}; + +#define MESSAGE_SIZE 64 + +#define SYS_CPUMERRSR_EL1 sys_reg(3, 1, 15, 2, 2) +#define SYS_L2MERRSR_EL1 sys_reg(3, 1, 15, 2, 3) + +static struct cpumask compat_mask; + +static void report_errors(struct edac_device_ctl_info *edac_ctl, int cpu, + struct merrsr *merrsr) +{ + char msg[MESSAGE_SIZE]; + u64 cpumerr = merrsr->cpumerr; + u64 l2merr = merrsr->l2merr; + + if (cpumerr & CPUMERRSR_EL1_VALID) { + const char *str; + bool fatal = cpumerr & CPUMERRSR_EL1_FATAL; + + switch (FIELD_GET(CPUMERRSR_EL1_RAMID, cpumerr)) { + case L1_I_TAG_RAM: + str = "L1-I Tag RAM"; + break; + case L1_I_DATA_RAM: + str = "L1-I Data RAM"; + break; + case L1_D_TAG_RAM: + str = "L1-D Tag RAM"; + break; + case L1_D_DATA_RAM: + str = "L1-D Data RAM"; + break; + case L1_D_DIRTY_RAM: + str = "L1 Dirty RAM"; + break; + case TLB_RAM: + str = "TLB RAM"; + break; + default: + str = "unknown"; + break; + } + + snprintf(msg, MESSAGE_SIZE, "%s %s error(s) on CPU %d", + str, fatal ? "fatal" : "correctable", cpu); + + if (fatal) + edac_device_handle_ue(edac_ctl, cpu, 0, msg); + else + edac_device_handle_ce(edac_ctl, cpu, 0, msg); + } + + if (l2merr & L2MERRSR_EL1_VALID) { + bool fatal = l2merr & L2MERRSR_EL1_FATAL; + + snprintf(msg, MESSAGE_SIZE, "L2 %s error(s) on CPU %d", + fatal ? "fatal" : "correctable", cpu); + if (fatal) + edac_device_handle_ue(edac_ctl, cpu, 1, msg); + else + edac_device_handle_ce(edac_ctl, cpu, 1, msg); + } +} + +static void read_errors(void *data) +{ + struct merrsr *merrsr = data; + + merrsr->cpumerr = read_sysreg_s(SYS_CPUMERRSR_EL1); + write_sysreg_s(0, SYS_CPUMERRSR_EL1); + merrsr->l2merr = read_sysreg_s(SYS_L2MERRSR_EL1); + write_sysreg_s(0, SYS_L2MERRSR_EL1); +} + +static void cortex_arm64_edac_check(struct edac_device_ctl_info *edac_ctl) +{ + struct merrsr merrsr; + int cpu; + + for_each_cpu_and(cpu, cpu_online_mask, &compat_mask) { + smp_call_function_single(cpu, read_errors, &merrsr, true); + report_errors(edac_ctl, cpu, &merrsr); + } +} + +static int cortex_arm64_edac_probe(struct platform_device *pdev) +{ + struct edac_device_ctl_info *edac_ctl; + struct device *dev = &pdev->dev; + int rc; + + edac_ctl = edac_device_alloc_ctl_info(0, "cpu", + num_possible_cpus(), "L", 2, 1, NULL, 0, + edac_device_alloc_index()); + if (!edac_ctl) + return -ENOMEM; + + edac_ctl->edac_check = cortex_arm64_edac_check; + edac_ctl->dev = dev; + edac_ctl->mod_name = dev_name(dev); + edac_ctl->dev_name = dev_name(dev); + edac_ctl->ctl_name = DRVNAME; + dev_set_drvdata(dev, edac_ctl); + + rc = edac_device_add_device(edac_ctl); + if (rc) + goto out_dev; + + return 0; + +out_dev: + edac_device_free_ctl_info(edac_ctl); + + return rc; +} + +static int cortex_arm64_edac_remove(struct platform_device *pdev) +{ + struct edac_device_ctl_info *edac_ctl = dev_get_drvdata(&pdev->dev); + + edac_device_del_device(edac_ctl->dev); + edac_device_free_ctl_info(edac_ctl); + + return 0; +} + +static const struct of_device_id cortex_arm64_edac_of_match[] = { + { .compatible = "arm,cortex-a53" }, + { .compatible = "arm,cortex-a57" }, + {} +}; +MODULE_DEVICE_TABLE(of, cortex_arm64_edac_of_match); + +static struct platform_driver cortex_arm64_edac_driver = { + .probe = cortex_arm64_edac_probe, + .remove = cortex_arm64_edac_remove, + .driver = { + .name = DRVNAME, + }, +}; + +static int __init cortex_arm64_edac_driver_init(void) +{ + struct device_node *np; + int cpu; + struct platform_device *pdev; + int err; + + for_each_possible_cpu(cpu) { + np = of_get_cpu_node(cpu, NULL); + + if (of_match_node(cortex_arm64_edac_of_match, np)) + cpumask_set_cpu(cpu, &compat_mask); + } + + if (cpumask_empty(&compat_mask)) + return 0; + + err = platform_driver_register(&cortex_arm64_edac_driver); + if (err) + return err; + + pdev = platform_device_register_simple(DRVNAME, -1, NULL, 0); + if (IS_ERR(pdev)) { + pr_err("failed to register cortex arm64 edac device\n"); + platform_driver_unregister(&cortex_arm64_edac_driver); + return PTR_ERR(pdev); + } + + return 0; +} + +device_initcall(cortex_arm64_edac_driver_init); + +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Sascha Hauer <s.hauer@pengutronix.de>"); +MODULE_DESCRIPTION("Cortex A57 and A53 L1 and L2 cache EDAC driver"); -- 2.29.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] drivers/edac: Add L1 and L2 error detection for A53 and A57 2021-04-01 11:06 ` [PATCH 1/2] drivers/edac: " Sascha Hauer @ 2021-04-02 10:06 ` Marc Zyngier 2021-04-15 10:15 ` Sascha Hauer 0 siblings, 1 reply; 11+ messages in thread From: Marc Zyngier @ 2021-04-02 10:06 UTC (permalink / raw) To: Sascha Hauer Cc: linux-edac, Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Rob Herring, Mark Rutland On Thu, 01 Apr 2021 12:06:14 +0100, Sascha Hauer <s.hauer@pengutronix.de> wrote: > > The Cortex A53 and A57 cores have error detection capabilities for the > L1/L2 Caches, this patch adds a driver for them. > > Unfortunately there is no robust way to inject errors into the caches, > so this driver doesn't contain any code to actually test it. It has > been tested though with code taken from an older version of this driver > found here: https://lkml.org/lkml/2018/3/14/1203. For reasons stated > in this thread the error injection code is not suitable for mainline, > so it is removed from the driver. > > Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> > --- > drivers/edac/Kconfig | 6 + > drivers/edac/Makefile | 1 + > drivers/edac/cortex_arm64_l1_l2.c | 218 ++++++++++++++++++++++++++++++ > 3 files changed, 225 insertions(+) > create mode 100644 drivers/edac/cortex_arm64_l1_l2.c > > diff --git a/drivers/edac/Kconfig b/drivers/edac/Kconfig > index 27d0c4cdc58d..b038aed35e93 100644 > --- a/drivers/edac/Kconfig > +++ b/drivers/edac/Kconfig > @@ -538,4 +538,10 @@ config EDAC_DMC520 > Support for error detection and correction on the > SoCs with ARM DMC-520 DRAM controller. > > +config EDAC_CORTEX_ARM64_L1_L2 > + tristate "ARM Cortex A57/A53" > + depends on ARM64 > + help > + Support for L1/L2 cache error detection on ARM Cortex A57 and A53. I went through the TRMs for a few other Cortex-A cores, and this feature looks more common than this comment suggests. At least A35 and A72 implement something similar (if not strictly identical), probably owing to their ancestry. > + > endif # EDAC > diff --git a/drivers/edac/Makefile b/drivers/edac/Makefile > index 2d1641a27a28..5849d8bb32ae 100644 > --- a/drivers/edac/Makefile > +++ b/drivers/edac/Makefile > @@ -84,3 +84,4 @@ obj-$(CONFIG_EDAC_QCOM) += qcom_edac.o > obj-$(CONFIG_EDAC_ASPEED) += aspeed_edac.o > obj-$(CONFIG_EDAC_BLUEFIELD) += bluefield_edac.o > obj-$(CONFIG_EDAC_DMC520) += dmc520_edac.o > +obj-$(CONFIG_EDAC_CORTEX_ARM64_L1_L2) += cortex_arm64_l1_l2.o > diff --git a/drivers/edac/cortex_arm64_l1_l2.c b/drivers/edac/cortex_arm64_l1_l2.c > new file mode 100644 > index 000000000000..3b1e2f3ccab6 > --- /dev/null > +++ b/drivers/edac/cortex_arm64_l1_l2.c > @@ -0,0 +1,218 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Cortex A57 and A53 EDAC L1 and L2 cache error detection > + * > + * Copyright (c) 2020 Pengutronix, Sascha Hauer <s.hauer@pengutronix.de> > + * > + * Based on Code from: > + * Copyright (c) 2018, NXP Semiconductor > + * Author: York Sun <york.sun@nxp.com> > + * > + */ > + > +#include <linux/module.h> > +#include <linux/of_platform.h> > +#include <linux/of_device.h> > +#include <linux/bitfield.h> > +#include <asm/smp_plat.h> > + > +#include "edac_module.h" > + > +#define DRVNAME "cortex-arm64-edac" > + > +#define CPUMERRSR_EL1_RAMID GENMASK(30, 24) > + > +#define CPUMERRSR_EL1_VALID BIT(31) > +#define CPUMERRSR_EL1_FATAL BIT(63) > + > +#define L1_I_TAG_RAM 0x00 > +#define L1_I_DATA_RAM 0x01 > +#define L1_D_TAG_RAM 0x08 > +#define L1_D_DATA_RAM 0x09 > +#define L1_D_DIRTY_RAM 0x14 > +#define TLB_RAM 0x18 > + > +#define L2MERRSR_EL1_VALID BIT(31) > +#define L2MERRSR_EL1_FATAL BIT(63) > + > +struct merrsr { > + u64 cpumerr; > + u64 l2merr; > +}; > + > +#define MESSAGE_SIZE 64 > + > +#define SYS_CPUMERRSR_EL1 sys_reg(3, 1, 15, 2, 2) > +#define SYS_L2MERRSR_EL1 sys_reg(3, 1, 15, 2, 3) > + > +static struct cpumask compat_mask; > + > +static void report_errors(struct edac_device_ctl_info *edac_ctl, int cpu, > + struct merrsr *merrsr) > +{ > + char msg[MESSAGE_SIZE]; > + u64 cpumerr = merrsr->cpumerr; > + u64 l2merr = merrsr->l2merr; > + > + if (cpumerr & CPUMERRSR_EL1_VALID) { > + const char *str; > + bool fatal = cpumerr & CPUMERRSR_EL1_FATAL; > + > + switch (FIELD_GET(CPUMERRSR_EL1_RAMID, cpumerr)) { > + case L1_I_TAG_RAM: > + str = "L1-I Tag RAM"; > + break; > + case L1_I_DATA_RAM: > + str = "L1-I Data RAM"; > + break; > + case L1_D_TAG_RAM: > + str = "L1-D Tag RAM"; > + break; > + case L1_D_DATA_RAM: > + str = "L1-D Data RAM"; > + break; > + case L1_D_DIRTY_RAM: > + str = "L1 Dirty RAM"; > + break; > + case TLB_RAM: > + str = "TLB RAM"; > + break; > + default: > + str = "unknown"; > + break; > + } > + > + snprintf(msg, MESSAGE_SIZE, "%s %s error(s) on CPU %d", > + str, fatal ? "fatal" : "correctable", cpu); > + > + if (fatal) > + edac_device_handle_ue(edac_ctl, cpu, 0, msg); > + else > + edac_device_handle_ce(edac_ctl, cpu, 0, msg); > + } > + > + if (l2merr & L2MERRSR_EL1_VALID) { > + bool fatal = l2merr & L2MERRSR_EL1_FATAL; > + > + snprintf(msg, MESSAGE_SIZE, "L2 %s error(s) on CPU %d", > + fatal ? "fatal" : "correctable", cpu); The shared nature of the L2 makes the CPU it has been detected on pretty much irrelevant. What you really want here is the CPUID+Way that is in the register data. > + if (fatal) > + edac_device_handle_ue(edac_ctl, cpu, 1, msg); > + else > + edac_device_handle_ce(edac_ctl, cpu, 1, msg); > + } > +} > + > +static void read_errors(void *data) > +{ > + struct merrsr *merrsr = data; > + > + merrsr->cpumerr = read_sysreg_s(SYS_CPUMERRSR_EL1); > + write_sysreg_s(0, SYS_CPUMERRSR_EL1); > + merrsr->l2merr = read_sysreg_s(SYS_L2MERRSR_EL1); > + write_sysreg_s(0, SYS_L2MERRSR_EL1); If an error happens between read and write, you lose it. That's not great. You could improve things by only writing 0 if you have found an error. You probably also need an isb after the write if you want it to take effect in a timely manner. I'm also not sure of how valuable it is to probe for L2 errors on each CPU, given that it is shared with up to 3 other cores. You probably want to use the cache topology information for this. Thanks, M. -- Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] drivers/edac: Add L1 and L2 error detection for A53 and A57 2021-04-02 10:06 ` Marc Zyngier @ 2021-04-15 10:15 ` Sascha Hauer 0 siblings, 0 replies; 11+ messages in thread From: Sascha Hauer @ 2021-04-15 10:15 UTC (permalink / raw) To: Marc Zyngier Cc: linux-edac, Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Rob Herring, Mark Rutland Hi Marc, Thanks for the input. On Fri, Apr 02, 2021 at 11:06:56AM +0100, Marc Zyngier wrote: > > +config EDAC_CORTEX_ARM64_L1_L2 > > + tristate "ARM Cortex A57/A53" > > + depends on ARM64 > > + help > > + Support for L1/L2 cache error detection on ARM Cortex A57 and A53. > > I went through the TRMs for a few other Cortex-A cores, and this > feature looks more common than this comment suggests. At least A35 and > A72 implement something similar (if not strictly identical), probably > owing to their ancestry. Ok, I'll add these to the description. > > + } > > + > > + snprintf(msg, MESSAGE_SIZE, "%s %s error(s) on CPU %d", > > + str, fatal ? "fatal" : "correctable", cpu); > > + > > + if (fatal) > > + edac_device_handle_ue(edac_ctl, cpu, 0, msg); > > + else > > + edac_device_handle_ce(edac_ctl, cpu, 0, msg); > > + } > > + > > + if (l2merr & L2MERRSR_EL1_VALID) { > > + bool fatal = l2merr & L2MERRSR_EL1_FATAL; > > + > > + snprintf(msg, MESSAGE_SIZE, "L2 %s error(s) on CPU %d", > > + fatal ? "fatal" : "correctable", cpu); > > The shared nature of the L2 makes the CPU it has been detected on > pretty much irrelevant. What you really want here is the CPUID+Way > that is in the register data. You are right. For the next round I added some more code to decode the CPUID/Way field. What's still missing then is information which L2 cache has errors in case there is more than one. I wonder if we should add get_cpu_cacheinfo(cpu)->id to the message or if there's more to it. > > > + if (fatal) > > + edac_device_handle_ue(edac_ctl, cpu, 1, msg); > > + else > > + edac_device_handle_ce(edac_ctl, cpu, 1, msg); > > + } > > +} > > + > > +static void read_errors(void *data) > > +{ > > + struct merrsr *merrsr = data; > > + > > + merrsr->cpumerr = read_sysreg_s(SYS_CPUMERRSR_EL1); > > + write_sysreg_s(0, SYS_CPUMERRSR_EL1); > > + merrsr->l2merr = read_sysreg_s(SYS_L2MERRSR_EL1); > > + write_sysreg_s(0, SYS_L2MERRSR_EL1); > > If an error happens between read and write, you lose it. That's not > great. You could improve things by only writing 0 if you have found an > error. You probably also need an isb after the write if you want it to > take effect in a timely manner. Ok, will change. > > I'm also not sure of how valuable it is to probe for L2 errors on each > CPU, given that it is shared with up to 3 other cores. You probably > want to use the cache topology information for this. I have no idea how l2merr is implemented. When there is only one register for all CPUs sharing the same L2 cache then it shouldn't do any harm to read it more than once. The expensive part is probably to schedule a function on all CPUs, and we have to do that anyway to read the L1 cache errors. Sascha -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 2/2] dt-bindings: arm: cpus: Add edac-enabled property 2021-04-01 11:06 [PATCH v5 0/2] Add L1 and L2 error detection for A53 and A57 Sascha Hauer 2021-04-01 11:06 ` [PATCH 1/2] drivers/edac: " Sascha Hauer @ 2021-04-01 11:06 ` Sascha Hauer 2021-04-01 15:37 ` Marc Zyngier 1 sibling, 1 reply; 11+ messages in thread From: Sascha Hauer @ 2021-04-01 11:06 UTC (permalink / raw) To: linux-edac Cc: Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Rob Herring, Mark Rutland, Marc Zyngier, Sascha Hauer Some CPUs like the Cortex-A53 and Cortex-A57 have Error Detection And Correction (EDAC) support on their L1 and L2 caches. This is implemented in implementation defined registers, so usage of this functionality is not safe in virtualized environments or when EL3 already uses these registers. This patch adds a edac-enabled flag which can be explicitly set when EDAC can be used. Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> --- Documentation/devicetree/bindings/arm/cpus.yaml | 6 ++++++ drivers/edac/cortex_arm64_l1_l2.c | 7 +++++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/Documentation/devicetree/bindings/arm/cpus.yaml b/Documentation/devicetree/bindings/arm/cpus.yaml index 26b886b20b27..74be19c0544a 100644 --- a/Documentation/devicetree/bindings/arm/cpus.yaml +++ b/Documentation/devicetree/bindings/arm/cpus.yaml @@ -270,6 +270,12 @@ properties: For PSCI based platforms, the name corresponding to the index of the PSCI PM domain provider, must be "psci". + edac-enabled: + $ref: '/schemas/types.yaml#/definitions/flag' + description: + Some CPUs support Error Detection And Correction (EDAC) on their L1 and + L2 caches. This flag marks this function as usable. + qcom,saw: $ref: '/schemas/types.yaml#/definitions/phandle' description: | diff --git a/drivers/edac/cortex_arm64_l1_l2.c b/drivers/edac/cortex_arm64_l1_l2.c index 3b1e2f3ccab6..6d5355bae80c 100644 --- a/drivers/edac/cortex_arm64_l1_l2.c +++ b/drivers/edac/cortex_arm64_l1_l2.c @@ -190,8 +190,11 @@ static int __init cortex_arm64_edac_driver_init(void) for_each_possible_cpu(cpu) { np = of_get_cpu_node(cpu, NULL); - if (of_match_node(cortex_arm64_edac_of_match, np)) - cpumask_set_cpu(cpu, &compat_mask); + if (!of_match_node(cortex_arm64_edac_of_match, np)) + continue; + if (!of_property_read_bool(np, "edac-enabled")) + continue; + cpumask_set_cpu(cpu, &compat_mask); } if (cpumask_empty(&compat_mask)) -- 2.29.2 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 2/2] dt-bindings: arm: cpus: Add edac-enabled property 2021-04-01 11:06 ` [PATCH 2/2] dt-bindings: arm: cpus: Add edac-enabled property Sascha Hauer @ 2021-04-01 15:37 ` Marc Zyngier 0 siblings, 0 replies; 11+ messages in thread From: Marc Zyngier @ 2021-04-01 15:37 UTC (permalink / raw) To: Sascha Hauer Cc: linux-edac, Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Rob Herring, Mark Rutland On Thu, 01 Apr 2021 12:06:15 +0100, Sascha Hauer <s.hauer@pengutronix.de> wrote: > > Some CPUs like the Cortex-A53 and Cortex-A57 have Error Detection And > Correction (EDAC) support on their L1 and L2 caches. This is implemented > in implementation defined registers, so usage of this functionality is > not safe in virtualized environments or when EL3 already uses these > registers. > This patch adds a edac-enabled flag which can be explicitly set when > EDAC can be used. > > Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> > --- > Documentation/devicetree/bindings/arm/cpus.yaml | 6 ++++++ > drivers/edac/cortex_arm64_l1_l2.c | 7 +++++-- > 2 files changed, 11 insertions(+), 2 deletions(-) > > diff --git a/Documentation/devicetree/bindings/arm/cpus.yaml b/Documentation/devicetree/bindings/arm/cpus.yaml > index 26b886b20b27..74be19c0544a 100644 > --- a/Documentation/devicetree/bindings/arm/cpus.yaml > +++ b/Documentation/devicetree/bindings/arm/cpus.yaml > @@ -270,6 +270,12 @@ properties: > For PSCI based platforms, the name corresponding to the index of the PSCI > PM domain provider, must be "psci". > > + edac-enabled: > + $ref: '/schemas/types.yaml#/definitions/flag' > + description: > + Some CPUs support Error Detection And Correction (EDAC) on their L1 and > + L2 caches. This flag marks this function as usable. > + > qcom,saw: > $ref: '/schemas/types.yaml#/definitions/phandle' > description: | > diff --git a/drivers/edac/cortex_arm64_l1_l2.c b/drivers/edac/cortex_arm64_l1_l2.c > index 3b1e2f3ccab6..6d5355bae80c 100644 > --- a/drivers/edac/cortex_arm64_l1_l2.c > +++ b/drivers/edac/cortex_arm64_l1_l2.c > @@ -190,8 +190,11 @@ static int __init cortex_arm64_edac_driver_init(void) > for_each_possible_cpu(cpu) { > np = of_get_cpu_node(cpu, NULL); > > - if (of_match_node(cortex_arm64_edac_of_match, np)) > - cpumask_set_cpu(cpu, &compat_mask); > + if (!of_match_node(cortex_arm64_edac_of_match, np)) > + continue; > + if (!of_property_read_bool(np, "edac-enabled")) > + continue; > + cpumask_set_cpu(cpu, &compat_mask); > } > > if (cpumask_empty(&compat_mask)) This last hunk must be part of the initial patch. Otherwise, it breaks exactly as described in the commit message. Thanks, M. -- Without deviation from the norm, progress is not possible. ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH iv4 0/2] Add L1 and L2 error detection for A53 and A57 @ 2021-02-01 11:57 Sascha Hauer 2021-02-01 11:57 ` [PATCH 1/2] drivers/edac: " Sascha Hauer 0 siblings, 1 reply; 11+ messages in thread From: Sascha Hauer @ 2021-02-01 11:57 UTC (permalink / raw) To: linux-edac Cc: Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Rob Herring, Sascha Hauer Hi All, As mentioned by Marc and Mark usage of the implementation defined registers is not generally safe, they can't be used in virtualized environments or when EL3 already uses the same registers. This is probably the last attempt to get this upstream, I added an additional property to the CPU device nodes to be set explicitly when using these registers is safe and desired. Sascha Changes since v3: - Add edac-enabled property to make EDAC support optional Changes since v2: - drop usage of virtual dt node (Robh) - use read_sysreg_s instead of open coded variant (James Morse) - separate error retrieving from error reporting - use smp_call_function_single rather than smp_call_function_single_async - make driver single instance and register all 'cpu' hierarchy up front once Changes since v1: - Split dt-binding into separate patch - Sort local function variables in reverse-xmas tree order - drop unnecessary comparison and make variable bool Sascha Hauer (2): drivers/edac: Add L1 and L2 error detection for A53 and A57 dt-bindings: arm: cpus: Add edac-enabled property .../devicetree/bindings/arm/cpus.yaml | 6 + drivers/edac/Kconfig | 6 + drivers/edac/Makefile | 1 + drivers/edac/cortex_arm64_l1_l2.c | 221 ++++++++++++++++++ 4 files changed, 234 insertions(+) create mode 100644 drivers/edac/cortex_arm64_l1_l2.c -- 2.20.1 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 1/2] drivers/edac: Add L1 and L2 error detection for A53 and A57 2021-02-01 11:57 [PATCH iv4 0/2] Add L1 and L2 error detection for A53 and A57 Sascha Hauer @ 2021-02-01 11:57 ` Sascha Hauer 0 siblings, 0 replies; 11+ messages in thread From: Sascha Hauer @ 2021-02-01 11:57 UTC (permalink / raw) To: linux-edac Cc: Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Rob Herring, Sascha Hauer The Cortex A53 and A57 cores have error detection capabilities for the L1/L2 Caches, this patch adds a driver for them. Unfortunately there is no robust way to inject errors into the caches, so this driver doesn't contain any code to actually test it. It has been tested though with code taken from an older version of this driver found here: https://lkml.org/lkml/2018/3/14/1203. For reasons stated in this thread the error injection code is not suitable for mainline, so it is removed from the driver. Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> --- drivers/edac/Kconfig | 6 + drivers/edac/Makefile | 1 + drivers/edac/cortex_arm64_l1_l2.c | 218 ++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+) create mode 100644 drivers/edac/cortex_arm64_l1_l2.c diff --git a/drivers/edac/Kconfig b/drivers/edac/Kconfig index 81c42664f21b..116ad56534a4 100644 --- a/drivers/edac/Kconfig +++ b/drivers/edac/Kconfig @@ -539,4 +539,10 @@ config EDAC_DMC520 Support for error detection and correction on the SoCs with ARM DMC-520 DRAM controller. +config EDAC_CORTEX_ARM64_L1_L2 + tristate "ARM Cortex A57/A53" + depends on ARM64 + help + Support for L1/L2 cache error detection on ARM Cortex A57 and A53. + endif # EDAC diff --git a/drivers/edac/Makefile b/drivers/edac/Makefile index 464d3d8d850a..a324fbc1693e 100644 --- a/drivers/edac/Makefile +++ b/drivers/edac/Makefile @@ -89,3 +89,4 @@ obj-$(CONFIG_EDAC_QCOM) += qcom_edac.o obj-$(CONFIG_EDAC_ASPEED) += aspeed_edac.o obj-$(CONFIG_EDAC_BLUEFIELD) += bluefield_edac.o obj-$(CONFIG_EDAC_DMC520) += dmc520_edac.o +obj-$(CONFIG_EDAC_CORTEX_ARM64_L1_L2) += cortex_arm64_l1_l2.o diff --git a/drivers/edac/cortex_arm64_l1_l2.c b/drivers/edac/cortex_arm64_l1_l2.c new file mode 100644 index 000000000000..3b1e2f3ccab6 --- /dev/null +++ b/drivers/edac/cortex_arm64_l1_l2.c @@ -0,0 +1,218 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Cortex A57 and A53 EDAC L1 and L2 cache error detection + * + * Copyright (c) 2020 Pengutronix, Sascha Hauer <s.hauer@pengutronix.de> + * + * Based on Code from: + * Copyright (c) 2018, NXP Semiconductor + * Author: York Sun <york.sun@nxp.com> + * + */ + +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/of_device.h> +#include <linux/bitfield.h> +#include <asm/smp_plat.h> + +#include "edac_module.h" + +#define DRVNAME "cortex-arm64-edac" + +#define CPUMERRSR_EL1_RAMID GENMASK(30, 24) + +#define CPUMERRSR_EL1_VALID BIT(31) +#define CPUMERRSR_EL1_FATAL BIT(63) + +#define L1_I_TAG_RAM 0x00 +#define L1_I_DATA_RAM 0x01 +#define L1_D_TAG_RAM 0x08 +#define L1_D_DATA_RAM 0x09 +#define L1_D_DIRTY_RAM 0x14 +#define TLB_RAM 0x18 + +#define L2MERRSR_EL1_VALID BIT(31) +#define L2MERRSR_EL1_FATAL BIT(63) + +struct merrsr { + u64 cpumerr; + u64 l2merr; +}; + +#define MESSAGE_SIZE 64 + +#define SYS_CPUMERRSR_EL1 sys_reg(3, 1, 15, 2, 2) +#define SYS_L2MERRSR_EL1 sys_reg(3, 1, 15, 2, 3) + +static struct cpumask compat_mask; + +static void report_errors(struct edac_device_ctl_info *edac_ctl, int cpu, + struct merrsr *merrsr) +{ + char msg[MESSAGE_SIZE]; + u64 cpumerr = merrsr->cpumerr; + u64 l2merr = merrsr->l2merr; + + if (cpumerr & CPUMERRSR_EL1_VALID) { + const char *str; + bool fatal = cpumerr & CPUMERRSR_EL1_FATAL; + + switch (FIELD_GET(CPUMERRSR_EL1_RAMID, cpumerr)) { + case L1_I_TAG_RAM: + str = "L1-I Tag RAM"; + break; + case L1_I_DATA_RAM: + str = "L1-I Data RAM"; + break; + case L1_D_TAG_RAM: + str = "L1-D Tag RAM"; + break; + case L1_D_DATA_RAM: + str = "L1-D Data RAM"; + break; + case L1_D_DIRTY_RAM: + str = "L1 Dirty RAM"; + break; + case TLB_RAM: + str = "TLB RAM"; + break; + default: + str = "unknown"; + break; + } + + snprintf(msg, MESSAGE_SIZE, "%s %s error(s) on CPU %d", + str, fatal ? "fatal" : "correctable", cpu); + + if (fatal) + edac_device_handle_ue(edac_ctl, cpu, 0, msg); + else + edac_device_handle_ce(edac_ctl, cpu, 0, msg); + } + + if (l2merr & L2MERRSR_EL1_VALID) { + bool fatal = l2merr & L2MERRSR_EL1_FATAL; + + snprintf(msg, MESSAGE_SIZE, "L2 %s error(s) on CPU %d", + fatal ? "fatal" : "correctable", cpu); + if (fatal) + edac_device_handle_ue(edac_ctl, cpu, 1, msg); + else + edac_device_handle_ce(edac_ctl, cpu, 1, msg); + } +} + +static void read_errors(void *data) +{ + struct merrsr *merrsr = data; + + merrsr->cpumerr = read_sysreg_s(SYS_CPUMERRSR_EL1); + write_sysreg_s(0, SYS_CPUMERRSR_EL1); + merrsr->l2merr = read_sysreg_s(SYS_L2MERRSR_EL1); + write_sysreg_s(0, SYS_L2MERRSR_EL1); +} + +static void cortex_arm64_edac_check(struct edac_device_ctl_info *edac_ctl) +{ + struct merrsr merrsr; + int cpu; + + for_each_cpu_and(cpu, cpu_online_mask, &compat_mask) { + smp_call_function_single(cpu, read_errors, &merrsr, true); + report_errors(edac_ctl, cpu, &merrsr); + } +} + +static int cortex_arm64_edac_probe(struct platform_device *pdev) +{ + struct edac_device_ctl_info *edac_ctl; + struct device *dev = &pdev->dev; + int rc; + + edac_ctl = edac_device_alloc_ctl_info(0, "cpu", + num_possible_cpus(), "L", 2, 1, NULL, 0, + edac_device_alloc_index()); + if (!edac_ctl) + return -ENOMEM; + + edac_ctl->edac_check = cortex_arm64_edac_check; + edac_ctl->dev = dev; + edac_ctl->mod_name = dev_name(dev); + edac_ctl->dev_name = dev_name(dev); + edac_ctl->ctl_name = DRVNAME; + dev_set_drvdata(dev, edac_ctl); + + rc = edac_device_add_device(edac_ctl); + if (rc) + goto out_dev; + + return 0; + +out_dev: + edac_device_free_ctl_info(edac_ctl); + + return rc; +} + +static int cortex_arm64_edac_remove(struct platform_device *pdev) +{ + struct edac_device_ctl_info *edac_ctl = dev_get_drvdata(&pdev->dev); + + edac_device_del_device(edac_ctl->dev); + edac_device_free_ctl_info(edac_ctl); + + return 0; +} + +static const struct of_device_id cortex_arm64_edac_of_match[] = { + { .compatible = "arm,cortex-a53" }, + { .compatible = "arm,cortex-a57" }, + {} +}; +MODULE_DEVICE_TABLE(of, cortex_arm64_edac_of_match); + +static struct platform_driver cortex_arm64_edac_driver = { + .probe = cortex_arm64_edac_probe, + .remove = cortex_arm64_edac_remove, + .driver = { + .name = DRVNAME, + }, +}; + +static int __init cortex_arm64_edac_driver_init(void) +{ + struct device_node *np; + int cpu; + struct platform_device *pdev; + int err; + + for_each_possible_cpu(cpu) { + np = of_get_cpu_node(cpu, NULL); + + if (of_match_node(cortex_arm64_edac_of_match, np)) + cpumask_set_cpu(cpu, &compat_mask); + } + + if (cpumask_empty(&compat_mask)) + return 0; + + err = platform_driver_register(&cortex_arm64_edac_driver); + if (err) + return err; + + pdev = platform_device_register_simple(DRVNAME, -1, NULL, 0); + if (IS_ERR(pdev)) { + pr_err("failed to register cortex arm64 edac device\n"); + platform_driver_unregister(&cortex_arm64_edac_driver); + return PTR_ERR(pdev); + } + + return 0; +} + +device_initcall(cortex_arm64_edac_driver_init); + +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Sascha Hauer <s.hauer@pengutronix.de>"); +MODULE_DESCRIPTION("Cortex A57 and A53 L1 and L2 cache EDAC driver"); -- 2.20.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 0/2] Add L1 and L2 error detection for A53 and A57 @ 2020-08-13 7:57 Sascha Hauer 2020-08-13 7:57 ` [PATCH 1/2] drivers/edac: " Sascha Hauer 0 siblings, 1 reply; 11+ messages in thread From: Sascha Hauer @ 2020-08-13 7:57 UTC (permalink / raw) To: linux-edac Cc: Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Sascha Hauer This driver is based on an earlier version from York Sun which can be found here: https://lkml.org/lkml/2018/3/14/1203. At that time the conclusion was that this driver is not suitable for mainline as it used IMPLEMENTATION DEFINED CPU registers and also NXP specific SMC calls. All this was used for the error injection only, for error reporting it is not needed. This is another try to get this driver to mainline. All error injection code has been removed (though it has initially been used to test this driver on an i.MX8 SoC), what's left is unfortunately not testable, but also doesn't contain none of the doubtful code anymore. Sascha Hauer (1): drivers/edac: Add L1 and L2 error detection for A53 and A57 York Sun (1): arm64: dts: ls104x: Add L1/L2 cache edac node .../bindings/edac/arm,cortex-a5x-edac.yaml | 32 +++ .../arm64/boot/dts/freescale/fsl-ls1043a.dtsi | 5 + .../arm64/boot/dts/freescale/fsl-ls1046a.dtsi | 5 + drivers/edac/Kconfig | 6 + drivers/edac/Makefile | 1 + drivers/edac/cortex_arm64_l1_l2.c | 208 ++++++++++++++++++ 6 files changed, 257 insertions(+) create mode 100644 Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml create mode 100644 drivers/edac/cortex_arm64_l1_l2.c -- 2.28.0 ^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 1/2] drivers/edac: Add L1 and L2 error detection for A53 and A57 2020-08-13 7:57 [PATCH 0/2] " Sascha Hauer @ 2020-08-13 7:57 ` Sascha Hauer 2020-08-26 8:41 ` Borislav Petkov 0 siblings, 1 reply; 11+ messages in thread From: Sascha Hauer @ 2020-08-13 7:57 UTC (permalink / raw) To: linux-edac Cc: Borislav Petkov, Mauro Carvalho Chehab, Tony Luck, James Morse, Robert Richter, York Sun, kernel, linux-arm-kernel, Sascha Hauer The Cortex A53 and A57 cores have error detection capabilities for the L1/L2 Caches, this patch adds a driver for them. Unfortunately there is no robust way to inject errors into the caches, so this driver doesn't contain any code to actually test it. It has been tested though with code taken from an older version of this driver found here: https://lkml.org/lkml/2018/3/14/1203. For reasons stated in this thread the error injection code is not suitable for mainline, so it is removed from the driver. Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> --- .../bindings/edac/arm,cortex-a5x-edac.yaml | 32 +++ drivers/edac/Kconfig | 6 + drivers/edac/Makefile | 1 + drivers/edac/cortex_arm64_l1_l2.c | 208 ++++++++++++++++++ 4 files changed, 247 insertions(+) create mode 100644 Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml create mode 100644 drivers/edac/cortex_arm64_l1_l2.c diff --git a/Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml b/Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml new file mode 100644 index 0000000000000..de9325b688a02 --- /dev/null +++ b/Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/edac/arm,cortex-a5x-edac.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: ARM cortex A53/A57 EDAC bindings + +description: |+ + This contains the binding to support error detection for the L1 and L2 caches + on ARM Cortex A53 and A57 cores. + +properties: + compatible: + items: + - const: arm,cortex-a53-edac + - const: arm,cortex-a57-edac + + cpus: + minItems: 1 + description: phandles to the cpu nodes this device handles + +required: + - compatible + - cpus + +examples: + - | + edac-a53 { + compatible = "arm,cortex-a53-edac"; + cpus = <&cpu0 &cpu1 &cpu2 &cpu3>; + }; diff --git a/drivers/edac/Kconfig b/drivers/edac/Kconfig index 7b6ec3014ba2d..19501e5529edd 100644 --- a/drivers/edac/Kconfig +++ b/drivers/edac/Kconfig @@ -530,4 +530,10 @@ config EDAC_DMC520 Support for error detection and correction on the SoCs with ARM DMC-520 DRAM controller. +config EDAC_CORTEX_ARM64_L1_L2 + tristate "ARM Cortex A57/A53" + depends on ARM64 + help + Support for error detection on ARM Cortex A57 and A53. + endif # EDAC diff --git a/drivers/edac/Makefile b/drivers/edac/Makefile index 269e15118ceac..3edba6bea3507 100644 --- a/drivers/edac/Makefile +++ b/drivers/edac/Makefile @@ -88,3 +88,4 @@ obj-$(CONFIG_EDAC_QCOM) += qcom_edac.o obj-$(CONFIG_EDAC_ASPEED) += aspeed_edac.o obj-$(CONFIG_EDAC_BLUEFIELD) += bluefield_edac.o obj-$(CONFIG_EDAC_DMC520) += dmc520_edac.o +obj-$(CONFIG_EDAC_CORTEX_ARM64_L1_L2) += cortex_arm64_l1_l2.o diff --git a/drivers/edac/cortex_arm64_l1_l2.c b/drivers/edac/cortex_arm64_l1_l2.c new file mode 100644 index 0000000000000..12b49a5af3c0f --- /dev/null +++ b/drivers/edac/cortex_arm64_l1_l2.c @@ -0,0 +1,208 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Cortex A57 and A53 EDAC L1 and L2 cache error detection + * + * Copyright (c) 2020 Pengutronix, Sascha Hauer <s.hauer@pengutronix.de> + * + * Based on Code from: + * Copyright (c) 2018, NXP Semiconductor + * Author: York Sun <york.sun@nxp.com> + * + */ + +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/of_device.h> +#include <linux/bitfield.h> +#include <asm/smp_plat.h> + +#include "edac_module.h" + +#define EDAC_MOD_STR "cortex_edac_l1_l2" + +#define CPUMERRSR_EL1_RAMID GENMASK(30, 24) + +#define CPUMERRSR_EL1_VALID BIT(31) +#define CPUMERRSR_EL1_FATAL BIT(63) + +#define L1_I_TAG_RAM 0x00 +#define L1_I_DATA_RAM 0x01 +#define L1_D_TAG_RAM 0x08 +#define L1_D_DATA_RAM 0x09 +#define L1_D_DIRTY_RAM 0x14 +#define TLB_RAM 0x18 + +#define L2MERRSR_EL1_VALID BIT(31) +#define L2MERRSR_EL1_FATAL BIT(63) + +struct arm64_pvt { + call_single_data_t __percpu *csd_check; + struct cpumask compat_mask; +}; + +#define MESSAGE_SIZE 64 + +static void read_errors(void *data) +{ + u64 cpumerr, l2merr; + int cpu = smp_processor_id(); + char msg[MESSAGE_SIZE]; + struct edac_device_ctl_info *edac_ctl = data; + + /* cpumerrsr_el1 */ + asm volatile("mrs %0, s3_1_c15_c2_2" : "=r" (cpumerr)); + asm volatile("msr s3_1_c15_c2_2, %0" :: "r" (0)); + + if (cpumerr & CPUMERRSR_EL1_VALID) { + const char *str; + int fatal = (cpumerr & CPUMERRSR_EL1_FATAL) != 0; + + switch (FIELD_GET(CPUMERRSR_EL1_RAMID, cpumerr)) { + case L1_I_TAG_RAM: + str = "L1-I Tag RAM"; + break; + case L1_I_DATA_RAM: + str = "L1-I Data RAM"; + break; + case L1_D_TAG_RAM: + str = "L1-D Tag RAM"; + break; + case L1_D_DATA_RAM: + str = "L1-D Data RAM"; + break; + case L1_D_DIRTY_RAM: + str = "L1 Dirty RAM"; + break; + case TLB_RAM: + str = "TLB RAM"; + break; + default: + str = "unknown"; + break; + } + + snprintf(msg, MESSAGE_SIZE, "%s %s error(s) on CPU %d", + str, fatal ? "fatal" : "correctable", cpu); + + if (fatal) + edac_device_handle_ue(edac_ctl, 0, 0, msg); + else + edac_device_handle_ce(edac_ctl, 0, 0, msg); + } + + /* l2merrsr_el1 */ + asm volatile("mrs %0, s3_1_c15_c2_3" : "=r" (l2merr)); + asm volatile("msr s3_1_c15_c2_3, %0" :: "r" (0)); + + if (l2merr & L2MERRSR_EL1_VALID) { + int fatal = (l2merr & L2MERRSR_EL1_FATAL) != 0; + + snprintf(msg, MESSAGE_SIZE, "L2 %s error(s) on CPU %d", + fatal ? "fatal" : "correctable", cpu); + if (fatal) + edac_device_handle_ue(edac_ctl, 0, 1, msg); + else + edac_device_handle_ce(edac_ctl, 0, 1, msg); + } +} + +static void cortex_arm64_edac_check(struct edac_device_ctl_info *edac_ctl) +{ + struct arm64_pvt *pvt = edac_ctl->pvt_info; + int cpu; + call_single_data_t *csd; + + get_online_cpus(); + for_each_cpu_and(cpu, cpu_online_mask, &pvt->compat_mask) { + csd = per_cpu_ptr(pvt->csd_check, cpu); + csd->func = read_errors; + csd->info = edac_ctl; + csd->flags = 0; + /* Read CPU L1/L2 errors */ + smp_call_function_single_async(cpu, csd); + /* Wait until flags cleared */ + smp_cond_load_acquire(&csd->flags, !VAL); + } + put_online_cpus(); +} + +static int cortex_arm64_edac_probe(struct platform_device *pdev) +{ + int rc, cpu; + struct edac_device_ctl_info *edac_ctl; + struct device *dev = &pdev->dev; + struct device_node *np, *dn = pdev->dev.of_node; + struct of_phandle_iterator it; + struct arm64_pvt *pvt; + + edac_ctl = edac_device_alloc_ctl_info(sizeof(*pvt), "cpu_cache", + 1, "L", 2, 1, NULL, 0, + edac_device_alloc_index()); + if (!edac_ctl) + return -ENOMEM; + + pvt = edac_ctl->pvt_info; + pvt->csd_check = devm_alloc_percpu(dev, call_single_data_t); + if (!pvt->csd_check) { + rc = -ENOMEM; + goto out_dev; + } + + of_for_each_phandle(&it, rc, dn, "cpus", NULL, 0) { + np = it.node; + cpu = of_cpu_node_to_id(np); + if (cpu < 0) + continue; + + cpumask_set_cpu(cpu, &pvt->compat_mask); + } + + edac_ctl->edac_check = cortex_arm64_edac_check; + edac_ctl->dev = dev; + edac_ctl->mod_name = dev_name(dev); + edac_ctl->dev_name = dev_name(dev); + edac_ctl->ctl_name = EDAC_MOD_STR; + dev_set_drvdata(dev, edac_ctl); + + rc = edac_device_add_device(edac_ctl); + if (rc) + goto out_dev; + + return 0; + +out_dev: + edac_device_free_ctl_info(edac_ctl); + + return rc; +} + +static int cortex_arm64_edac_remove(struct platform_device *pdev) +{ + struct edac_device_ctl_info *edac_ctl = dev_get_drvdata(&pdev->dev); + + edac_device_del_device(edac_ctl->dev); + edac_device_free_ctl_info(edac_ctl); + + return 0; +} + +static const struct of_device_id cortex_arm64_edac_of_match[] = { + { .compatible = "arm,cortex-a53-edac" }, + { .compatible = "arm,cortex-a57-edac" }, + {} +}; +MODULE_DEVICE_TABLE(of, cortex_arm64_edac_of_match); + +static struct platform_driver cortex_arm64_edac_driver = { + .probe = cortex_arm64_edac_probe, + .remove = cortex_arm64_edac_remove, + .driver = { + .name = EDAC_MOD_STR, + .of_match_table = cortex_arm64_edac_of_match, + }, +}; +module_platform_driver(cortex_arm64_edac_driver); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Sascha Hauer <s.hauer@pengutronix.de>"); +MODULE_DESCRIPTION("Cortex A57 and A53 L1 and L2 cache EDAC driver"); -- 2.28.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] drivers/edac: Add L1 and L2 error detection for A53 and A57 2020-08-13 7:57 ` [PATCH 1/2] drivers/edac: " Sascha Hauer @ 2020-08-26 8:41 ` Borislav Petkov 2020-10-13 11:13 ` Sascha Hauer 0 siblings, 1 reply; 11+ messages in thread From: Borislav Petkov @ 2020-08-26 8:41 UTC (permalink / raw) To: Sascha Hauer, James Morse Cc: linux-edac, Mauro Carvalho Chehab, Tony Luck, Robert Richter, York Sun, kernel, linux-arm-kernel On Thu, Aug 13, 2020 at 09:57:20AM +0200, Sascha Hauer wrote: > The Cortex A53 and A57 cores have error detection capabilities for the > L1/L2 Caches, this patch adds a driver for them. > > Unfortunately there is no robust way to inject errors into the caches, > so this driver doesn't contain any code to actually test it. It has > been tested though with code taken from an older version of this driver > found here: https://lkml.org/lkml/2018/3/14/1203. For reasons stated > in this thread the error injection code is not suitable for mainline, > so it is removed from the driver. > > Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> > --- > .../bindings/edac/arm,cortex-a5x-edac.yaml | 32 +++ > drivers/edac/Kconfig | 6 + > drivers/edac/Makefile | 1 + > drivers/edac/cortex_arm64_l1_l2.c | 208 ++++++++++++++++++ > 4 files changed, 247 insertions(+) > create mode 100644 Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml > create mode 100644 drivers/edac/cortex_arm64_l1_l2.c Just nitpicks below. James'd need to look at this too before it goes anywhere. Checkpatch is trying to tell me something here: WARNING: DT compatible string "arm,cortex-a53-edac" appears un-documented -- check ./Documentation/devicetree/bindings/ #296: FILE: drivers/edac/cortex_arm64_l1_l2.c:190: + { .compatible = "arm,cortex-a53-edac" }, WARNING: DT compatible string "arm,cortex-a57-edac" appears un-documented -- check ./Documentation/devicetree/bindings/ #297: FILE: drivers/edac/cortex_arm64_l1_l2.c:191: + { .compatible = "arm,cortex-a57-edac" }, for 2/2 too: WARNING: DT compatible string "arm,cortex-a53-edac" appears un-documented -- check ./Documentation/devicetree/bindings/ #39: FILE: arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi:842: + compatible = "arm,cortex-a53-edac"; WARNING: DT compatible string "arm,cortex-a57-edac" appears un-documented -- check ./Documentation/devicetree/bindings/ #56: FILE: arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi:805: + compatible = "arm,cortex-a57-edac"; False positive or valid? ... > +static void read_errors(void *data) > +{ > + u64 cpumerr, l2merr; > + int cpu = smp_processor_id(); > + char msg[MESSAGE_SIZE]; > + struct edac_device_ctl_info *edac_ctl = data; Please sort function local variables declaration in a reverse christmas tree order: <type A> longest_variable_name; <type B> shorter_var_name; <type C> even_shorter; <type D> i; Check your other functions too pls. > + /* cpumerrsr_el1 */ > + asm volatile("mrs %0, s3_1_c15_c2_2" : "=r" (cpumerr)); > + asm volatile("msr s3_1_c15_c2_2, %0" :: "r" (0)); > + > + if (cpumerr & CPUMERRSR_EL1_VALID) { > + const char *str; > + int fatal = (cpumerr & CPUMERRSR_EL1_FATAL) != 0; Don't need "!= 0" and fatal can be bool. > + switch (FIELD_GET(CPUMERRSR_EL1_RAMID, cpumerr)) { > + case L1_I_TAG_RAM: > + str = "L1-I Tag RAM"; > + break; > + case L1_I_DATA_RAM: > + str = "L1-I Data RAM"; > + break; > + case L1_D_TAG_RAM: > + str = "L1-D Tag RAM"; > + break; > + case L1_D_DATA_RAM: > + str = "L1-D Data RAM"; > + break; > + case L1_D_DIRTY_RAM: > + str = "L1 Dirty RAM"; > + break; > + case TLB_RAM: > + str = "TLB RAM"; > + break; > + default: > + str = "unknown"; > + break; > + } > + > + snprintf(msg, MESSAGE_SIZE, "%s %s error(s) on CPU %d", > + str, fatal ? "fatal" : "correctable", cpu); > + > + if (fatal) > + edac_device_handle_ue(edac_ctl, 0, 0, msg); > + else > + edac_device_handle_ce(edac_ctl, 0, 0, msg); > + } > + > + /* l2merrsr_el1 */ > + asm volatile("mrs %0, s3_1_c15_c2_3" : "=r" (l2merr)); > + asm volatile("msr s3_1_c15_c2_3, %0" :: "r" (0)); > + > + if (l2merr & L2MERRSR_EL1_VALID) { > + int fatal = (l2merr & L2MERRSR_EL1_FATAL) != 0; See above. > + > + snprintf(msg, MESSAGE_SIZE, "L2 %s error(s) on CPU %d", > + fatal ? "fatal" : "correctable", cpu); > + if (fatal) > + edac_device_handle_ue(edac_ctl, 0, 1, msg); > + else > + edac_device_handle_ce(edac_ctl, 0, 1, msg); > + } > +} ... -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] drivers/edac: Add L1 and L2 error detection for A53 and A57 2020-08-26 8:41 ` Borislav Petkov @ 2020-10-13 11:13 ` Sascha Hauer 2020-10-13 11:31 ` Borislav Petkov 0 siblings, 1 reply; 11+ messages in thread From: Sascha Hauer @ 2020-10-13 11:13 UTC (permalink / raw) To: Borislav Petkov Cc: James Morse, linux-edac, Mauro Carvalho Chehab, Tony Luck, Robert Richter, York Sun, kernel, linux-arm-kernel Hi Boris, Sorry for the long delay, your mail was buried in my inbox. On Wed, Aug 26, 2020 at 10:41:35AM +0200, Borislav Petkov wrote: > On Thu, Aug 13, 2020 at 09:57:20AM +0200, Sascha Hauer wrote: > > The Cortex A53 and A57 cores have error detection capabilities for the > > L1/L2 Caches, this patch adds a driver for them. > > > > Unfortunately there is no robust way to inject errors into the caches, > > so this driver doesn't contain any code to actually test it. It has > > been tested though with code taken from an older version of this driver > > found here: https://lkml.org/lkml/2018/3/14/1203. For reasons stated > > in this thread the error injection code is not suitable for mainline, > > so it is removed from the driver. > > > > Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de> > > --- > > .../bindings/edac/arm,cortex-a5x-edac.yaml | 32 +++ > > drivers/edac/Kconfig | 6 + > > drivers/edac/Makefile | 1 + > > drivers/edac/cortex_arm64_l1_l2.c | 208 ++++++++++++++++++ > > 4 files changed, 247 insertions(+) > > create mode 100644 Documentation/devicetree/bindings/edac/arm,cortex-a5x-edac.yaml > > create mode 100644 drivers/edac/cortex_arm64_l1_l2.c > > Just nitpicks below. James'd need to look at this too before it goes > anywhere. > > Checkpatch is trying to tell me something here: > > WARNING: DT compatible string "arm,cortex-a53-edac" appears un-documented -- check ./Documentation/devicetree/bindings/ > #296: FILE: drivers/edac/cortex_arm64_l1_l2.c:190: > + { .compatible = "arm,cortex-a53-edac" }, > > WARNING: DT compatible string "arm,cortex-a57-edac" appears un-documented -- check ./Documentation/devicetree/bindings/ > #297: FILE: drivers/edac/cortex_arm64_l1_l2.c:191: > + { .compatible = "arm,cortex-a57-edac" }, > > for 2/2 too: > > WARNING: DT compatible string "arm,cortex-a53-edac" appears un-documented -- check ./Documentation/devicetree/bindings/ > #39: FILE: arch/arm64/boot/dts/freescale/fsl-ls1043a.dtsi:842: > + compatible = "arm,cortex-a53-edac"; > > WARNING: DT compatible string "arm,cortex-a57-edac" appears un-documented -- check ./Documentation/devicetree/bindings/ > #56: FILE: arch/arm64/boot/dts/freescale/fsl-ls1046a.dtsi:805: > + compatible = "arm,cortex-a57-edac"; > > > False positive or valid? Have you run checkpatch on a tree with this patch applied? If not, then yes, it's undocumented as the docs are added with this patch. > > ... > > > +static void read_errors(void *data) > > +{ > > + u64 cpumerr, l2merr; > > + int cpu = smp_processor_id(); > > + char msg[MESSAGE_SIZE]; > > + struct edac_device_ctl_info *edac_ctl = data; > > Please sort function local variables declaration in a reverse christmas > tree order: > > <type A> longest_variable_name; > <type B> shorter_var_name; > <type C> even_shorter; > <type D> i; I never heard of such a requirement. How is the length defined? Is it only the length of the variable name or is it the length of the name including the type? Including the array braces [] or not? What if a variable shall be initialized with the value of an earlier declared variable, do I have to make up variable names with a suitable length in that case? What if shorter_var_name and even_shorter are of same type, can I still write them in a single line? Finally, Is this documented somewhere? I hope that was a joke from you that I didn't understand. > > Check your other functions too pls. > > > + /* cpumerrsr_el1 */ > > + asm volatile("mrs %0, s3_1_c15_c2_2" : "=r" (cpumerr)); > > + asm volatile("msr s3_1_c15_c2_2, %0" :: "r" (0)); > > + > > + if (cpumerr & CPUMERRSR_EL1_VALID) { > > + const char *str; > > + int fatal = (cpumerr & CPUMERRSR_EL1_FATAL) != 0; > > Don't need "!= 0" and fatal can be bool. Ok. Sascha -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/2] drivers/edac: Add L1 and L2 error detection for A53 and A57 2020-10-13 11:13 ` Sascha Hauer @ 2020-10-13 11:31 ` Borislav Petkov 0 siblings, 0 replies; 11+ messages in thread From: Borislav Petkov @ 2020-10-13 11:31 UTC (permalink / raw) To: Sascha Hauer Cc: James Morse, linux-edac, Mauro Carvalho Chehab, Tony Luck, Robert Richter, York Sun, kernel, linux-arm-kernel On Tue, Oct 13, 2020 at 01:13:46PM +0200, Sascha Hauer wrote: > Have you run checkpatch on a tree with this patch applied? If not, then > yes, it's undocumented as the docs are added with this patch. Aha, I'm usually running checkpatch on the patch itself so if this patch adds it then checkpatch is doing it wrong. Oh well. > I never heard of such a requirement. How is the length defined? Is it > only the length of the variable name or is it the length of the name > including the type? Including the array braces [] or not? The total textual length. > What if a variable shall be initialized with the value of an earlier > declared variable, do I have to make up variable names with a suitable > length in that case? In that case you should order it so that the initialization is correct. > What if shorter_var_name and even_shorter are of same type, can I > still write them in a single line? You simply sort them so that it remains a reversed xmas tree. > Finally, Is this documented somewhere? No, would it make any difference if it were? > I hope that was a joke from you that I didn't understand. This is the convention that is used in the tip tree and I'm using it in the EDAC tree too. Just think of it as a subsystem formatting rule. Thx. -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-04-15 10:15 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-04-01 11:06 [PATCH v5 0/2] Add L1 and L2 error detection for A53 and A57 Sascha Hauer 2021-04-01 11:06 ` [PATCH 1/2] drivers/edac: " Sascha Hauer 2021-04-02 10:06 ` Marc Zyngier 2021-04-15 10:15 ` Sascha Hauer 2021-04-01 11:06 ` [PATCH 2/2] dt-bindings: arm: cpus: Add edac-enabled property Sascha Hauer 2021-04-01 15:37 ` Marc Zyngier -- strict thread matches above, loose matches on Subject: below -- 2021-02-01 11:57 [PATCH iv4 0/2] Add L1 and L2 error detection for A53 and A57 Sascha Hauer 2021-02-01 11:57 ` [PATCH 1/2] drivers/edac: " Sascha Hauer 2020-08-13 7:57 [PATCH 0/2] " Sascha Hauer 2020-08-13 7:57 ` [PATCH 1/2] drivers/edac: " Sascha Hauer 2020-08-26 8:41 ` Borislav Petkov 2020-10-13 11:13 ` Sascha Hauer 2020-10-13 11:31 ` Borislav Petkov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).