* [PATCH v3 0/3] AMD Address Translation Library
@ 2023-12-10 19:49 Yazen Ghannam
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
` (2 more replies)
0 siblings, 3 replies; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-10 19:49 UTC (permalink / raw)
To: linux-edac
Cc: linux-kernel, tony.luck, x86, avadhut.naik, john.allen,
william.roche, muralidhara.mk, Yazen Ghannam
Hi all,
This revision addresses comments from Boris for v2. The most substantial
change is how module loading/unloading is done for the library.
Thanks,
Yazen
Yazen Ghannam (3):
RAS: Introduce AMD Address Translation Library
EDAC/amd64: Use new AMD Address Translation Library
Documentation: RAS: Add index and address translation section
Documentation/RAS/address-translation.rst | 24 +
.../RAS/{ras.rst => error-decoding.rst} | 11 +-
Documentation/RAS/index.rst | 14 +
Documentation/index.rst | 2 +-
MAINTAINERS | 8 +
arch/x86/include/asm/amd_atl.h | 18 +
drivers/edac/Kconfig | 1 +
drivers/edac/amd64_edac.c | 278 +-------
drivers/ras/Kconfig | 1 +
drivers/ras/Makefile | 2 +
drivers/ras/amd/atl/Kconfig | 41 ++
drivers/ras/amd/atl/Makefile | 20 +
drivers/ras/amd/atl/access.c | 106 +++
drivers/ras/amd/atl/core.c | 217 ++++++
drivers/ras/amd/atl/dehash.c | 446 ++++++++++++
drivers/ras/amd/atl/denormalize.c | 616 ++++++++++++++++
drivers/ras/amd/atl/internal.h | 312 +++++++++
drivers/ras/amd/atl/map.c | 659 ++++++++++++++++++
drivers/ras/amd/atl/reg_fields.h | 603 ++++++++++++++++
drivers/ras/amd/atl/stub.c | 78 +++
drivers/ras/amd/atl/stub.h | 34 +
drivers/ras/amd/atl/system.c | 283 ++++++++
drivers/ras/amd/atl/umc.c | 45 ++
23 files changed, 3534 insertions(+), 285 deletions(-)
create mode 100644 Documentation/RAS/address-translation.rst
rename Documentation/RAS/{ras.rst => error-decoding.rst} (73%)
create mode 100644 Documentation/RAS/index.rst
create mode 100644 arch/x86/include/asm/amd_atl.h
create mode 100644 drivers/ras/amd/atl/Kconfig
create mode 100644 drivers/ras/amd/atl/Makefile
create mode 100644 drivers/ras/amd/atl/access.c
create mode 100644 drivers/ras/amd/atl/core.c
create mode 100644 drivers/ras/amd/atl/dehash.c
create mode 100644 drivers/ras/amd/atl/denormalize.c
create mode 100644 drivers/ras/amd/atl/internal.h
create mode 100644 drivers/ras/amd/atl/map.c
create mode 100644 drivers/ras/amd/atl/reg_fields.h
create mode 100644 drivers/ras/amd/atl/stub.c
create mode 100644 drivers/ras/amd/atl/stub.h
create mode 100644 drivers/ras/amd/atl/system.c
create mode 100644 drivers/ras/amd/atl/umc.c
base-commit: ec00e2ed1bc2d6ed6a65ef2b1030ba3ab1575733
--
2.34.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-10 19:49 [PATCH v3 0/3] AMD Address Translation Library Yazen Ghannam
@ 2023-12-10 19:49 ` Yazen Ghannam
2023-12-11 14:20 ` Borislav Petkov
` (4 more replies)
2023-12-10 19:49 ` [PATCH v3 2/3] EDAC/amd64: Use new " Yazen Ghannam
2023-12-10 19:49 ` [PATCH v3 3/3] Documentation: RAS: Add index and address translation section Yazen Ghannam
2 siblings, 5 replies; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-10 19:49 UTC (permalink / raw)
To: linux-edac
Cc: linux-kernel, tony.luck, x86, avadhut.naik, john.allen,
william.roche, muralidhara.mk, Yazen Ghannam
AMD Zen-based systems report memory errors through Machine Check banks
representing Unified Memory Controllers (UMCs). The address value
reported for DRAM ECC errors is a "normalized address" that is relative
to the UMC. This normalized address must be converted to a system
physical address to be usable by the OS.
Support for this address translation was introduced to the MCA subsystem
with Zen1 systems. The code was later moved to the AMD64 EDAC module,
since this was the only user of the code at the time.
However, there are uses for this translation outside of EDAC. The system
physical address can be used in MCA for preemptive page offlining as done
in some MCA notifier functions. Also, this translation is needed as the
basis of similar functionality needed for some CXL configurations on AMD
systems.
Introduce a common address translation library that can be used for
multiple subsystems including MCA, EDAC, and CXL.
Include support for UMC normalized to system physical address
translation for current CPU systems.
The Data Fabric Indirect register access offsets and one of the register
fields were changed. Default to the current offsets and register field
definition. And fallback to the older values if running on a "legacy"
system.
Provide a built-in "stub" to facilitate loading and unloading of the
library module without affecting other modules or built-in code.
Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lore.kernel.org/r/20231005173526.42831-2-yazen.ghannam@amd.com
v2->v3:
* Remove "future work" list from commit message.
* Add "X86_64" Kconfig dependency.
* Remove code comment and update commit message for "legacy" DF indirect
access changes.
* Move amd_atl.h to x86/asm.
* Remove "TODO" comments and code.
* Remove placeholder helper functions.
* Rework module dependencies using kref and function pointers.
* Remove extra line breaks in switch statements.
* Rework warnings.
* Various code formatting adjustments.
v1->v2:
* Move to drivers/ras.
* Provide inline helper to do "config reachable" check.
MAINTAINERS | 7 +
arch/x86/include/asm/amd_atl.h | 18 +
drivers/ras/Kconfig | 1 +
drivers/ras/Makefile | 2 +
drivers/ras/amd/atl/Kconfig | 41 ++
drivers/ras/amd/atl/Makefile | 20 +
drivers/ras/amd/atl/access.c | 106 +++++
drivers/ras/amd/atl/core.c | 217 ++++++++++
drivers/ras/amd/atl/dehash.c | 446 ++++++++++++++++++++
drivers/ras/amd/atl/denormalize.c | 616 ++++++++++++++++++++++++++++
drivers/ras/amd/atl/internal.h | 312 ++++++++++++++
drivers/ras/amd/atl/map.c | 659 ++++++++++++++++++++++++++++++
drivers/ras/amd/atl/reg_fields.h | 603 +++++++++++++++++++++++++++
drivers/ras/amd/atl/stub.c | 78 ++++
drivers/ras/amd/atl/stub.h | 34 ++
drivers/ras/amd/atl/system.c | 283 +++++++++++++
drivers/ras/amd/atl/umc.c | 45 ++
17 files changed, 3488 insertions(+)
create mode 100644 arch/x86/include/asm/amd_atl.h
create mode 100644 drivers/ras/amd/atl/Kconfig
create mode 100644 drivers/ras/amd/atl/Makefile
create mode 100644 drivers/ras/amd/atl/access.c
create mode 100644 drivers/ras/amd/atl/core.c
create mode 100644 drivers/ras/amd/atl/dehash.c
create mode 100644 drivers/ras/amd/atl/denormalize.c
create mode 100644 drivers/ras/amd/atl/internal.h
create mode 100644 drivers/ras/amd/atl/map.c
create mode 100644 drivers/ras/amd/atl/reg_fields.h
create mode 100644 drivers/ras/amd/atl/stub.c
create mode 100644 drivers/ras/amd/atl/stub.h
create mode 100644 drivers/ras/amd/atl/system.c
create mode 100644 drivers/ras/amd/atl/umc.c
diff --git a/MAINTAINERS b/MAINTAINERS
index ceb4263ec140..b9b060917680 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -881,6 +881,13 @@ Q: https://patchwork.kernel.org/project/linux-rdma/list/
F: drivers/infiniband/hw/efa/
F: include/uapi/rdma/efa-abi.h
+AMD ADDRESS TRANSLATION LIBRARY (ATL)
+M: Yazen Ghannam <Yazen.Ghannam@amd.com>
+L: linux-edac@vger.kernel.org
+S: Supported
+F: arch/x86/include/asm/amd-atl.h
+F: drivers/ras/amd/atl/*
+
AMD CDX BUS DRIVER
M: Nipun Gupta <nipun.gupta@amd.com>
M: Nikhil Agarwal <nikhil.agarwal@amd.com>
diff --git a/arch/x86/include/asm/amd_atl.h b/arch/x86/include/asm/amd_atl.h
new file mode 100644
index 000000000000..4a51db4e107a
--- /dev/null
+++ b/arch/x86/include/asm/amd_atl.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * AMD Address Translation Library
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#ifndef _AMD_ATL_H
+#define _AMD_ATL_H
+
+#include <uapi/asm/mce.h>
+
+int amd_convert_umc_mca_addr_to_sys_addr(struct mce *m, u64 *sys_addr);
+
+#endif /* _AMD_ATL_H */
diff --git a/drivers/ras/Kconfig b/drivers/ras/Kconfig
index c2a236f2e846..2e969f59c0ca 100644
--- a/drivers/ras/Kconfig
+++ b/drivers/ras/Kconfig
@@ -32,5 +32,6 @@ menuconfig RAS
if RAS
source "arch/x86/ras/Kconfig"
+source "drivers/ras/amd/atl/Kconfig"
endif
diff --git a/drivers/ras/Makefile b/drivers/ras/Makefile
index 6f0404f50107..3fac80f58005 100644
--- a/drivers/ras/Makefile
+++ b/drivers/ras/Makefile
@@ -2,3 +2,5 @@
obj-$(CONFIG_RAS) += ras.o
obj-$(CONFIG_DEBUG_FS) += debugfs.o
obj-$(CONFIG_RAS_CEC) += cec.o
+
+obj-y += amd/atl/
diff --git a/drivers/ras/amd/atl/Kconfig b/drivers/ras/amd/atl/Kconfig
new file mode 100644
index 000000000000..fcff387faa33
--- /dev/null
+++ b/drivers/ras/amd/atl/Kconfig
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# AMD Address Translation Library Kconfig
+#
+# Copyright (c) 2023, Advanced Micro Devices, Inc.
+# All Rights Reserved.
+#
+# Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+
+config AMD_ATL
+ tristate "AMD Address Translation Library"
+ depends on AMD_NB
+ imply AMD_ATL_MOD
+ select AMD_ATL_STUB
+ default N
+ help
+ This library includes support for implementation-specific
+ address translation procedures needed for various error
+ handling cases.
+
+ Enable this option if using DRAM ECC on Zen-based systems
+ and OS-based error handling.
+
+config AMD_ATL_MOD
+ tristate "AMD Address Translation Library module"
+ depends on X86_64
+ default N
+ help
+ This option provides the module component of the AMD Address
+ Translation Library. It will be selected automatically if
+ building the library. Minimal dependencies are given to avoid
+ build errors for modules using the library.
+
+config AMD_ATL_STUB
+ bool "AMD Address Translation Library stub"
+ default N
+ help
+ This option provides minimal built-in code to support the
+ AMD Address Translation Library module. It will be selected
+ automatically if building the library. No dependencies are given
+ to avoid build errors for modules using the library.
diff --git a/drivers/ras/amd/atl/Makefile b/drivers/ras/amd/atl/Makefile
new file mode 100644
index 000000000000..dc9fdbc77e91
--- /dev/null
+++ b/drivers/ras/amd/atl/Makefile
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# AMD Address Translation Library Makefile
+#
+# Copyright (c) 2023, Advanced Micro Devices, Inc.
+# All Rights Reserved.
+#
+# Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+
+obj-$(CONFIG_AMD_ATL_STUB) += stub.o
+
+amd_atl-y := access.o
+amd_atl-y += core.o
+amd_atl-y += dehash.o
+amd_atl-y += denormalize.o
+amd_atl-y += map.o
+amd_atl-y += system.o
+amd_atl-y += umc.o
+
+obj-$(CONFIG_AMD_ATL_MOD) += amd_atl.o
diff --git a/drivers/ras/amd/atl/access.c b/drivers/ras/amd/atl/access.c
new file mode 100644
index 000000000000..1de0460f5e03
--- /dev/null
+++ b/drivers/ras/amd/atl/access.c
@@ -0,0 +1,106 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AMD Address Translation Library
+ *
+ * access.c : DF Indirect Access functions
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#include "internal.h"
+
+/* Protect the PCI config register pairs used for DF indirect access. */
+static DEFINE_MUTEX(df_indirect_mutex);
+
+/*
+ * Data Fabric Indirect Access uses FICAA/FICAD.
+ *
+ * Fabric Indirect Configuration Access Address (FICAA): Constructed based
+ * on the device's Instance Id and the PCI function and register offset of
+ * the desired register.
+ *
+ * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
+ * and FICAD HI registers but so far we only need the LO register.
+ *
+ * Use Instance Id 0xFF to indicate a broadcast read.
+ */
+#define DF_BROADCAST 0xFF
+
+#define DF_FICAA_INST_EN BIT(0)
+#define DF_FICAA_REG_NUM GENMASK(10, 1)
+#define DF_FICAA_FUNC_NUM GENMASK(13, 11)
+#define DF_FICAA_INST_ID GENMASK(23, 16)
+
+#define DF_FICAA_REG_NUM_LEGACY GENMASK(10, 2)
+
+static int __df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
+{
+ u32 ficaa_addr = 0x8C, ficad_addr = 0xB8;
+ struct pci_dev *F4;
+ int err = -ENODEV;
+ u32 ficaa = 0;
+
+ if (node >= amd_nb_num())
+ goto out;
+
+ F4 = node_to_amd_nb(node)->link;
+ if (!F4)
+ goto out;
+
+ /* Enable instance-specific access. */
+ if (instance_id != DF_BROADCAST) {
+ ficaa |= FIELD_PREP(DF_FICAA_INST_EN, 1);
+ ficaa |= FIELD_PREP(DF_FICAA_INST_ID, instance_id);
+ }
+
+ /*
+ * The two least-significant bits are masked when inputing the
+ * register offset to FICAA.
+ */
+ reg >>= 2;
+
+ if (df_cfg.flags.legacy_ficaa) {
+ ficaa_addr = 0x5C;
+ ficad_addr = 0x98;
+
+ ficaa |= FIELD_PREP(DF_FICAA_REG_NUM_LEGACY, reg);
+ } else {
+ ficaa |= FIELD_PREP(DF_FICAA_REG_NUM, reg);
+ }
+
+ ficaa |= FIELD_PREP(DF_FICAA_FUNC_NUM, func);
+
+ mutex_lock(&df_indirect_mutex);
+
+ err = pci_write_config_dword(F4, ficaa_addr, ficaa);
+ if (err) {
+ pr_warn("Error writing DF Indirect FICAA, FICAA=0x%x\n", ficaa);
+ goto out_unlock;
+ }
+
+ err = pci_read_config_dword(F4, ficad_addr, lo);
+ if (err)
+ pr_warn("Error reading DF Indirect FICAD LO, FICAA=0x%x.\n", ficaa);
+
+ pr_debug("node=%u inst=0x%x func=0x%x reg=0x%x val=0x%x",
+ node, instance_id, func, reg << 2, *lo);
+
+out_unlock:
+ mutex_unlock(&df_indirect_mutex);
+
+out:
+ return err;
+}
+
+int df_indirect_read_instance(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
+{
+ return __df_indirect_read(node, func, reg, instance_id, lo);
+}
+
+int df_indirect_read_broadcast(u16 node, u8 func, u16 reg, u32 *lo)
+{
+ return __df_indirect_read(node, func, reg, DF_BROADCAST, lo);
+}
diff --git a/drivers/ras/amd/atl/core.c b/drivers/ras/amd/atl/core.c
new file mode 100644
index 000000000000..6a6220fef81f
--- /dev/null
+++ b/drivers/ras/amd/atl/core.c
@@ -0,0 +1,217 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AMD Address Translation Library
+ *
+ * core.c : Module init and base translation functions
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#include <linux/module.h>
+#include <asm/cpu_device_id.h>
+
+#include "internal.h"
+
+struct df_config df_cfg __read_mostly;
+
+static int addr_over_limit(struct addr_ctx *ctx)
+{
+ u64 dram_limit_addr;
+
+ if (df_cfg.rev >= DF4)
+ dram_limit_addr = FIELD_GET(DF4_DRAM_LIMIT_ADDR, ctx->map.limit);
+ else
+ dram_limit_addr = FIELD_GET(DF2_DRAM_LIMIT_ADDR, ctx->map.limit);
+
+ dram_limit_addr <<= DF_DRAM_BASE_LIMIT_LSB;
+ dram_limit_addr |= GENMASK(DF_DRAM_BASE_LIMIT_LSB - 1, 0);
+
+ /* Is calculated system address above DRAM limit address? */
+ if (ctx->ret_addr > dram_limit_addr) {
+ warn_on_assert("Calculated address (0x%016llx) > DRAM limit (0x%016llx)",
+ ctx->ret_addr, dram_limit_addr);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static bool legacy_hole_en(struct addr_ctx *ctx)
+{
+ u32 reg = ctx->map.base;
+
+ if (df_cfg.rev >= DF4)
+ reg = ctx->map.ctl;
+
+ return FIELD_GET(DF_LEGACY_MMIO_HOLE_EN, reg);
+}
+
+static int add_legacy_hole(struct addr_ctx *ctx)
+{
+ u32 dram_hole_base;
+ u8 func = 0;
+
+ if (!legacy_hole_en(ctx))
+ return 0;
+
+ if (df_cfg.rev >= DF4)
+ func = 7;
+
+ if (df_indirect_read_broadcast(ctx->node_id, func, 0x104, &dram_hole_base))
+ return -EINVAL;
+
+ dram_hole_base &= DF_DRAM_HOLE_BASE_MASK;
+
+ if (ctx->ret_addr >= dram_hole_base)
+ ctx->ret_addr += (BIT_ULL(32) - dram_hole_base);
+
+ return 0;
+}
+
+static u64 get_base_addr(struct addr_ctx *ctx)
+{
+ u64 base_addr;
+
+ if (df_cfg.rev >= DF4)
+ base_addr = FIELD_GET(DF4_BASE_ADDR, ctx->map.base);
+ else
+ base_addr = FIELD_GET(DF2_BASE_ADDR, ctx->map.base);
+
+ return base_addr << DF_DRAM_BASE_LIMIT_LSB;
+}
+
+static int add_base_and_hole(struct addr_ctx *ctx)
+{
+ ctx->ret_addr += get_base_addr(ctx);
+
+ if (add_legacy_hole(ctx))
+ return -EINVAL;
+
+ return 0;
+}
+
+static bool late_hole_remove(struct addr_ctx *ctx)
+{
+ if (df_cfg.rev == DF3p5)
+ return true;
+
+ if (df_cfg.rev == DF4)
+ return true;
+
+ if (ctx->map.intlv_mode == DF3_6CHAN)
+ return true;
+
+ return false;
+}
+
+int norm_to_sys_addr(u8 socket_id, u8 die_id, u8 cs_inst_id, u64 *addr)
+{
+ struct addr_ctx ctx;
+
+ if (df_cfg.rev == UNKNOWN)
+ return -EINVAL;
+
+ memset(&ctx, 0, sizeof(ctx));
+
+ /* We start from the normalized address */
+ ctx.ret_addr = *addr;
+ ctx.inst_id = cs_inst_id;
+
+ ctx.inputs.norm_addr = *addr;
+ ctx.inputs.socket_id = socket_id;
+ ctx.inputs.die_id = die_id;
+ ctx.inputs.cs_inst_id = cs_inst_id;
+
+ if (determine_node_id(&ctx, socket_id, die_id))
+ return -EINVAL;
+
+ if (get_address_map(&ctx))
+ return -EINVAL;
+
+ if (denormalize_address(&ctx))
+ return -EINVAL;
+
+ if (!late_hole_remove(&ctx) && add_base_and_hole(&ctx))
+ return -EINVAL;
+
+ if (dehash_address(&ctx))
+ return -EINVAL;
+
+ if (late_hole_remove(&ctx) && add_base_and_hole(&ctx))
+ return -EINVAL;
+
+ if (addr_over_limit(&ctx))
+ return -EINVAL;
+
+ *addr = ctx.ret_addr;
+ return 0;
+}
+
+static void check_for_legacy_df_access(void)
+{
+ /*
+ * All Zen-based systems before Family 19h use the legacy
+ * DF Indirect Access (FICAA/FICAD) offsets.
+ */
+ if (boot_cpu_data.x86 < 0x19) {
+ df_cfg.flags.legacy_ficaa = true;
+ return;
+ }
+
+ /* All systems after Family 19h use the current offsets. */
+ if (boot_cpu_data.x86 > 0x19)
+ return;
+
+ /* Some Family 19h systems use the legacy offsets. */
+ switch (boot_cpu_data.x86_model) {
+ case 0x00 ... 0x0f:
+ case 0x20 ... 0x5f:
+ df_cfg.flags.legacy_ficaa = true;
+ }
+}
+
+static const struct x86_cpu_id amd_atl_cpuids[] = {
+ X86_MATCH_FEATURE(X86_FEATURE_SMCA, NULL),
+ X86_MATCH_FEATURE(X86_FEATURE_ZEN, NULL),
+ X86_MATCH_FEATURE(X86_FEATURE_ZEN2, NULL),
+ X86_MATCH_FEATURE(X86_FEATURE_ZEN3, NULL),
+ X86_MATCH_FEATURE(X86_FEATURE_ZEN4, NULL),
+ { }
+};
+MODULE_DEVICE_TABLE(x86cpu, amd_atl_cpuids);
+
+static struct amd_atl_ops ops = {
+ .convert_umc_na_to_spa = convert_umc_mca_addr_to_sys_addr,
+};
+
+static int __init amd_atl_init(void)
+{
+ if (!x86_match_cpu(amd_atl_cpuids))
+ return -ENODEV;
+
+ if (!amd_nb_num())
+ return -ENODEV;
+
+ check_for_legacy_df_access();
+
+ if (get_df_system_info())
+ return -ENODEV;
+
+ amd_atl_pvt.initialize(&ops);
+
+ pr_info("AMD Address Translation Library initialized");
+ return 0;
+}
+
+static void __exit amd_atl_exit(void)
+{
+ amd_atl_pvt.release();
+}
+
+module_init(amd_atl_init);
+module_exit(amd_atl_exit);
+
+MODULE_LICENSE("GPL");
diff --git a/drivers/ras/amd/atl/dehash.c b/drivers/ras/amd/atl/dehash.c
new file mode 100644
index 000000000000..84fe9793694e
--- /dev/null
+++ b/drivers/ras/amd/atl/dehash.c
@@ -0,0 +1,446 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AMD Address Translation Library
+ *
+ * dehash.c : Functions to account for hashing bits
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#include "internal.h"
+
+static inline bool assert_intlv_bit(struct addr_ctx *ctx, u8 bit1, u8 bit2)
+{
+ if (ctx->map.intlv_bit_pos == bit1 || ctx->map.intlv_bit_pos == bit2)
+ return false;
+
+ warn_on_assert("%s: Invalid interleave bit: %u",
+ __func__, ctx->map.intlv_bit_pos);
+
+ return true;
+}
+
+static inline bool assert_num_intlv_dies(struct addr_ctx *ctx, u8 num_intlv_dies)
+{
+ if (ctx->map.num_intlv_dies <= num_intlv_dies)
+ return false;
+
+ warn_on_assert("%s: Invalid number of interleave dies: %u",
+ __func__, ctx->map.num_intlv_dies);
+
+ return true;
+}
+
+static inline bool assert_num_intlv_sockets(struct addr_ctx *ctx, u8 num_intlv_sockets)
+{
+ if (ctx->map.num_intlv_sockets <= num_intlv_sockets)
+ return false;
+
+ warn_on_assert("%s: Invalid number of interleave sockets: %u",
+ __func__, ctx->map.num_intlv_sockets);
+
+ return true;
+}
+
+static int df2_dehash_addr(struct addr_ctx *ctx)
+{
+ u8 hashed_bit, intlv_bit, intlv_bit_pos;
+
+ /* Assert that interleave bit is 8 or 9. */
+ if (assert_intlv_bit(ctx, 8, 9))
+ return -EINVAL;
+
+ /* Assert that die and socket interleaving are disabled. */
+ if (assert_num_intlv_dies(ctx, 1))
+ return -EINVAL;
+
+ if (assert_num_intlv_sockets(ctx, 1))
+ return -EINVAL;
+
+ intlv_bit_pos = ctx->map.intlv_bit_pos;
+ intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(12), ctx->ret_addr);
+ hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr);
+ hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr);
+ hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr);
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
+
+ return 0;
+}
+
+static int df3_dehash_addr(struct addr_ctx *ctx)
+{
+ bool hash_ctl_64k, hash_ctl_2M, hash_ctl_1G;
+ u8 hashed_bit, intlv_bit, intlv_bit_pos;
+
+ /* Assert that interleave bit is 8 or 9. */
+ if (assert_intlv_bit(ctx, 8, 9))
+ return -EINVAL;
+
+ /* Assert that die and socket interleaving are disabled. */
+ if (assert_num_intlv_dies(ctx, 1))
+ return -EINVAL;
+
+ if (assert_num_intlv_sockets(ctx, 1))
+ return -EINVAL;
+
+ hash_ctl_64k = FIELD_GET(DF3_HASH_CTL_64K, ctx->map.ctl);
+ hash_ctl_2M = FIELD_GET(DF3_HASH_CTL_2M, ctx->map.ctl);
+ hash_ctl_1G = FIELD_GET(DF3_HASH_CTL_1G, ctx->map.ctl);
+
+ intlv_bit_pos = ctx->map.intlv_bit_pos;
+ intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(14), ctx->ret_addr);
+ hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
+
+ /* Calculation complete for 2 channels. Continue for 4 and 8 channels. */
+ if (ctx->map.intlv_mode == DF3_COD4_2CHAN_HASH)
+ return 0;
+
+ intlv_bit = FIELD_GET(BIT_ULL(12), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(16), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(12);
+
+ /* Calculation complete for 4 channels. Continue for 8 channels. */
+ if (ctx->map.intlv_mode == DF3_COD2_4CHAN_HASH)
+ return 0;
+
+ intlv_bit = FIELD_GET(BIT_ULL(13), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(17), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(13);
+
+ return 0;
+}
+
+static int df3_6chan_dehash_addr(struct addr_ctx *ctx)
+{
+ u8 intlv_bit_pos = ctx->map.intlv_bit_pos;
+ u8 hashed_bit, intlv_bit, num_intlv_bits;
+ bool hash_ctl_2M, hash_ctl_1G;
+
+ if (ctx->map.intlv_mode != DF3_6CHAN) {
+ warn_on_bad_intlv_mode(ctx);
+ return -EINVAL;
+ }
+
+ num_intlv_bits = ilog2(ctx->map.num_intlv_chan) + 1;
+
+ hash_ctl_2M = FIELD_GET(DF3_HASH_CTL_2M, ctx->map.ctl);
+ hash_ctl_1G = FIELD_GET(DF3_HASH_CTL_1G, ctx->map.ctl);
+
+ intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= atl_get_bit((intlv_bit_pos + num_intlv_bits), ctx->ret_addr);
+ hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
+
+ intlv_bit_pos++;
+ intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
+
+ intlv_bit_pos++;
+ intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
+
+ return 0;
+}
+
+static int df4_dehash_addr(struct addr_ctx *ctx)
+{
+ bool hash_ctl_64k, hash_ctl_2M, hash_ctl_1G;
+ u8 hashed_bit, intlv_bit;
+
+ /* Assert that interleave bit is 8. */
+ if (assert_intlv_bit(ctx, 8, 8))
+ return -EINVAL;
+
+ /* Assert that die interleaving is disabled. */
+ if (assert_num_intlv_dies(ctx, 1))
+ return -EINVAL;
+
+ /* Assert that no more than 2 sockets are interleaved. */
+ if (assert_num_intlv_sockets(ctx, 2))
+ return -EINVAL;
+
+ hash_ctl_64k = FIELD_GET(DF4_HASH_CTL_64K, ctx->map.ctl);
+ hash_ctl_2M = FIELD_GET(DF4_HASH_CTL_2M, ctx->map.ctl);
+ hash_ctl_1G = FIELD_GET(DF4_HASH_CTL_1G, ctx->map.ctl);
+
+ intlv_bit = FIELD_GET(BIT_ULL(8), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(16), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
+
+ if (ctx->map.num_intlv_sockets == 1)
+ hashed_bit ^= FIELD_GET(BIT_ULL(14), ctx->ret_addr);
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(8);
+
+ /*
+ * Hashing is possible with socket interleaving, so check the total number
+ * of channels in the system rather than DRAM map interleaving mode.
+ *
+ * Calculation complete for 2 channels. Continue for 4, 8, and 16 channels.
+ */
+ if (ctx->map.total_intlv_chan <= 2)
+ return 0;
+
+ intlv_bit = FIELD_GET(BIT_ULL(12), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(17), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(12);
+
+ /* Calculation complete for 4 channels. Continue for 8 and 16 channels. */
+ if (ctx->map.total_intlv_chan <= 4)
+ return 0;
+
+ intlv_bit = FIELD_GET(BIT_ULL(13), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(13);
+
+ /* Calculation complete for 8 channels. Continue for 16 channels. */
+ if (ctx->map.total_intlv_chan <= 8)
+ return 0;
+
+ intlv_bit = FIELD_GET(BIT_ULL(14), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(19), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(24), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(33), ctx->ret_addr) & hash_ctl_1G;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(14);
+
+ return 0;
+}
+
+static int df4p5_dehash_addr(struct addr_ctx *ctx)
+{
+ bool hash_ctl_64k, hash_ctl_2M, hash_ctl_1G, hash_ctl_1T;
+ u8 hashed_bit, intlv_bit;
+ u64 rehash_vector;
+
+ /* Assert that interleave bit is 8. */
+ if (assert_intlv_bit(ctx, 8, 8))
+ return -EINVAL;
+
+ /* Assert that die interleaving is disabled. */
+ if (assert_num_intlv_dies(ctx, 1))
+ return -EINVAL;
+
+ /* Assert that no more than 2 sockets are interleaved. */
+ if (assert_num_intlv_sockets(ctx, 2))
+ return -EINVAL;
+
+ hash_ctl_64k = FIELD_GET(DF4_HASH_CTL_64K, ctx->map.ctl);
+ hash_ctl_2M = FIELD_GET(DF4_HASH_CTL_2M, ctx->map.ctl);
+ hash_ctl_1G = FIELD_GET(DF4_HASH_CTL_1G, ctx->map.ctl);
+ hash_ctl_1T = FIELD_GET(DF4_HASH_CTL_1T, ctx->map.ctl);
+
+ /*
+ * Generate a unique address to determine which bits
+ * need to be dehashed.
+ *
+ * Start with a contiguous bitmask for the total
+ * number of channels starting at bit 8.
+ *
+ * Then make a gap in the proper place based on
+ * interleave mode.
+ */
+ rehash_vector = ctx->map.total_intlv_chan - 1;
+ rehash_vector <<= 8;
+
+ if (ctx->map.intlv_mode == DF4p5_NPS2_4CHAN_1K_HASH ||
+ ctx->map.intlv_mode == DF4p5_NPS1_8CHAN_1K_HASH ||
+ ctx->map.intlv_mode == DF4p5_NPS1_16CHAN_1K_HASH)
+ rehash_vector = expand_bits(10, 2, rehash_vector);
+ else
+ rehash_vector = expand_bits(9, 3, rehash_vector);
+
+ if (rehash_vector & BIT_ULL(8)) {
+ intlv_bit = FIELD_GET(BIT_ULL(8), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(16), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
+ hashed_bit ^= FIELD_GET(BIT_ULL(40), ctx->ret_addr) & hash_ctl_1T;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(8);
+ }
+
+ if (rehash_vector & BIT_ULL(9)) {
+ intlv_bit = FIELD_GET(BIT_ULL(9), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(17), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
+ hashed_bit ^= FIELD_GET(BIT_ULL(41), ctx->ret_addr) & hash_ctl_1T;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(9);
+ }
+
+ if (rehash_vector & BIT_ULL(12)) {
+ intlv_bit = FIELD_GET(BIT_ULL(12), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
+ hashed_bit ^= FIELD_GET(BIT_ULL(42), ctx->ret_addr) & hash_ctl_1T;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(12);
+ }
+
+ if (rehash_vector & BIT_ULL(13)) {
+ intlv_bit = FIELD_GET(BIT_ULL(13), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(19), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(24), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(33), ctx->ret_addr) & hash_ctl_1G;
+ hashed_bit ^= FIELD_GET(BIT_ULL(43), ctx->ret_addr) & hash_ctl_1T;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(13);
+ }
+
+ if (rehash_vector & BIT_ULL(14)) {
+ intlv_bit = FIELD_GET(BIT_ULL(14), ctx->ret_addr);
+
+ hashed_bit = intlv_bit;
+ hashed_bit ^= FIELD_GET(BIT_ULL(20), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(25), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(34), ctx->ret_addr) & hash_ctl_1G;
+ hashed_bit ^= FIELD_GET(BIT_ULL(44), ctx->ret_addr) & hash_ctl_1T;
+
+ if (hashed_bit != intlv_bit)
+ ctx->ret_addr ^= BIT_ULL(14);
+ }
+
+ return 0;
+}
+
+int dehash_address(struct addr_ctx *ctx)
+{
+ switch (ctx->map.intlv_mode) {
+ /* No hashing cases. */
+ case NONE:
+ case NOHASH_2CHAN:
+ case NOHASH_4CHAN:
+ case NOHASH_8CHAN:
+ case NOHASH_16CHAN:
+ case NOHASH_32CHAN:
+ /* Hashing bits handled earlier during CS ID calculation. */
+ case DF4_NPS4_3CHAN_HASH:
+ case DF4_NPS2_5CHAN_HASH:
+ case DF4_NPS2_6CHAN_HASH:
+ case DF4_NPS1_10CHAN_HASH:
+ case DF4_NPS1_12CHAN_HASH:
+ case DF4p5_NPS2_6CHAN_1K_HASH:
+ case DF4p5_NPS2_6CHAN_2K_HASH:
+ case DF4p5_NPS1_10CHAN_1K_HASH:
+ case DF4p5_NPS1_10CHAN_2K_HASH:
+ case DF4p5_NPS1_12CHAN_1K_HASH:
+ case DF4p5_NPS1_12CHAN_2K_HASH:
+ case DF4p5_NPS0_24CHAN_1K_HASH:
+ case DF4p5_NPS0_24CHAN_2K_HASH:
+ /* No hash physical address bits, so nothing to do. */
+ case DF4p5_NPS4_3CHAN_1K_HASH:
+ case DF4p5_NPS4_3CHAN_2K_HASH:
+ case DF4p5_NPS2_5CHAN_1K_HASH:
+ case DF4p5_NPS2_5CHAN_2K_HASH:
+ return 0;
+
+ case DF2_2CHAN_HASH:
+ return df2_dehash_addr(ctx);
+
+ case DF3_COD4_2CHAN_HASH:
+ case DF3_COD2_4CHAN_HASH:
+ case DF3_COD1_8CHAN_HASH:
+ return df3_dehash_addr(ctx);
+
+ case DF3_6CHAN:
+ return df3_6chan_dehash_addr(ctx);
+
+ case DF4_NPS4_2CHAN_HASH:
+ case DF4_NPS2_4CHAN_HASH:
+ case DF4_NPS1_8CHAN_HASH:
+ return df4_dehash_addr(ctx);
+
+ case DF4p5_NPS4_2CHAN_1K_HASH:
+ case DF4p5_NPS4_2CHAN_2K_HASH:
+ case DF4p5_NPS2_4CHAN_2K_HASH:
+ case DF4p5_NPS2_4CHAN_1K_HASH:
+ case DF4p5_NPS1_8CHAN_1K_HASH:
+ case DF4p5_NPS1_8CHAN_2K_HASH:
+ case DF4p5_NPS1_16CHAN_1K_HASH:
+ case DF4p5_NPS1_16CHAN_2K_HASH:
+ return df4p5_dehash_addr(ctx);
+
+ default:
+ warn_on_bad_intlv_mode(ctx);
+ return -EINVAL;
+ }
+}
diff --git a/drivers/ras/amd/atl/denormalize.c b/drivers/ras/amd/atl/denormalize.c
new file mode 100644
index 000000000000..f84a370efe90
--- /dev/null
+++ b/drivers/ras/amd/atl/denormalize.c
@@ -0,0 +1,616 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AMD Address Translation Library
+ *
+ * denormalize.c : Functions to account for interleaving bits
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#include "internal.h"
+
+/*
+ * Returns the Destination Fabric ID. This is the first (lowest)
+ * CS Fabric ID used within a DRAM Address map.
+ */
+static u16 get_dst_fabric_id(struct addr_ctx *ctx)
+{
+ switch (df_cfg.rev) {
+ case DF2: return FIELD_GET(DF2_DST_FABRIC_ID, ctx->map.limit);
+ case DF3: return FIELD_GET(DF3_DST_FABRIC_ID, ctx->map.limit);
+ case DF3p5: return FIELD_GET(DF3p5_DST_FABRIC_ID, ctx->map.limit);
+ case DF4: return FIELD_GET(DF4_DST_FABRIC_ID, ctx->map.ctl);
+ case DF4p5: return FIELD_GET(DF4p5_DST_FABRIC_ID, ctx->map.ctl);
+ default:
+ warn_on_bad_df_rev();
+ return 0;
+ }
+}
+
+/*
+ * Make a contiguous gap in address for N bits starting at bit P.
+ *
+ * Example:
+ * address bits: [20:0]
+ * # of interleave bits (n): 3
+ * starting interleave bit (p): 8
+ *
+ * expanded address bits: [20+n : n+p][n+p-1 : p][p-1 : 0]
+ * [23 : 11][10 : 8][7 : 0]
+ */
+static u64 make_space_for_cs_id_at_intlv_bit(struct addr_ctx *ctx)
+{
+ return expand_bits(ctx->map.intlv_bit_pos,
+ ctx->map.total_intlv_bits,
+ ctx->ret_addr);
+}
+
+/*
+ * Make two gaps in address for N bits.
+ * First gap is a single bit at bit P.
+ * Second gap is the remaining N-1 bits at bit 12.
+ *
+ * Example:
+ * address bits: [20:0]
+ * # of interleave bits (n): 3
+ * starting interleave bit (p): 8
+ *
+ * First gap
+ * expanded address bits: [20+1 : p+1][p][p-1 : 0]
+ * [21 : 9][8][7 : 0]
+ *
+ * Second gap uses result from first.
+ * r = n - 1; remaining interleave bits
+ * expanded address bits: [21+r : 12+r][12+r-1: 12][11 : 0]
+ * [23 : 14][13 : 12][11 : 0]
+ */
+static u64 make_space_for_cs_id_split_2_1(struct addr_ctx *ctx)
+{
+ /* Make a single space at the interleave bit. */
+ u64 denorm_addr = expand_bits(ctx->map.intlv_bit_pos, 1, ctx->ret_addr);
+
+ /* Done if there's only a single interleave bit. */
+ if (ctx->map.total_intlv_bits <= 1)
+ return denorm_addr;
+
+ /* Make spaces for the remaining interleave bits starting at bit 12. */
+ return expand_bits(12, ctx->map.total_intlv_bits - 1, denorm_addr);
+}
+
+/*
+ * Take the current calculated address and shift enough bits in the middle
+ * to make a gap where the interleave bits will be inserted.
+ */
+static u64 make_space_for_cs_id(struct addr_ctx *ctx)
+{
+ switch (ctx->map.intlv_mode) {
+ case NOHASH_2CHAN:
+ case NOHASH_4CHAN:
+ case NOHASH_8CHAN:
+ case NOHASH_16CHAN:
+ case NOHASH_32CHAN:
+ case DF2_2CHAN_HASH:
+ return make_space_for_cs_id_at_intlv_bit(ctx);
+
+ case DF3_COD4_2CHAN_HASH:
+ case DF3_COD2_4CHAN_HASH:
+ case DF3_COD1_8CHAN_HASH:
+ case DF4_NPS4_2CHAN_HASH:
+ case DF4_NPS2_4CHAN_HASH:
+ case DF4_NPS1_8CHAN_HASH:
+ case DF4p5_NPS4_2CHAN_1K_HASH:
+ case DF4p5_NPS4_2CHAN_2K_HASH:
+ case DF4p5_NPS2_4CHAN_2K_HASH:
+ case DF4p5_NPS1_8CHAN_2K_HASH:
+ case DF4p5_NPS1_16CHAN_2K_HASH:
+ return make_space_for_cs_id_split_2_1(ctx);
+ default:
+ warn_on_bad_intlv_mode(ctx);
+ return ~0ULL;
+ }
+}
+
+static u16 get_cs_id_df2(struct addr_ctx *ctx)
+{
+ u8 num_socket_intlv_bits = ilog2(ctx->map.num_intlv_sockets);
+ u8 num_die_intlv_bits = ilog2(ctx->map.num_intlv_dies);
+ u8 num_intlv_bits;
+ u16 cs_id, mask;
+
+ cs_id = ctx->cs_fabric_id - get_dst_fabric_id(ctx);
+
+ /* Channel interleave bits */
+ num_intlv_bits = order_base_2(ctx->map.num_intlv_chan);
+ mask = GENMASK(num_intlv_bits - 1, 0);
+ cs_id &= mask;
+
+ /* Die interleave bits */
+ if (num_die_intlv_bits) {
+ u16 die_bits;
+
+ mask = GENMASK(num_die_intlv_bits - 1, 0);
+ die_bits = ctx->cs_fabric_id & df_cfg.die_id_mask;
+ die_bits >>= df_cfg.die_id_shift;
+
+ cs_id |= (die_bits & mask) << num_intlv_bits;
+ num_intlv_bits += num_die_intlv_bits;
+ }
+
+ /* Socket interleave bits */
+ if (num_socket_intlv_bits) {
+ u16 socket_bits;
+
+ mask = GENMASK(num_socket_intlv_bits - 1, 0);
+ socket_bits = ctx->cs_fabric_id & df_cfg.socket_id_mask;
+ socket_bits >>= df_cfg.socket_id_shift;
+
+ cs_id |= (socket_bits & mask) << num_intlv_bits;
+ }
+
+ return cs_id;
+}
+
+static u16 get_cs_id_df4(struct addr_ctx *ctx)
+{
+ /*
+ * Start with the original component mask and the number of interleave
+ * bits for the channels in this map.
+ */
+ u8 num_intlv_bits = ilog2(ctx->map.num_intlv_chan);
+ u16 mask = df_cfg.component_id_mask;
+
+ u16 socket_bits;
+
+ /* Set the derived CS ID to the input CS Fabric ID. */
+ u16 cs_id = ctx->cs_fabric_id & mask;
+
+ /*
+ * Subtract the "base" Destination Fabric ID.
+ * This accounts for systems with disabled Coherent Stations.
+ */
+ cs_id -= get_dst_fabric_id(ctx) & mask;
+
+ /*
+ * Generate and use a new mask based on the number of bits
+ * needed for channel interleaving in this map.
+ */
+ mask = GENMASK(num_intlv_bits - 1, 0);
+ cs_id &= mask;
+
+ /* Done if socket interleaving is not enabled. */
+ if (ctx->map.num_intlv_sockets <= 1)
+ return cs_id;
+
+ /*
+ * Figure out how many bits are needed for the number of
+ * interleaved sockets. And shift the derived CS ID to account
+ * for these.
+ */
+ num_intlv_bits = ilog2(ctx->map.num_intlv_sockets);
+ cs_id <<= num_intlv_bits;
+
+ /* Generate a new mask for the socket interleaving bits. */
+ mask = GENMASK(num_intlv_bits - 1, 0);
+
+ /* Get the socket interleave bits from the original CS Fabric ID. */
+ socket_bits = (ctx->cs_fabric_id & df_cfg.socket_id_mask) >> df_cfg.socket_id_shift;
+
+ /* Apply the appropriate socket bits to the derived CS ID. */
+ cs_id |= socket_bits & mask;
+
+ return cs_id;
+}
+
+/*
+ * Derive the correct CS ID that represents the interleave bits
+ * used within the system physical address. This accounts for the
+ * interleave mode, number of interleaved channels/dies/sockets, and
+ * other system/mode-specific bit swizzling.
+ *
+ * Returns: CS ID on success.
+ * All bits set on error.
+ */
+static u16 calculate_cs_id(struct addr_ctx *ctx)
+{
+ switch (ctx->map.intlv_mode) {
+ case NOHASH_2CHAN:
+ case NOHASH_4CHAN:
+ case NOHASH_8CHAN:
+ case NOHASH_16CHAN:
+ case NOHASH_32CHAN:
+ case DF3_COD4_2CHAN_HASH:
+ case DF3_COD2_4CHAN_HASH:
+ case DF3_COD1_8CHAN_HASH:
+ case DF2_2CHAN_HASH:
+ return get_cs_id_df2(ctx);
+
+ case DF4_NPS4_2CHAN_HASH:
+ case DF4_NPS2_4CHAN_HASH:
+ case DF4_NPS1_8CHAN_HASH:
+ case DF4p5_NPS4_2CHAN_1K_HASH:
+ case DF4p5_NPS4_2CHAN_2K_HASH:
+ case DF4p5_NPS2_4CHAN_2K_HASH:
+ case DF4p5_NPS1_8CHAN_2K_HASH:
+ case DF4p5_NPS1_16CHAN_2K_HASH:
+ return get_cs_id_df4(ctx);
+
+ /* CS ID is simply the CS Fabric ID adjusted by the Destination Fabric ID. */
+ case DF4p5_NPS2_4CHAN_1K_HASH:
+ case DF4p5_NPS1_8CHAN_1K_HASH:
+ case DF4p5_NPS1_16CHAN_1K_HASH:
+ return ctx->cs_fabric_id - get_dst_fabric_id(ctx);
+
+ default:
+ warn_on_bad_intlv_mode(ctx);
+ return ~0;
+ }
+}
+
+static u64 insert_cs_id_at_intlv_bit(struct addr_ctx *ctx, u64 denorm_addr, u16 cs_id)
+{
+ return denorm_addr | (cs_id << ctx->map.intlv_bit_pos);
+}
+
+static u64 insert_cs_id_split_2_1(struct addr_ctx *ctx, u64 denorm_addr, u16 cs_id)
+{
+ /* Insert cs_id[0] at the interleave bit. */
+ denorm_addr |= (cs_id & BIT(0)) << ctx->map.intlv_bit_pos;
+
+ /* Insert cs_id[2:1] at bit 12. */
+ denorm_addr |= (cs_id & GENMASK(2, 1)) << 11;
+
+ return denorm_addr;
+}
+
+static u64 insert_cs_id_split_2_2(struct addr_ctx *ctx, u64 denorm_addr, u16 cs_id)
+{
+ /* Insert cs_id[1:0] at bit 8. */
+ denorm_addr |= (cs_id & GENMASK(1, 0)) << 8;
+
+ /*
+ * Insert cs_id[n:2] at bit 12. 'n' could be 2 or 3.
+ * Grab both because bit 3 will be clear if unused.
+ */
+ denorm_addr |= (cs_id & GENMASK(3, 2)) << 10;
+
+ return denorm_addr;
+}
+
+static u64 insert_cs_id(struct addr_ctx *ctx, u64 denorm_addr, u16 cs_id)
+{
+ switch (ctx->map.intlv_mode) {
+ case NOHASH_2CHAN:
+ case NOHASH_4CHAN:
+ case NOHASH_8CHAN:
+ case NOHASH_16CHAN:
+ case NOHASH_32CHAN:
+ case DF2_2CHAN_HASH:
+ return insert_cs_id_at_intlv_bit(ctx, denorm_addr, cs_id);
+
+ case DF3_COD4_2CHAN_HASH:
+ case DF3_COD2_4CHAN_HASH:
+ case DF3_COD1_8CHAN_HASH:
+ case DF4_NPS4_2CHAN_HASH:
+ case DF4_NPS2_4CHAN_HASH:
+ case DF4_NPS1_8CHAN_HASH:
+ case DF4p5_NPS4_2CHAN_1K_HASH:
+ case DF4p5_NPS4_2CHAN_2K_HASH:
+ case DF4p5_NPS2_4CHAN_2K_HASH:
+ case DF4p5_NPS1_8CHAN_2K_HASH:
+ case DF4p5_NPS1_16CHAN_2K_HASH:
+ return insert_cs_id_split_2_1(ctx, denorm_addr, cs_id);
+
+ case DF4p5_NPS2_4CHAN_1K_HASH:
+ case DF4p5_NPS1_8CHAN_1K_HASH:
+ case DF4p5_NPS1_16CHAN_1K_HASH:
+ return insert_cs_id_split_2_2(ctx, denorm_addr, cs_id);
+
+ default:
+ warn_on_bad_intlv_mode(ctx);
+ return ~0ULL;
+ }
+}
+
+static u16 get_logical_cs_fabric_id(struct addr_ctx *ctx)
+{
+ u16 cs_component_id, log_cs_fabric_id;
+
+ /* Start with the physical CS Fabric ID. */
+ u16 phys_cs_fabric_id = ctx->cs_fabric_id;
+
+ /* Skip logical ID lookup if remapping is disabled. */
+ if (!FIELD_GET(DF4_REMAP_EN, ctx->map.ctl) &&
+ ctx->map.intlv_mode != DF3_6CHAN)
+ return phys_cs_fabric_id;
+
+ /* Mask off the Node ID bits to get the "local" Component ID. */
+ cs_component_id = phys_cs_fabric_id & df_cfg.component_id_mask;
+
+ /*
+ * Search the list of logical Component IDs for the one that
+ * matches this physical Component ID.
+ */
+ for (log_cs_fabric_id = 0; log_cs_fabric_id < MAX_CS_CHANNELS; log_cs_fabric_id++) {
+ if (ctx->map.remap_array[log_cs_fabric_id] == cs_component_id)
+ break;
+ }
+
+ if (log_cs_fabric_id == MAX_CS_CHANNELS)
+ warn_on_assert("CS remap entry not found for 0x%x", log_cs_fabric_id);
+
+ /* Get the Node ID bits from the physical and apply to the logical. */
+ return (phys_cs_fabric_id & df_cfg.node_id_mask) | log_cs_fabric_id;
+}
+
+static int denorm_addr_common(struct addr_ctx *ctx)
+{
+ u64 denorm_addr;
+ u16 cs_id;
+
+ /*
+ * Convert the original physical CS Fabric ID to a logical value.
+ * This is required for non-power-of-two and other interleaving modes.
+ */
+ ctx->cs_fabric_id = get_logical_cs_fabric_id(ctx);
+
+ denorm_addr = make_space_for_cs_id(ctx);
+ cs_id = calculate_cs_id(ctx);
+ ctx->ret_addr = insert_cs_id(ctx, denorm_addr, cs_id);
+ return 0;
+}
+
+static int denorm_addr_df3_6chan(struct addr_ctx *ctx)
+{
+ u16 cs_id = ctx->cs_fabric_id & df_cfg.component_id_mask;
+ u8 total_intlv_bits = ctx->map.total_intlv_bits;
+ u8 low_bit, intlv_bit = ctx->map.intlv_bit_pos;
+ u64 msb_intlv_bits, temp_addr_a, temp_addr_b;
+ u8 np2_bits = ctx->map.np2_bits;
+
+ if (ctx->map.intlv_mode != DF3_6CHAN)
+ return -EINVAL;
+
+ /*
+ * 'np2_bits' holds the number of bits needed to cover the
+ * amount of memory (rounded up) in this map using 64K chunks.
+ *
+ * Example:
+ * Total memory in map: 6GB
+ * Rounded up to next power-of-2: 8GB
+ * Number of 64K chunks: 0x20000
+ * np2_bits = log2(# of chunks): 17
+ *
+ * Get the two most-significant interleave bits from the
+ * input address based on the following:
+ *
+ * [15 + np2_bits - total_intlv_bits : 14 + np2_bits - total_intlv_bits]
+ */
+ low_bit = 14 + np2_bits - total_intlv_bits;
+ msb_intlv_bits = ctx->ret_addr >> low_bit;
+ msb_intlv_bits &= 0x3;
+
+ /*
+ * If MSB are 11b, then logical CS ID is 6 or 7.
+ * Need to adjust based on the mod3 result.
+ */
+ if (msb_intlv_bits == 3) {
+ u8 addr_mod, phys_addr_msb, msb_cs_id;
+
+ /* Get the remaining interleave bits from the input address. */
+ temp_addr_b = GENMASK_ULL(low_bit - 1, intlv_bit) & ctx->ret_addr;
+ temp_addr_b >>= intlv_bit;
+
+ /* Calculate the logical CS offset based on mod3. */
+ addr_mod = temp_addr_b % 3;
+
+ /* Get CS ID bits [2:1]. */
+ msb_cs_id = (cs_id >> 1) & 0x3;
+
+ /* Get the bit that starts the physical address bits. */
+ phys_addr_msb = (intlv_bit + np2_bits + 1);
+ phys_addr_msb &= BIT(0);
+ phys_addr_msb++;
+ phys_addr_msb *= 3 - addr_mod + msb_cs_id;
+ phys_addr_msb %= 3;
+
+ /* Move the physical address MSB to the correct place. */
+ temp_addr_b |= phys_addr_msb << (low_bit - total_intlv_bits - intlv_bit);
+
+ /* Generate a new CS ID as follows: cs_id = [1, 1, cs_id[0]] */
+ cs_id &= BIT(0);
+ cs_id |= GENMASK(2, 1);
+ } else {
+ temp_addr_b = GENMASK_ULL(63, intlv_bit) & ctx->ret_addr;
+ temp_addr_b >>= intlv_bit;
+ }
+
+ temp_addr_a = GENMASK_ULL(intlv_bit - 1, 0) & ctx->ret_addr;
+ temp_addr_b <<= intlv_bit + total_intlv_bits;
+
+ ctx->ret_addr = temp_addr_a | temp_addr_b;
+ ctx->ret_addr |= cs_id << intlv_bit;
+ return 0;
+}
+
+static int denorm_addr_df4_np2(struct addr_ctx *ctx)
+{
+ bool hash_ctl_64k, hash_ctl_2M, hash_ctl_1G;
+ u16 group, group_offset, log_cs_offset;
+ unsigned int mod_value, shift_value;
+ u16 mask = df_cfg.component_id_mask;
+ u64 temp_addr_a, temp_addr_b;
+ u8 hash_pa8, hashed_bit;
+
+ switch (ctx->map.intlv_mode) {
+ case DF4_NPS4_3CHAN_HASH:
+ mod_value = 3;
+ shift_value = 13;
+ break;
+ case DF4_NPS2_6CHAN_HASH:
+ mod_value = 3;
+ shift_value = 12;
+ break;
+ case DF4_NPS1_12CHAN_HASH:
+ mod_value = 3;
+ shift_value = 11;
+ break;
+ case DF4_NPS2_5CHAN_HASH:
+ mod_value = 5;
+ shift_value = 13;
+ break;
+ case DF4_NPS1_10CHAN_HASH:
+ mod_value = 5;
+ shift_value = 12;
+ break;
+ default:
+ warn_on_bad_intlv_mode(ctx);
+ return -EINVAL;
+ };
+
+ if (ctx->map.num_intlv_sockets == 1) {
+ hash_pa8 = BIT_ULL(shift_value) & ctx->ret_addr;
+ temp_addr_a = remove_bits(shift_value, shift_value, ctx->ret_addr);
+ } else {
+ hash_pa8 = (ctx->cs_fabric_id & df_cfg.socket_id_mask);
+ hash_pa8 >>= df_cfg.socket_id_shift;
+ temp_addr_a = ctx->ret_addr;
+ }
+
+ /* Make a gap for the real bit [8]. */
+ temp_addr_a = expand_bits(8, 1, temp_addr_a);
+
+ /* Make an additional gap for bits [13:12], as appropriate.*/
+ if (ctx->map.intlv_mode == DF4_NPS2_6CHAN_HASH ||
+ ctx->map.intlv_mode == DF4_NPS1_10CHAN_HASH) {
+ temp_addr_a = expand_bits(13, 1, temp_addr_a);
+ } else if (ctx->map.intlv_mode == DF4_NPS1_12CHAN_HASH) {
+ temp_addr_a = expand_bits(12, 2, temp_addr_a);
+ }
+
+ /* Keep bits [13:0]. */
+ temp_addr_a &= GENMASK_ULL(13, 0);
+
+ /* Get the appropriate high bits. */
+ shift_value += 1 - ilog2(ctx->map.num_intlv_sockets);
+ temp_addr_b = GENMASK_ULL(63, shift_value) & ctx->ret_addr;
+ temp_addr_b >>= shift_value;
+ temp_addr_b *= mod_value;
+
+ /*
+ * Coherent Stations are divided into groups.
+ *
+ * Multiples of 3 (mod3) are divided into quadrants.
+ * e.g. NP4_3CHAN -> [0, 1, 2] [6, 7, 8]
+ * [3, 4, 5] [9, 10, 11]
+ *
+ * Multiples of 5 (mod5) are divided into sides.
+ * e.g. NP2_5CHAN -> [0, 1, 2, 3, 4] [5, 6, 7, 8, 9]
+ */
+
+ /*
+ * Calculate the logical offset for the CS within its DRAM Address map.
+ * e.g. if map includes [5, 6, 7, 8, 9] and target instance is '8', then
+ * log_cs_offset = 8 - 5 = 3
+ */
+ log_cs_offset = (ctx->cs_fabric_id & mask) - (get_dst_fabric_id(ctx) & mask);
+
+ /*
+ * Figure out the group number.
+ *
+ * Following above example,
+ * log_cs_offset = 3
+ * mod_value = 5
+ * group = 3 / 5 = 0
+ */
+ group = log_cs_offset / mod_value;
+
+ /*
+ * Figure out the offset within the group.
+ *
+ * Following above example,
+ * log_cs_offset = 3
+ * mod_value = 5
+ * group_offset = 3 % 5 = 3
+ */
+ group_offset = log_cs_offset % mod_value;
+
+ /* Adjust group_offset if the hashed bit [8] is set. */
+ if (hash_pa8) {
+ if (!group_offset)
+ group_offset = mod_value - 1;
+ else
+ group_offset--;
+ }
+
+ /* Add in the group offset to the high bits. */
+ temp_addr_b += group_offset;
+
+ /* Shift the high bits to the proper starting position. */
+ temp_addr_b <<= 14;
+
+ /* Combine the high and low bits together. */
+ ctx->ret_addr = temp_addr_a | temp_addr_b;
+
+ /* Account for hashing here instead of in dehash_address(). */
+ hash_ctl_64k = FIELD_GET(DF4_HASH_CTL_64K, ctx->map.ctl);
+ hash_ctl_2M = FIELD_GET(DF4_HASH_CTL_2M, ctx->map.ctl);
+ hash_ctl_1G = FIELD_GET(DF4_HASH_CTL_1G, ctx->map.ctl);
+
+ hashed_bit = !!hash_pa8;
+ hashed_bit ^= FIELD_GET(BIT_ULL(14), ctx->ret_addr);
+ hashed_bit ^= FIELD_GET(BIT_ULL(16), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
+
+ ctx->ret_addr |= hashed_bit << 8;
+
+ /* Done for 3 and 5 channel. */
+ if (ctx->map.intlv_mode == DF4_NPS4_3CHAN_HASH ||
+ ctx->map.intlv_mode == DF4_NPS2_5CHAN_HASH)
+ return 0;
+
+ /* Select the proper 'group' bit to use for Bit 13. */
+ if (ctx->map.intlv_mode == DF4_NPS1_12CHAN_HASH)
+ hashed_bit = !!(group & BIT(1));
+ else
+ hashed_bit = group & BIT(0);
+
+ hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
+
+ ctx->ret_addr |= hashed_bit << 13;
+
+ /* Done for 6 and 10 channel. */
+ if (ctx->map.intlv_mode != DF4_NPS1_12CHAN_HASH)
+ return 0;
+
+ hashed_bit = group & BIT(0);
+ hashed_bit ^= FIELD_GET(BIT_ULL(17), ctx->ret_addr) & hash_ctl_64k;
+ hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
+ hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
+
+ ctx->ret_addr |= hashed_bit << 12;
+ return 0;
+}
+
+int denormalize_address(struct addr_ctx *ctx)
+{
+ switch (ctx->map.intlv_mode) {
+ case NONE:
+ return 0;
+ case DF4_NPS4_3CHAN_HASH:
+ case DF4_NPS2_6CHAN_HASH:
+ case DF4_NPS1_12CHAN_HASH:
+ case DF4_NPS2_5CHAN_HASH:
+ case DF4_NPS1_10CHAN_HASH:
+ return denorm_addr_df4_np2(ctx);
+ case DF3_6CHAN:
+ return denorm_addr_df3_6chan(ctx);
+ default:
+ return denorm_addr_common(ctx);
+ }
+}
diff --git a/drivers/ras/amd/atl/internal.h b/drivers/ras/amd/atl/internal.h
new file mode 100644
index 000000000000..08bc46f7cabf
--- /dev/null
+++ b/drivers/ras/amd/atl/internal.h
@@ -0,0 +1,312 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AMD Address Translation Library
+ *
+ * internal.h : Helper functions and common defines
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#ifndef __AMD_ATL_INTERNAL_H__
+#define __AMD_ATL_INTERNAL_H__
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+
+#include <asm/amd_nb.h>
+
+#include "reg_fields.h"
+#include "stub.h"
+
+/* Maximum possible number of Coherent Stations within a single Data Fabric. */
+#define MAX_CS_CHANNELS 32
+
+/* PCI ID for Zen4 Server DF Function 0. */
+#define DF_FUNC0_ID_ZEN4_SERVER 0x14AD1022
+
+/* Shift needed for adjusting register values to true values. */
+#define DF_DRAM_BASE_LIMIT_LSB 28
+
+enum df_revisions {
+ UNKNOWN,
+ DF2,
+ DF3,
+ DF3p5,
+ DF4,
+ DF4p5,
+};
+
+/* These are mapped 1:1 to the hardware values. Special cases are set at > 0x20. */
+enum intlv_modes {
+ NONE = 0x00,
+ NOHASH_2CHAN = 0x01,
+ NOHASH_4CHAN = 0x03,
+ NOHASH_8CHAN = 0x05,
+ DF3_6CHAN = 0x06,
+ NOHASH_16CHAN = 0x07,
+ NOHASH_32CHAN = 0x08,
+ DF3_COD4_2CHAN_HASH = 0x0C,
+ DF3_COD2_4CHAN_HASH = 0x0D,
+ DF3_COD1_8CHAN_HASH = 0x0E,
+ DF4_NPS4_2CHAN_HASH = 0x10,
+ DF4_NPS2_4CHAN_HASH = 0x11,
+ DF4_NPS1_8CHAN_HASH = 0x12,
+ DF4_NPS4_3CHAN_HASH = 0x13,
+ DF4_NPS2_6CHAN_HASH = 0x14,
+ DF4_NPS1_12CHAN_HASH = 0x15,
+ DF4_NPS2_5CHAN_HASH = 0x16,
+ DF4_NPS1_10CHAN_HASH = 0x17,
+ DF2_2CHAN_HASH = 0x21,
+ /* DF4.5 modes are all IntLvNumChan + 0x20 */
+ DF4p5_NPS1_16CHAN_1K_HASH = 0x2C,
+ DF4p5_NPS0_24CHAN_1K_HASH = 0x2E,
+ DF4p5_NPS4_2CHAN_1K_HASH = 0x30,
+ DF4p5_NPS2_4CHAN_1K_HASH = 0x31,
+ DF4p5_NPS1_8CHAN_1K_HASH = 0x32,
+ DF4p5_NPS4_3CHAN_1K_HASH = 0x33,
+ DF4p5_NPS2_6CHAN_1K_HASH = 0x34,
+ DF4p5_NPS1_12CHAN_1K_HASH = 0x35,
+ DF4p5_NPS2_5CHAN_1K_HASH = 0x36,
+ DF4p5_NPS1_10CHAN_1K_HASH = 0x37,
+ DF4p5_NPS4_2CHAN_2K_HASH = 0x40,
+ DF4p5_NPS2_4CHAN_2K_HASH = 0x41,
+ DF4p5_NPS1_8CHAN_2K_HASH = 0x42,
+ DF4p5_NPS1_16CHAN_2K_HASH = 0x43,
+ DF4p5_NPS4_3CHAN_2K_HASH = 0x44,
+ DF4p5_NPS2_6CHAN_2K_HASH = 0x45,
+ DF4p5_NPS1_12CHAN_2K_HASH = 0x46,
+ DF4p5_NPS0_24CHAN_2K_HASH = 0x47,
+ DF4p5_NPS2_5CHAN_2K_HASH = 0x48,
+ DF4p5_NPS1_10CHAN_2K_HASH = 0x49,
+};
+
+struct df_flags {
+ __u8 legacy_ficaa : 1,
+ socket_id_shift_quirk : 1,
+ __reserved_0 : 6;
+};
+
+struct df_config {
+ enum df_revisions rev;
+
+ /*
+ * These masks operate on the 16-bit CS IDs,
+ * e.g. Instance, Fabric, Destination, etc.
+ */
+ u16 component_id_mask;
+ u16 die_id_mask;
+ u16 node_id_mask;
+ u16 socket_id_mask;
+
+ /*
+ * Least-significant bit of Node ID portion of the
+ * system-wide CS Fabric ID.
+ */
+ u8 node_id_shift;
+
+ /*
+ * Least-significant bit of Die portion of the Node ID.
+ * Adjusted to include the Node ID shift in order to apply
+ * to the CS Fabric ID.
+ */
+ u8 die_id_shift;
+
+ /*
+ * Least-significant bit of Socket portion of the Node ID.
+ * Adjusted to include the Node ID shift in order to apply
+ * to the CS Fabric ID.
+ */
+ u8 socket_id_shift;
+
+ /* Number of DRAM Address maps visible in a CS. */
+ u8 num_cs_maps;
+
+ /* Global flags to handle special cases. */
+ struct df_flags flags;
+};
+
+extern struct df_config df_cfg;
+
+struct dram_addr_map {
+ /*
+ * Each DRAM Address Map can operate independently
+ * in different interleaving modes.
+ */
+ enum intlv_modes intlv_mode;
+
+ /* System-wide number for this address map. */
+ u8 num;
+
+ /* Raw register values */
+ u32 base;
+ u32 limit;
+ u32 ctl;
+ u32 intlv;
+
+ /*
+ * Logical to Physical CS Remapping array
+ *
+ * Index: Logical CS Instance ID
+ * Value: Physical CS Instance ID
+ *
+ * phys_cs_inst_id = remap_array[log_cs_inst_id]
+ */
+ u8 remap_array[MAX_CS_CHANNELS];
+
+ /*
+ * Number of bits covering DRAM Address map 0
+ * when interleaving is non-power-of-2.
+ *
+ * Used only for DF3_6CHAN.
+ */
+ u8 np2_bits;
+
+ /* Position of the 'interleave bit'. */
+ u8 intlv_bit_pos;
+ /* Number of channels interleaved in this map. */
+ u8 num_intlv_chan;
+ /* Number of dies interleaved in this map. */
+ u8 num_intlv_dies;
+ /* Number of sockets interleaved in this map. */
+ u8 num_intlv_sockets;
+ /*
+ * Total number of channels interleaved accounting
+ * for die and socket interleaving.
+ */
+ u8 total_intlv_chan;
+ /* Total bits needed to cover 'total_intlv_chan'. */
+ u8 total_intlv_bits;
+};
+
+/* Original input values cached for debug printing. */
+struct addr_ctx_inputs {
+ u64 norm_addr;
+ u8 socket_id;
+ u8 die_id;
+ u8 cs_inst_id;
+};
+
+struct addr_ctx {
+ u64 ret_addr;
+
+ struct addr_ctx_inputs inputs;
+ struct dram_addr_map map;
+
+ /* AMD Node ID calculated from Socket and Die IDs. */
+ u8 node_id;
+
+ /*
+ * CS Instance ID
+ * Local ID used within a 'node'.
+ */
+ u16 inst_id;
+
+ /*
+ * CS Fabric ID
+ * System-wide ID that includes 'node' bits.
+ */
+ u16 cs_fabric_id;
+};
+
+int df_indirect_read_instance(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo);
+int df_indirect_read_broadcast(u16 node, u8 func, u16 reg, u32 *lo);
+
+int get_df_system_info(void);
+int determine_node_id(struct addr_ctx *ctx, u8 socket_num, u8 die_num);
+
+int get_address_map(struct addr_ctx *ctx);
+
+int denormalize_address(struct addr_ctx *ctx);
+int dehash_address(struct addr_ctx *ctx);
+
+int norm_to_sys_addr(u8 socket_id, u8 die_id, u8 cs_inst_id, u64 *addr);
+int convert_umc_mca_addr_to_sys_addr(struct mce *m, u64 *sys_addr);
+
+/*
+ * Helper to use test_bit() without needing to do
+ * a cast to "unsigned long *" everywhere.
+ *
+ * Use this for dynamic checks, i.e. when FIELD_GET()
+ * won't work.
+ */
+static inline bool atl_get_bit(u8 bit_num, u64 data)
+{
+ return test_bit(bit_num, (unsigned long *)&data);
+}
+
+/*
+ * Make a gap in 'data' that is 'num_bits' long starting at 'bit_num.
+ * e.g. data = 11111111'b
+ * bit_num = 3
+ * num_bits = 2
+ * result = 1111100111'b
+ */
+static inline u64 expand_bits(u8 bit_num, u8 num_bits, u64 data)
+{
+ u64 temp1, temp2;
+
+ /*
+ * Return the original data if the "space" needed is '0'.
+ * This helps avoid the need to check for '0' at each
+ * caller.
+ */
+ if (!num_bits)
+ return data;
+
+ if (!bit_num)
+ return data << num_bits;
+
+ temp1 = data & GENMASK_ULL(bit_num - 1, 0);
+
+ temp2 = data & GENMASK_ULL(63, bit_num);
+ temp2 <<= num_bits;
+
+ return temp1 | temp2;
+}
+
+/*
+ * Remove bits in 'data' between low_bit and high_bit inclusive.
+ * e.g. data = XXXYYZZZ'b
+ * low_bit = 3
+ * high_bit = 4
+ * result = XXXZZZ'b
+ */
+static inline u64 remove_bits(u8 low_bit, u8 high_bit, u64 data)
+{
+ u64 temp1, temp2;
+
+ if (high_bit >= 64 || low_bit >= 64 || low_bit > high_bit) {
+ pr_warn("%s: Invalid inputs: low_bit=%u high_bit=%u",
+ __func__, low_bit, high_bit);
+ return 0;
+ }
+
+ if (!low_bit)
+ return data >> (high_bit++);
+
+ temp1 = GENMASK_ULL(low_bit - 1, 0) & data;
+ temp2 = GENMASK_ULL(63, high_bit + 1) & data;
+ temp2 >>= high_bit - low_bit + 1;
+
+ return temp1 | temp2;
+}
+
+#define warn_on_assert(fmt, arg...) \
+ pr_warn("amd_atl: socket_id=%u die_id=%u cs_inst_id=%u norm_addr=0x%016llx: " fmt,\
+ ctx->inputs.socket_id, ctx->inputs.die_id,\
+ ctx->inputs.cs_inst_id, ctx->inputs.norm_addr, ##arg)
+
+static inline void warn_on_bad_df_rev(void)
+{
+ pr_warn("amd_atl:%s: Unrecognized DF rev: %u", __func__, df_cfg.rev);
+}
+
+static inline void warn_on_bad_intlv_mode(struct addr_ctx *ctx)
+{
+ warn_on_assert("Unrecognized interleave mode: %u", ctx->map.intlv_mode);
+}
+
+#endif /* __AMD_ATL_INTERNAL_H__ */
diff --git a/drivers/ras/amd/atl/map.c b/drivers/ras/amd/atl/map.c
new file mode 100644
index 000000000000..1de723178cb5
--- /dev/null
+++ b/drivers/ras/amd/atl/map.c
@@ -0,0 +1,659 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AMD Address Translation Library
+ *
+ * map.c : Functions to read and decode DRAM address maps
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#include "internal.h"
+
+static int df2_get_intlv_mode(struct addr_ctx *ctx)
+{
+ ctx->map.intlv_mode = FIELD_GET(DF2_INTLV_NUM_CHAN, ctx->map.base);
+
+ if (ctx->map.intlv_mode == 8)
+ ctx->map.intlv_mode = DF2_2CHAN_HASH;
+
+ if (ctx->map.intlv_mode != NONE &&
+ ctx->map.intlv_mode != NOHASH_2CHAN &&
+ ctx->map.intlv_mode != DF2_2CHAN_HASH) {
+ warn_on_bad_intlv_mode(ctx);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int df3_get_intlv_mode(struct addr_ctx *ctx)
+{
+ ctx->map.intlv_mode = FIELD_GET(DF3_INTLV_NUM_CHAN, ctx->map.base);
+ return 0;
+}
+
+static int df3p5_get_intlv_mode(struct addr_ctx *ctx)
+{
+ ctx->map.intlv_mode = FIELD_GET(DF3p5_INTLV_NUM_CHAN, ctx->map.base);
+
+ if (ctx->map.intlv_mode == DF3_6CHAN) {
+ warn_on_bad_intlv_mode(ctx);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int df4_get_intlv_mode(struct addr_ctx *ctx)
+{
+ ctx->map.intlv_mode = FIELD_GET(DF4_INTLV_NUM_CHAN, ctx->map.intlv);
+
+ if (ctx->map.intlv_mode == DF3_COD4_2CHAN_HASH ||
+ ctx->map.intlv_mode == DF3_COD2_4CHAN_HASH ||
+ ctx->map.intlv_mode == DF3_COD1_8CHAN_HASH ||
+ ctx->map.intlv_mode == DF3_6CHAN) {
+ warn_on_bad_intlv_mode(ctx);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int df4p5_get_intlv_mode(struct addr_ctx *ctx)
+{
+ ctx->map.intlv_mode = FIELD_GET(DF4p5_INTLV_NUM_CHAN, ctx->map.intlv);
+
+ if (ctx->map.intlv_mode <= NOHASH_32CHAN)
+ return 0;
+
+ /*
+ * Modes matching the ranges above are returned as-is.
+ *
+ * All other modes are "fixed up" by adding 20h to make a unique value.
+ */
+ ctx->map.intlv_mode += 0x20;
+
+ return 0;
+}
+
+/*
+ * Some, but not all, cases have asserts.
+ * So use return values to indicate failure where needed.
+ */
+static int get_intlv_mode(struct addr_ctx *ctx)
+{
+ switch (df_cfg.rev) {
+ case DF2: return df2_get_intlv_mode(ctx);
+ case DF3: return df3_get_intlv_mode(ctx);
+ case DF3p5: return df3p5_get_intlv_mode(ctx);
+ case DF4: return df4_get_intlv_mode(ctx);
+ case DF4p5: return df4p5_get_intlv_mode(ctx);
+ default:
+ warn_on_bad_df_rev();
+ return -EINVAL;
+ }
+}
+
+static u64 get_hi_addr_offset(u32 reg_dram_offset)
+{
+ u8 shift = DF_DRAM_BASE_LIMIT_LSB;
+ u64 hi_addr_offset = 0;
+
+ switch (df_cfg.rev) {
+ case DF2:
+ hi_addr_offset = FIELD_GET(DF2_HI_ADDR_OFFSET, reg_dram_offset);
+ break;
+ case DF3:
+ case DF3p5:
+ hi_addr_offset = FIELD_GET(DF3_HI_ADDR_OFFSET, reg_dram_offset);
+ break;
+ case DF4:
+ case DF4p5:
+ hi_addr_offset = FIELD_GET(DF4_HI_ADDR_OFFSET, reg_dram_offset);
+ break;
+ default:
+ warn_on_bad_df_rev();
+ }
+
+ return hi_addr_offset << shift;
+}
+
+static int get_dram_offset(struct addr_ctx *ctx, bool *enabled, u64 *norm_offset)
+{
+ u32 reg_dram_offset;
+ u8 map_num;
+
+ /* Should not be called for map 0. */
+ if (!ctx->map.num) {
+ warn_on_assert("Trying to find DRAM offset for map 0");
+ return -EINVAL;
+ }
+
+ /*
+ * DramOffset registers don't exist for map 0, so the base register
+ * actually refers to map 1.
+ * Adjust the map_num for the register offsets.
+ */
+ map_num = ctx->map.num - 1;
+
+ if (df_cfg.rev >= DF4) {
+ /* Read D18F7x140 (DramOffset) */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x140 + (4 * map_num),
+ ctx->inst_id, ®_dram_offset))
+ return -EINVAL;
+
+ } else {
+ /* Read D18F0x1B4 (DramOffset) */
+ if (df_indirect_read_instance(ctx->node_id, 0, 0x1B4 + (4 * map_num),
+ ctx->inst_id, ®_dram_offset))
+ return -EINVAL;
+ }
+
+ if (!FIELD_GET(DF_HI_ADDR_OFFSET_EN, reg_dram_offset))
+ return 0;
+
+ *enabled = true;
+ *norm_offset = get_hi_addr_offset(reg_dram_offset);
+
+ return 0;
+}
+
+static int df3_6ch_get_dram_addr_map(struct addr_ctx *ctx)
+{
+ u16 dst_fabric_id = FIELD_GET(DF3_DST_FABRIC_ID, ctx->map.limit);
+ u8 i, j, shift = 4, mask = 0xF;
+ u32 reg, offset = 0x60;
+ u16 dst_node_id;
+
+ /* Get Socket 1 register. */
+ if (dst_fabric_id & df_cfg.socket_id_mask)
+ offset = 0x68;
+
+ /* Read D18F0x06{0,8} (DF::Skt0CsTargetRemap0)/(DF::Skt0CsTargetRemap1) */
+ if (df_indirect_read_broadcast(ctx->node_id, 0, offset, ®))
+ return -EINVAL;
+
+ /* Save 8 remap entries. */
+ for (i = 0, j = 0; i < 8; i++, j++)
+ ctx->map.remap_array[i] = (reg >> (j * shift)) & mask;
+
+ dst_node_id = dst_fabric_id & df_cfg.node_id_mask;
+ dst_node_id >>= df_cfg.node_id_shift;
+
+ /* Read D18F2x090 (DF::Np2ChannelConfig) */
+ if (df_indirect_read_broadcast(dst_node_id, 2, 0x90, ®))
+ return -EINVAL;
+
+ ctx->map.np2_bits = FIELD_GET(DF_LOG2_ADDR_64K_SPACE0, reg);
+ return 0;
+}
+
+static int df2_get_dram_addr_map(struct addr_ctx *ctx)
+{
+ /* Read D18F0x110 (DramBaseAddress). */
+ if (df_indirect_read_instance(ctx->node_id, 0, 0x110 + (8 * ctx->map.num),
+ ctx->inst_id, &ctx->map.base))
+ return -EINVAL;
+
+ /* Read D18F0x114 (DramLimitAddress). */
+ if (df_indirect_read_instance(ctx->node_id, 0, 0x114 + (8 * ctx->map.num),
+ ctx->inst_id, &ctx->map.limit))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int df3_get_dram_addr_map(struct addr_ctx *ctx)
+{
+ if (df2_get_dram_addr_map(ctx))
+ return -EINVAL;
+
+ /* Read D18F0x3F8 (DfGlobalCtl). */
+ if (df_indirect_read_instance(ctx->node_id, 0, 0x3F8,
+ ctx->inst_id, &ctx->map.ctl))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int df4_get_dram_addr_map(struct addr_ctx *ctx)
+{
+ u8 remap_sel, i, j, shift = 4, mask = 0xF;
+ u32 remap_reg;
+
+ /* Read D18F7xE00 (DramBaseAddress). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0xE00 + (16 * ctx->map.num),
+ ctx->inst_id, &ctx->map.base))
+ return -EINVAL;
+
+ /* Read D18F7xE04 (DramLimitAddress). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0xE04 + (16 * ctx->map.num),
+ ctx->inst_id, &ctx->map.limit))
+ return -EINVAL;
+
+ /* Read D18F7xE08 (DramAddressCtl). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0xE08 + (16 * ctx->map.num),
+ ctx->inst_id, &ctx->map.ctl))
+ return -EINVAL;
+
+ /* Read D18F7xE0C (DramAddressIntlv). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0xE0C + (16 * ctx->map.num),
+ ctx->inst_id, &ctx->map.intlv))
+ return -EINVAL;
+
+ /* Check if Remap Enable bit is valid. */
+ if (!FIELD_GET(DF4_REMAP_EN, ctx->map.ctl))
+ return 0;
+
+ /* Fill with bogus values, because '0' is a valid value. */
+ memset(&ctx->map.remap_array, 0xFF, sizeof(ctx->map.remap_array));
+
+ /* Get Remap registers. */
+ remap_sel = FIELD_GET(DF4_REMAP_SEL, ctx->map.ctl);
+
+ /* Read D18F7x180 (CsTargetRemap0A). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x180 + (8 * remap_sel),
+ ctx->inst_id, &remap_reg))
+ return -EINVAL;
+
+ /* Save first 8 remap entries. */
+ for (i = 0, j = 0; i < 8; i++, j++)
+ ctx->map.remap_array[i] = (remap_reg >> (j * shift)) & mask;
+
+ /* Read D18F7x184 (CsTargetRemap0B). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x184 + (8 * remap_sel),
+ ctx->inst_id, &remap_reg))
+ return -EINVAL;
+
+ /* Save next 8 remap entries. */
+ for (i = 8, j = 0; i < 16; i++, j++)
+ ctx->map.remap_array[i] = (remap_reg >> (j * shift)) & mask;
+
+ return 0;
+}
+
+static int df4p5_get_dram_addr_map(struct addr_ctx *ctx)
+{
+ u8 remap_sel, i, j, shift = 5, mask = 0x1F;
+ u32 remap_reg;
+
+ /* Read D18F7x200 (DramBaseAddress). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x200 + (16 * ctx->map.num),
+ ctx->inst_id, &ctx->map.base))
+ return -EINVAL;
+
+ /* Read D18F7x204 (DramLimitAddress). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x204 + (16 * ctx->map.num),
+ ctx->inst_id, &ctx->map.limit))
+ return -EINVAL;
+
+ /* Read D18F7x208 (DramAddressCtl). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x208 + (16 * ctx->map.num),
+ ctx->inst_id, &ctx->map.ctl))
+ return -EINVAL;
+
+ /* Read D18F7x20C (DramAddressIntlv). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x20C + (16 * ctx->map.num),
+ ctx->inst_id, &ctx->map.intlv))
+ return -EINVAL;
+
+ /* Check if Remap Enable bit is valid. */
+ if (!FIELD_GET(DF4_REMAP_EN, ctx->map.ctl))
+ return 0;
+
+ /* Fill with bogus values, because '0' is a valid value. */
+ memset(&ctx->map.remap_array, 0xFF, sizeof(ctx->map.remap_array));
+
+ /* Get Remap registers. */
+ remap_sel = FIELD_GET(DF4_REMAP_SEL, ctx->map.ctl);
+
+ /* Read D18F7x180 (CsTargetRemap0A). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x180 + (24 * remap_sel),
+ ctx->inst_id, &remap_reg))
+ return -EINVAL;
+
+ /* Save first 6 remap entries. */
+ for (i = 0, j = 0; i < 6; i++, j++)
+ ctx->map.remap_array[i] = (remap_reg >> (j * shift)) & mask;
+
+ /* Read D18F7x184 (CsTargetRemap0B). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x184 + (24 * remap_sel),
+ ctx->inst_id, &remap_reg))
+ return -EINVAL;
+
+ /* Save next 6 remap entries. */
+ for (i = 6, j = 0; i < 12; i++, j++)
+ ctx->map.remap_array[i] = (remap_reg >> (j * shift)) & mask;
+
+ /* Read D18F7x188 (CsTargetRemap0C). */
+ if (df_indirect_read_instance(ctx->node_id, 7, 0x188 + (24 * remap_sel),
+ ctx->inst_id, &remap_reg))
+ return -EINVAL;
+
+ /* Save next 6 remap entries. */
+ for (i = 12, j = 0; i < 18; i++, j++)
+ ctx->map.remap_array[i] = (remap_reg >> (j * shift)) & mask;
+
+ return 0;
+}
+
+static int get_dram_addr_map(struct addr_ctx *ctx)
+{
+ switch (df_cfg.rev) {
+ case DF2: return df2_get_dram_addr_map(ctx);
+ case DF3:
+ case DF3p5: return df3_get_dram_addr_map(ctx);
+ case DF4: return df4_get_dram_addr_map(ctx);
+ case DF4p5: return df4p5_get_dram_addr_map(ctx);
+ default:
+ warn_on_bad_df_rev();
+ return -EINVAL;
+ }
+}
+
+static int lookup_cs_fabric_id(struct addr_ctx *ctx)
+{
+ u32 reg;
+
+ /* Read D18F0x50 (FabricBlockInstanceInformation3). */
+ if (df_indirect_read_instance(ctx->node_id, 0, 0x50, ctx->inst_id, ®))
+ return -EINVAL;
+
+ if (df_cfg.rev < DF4p5)
+ ctx->cs_fabric_id = FIELD_GET(DF2_CS_FABRIC_ID, reg);
+ else
+ ctx->cs_fabric_id = FIELD_GET(DF4p5_CS_FABRIC_ID, reg);
+
+ return 0;
+}
+
+static int get_cs_fabric_id(struct addr_ctx *ctx)
+{
+ return lookup_cs_fabric_id(ctx);
+}
+
+static int find_normalized_offset(struct addr_ctx *ctx, u64 *norm_offset)
+{
+ u64 last_offset = 0;
+ bool enabled;
+
+ for (ctx->map.num = 1; ctx->map.num < df_cfg.num_cs_maps; ctx->map.num++) {
+ enabled = false;
+
+ if (get_dram_offset(ctx, &enabled, norm_offset))
+ return -EINVAL;
+
+ /* Continue search if this map's offset is not enabled. */
+ if (!enabled)
+ continue;
+
+ /* Enabled offsets should never be 0. */
+ if (*norm_offset == 0) {
+ warn_on_assert("Enabled map %u offset is 0", ctx->map.num);
+ return -EINVAL;
+ }
+
+ /* Offsets should always increase from one map to the next. */
+ if (*norm_offset <= last_offset) {
+ warn_on_assert("Map %u offset (0x%016llx) <= previous (0x%016llx)",
+ ctx->map.num, *norm_offset, last_offset);
+ return -EINVAL;
+ }
+
+ /* Match if this map's offset is less than the current calculated address. */
+ if (ctx->ret_addr >= *norm_offset)
+ break;
+
+ last_offset = *norm_offset;
+ }
+
+ /*
+ * Finished search without finding a match.
+ * Reset to map 0 and no offset.
+ */
+ if (ctx->map.num >= df_cfg.num_cs_maps) {
+ ctx->map.num = 0;
+ *norm_offset = 0;
+ }
+
+ return 0;
+}
+
+static bool valid_map(struct addr_ctx *ctx)
+{
+ if (df_cfg.rev >= DF4)
+ return FIELD_GET(DF_ADDR_RANGE_VAL, ctx->map.ctl);
+
+ return FIELD_GET(DF_ADDR_RANGE_VAL, ctx->map.base);
+}
+
+static int get_address_map_common(struct addr_ctx *ctx)
+{
+ u64 norm_offset = 0;
+
+ if (get_cs_fabric_id(ctx))
+ return -EINVAL;
+
+ if (find_normalized_offset(ctx, &norm_offset))
+ return -EINVAL;
+
+ if (get_dram_addr_map(ctx))
+ return -EINVAL;
+
+ if (!valid_map(ctx))
+ return -EINVAL;
+
+ ctx->ret_addr -= norm_offset;
+
+ return 0;
+}
+
+static u8 get_num_intlv_chan(struct addr_ctx *ctx)
+{
+ switch (ctx->map.intlv_mode) {
+ case NONE:
+ return 1;
+ case NOHASH_2CHAN:
+ case DF2_2CHAN_HASH:
+ case DF3_COD4_2CHAN_HASH:
+ case DF4_NPS4_2CHAN_HASH:
+ case DF4p5_NPS4_2CHAN_1K_HASH:
+ case DF4p5_NPS4_2CHAN_2K_HASH:
+ return 2;
+ case DF4_NPS4_3CHAN_HASH:
+ case DF4p5_NPS4_3CHAN_1K_HASH:
+ case DF4p5_NPS4_3CHAN_2K_HASH:
+ return 3;
+ case NOHASH_4CHAN:
+ case DF3_COD2_4CHAN_HASH:
+ case DF4_NPS2_4CHAN_HASH:
+ case DF4p5_NPS2_4CHAN_1K_HASH:
+ case DF4p5_NPS2_4CHAN_2K_HASH:
+ return 4;
+ case DF4_NPS2_5CHAN_HASH:
+ case DF4p5_NPS2_5CHAN_1K_HASH:
+ case DF4p5_NPS2_5CHAN_2K_HASH:
+ return 5;
+ case DF3_6CHAN:
+ case DF4_NPS2_6CHAN_HASH:
+ case DF4p5_NPS2_6CHAN_1K_HASH:
+ case DF4p5_NPS2_6CHAN_2K_HASH:
+ return 6;
+ case NOHASH_8CHAN:
+ case DF3_COD1_8CHAN_HASH:
+ case DF4_NPS1_8CHAN_HASH:
+ case DF4p5_NPS1_8CHAN_1K_HASH:
+ case DF4p5_NPS1_8CHAN_2K_HASH:
+ return 8;
+ case DF4_NPS1_10CHAN_HASH:
+ case DF4p5_NPS1_10CHAN_1K_HASH:
+ case DF4p5_NPS1_10CHAN_2K_HASH:
+ return 10;
+ case DF4_NPS1_12CHAN_HASH:
+ case DF4p5_NPS1_12CHAN_1K_HASH:
+ case DF4p5_NPS1_12CHAN_2K_HASH:
+ return 12;
+ case NOHASH_16CHAN:
+ case DF4p5_NPS1_16CHAN_1K_HASH:
+ case DF4p5_NPS1_16CHAN_2K_HASH:
+ return 16;
+ case DF4p5_NPS0_24CHAN_1K_HASH:
+ case DF4p5_NPS0_24CHAN_2K_HASH:
+ return 24;
+ case NOHASH_32CHAN:
+ return 32;
+ default:
+ warn_on_bad_intlv_mode(ctx);
+ return 0;
+ }
+}
+
+static void calculate_intlv_bits(struct addr_ctx *ctx)
+{
+ ctx->map.num_intlv_chan = get_num_intlv_chan(ctx);
+
+ ctx->map.total_intlv_chan = ctx->map.num_intlv_chan;
+ ctx->map.total_intlv_chan *= ctx->map.num_intlv_dies;
+ ctx->map.total_intlv_chan *= ctx->map.num_intlv_sockets;
+
+ /*
+ * Get the number of bits needed to cover this many channels.
+ * order_base_2() rounds up automatically.
+ */
+ ctx->map.total_intlv_bits = order_base_2(ctx->map.total_intlv_chan);
+}
+
+static u8 get_intlv_bit_pos(struct addr_ctx *ctx)
+{
+ u8 addr_sel = 0;
+
+ switch (df_cfg.rev) {
+ case DF2:
+ addr_sel = FIELD_GET(DF2_INTLV_ADDR_SEL, ctx->map.base);
+ break;
+ case DF3:
+ case DF3p5:
+ addr_sel = FIELD_GET(DF3_INTLV_ADDR_SEL, ctx->map.base);
+ break;
+ case DF4:
+ case DF4p5:
+ addr_sel = FIELD_GET(DF4_INTLV_ADDR_SEL, ctx->map.intlv);
+ break;
+ default:
+ warn_on_bad_df_rev();
+ break;
+ }
+
+ /* Add '8' to get the 'interleave bit position'. */
+ return addr_sel + 8;
+}
+
+static u8 get_num_intlv_dies(struct addr_ctx *ctx)
+{
+ u8 dies = 0;
+
+ switch (df_cfg.rev) {
+ case DF2:
+ dies = FIELD_GET(DF2_INTLV_NUM_DIES, ctx->map.limit);
+ break;
+ case DF3:
+ dies = FIELD_GET(DF3_INTLV_NUM_DIES, ctx->map.base);
+ break;
+ case DF3p5:
+ dies = FIELD_GET(DF3p5_INTLV_NUM_DIES, ctx->map.base);
+ break;
+ case DF4:
+ case DF4p5:
+ dies = FIELD_GET(DF4_INTLV_NUM_DIES, ctx->map.intlv);
+ break;
+ default:
+ warn_on_bad_df_rev();
+ break;
+ }
+
+ /* Register value is log2, e.g. 0 -> 1 die, 1 -> 2 dies, etc. */
+ return 1 << dies;
+}
+
+static u8 get_num_intlv_sockets(struct addr_ctx *ctx)
+{
+ u8 sockets = 0;
+
+ switch (df_cfg.rev) {
+ case DF2:
+ sockets = FIELD_GET(DF2_INTLV_NUM_SOCKETS, ctx->map.limit);
+ break;
+ case DF3:
+ case DF3p5:
+ sockets = FIELD_GET(DF2_INTLV_NUM_SOCKETS, ctx->map.base);
+ break;
+ case DF4:
+ case DF4p5:
+ sockets = FIELD_GET(DF4_INTLV_NUM_SOCKETS, ctx->map.intlv);
+ break;
+ default:
+ warn_on_bad_df_rev();
+ break;
+ }
+
+ /* Register value is log2, e.g. 0 -> 1 sockets, 1 -> 2 sockets, etc. */
+ return 1 << sockets;
+}
+
+static int get_global_map_data(struct addr_ctx *ctx)
+{
+ if (get_intlv_mode(ctx))
+ return -EINVAL;
+
+ if (ctx->map.intlv_mode == DF3_6CHAN &&
+ df3_6ch_get_dram_addr_map(ctx))
+ return -EINVAL;
+
+ ctx->map.intlv_bit_pos = get_intlv_bit_pos(ctx);
+ ctx->map.num_intlv_dies = get_num_intlv_dies(ctx);
+ ctx->map.num_intlv_sockets = get_num_intlv_sockets(ctx);
+ calculate_intlv_bits(ctx);
+
+ return 0;
+}
+
+static void dump_address_map(struct dram_addr_map *map)
+{
+ u8 i;
+
+ pr_debug("intlv_mode=0x%x", map->intlv_mode);
+ pr_debug("num=0x%x", map->num);
+ pr_debug("base=0x%x", map->base);
+ pr_debug("limit=0x%x", map->limit);
+ pr_debug("ctl=0x%x", map->ctl);
+ pr_debug("intlv=0x%x", map->intlv);
+
+ for (i = 0; i < MAX_CS_CHANNELS; i++)
+ pr_debug("remap_array[%u]=0x%x", i, map->remap_array[i]);
+
+ pr_debug("intlv_bit_pos=%u", map->intlv_bit_pos);
+ pr_debug("num_intlv_chan=%u", map->num_intlv_chan);
+ pr_debug("num_intlv_dies=%u", map->num_intlv_dies);
+ pr_debug("num_intlv_sockets=%u", map->num_intlv_sockets);
+ pr_debug("total_intlv_chan=%u", map->total_intlv_chan);
+ pr_debug("total_intlv_bits=%u", map->total_intlv_bits);
+}
+
+int get_address_map(struct addr_ctx *ctx)
+{
+ int ret = 0;
+
+ ret = get_address_map_common(ctx);
+ if (ret)
+ return ret;
+
+ if (get_global_map_data(ctx))
+ return -EINVAL;
+
+ dump_address_map(&ctx->map);
+
+ return ret;
+}
diff --git a/drivers/ras/amd/atl/reg_fields.h b/drivers/ras/amd/atl/reg_fields.h
new file mode 100644
index 000000000000..d48470e12498
--- /dev/null
+++ b/drivers/ras/amd/atl/reg_fields.h
@@ -0,0 +1,603 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AMD Address Translation Library
+ *
+ * reg_fields.h : Register field definitions
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+/*
+ * Notes on naming:
+ * 1) Use "DF_" prefix for fields that are the same for all revisions.
+ * 2) Use "DFx_" prefix for fields that differ between revisions.
+ * a) "x" is the first major revision where the new field appears.
+ * b) E.g., if DF2 and DF3 have the same field, then call it DF2.
+ * c) E.g., if DF3p5 and DF4 have the same field, then call it DF4.
+ */
+
+/*
+ * Coherent Station Fabric ID
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x50 [Fabric Block Instance Information 3]
+ * DF2 BlockFabricId [19:8]
+ * DF3 BlockFabricId [19:8]
+ * DF3p5 BlockFabricId [19:8]
+ * DF4 BlockFabricId [19:8]
+ * DF4p5 BlockFabricId [15:8]
+ */
+#define DF2_CS_FABRIC_ID GENMASK(19, 8)
+#define DF4p5_CS_FABRIC_ID GENMASK(15, 8)
+
+/*
+ * Component ID Mask
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ *
+ * D18F1x208 [System Fabric ID Mask 0]
+ * DF3 ComponentIdMask [9:0]
+ *
+ * D18F1x150 [System Fabric ID Mask 0]
+ * DF3p5 ComponentIdMask [15:0]
+ *
+ * D18F4x1B0 [System Fabric ID Mask 0]
+ * DF4 ComponentIdMask [15:0]
+ * DF4p5 ComponentIdMask [15:0]
+ */
+#define DF3_COMPONENT_ID_MASK GENMASK(9, 0)
+#define DF4_COMPONENT_ID_MASK GENMASK(15, 0)
+
+/*
+ * Destination Fabric ID
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x114 [DRAM Limit Address]
+ * DF2 DstFabricID [7:0]
+ * DF3 DstFabricID [9:0]
+ * DF3 DstFabricID [11:0]
+ *
+ * D18F7xE08 [DRAM Address Control]
+ * DF4 DstFabricID [27:16]
+ *
+ * D18F7x208 [DRAM Address Control]
+ * DF4p5 DstFabricID [23:16]
+ */
+#define DF2_DST_FABRIC_ID GENMASK(7, 0)
+#define DF3_DST_FABRIC_ID GENMASK(9, 0)
+#define DF3p5_DST_FABRIC_ID GENMASK(11, 0)
+#define DF4_DST_FABRIC_ID GENMASK(27, 16)
+#define DF4p5_DST_FABRIC_ID GENMASK(23, 16)
+
+/*
+ * Die ID Mask
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F1x208 [System Fabric ID Mask]
+ * DF2 DieIdMask [15:8]
+ *
+ * D18F1x20C [System Fabric ID Mask 1]
+ * DF3 DieIdMask [18:16]
+ *
+ * D18F1x158 [System Fabric ID Mask 2]
+ * DF3p5 DieIdMask [15:0]
+ *
+ * D18F4x1B8 [System Fabric ID Mask 2]
+ * DF4 DieIdMask [15:0]
+ * DF4p5 DieIdMask [15:0]
+ */
+#define DF2_DIE_ID_MASK GENMASK(15, 8)
+#define DF3_DIE_ID_MASK GENMASK(18, 16)
+#define DF4_DIE_ID_MASK GENMASK(15, 0)
+
+/*
+ * Die ID Shift
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F1x208 [System Fabric ID Mask]
+ * DF2 DieIdShift [27:24]
+ *
+ * DF3 N/A
+ * DF3p5 N/A
+ * DF4 N/A
+ * DF4p5 N/A
+ */
+#define DF2_DIE_ID_SHIFT GENMASK(27, 24)
+
+/*
+ * DRAM Address Range Valid
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x110 [DRAM Base Address]
+ * DF2 AddrRngVal [0]
+ * DF3 AddrRngVal [0]
+ * DF3p5 AddrRngVal [0]
+ *
+ * D18F7xE08 [DRAM Address Control]
+ * DF4 AddrRngVal [0]
+ *
+ * D18F7x208 [DRAM Address Control]
+ * DF4p5 AddrRngVal [0]
+ */
+#define DF_ADDR_RANGE_VAL BIT(0)
+
+/*
+ * DRAM Base Address
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x110 [DRAM Base Address]
+ * DF2 DramBaseAddr [31:12]
+ * DF3 DramBaseAddr [31:12]
+ * DF3p5 DramBaseAddr [31:12]
+ *
+ * D18F7xE00 [DRAM Base Address]
+ * DF4 DramBaseAddr [27:0]
+ *
+ * D18F7x200 [DRAM Base Address]
+ * DF4p5 DramBaseAddr [27:0]
+ */
+#define DF2_BASE_ADDR GENMASK(31, 12)
+#define DF4_BASE_ADDR GENMASK(27, 0)
+
+/*
+ * DRAM Hole Base
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x104 [DRAM Hole Control]
+ * DF2 DramHoleBase [31:24]
+ * DF3 DramHoleBase [31:24]
+ * DF3p5 DramHoleBase [31:24]
+ *
+ * D18F7x104 [DRAM Hole Control]
+ * DF4 DramHoleBase [31:24]
+ * DF4p5 DramHoleBase [31:24]
+ */
+#define DF_DRAM_HOLE_BASE_MASK GENMASK(31, 24)
+
+/*
+ * DRAM Limit Address
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x114 [DRAM Limit Address]
+ * DF2 DramLimitAddr [31:12]
+ * DF3 DramLimitAddr [31:12]
+ * DF3p5 DramLimitAddr [31:12]
+ *
+ * D18F7xE04 [DRAM Limit Address]
+ * DF4 DramLimitAddr [27:0]
+ *
+ * D18F7x204 [DRAM Limit Address]
+ * DF4p5 DramLimitAddr [27:0]
+ */
+#define DF2_DRAM_LIMIT_ADDR GENMASK(31, 12)
+#define DF4_DRAM_LIMIT_ADDR GENMASK(27, 0)
+
+/*
+ * Hash Interleave Controls
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ *
+ * D18F0x3F8 [DF Global Control]
+ * DF3 GlbHashIntlvCtl64K [20]
+ * GlbHashIntlvCtl2M [21]
+ * GlbHashIntlvCtl1G [22]
+ *
+ * DF3p5 GlbHashIntlvCtl64K [20]
+ * GlbHashIntlvCtl2M [21]
+ * GlbHashIntlvCtl1G [22]
+ *
+ * D18F7xE08 [DRAM Address Control]
+ * DF4 HashIntlvCtl64K [8]
+ * HashIntlvCtl2M [9]
+ * HashIntlvCtl1G [10]
+ *
+ * D18F7x208 [DRAM Address Control]
+ * DF4p5 HashIntlvCtl4K [7]
+ * HashIntlvCtl64K [8]
+ * HashIntlvCtl2M [9]
+ * HashIntlvCtl1G [10]
+ * HashIntlvCtl1T [15]
+ */
+#define DF3_HASH_CTL_64K BIT(20)
+#define DF3_HASH_CTL_2M BIT(21)
+#define DF3_HASH_CTL_1G BIT(22)
+#define DF4_HASH_CTL_4K BIT(7)
+#define DF4_HASH_CTL_64K BIT(8)
+#define DF4_HASH_CTL_2M BIT(9)
+#define DF4_HASH_CTL_1G BIT(10)
+#define DF4_HASH_CTL_1T BIT(15)
+
+/*
+ * High Address Offset
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x1B4 [DRAM Offset]
+ * DF2 HiAddrOffset [31:20]
+ * DF3 HiAddrOffset [31:12]
+ * DF3p5 HiAddrOffset [31:12]
+ *
+ * D18F7x140 [DRAM Offset]
+ * DF4 HiAddrOffset [24:1]
+ * DF4p5 HiAddrOffset [24:1]
+ */
+#define DF2_HI_ADDR_OFFSET GENMASK(31, 20)
+#define DF3_HI_ADDR_OFFSET GENMASK(31, 12)
+#define DF4_HI_ADDR_OFFSET GENMASK(24, 1)
+
+/*
+ * High Address Offset Enable
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x1B4 [DRAM Offset]
+ * DF2 HiAddrOffsetEn [0]
+ * DF3 HiAddrOffsetEn [0]
+ * DF3p5 HiAddrOffsetEn [0]
+ *
+ * D18F7x140 [DRAM Offset]
+ * DF4 HiAddrOffsetEn [0]
+ * DF4p5 HiAddrOffsetEn [0]
+ */
+#define DF_HI_ADDR_OFFSET_EN BIT(0)
+
+/*
+ * Interleave Address Select
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x110 [DRAM Base Address]
+ * DF2 IntLvAddrSel [10:8]
+ * DF3 IntLvAddrSel [11:9]
+ * DF3p5 IntLvAddrSel [11:9]
+ *
+ * D18F7xE0C [DRAM Address Interleave]
+ * DF4 IntLvAddrSel [2:0]
+ *
+ * D18F7x20C [DRAM Address Interleave]
+ * DF4p5 IntLvAddrSel [2:0]
+ */
+#define DF2_INTLV_ADDR_SEL GENMASK(10, 8)
+#define DF3_INTLV_ADDR_SEL GENMASK(11, 9)
+#define DF4_INTLV_ADDR_SEL GENMASK(2, 0)
+
+/*
+ * Interleave Number of Channels
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x110 [DRAM Base Address]
+ * DF2 IntLvNumChan [7:4]
+ * DF3 IntLvNumChan [5:2]
+ * DF3p5 IntLvNumChan [6:2]
+ *
+ * D18F7xE0C [DRAM Address Interleave]
+ * DF4 IntLvNumChan [8:4]
+ *
+ * D18F7x20C [DRAM Address Interleave]
+ * DF4p5 IntLvNumChan [9:4]
+ */
+#define DF2_INTLV_NUM_CHAN GENMASK(7, 4)
+#define DF3_INTLV_NUM_CHAN GENMASK(5, 2)
+#define DF3p5_INTLV_NUM_CHAN GENMASK(6, 2)
+#define DF4_INTLV_NUM_CHAN GENMASK(8, 4)
+#define DF4p5_INTLV_NUM_CHAN GENMASK(9, 4)
+
+/*
+ * Interleave Number of Dies
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x114 [DRAM Limit Address]
+ * DF2 IntLvNumDies [11:10]
+ *
+ * D18F0x110 [DRAM Base Address]
+ * DF3 IntLvNumDies [7:6]
+ * DF3p5 IntLvNumDies [7]
+ *
+ * D18F7xE0C [DRAM Address Interleave]
+ * DF4 IntLvNumDies [13:12]
+ *
+ * D18F7x20C [DRAM Address Interleave]
+ * DF4p5 IntLvNumDies [13:12]
+ */
+#define DF2_INTLV_NUM_DIES GENMASK(11, 10)
+#define DF3_INTLV_NUM_DIES GENMASK(7, 6)
+#define DF3p5_INTLV_NUM_DIES BIT(7)
+#define DF4_INTLV_NUM_DIES GENMASK(13, 12)
+
+/*
+ * Interleave Number of Sockets
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x114 [DRAM Limit Address]
+ * DF2 IntLvNumSockets [8]
+ *
+ * D18F0x110 [DRAM Base Address]
+ * DF3 IntLvNumSockets [8]
+ * DF3p5 IntLvNumSockets [8]
+ *
+ * D18F7xE0C [DRAM Address Interleave]
+ * DF4 IntLvNumSockets [18]
+ *
+ * D18F7x20C [DRAM Address Interleave]
+ * DF4p5 IntLvNumSockets [18]
+ */
+#define DF2_INTLV_NUM_SOCKETS BIT(8)
+#define DF4_INTLV_NUM_SOCKETS BIT(18)
+
+/*
+ * Legacy MMIO Hole Enable
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F0x110 [DRAM Base Address]
+ * DF2 LgcyMmioHoleEn [1]
+ * DF3 LgcyMmioHoleEn [1]
+ * DF3p5 LgcyMmioHoleEn [1]
+ *
+ * D18F7xE08 [DRAM Address Control]
+ * DF4 LgcyMmioHoleEn [1]
+ *
+ * D18F7x208 [DRAM Address Control]
+ * DF4p5 LgcyMmioHoleEn [1]
+ */
+#define DF_LEGACY_MMIO_HOLE_EN BIT(1)
+
+/*
+ * Log2 Address 64K Space 0
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ *
+ * D18F2x90 [Non-power-of-2 channel Configuration Register for CS DRAM Address Maps]
+ * DF3 Log2Addr64KSpace0 [5:0]
+ *
+ * DF3p5 N/A
+ * DF4 N/A
+ * DF4p5 N/A
+ */
+#define DF_LOG2_ADDR_64K_SPACE0 GENMASK(5, 0)
+
+/*
+ * Major Revision
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ * DF3 N/A
+ * DF3p5 N/A
+ *
+ * D18F0x040 [Fabric Block Instance Count]
+ * DF4 MajorRevision [27:24]
+ * DF4p5 MajorRevision [27:24]
+ */
+#define DF_MAJOR_REVISION GENMASK(27, 24)
+
+/*
+ * Minor Revision
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ * DF3 N/A
+ * DF3p5 N/A
+ *
+ * D18F0x040 [Fabric Block Instance Count]
+ * DF4 MinorRevision [23:16]
+ * DF4p5 MinorRevision [23:16]
+ */
+#define DF_MINOR_REVISION GENMASK(23, 16)
+
+/*
+ * Node ID Mask
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ *
+ * D18F1x208 [System Fabric ID Mask 0]
+ * DF3 NodeIdMask [25:16]
+ *
+ * D18F1x150 [System Fabric ID Mask 0]
+ * DF3p5 NodeIdMask [31:16]
+ *
+ * D18F4x1B0 [System Fabric ID Mask 0]
+ * DF4 NodeIdMask [31:16]
+ * DF4p5 NodeIdMask [31:16]
+ */
+#define DF3_NODE_ID_MASK GENMASK(25, 16)
+#define DF4_NODE_ID_MASK GENMASK(31, 16)
+
+/*
+ * Node ID Shift
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ *
+ * D18F1x20C [System Fabric ID Mask 1]
+ * DF3 NodeIdShift [3:0]
+ *
+ * D18F1x154 [System Fabric ID Mask 1]
+ * DF3p5 NodeIdShift [3:0]
+ *
+ * D18F4x1B4 [System Fabric ID Mask 1]
+ * DF4 NodeIdShift [3:0]
+ * DF4p5 NodeIdShift [3:0]
+ */
+#define DF3_NODE_ID_SHIFT GENMASK(3, 0)
+
+/*
+ * Remap Enable
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ * DF3 N/A
+ * DF3p5 N/A
+ *
+ * D18F7xE08 [DRAM Address Control]
+ * DF4 RemapEn [4]
+ *
+ * D18F7x208 [DRAM Address Control]
+ * DF4p5 RemapEn [4]
+ */
+#define DF4_REMAP_EN BIT(4)
+
+/*
+ * Remap Select
+ *
+ * Access type: Instance
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * DF2 N/A
+ * DF3 N/A
+ * DF3p5 N/A
+ *
+ * D18F7xE08 [DRAM Address Control]
+ * DF4 RemapSel [7:5]
+ *
+ * D18F7x208 [DRAM Address Control]
+ * DF4p5 RemapSel [6:5]
+ */
+#define DF4_REMAP_SEL GENMASK(7, 5)
+#define DF4p5_REMAP_SEL GENMASK(6, 5)
+
+/*
+ * Socket ID Mask
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F1x208 [System Fabric ID Mask]
+ * DF2 SocketIdMask [23:16]
+ *
+ * D18F1x20C [System Fabric ID Mask 1]
+ * DF3 SocketIdMask [26:24]
+ *
+ * D18F1x158 [System Fabric ID Mask 2]
+ * DF3p5 SocketIdMask [31:16]
+ *
+ * D18F4x1B8 [System Fabric ID Mask 2]
+ * DF4 SocketIdMask [31:16]
+ * DF4p5 SocketIdMask [31:16]
+ */
+#define DF2_SOCKET_ID_MASK GENMASK(23, 16)
+#define DF3_SOCKET_ID_MASK GENMASK(26, 24)
+#define DF4_SOCKET_ID_MASK GENMASK(31, 16)
+
+/*
+ * Socket ID Shift
+ *
+ * Access type: Broadcast
+ *
+ * Register
+ * Rev Fieldname Bits
+ *
+ * D18F1x208 [System Fabric ID Mask]
+ * DF2 SocketIdShift [31:28]
+ *
+ * D18F1x20C [System Fabric ID Mask 1]
+ * DF3 SocketIdShift [9:8]
+ *
+ * D18F1x158 [System Fabric ID Mask 2]
+ * DF3p5 SocketIdShift [11:8]
+ *
+ * D18F4x1B4 [System Fabric ID Mask 1]
+ * DF4 SocketIdShift [11:8]
+ * DF4p5 SocketIdShift [11:8]
+ */
+#define DF2_SOCKET_ID_SHIFT GENMASK(31, 28)
+#define DF3_SOCKET_ID_SHIFT GENMASK(9, 8)
+#define DF4_SOCKET_ID_SHIFT GENMASK(11, 8)
diff --git a/drivers/ras/amd/atl/stub.c b/drivers/ras/amd/atl/stub.c
new file mode 100644
index 000000000000..7f8b793ff56a
--- /dev/null
+++ b/drivers/ras/amd/atl/stub.c
@@ -0,0 +1,78 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AMD Address Translation Library
+ *
+ * stub.c : Minimal support code to be built-in.
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#include "stub.h"
+
+static DEFINE_MUTEX(amd_atl_ops_lock);
+
+static int default_convert_umc_na_to_spa(struct mce *m, u64 *sys_addr) { return -EINVAL; }
+
+static struct amd_atl_ops default_ops = {
+ .convert_umc_na_to_spa = default_convert_umc_na_to_spa,
+};
+
+/*
+ * Kref goes to 0 when ATL module is unloaded.
+ * Switch back to default function pointers in this case.
+ */
+static void release_amd_atl_ops(struct kref *refcount)
+{
+ amd_atl_pvt.ops = &default_ops;
+}
+
+static void initialize_ops(struct amd_atl_ops *ops)
+{
+ mutex_lock(&amd_atl_ops_lock);
+ amd_atl_pvt.ops = ops;
+ kref_init(&amd_atl_pvt.refcount);
+ mutex_unlock(&amd_atl_ops_lock);
+}
+
+static void release_ops(void)
+{
+ mutex_lock(&amd_atl_ops_lock);
+ kref_put(&amd_atl_pvt.refcount, release_amd_atl_ops);
+ mutex_unlock(&amd_atl_ops_lock);
+}
+
+/* This private structure is exported solely for use by the AMD ATL module. */
+struct amd_atl_pvt amd_atl_pvt = {
+ .initialize = initialize_ops,
+ .release = release_ops,
+ .ops = &default_ops,
+};
+EXPORT_SYMBOL_GPL(amd_atl_pvt);
+
+int amd_convert_umc_mca_addr_to_sys_addr(struct mce *m, u64 *sys_addr)
+{
+ int ret = -EINVAL;
+
+ mutex_lock(&amd_atl_ops_lock);
+
+ /* Kref only gets initialized when ATL module successfully loads. */
+ if (!kref_get_unless_zero(&amd_atl_pvt.refcount)) {
+ pr_warn("AMD Address Translation Library not enabled");
+ goto unlock;
+ }
+
+ mutex_unlock(&amd_atl_ops_lock);
+
+ ret = amd_atl_pvt.ops->convert_umc_na_to_spa(m, sys_addr);
+
+ mutex_lock(&amd_atl_ops_lock);
+ kref_put(&amd_atl_pvt.refcount, release_amd_atl_ops);
+
+unlock:
+ mutex_unlock(&amd_atl_ops_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(amd_convert_umc_mca_addr_to_sys_addr);
diff --git a/drivers/ras/amd/atl/stub.h b/drivers/ras/amd/atl/stub.h
new file mode 100644
index 000000000000..ad07c46534e7
--- /dev/null
+++ b/drivers/ras/amd/atl/stub.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AMD Address Translation Library
+ *
+ * stub.h : Common structs and defines used by library stub and module
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#ifndef __AMD_ATL_STUB_H__
+#define __AMD_ATL_STUB_H__
+
+#include <linux/kref.h>
+#include <linux/mutex.h>
+
+#include <asm/amd_atl.h>
+
+struct amd_atl_ops {
+ int (*convert_umc_na_to_spa)(struct mce *m, u64 *sys_addr);
+};
+
+struct amd_atl_pvt {
+ struct kref refcount;
+ void (*initialize)(struct amd_atl_ops *ops);
+ void (*release)(void);
+ struct amd_atl_ops *ops;
+};
+
+extern struct amd_atl_pvt amd_atl_pvt;
+
+#endif /* __AMD_ATL_STUB_H__ */
diff --git a/drivers/ras/amd/atl/system.c b/drivers/ras/amd/atl/system.c
new file mode 100644
index 000000000000..63dd320d0758
--- /dev/null
+++ b/drivers/ras/amd/atl/system.c
@@ -0,0 +1,283 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AMD Address Translation Library
+ *
+ * system.c : Functions to read and save system-wide data
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#include "internal.h"
+
+int determine_node_id(struct addr_ctx *ctx, u8 socket_id, u8 die_id)
+{
+ u16 socket_id_bits, die_id_bits;
+
+ if (socket_id > 0 && df_cfg.socket_id_mask == 0) {
+ warn_on_assert("Invalid socket inputs: socket_id=%u socket_id_mask=0x%x",
+ socket_id, df_cfg.socket_id_mask);
+ return -EINVAL;
+ }
+
+ /* Do each step independently to avoid shift out-of-bounds issues. */
+ socket_id_bits = socket_id;
+ socket_id_bits <<= df_cfg.socket_id_shift;
+ socket_id_bits &= df_cfg.socket_id_mask;
+
+ if (die_id > 0 && df_cfg.die_id_mask == 0) {
+ warn_on_assert("Invalid die inputs: die_id=%u die_id_mask=0x%x",
+ die_id, df_cfg.die_id_mask);
+ return -EINVAL;
+ }
+
+ /* Do each step independently to avoid shift out-of-bounds issues. */
+ die_id_bits = die_id;
+ die_id_bits <<= df_cfg.die_id_shift;
+ die_id_bits &= df_cfg.die_id_mask;
+
+ ctx->node_id = (socket_id_bits | die_id_bits) >> df_cfg.node_id_shift;
+ return 0;
+}
+
+static void df2_get_masks_shifts(u32 mask0)
+{
+ df_cfg.socket_id_shift = FIELD_GET(DF2_SOCKET_ID_SHIFT, mask0);
+ df_cfg.socket_id_mask = FIELD_GET(DF2_SOCKET_ID_MASK, mask0);
+ df_cfg.die_id_shift = FIELD_GET(DF2_DIE_ID_SHIFT, mask0);
+ df_cfg.die_id_mask = FIELD_GET(DF2_DIE_ID_MASK, mask0);
+ df_cfg.node_id_shift = df_cfg.die_id_shift;
+ df_cfg.node_id_mask = df_cfg.socket_id_mask | df_cfg.die_id_mask;
+ df_cfg.component_id_mask = ~df_cfg.node_id_mask;
+}
+
+static void df3_get_masks_shifts(u32 mask0, u32 mask1)
+{
+ df_cfg.component_id_mask = FIELD_GET(DF3_COMPONENT_ID_MASK, mask0);
+ df_cfg.node_id_mask = FIELD_GET(DF3_NODE_ID_MASK, mask0);
+
+ df_cfg.node_id_shift = FIELD_GET(DF3_NODE_ID_SHIFT, mask1);
+ df_cfg.socket_id_shift = FIELD_GET(DF3_SOCKET_ID_SHIFT, mask1);
+ df_cfg.socket_id_mask = FIELD_GET(DF3_SOCKET_ID_MASK, mask1);
+ df_cfg.die_id_mask = FIELD_GET(DF3_DIE_ID_MASK, mask1);
+}
+
+static void df3p5_get_masks_shifts(u32 mask0, u32 mask1, u32 mask2)
+{
+ df_cfg.component_id_mask = FIELD_GET(DF4_COMPONENT_ID_MASK, mask0);
+ df_cfg.node_id_mask = FIELD_GET(DF4_NODE_ID_MASK, mask0);
+
+ df_cfg.node_id_shift = FIELD_GET(DF3_NODE_ID_SHIFT, mask1);
+ df_cfg.socket_id_shift = FIELD_GET(DF4_SOCKET_ID_SHIFT, mask1);
+
+ df_cfg.socket_id_mask = FIELD_GET(DF4_SOCKET_ID_MASK, mask2);
+ df_cfg.die_id_mask = FIELD_GET(DF4_DIE_ID_MASK, mask2);
+}
+
+static void df4_get_masks_shifts(u32 mask0, u32 mask1, u32 mask2)
+{
+ df3p5_get_masks_shifts(mask0, mask1, mask2);
+
+ if (!(df_cfg.flags.socket_id_shift_quirk && df_cfg.socket_id_shift == 1))
+ return;
+
+ df_cfg.socket_id_shift = 0;
+ df_cfg.socket_id_mask = 1;
+ df_cfg.die_id_shift = 0;
+ df_cfg.die_id_mask = 0;
+ df_cfg.node_id_shift = 8;
+ df_cfg.node_id_mask = 0x100;
+}
+
+static int df4_get_fabric_id_mask_registers(void)
+{
+ u32 mask0, mask1, mask2;
+
+ /* Read D18F4x1B0 (SystemFabricIdMask0) */
+ if (df_indirect_read_broadcast(0, 4, 0x1B0, &mask0))
+ return -EINVAL;
+
+ /* Read D18F4x1B4 (SystemFabricIdMask1) */
+ if (df_indirect_read_broadcast(0, 4, 0x1B4, &mask1))
+ return -EINVAL;
+
+ /* Read D18F4x1B8 (SystemFabricIdMask2) */
+ if (df_indirect_read_broadcast(0, 4, 0x1B8, &mask2))
+ return -EINVAL;
+
+ df4_get_masks_shifts(mask0, mask1, mask2);
+ return 0;
+}
+
+static int df4_determine_df_rev(u32 reg)
+{
+ df_cfg.rev = FIELD_GET(DF_MINOR_REVISION, reg) < 5 ? DF4 : DF4p5;
+
+ /* Check for special cases or quirks based on Device/Vendor IDs.*/
+
+ /* Read D18F0x000 (DeviceVendorId0) */
+ if (df_indirect_read_broadcast(0, 0, 0, ®))
+ return -EINVAL;
+
+ if (reg == DF_FUNC0_ID_ZEN4_SERVER)
+ df_cfg.flags.socket_id_shift_quirk = 1;
+
+ return df4_get_fabric_id_mask_registers();
+}
+
+static int determine_df_rev_legacy(void)
+{
+ u32 fabric_id_mask0, fabric_id_mask1, fabric_id_mask2;
+
+ /*
+ * Check for DF3.5.
+ *
+ * Component ID Mask must be non-zero. Register D18F1x150 is
+ * reserved pre-DF3.5, so value will be Read-as-Zero.
+ */
+
+ /* Read D18F1x150 (SystemFabricIdMask0). */
+ if (df_indirect_read_broadcast(0, 1, 0x150, &fabric_id_mask0))
+ return -EINVAL;
+
+ if (FIELD_GET(DF4_COMPONENT_ID_MASK, fabric_id_mask0)) {
+ df_cfg.rev = DF3p5;
+
+ /* Read D18F1x154 (SystemFabricIdMask1) */
+ if (df_indirect_read_broadcast(0, 1, 0x154, &fabric_id_mask1))
+ return -EINVAL;
+
+ /* Read D18F1x158 (SystemFabricIdMask2) */
+ if (df_indirect_read_broadcast(0, 1, 0x158, &fabric_id_mask2))
+ return -EINVAL;
+
+ df3p5_get_masks_shifts(fabric_id_mask0, fabric_id_mask1, fabric_id_mask2);
+ return 0;
+ }
+
+ /*
+ * Check for DF3.
+ *
+ * Component ID Mask must be non-zero. Field is Read-as-Zero on DF2.
+ */
+
+ /* Read D18F1x208 (SystemFabricIdMask). */
+ if (df_indirect_read_broadcast(0, 1, 0x208, &fabric_id_mask0))
+ return -EINVAL;
+
+ if (FIELD_GET(DF3_COMPONENT_ID_MASK, fabric_id_mask0)) {
+ df_cfg.rev = DF3;
+
+ /* Read D18F1x20C (SystemFabricIdMask1) */
+ if (df_indirect_read_broadcast(0, 1, 0x20C, &fabric_id_mask1))
+ return -EINVAL;
+
+ df3_get_masks_shifts(fabric_id_mask0, fabric_id_mask1);
+ return 0;
+ }
+
+ /* Default to DF2. */
+ df_cfg.rev = DF2;
+ df2_get_masks_shifts(fabric_id_mask0);
+ return 0;
+}
+
+static int determine_df_rev(void)
+{
+ u32 reg;
+ u8 rev;
+
+ if (df_cfg.rev != UNKNOWN)
+ return 0;
+
+ /* Read D18F0x40 (FabricBlockInstanceCount). */
+ if (df_indirect_read_broadcast(0, 0, 0x40, ®))
+ return -EINVAL;
+
+ /*
+ * Revision fields added for DF4 and later.
+ *
+ * Major revision of '0' is found pre-DF4. Field is Read-as-Zero.
+ */
+ rev = FIELD_GET(DF_MAJOR_REVISION, reg);
+ if (!rev)
+ return determine_df_rev_legacy();
+
+ /*
+ * Fail out for major revisions other than '4'.
+ *
+ * Explicit support should be added for newer systems to avoid issues.
+ */
+ if (rev == 4)
+ return df4_determine_df_rev(reg);
+
+ return -EINVAL;
+}
+
+static void get_num_maps(void)
+{
+ switch (df_cfg.rev) {
+ case DF2:
+ case DF3:
+ case DF3p5:
+ df_cfg.num_cs_maps = 2;
+ break;
+ case DF4:
+ df_cfg.num_cs_maps = 4;
+ break;
+ case DF4p5:
+ df_cfg.num_cs_maps = 4;
+ break;
+ default:
+ warn_on_bad_df_rev();
+ }
+}
+
+static void apply_node_id_shift(void)
+{
+ if (df_cfg.rev == DF2)
+ return;
+
+ df_cfg.die_id_shift = df_cfg.node_id_shift;
+ df_cfg.die_id_mask <<= df_cfg.node_id_shift;
+ df_cfg.socket_id_mask <<= df_cfg.node_id_shift;
+ df_cfg.socket_id_shift += df_cfg.node_id_shift;
+}
+
+static void dump_df_cfg(void)
+{
+ pr_debug("rev=0x%x", df_cfg.rev);
+
+ pr_debug("component_id_mask=0x%x", df_cfg.component_id_mask);
+ pr_debug("die_id_mask=0x%x", df_cfg.die_id_mask);
+ pr_debug("node_id_mask=0x%x", df_cfg.node_id_mask);
+ pr_debug("socket_id_mask=0x%x", df_cfg.socket_id_mask);
+
+ pr_debug("die_id_shift=0x%x", df_cfg.die_id_shift);
+ pr_debug("node_id_shift=0x%x", df_cfg.node_id_shift);
+ pr_debug("socket_id_shift=0x%x", df_cfg.socket_id_shift);
+
+ pr_debug("num_cs_maps=%u", df_cfg.num_cs_maps);
+
+ pr_debug("flags.legacy_ficaa=%u", df_cfg.flags.legacy_ficaa);
+ pr_debug("flags.socket_id_shift_quirk=%u", df_cfg.flags.socket_id_shift_quirk);
+}
+
+int get_df_system_info(void)
+{
+ if (determine_df_rev()) {
+ pr_warn("amd_atl: Failed to determine DF Revision");
+ df_cfg.rev = UNKNOWN;
+ return -EINVAL;
+ }
+
+ apply_node_id_shift();
+
+ get_num_maps();
+
+ dump_df_cfg();
+
+ return 0;
+}
diff --git a/drivers/ras/amd/atl/umc.c b/drivers/ras/amd/atl/umc.c
new file mode 100644
index 000000000000..af2ed33414bc
--- /dev/null
+++ b/drivers/ras/amd/atl/umc.c
@@ -0,0 +1,45 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * AMD Address Translation Library
+ *
+ * umc.c : Unified Memory Controller (UMC) topology helpers
+ *
+ * Copyright (c) 2023, Advanced Micro Devices, Inc.
+ * All Rights Reserved.
+ *
+ * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
+ */
+
+#include "internal.h"
+
+static u8 get_die_id(struct mce *m)
+{
+ /*
+ * For CPUs, this is the AMD Node ID modulo the number
+ * of AMD Nodes per socket.
+ */
+ return topology_die_id(m->extcpu) % amd_get_nodes_per_socket();
+}
+
+#define UMC_CHANNEL_NUM GENMASK(31, 20)
+static u8 get_cs_inst_id(struct mce *m)
+{
+ return FIELD_GET(UMC_CHANNEL_NUM, m->ipid);
+}
+
+int convert_umc_mca_addr_to_sys_addr(struct mce *m, u64 *sys_addr)
+{
+ u8 cs_inst_id = get_cs_inst_id(m);
+ u8 socket_id = m->socketid;
+ u8 die_id = get_die_id(m);
+ u64 addr = m->addr;
+
+ pr_debug("socket_id=0x%x die_id=0x%x cs_inst_id=0x%x addr=0x%016llx",
+ socket_id, die_id, cs_inst_id, addr);
+
+ if (norm_to_sys_addr(socket_id, die_id, cs_inst_id, &addr))
+ return -EINVAL;
+
+ *sys_addr = addr;
+ return 0;
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v3 2/3] EDAC/amd64: Use new AMD Address Translation Library
2023-12-10 19:49 [PATCH v3 0/3] AMD Address Translation Library Yazen Ghannam
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
@ 2023-12-10 19:49 ` Yazen Ghannam
2023-12-10 19:49 ` [PATCH v3 3/3] Documentation: RAS: Add index and address translation section Yazen Ghannam
2 siblings, 0 replies; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-10 19:49 UTC (permalink / raw)
To: linux-edac
Cc: linux-kernel, tony.luck, x86, avadhut.naik, john.allen,
william.roche, muralidhara.mk, Yazen Ghannam
Remove old address translation code and use the new AMD Address
Translation Library.
Use "imply" in Kconfig so that the "AMD_ATL" config option takes the
value of "EDAC_AMD64" as its default.
Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lore.kernel.org/r/20231005173526.42831-3-yazen.ghannam@amd.com
v2->v3:
* Adjust header include and library function name.
v1->v2:
* Drop the "config reachable" check.
drivers/edac/Kconfig | 1 +
drivers/edac/amd64_edac.c | 278 +-------------------------------------
2 files changed, 3 insertions(+), 276 deletions(-)
diff --git a/drivers/edac/Kconfig b/drivers/edac/Kconfig
index 5a7f3fabee22..16c8de5050e5 100644
--- a/drivers/edac/Kconfig
+++ b/drivers/edac/Kconfig
@@ -78,6 +78,7 @@ config EDAC_GHES
config EDAC_AMD64
tristate "AMD64 (Opteron, Athlon64)"
depends on AMD_NB && EDAC_DECODE_MCE
+ imply AMD_ATL
help
Support for error detection and correction of DRAM ECC errors on
the AMD64 families (>= K8) of memory controllers.
diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 9b6642d00871..70bdaca78545 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
#include "amd64_edac.h"
#include <asm/amd_nb.h>
+#include <asm/amd_atl.h>
static struct edac_pci_ctl_info *pci_ctl;
@@ -1043,281 +1044,6 @@ static int fixup_node_id(int node_id, struct mce *m)
return nid - gpu_node_map.base_node_id + 1;
}
-/* Protect the PCI config register pairs used for DF indirect access. */
-static DEFINE_MUTEX(df_indirect_mutex);
-
-/*
- * Data Fabric Indirect Access uses FICAA/FICAD.
- *
- * Fabric Indirect Configuration Access Address (FICAA): Constructed based
- * on the device's Instance Id and the PCI function and register offset of
- * the desired register.
- *
- * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
- * and FICAD HI registers but so far we only need the LO register.
- *
- * Use Instance Id 0xFF to indicate a broadcast read.
- */
-#define DF_BROADCAST 0xFF
-static int __df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
-{
- struct pci_dev *F4;
- u32 ficaa;
- int err = -ENODEV;
-
- if (node >= amd_nb_num())
- goto out;
-
- F4 = node_to_amd_nb(node)->link;
- if (!F4)
- goto out;
-
- ficaa = (instance_id == DF_BROADCAST) ? 0 : 1;
- ficaa |= reg & 0x3FC;
- ficaa |= (func & 0x7) << 11;
- ficaa |= instance_id << 16;
-
- mutex_lock(&df_indirect_mutex);
-
- err = pci_write_config_dword(F4, 0x5C, ficaa);
- if (err) {
- pr_warn("Error writing DF Indirect FICAA, FICAA=0x%x\n", ficaa);
- goto out_unlock;
- }
-
- err = pci_read_config_dword(F4, 0x98, lo);
- if (err)
- pr_warn("Error reading DF Indirect FICAD LO, FICAA=0x%x.\n", ficaa);
-
-out_unlock:
- mutex_unlock(&df_indirect_mutex);
-
-out:
- return err;
-}
-
-static int df_indirect_read_instance(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
-{
- return __df_indirect_read(node, func, reg, instance_id, lo);
-}
-
-static int df_indirect_read_broadcast(u16 node, u8 func, u16 reg, u32 *lo)
-{
- return __df_indirect_read(node, func, reg, DF_BROADCAST, lo);
-}
-
-struct addr_ctx {
- u64 ret_addr;
- u32 tmp;
- u16 nid;
- u8 inst_id;
-};
-
-static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
-{
- u64 dram_base_addr, dram_limit_addr, dram_hole_base;
-
- u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
- u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
- u8 intlv_addr_sel, intlv_addr_bit;
- u8 num_intlv_bits, hashed_bit;
- u8 lgcy_mmio_hole_en, base = 0;
- u8 cs_mask, cs_id = 0;
- bool hash_enabled = false;
-
- struct addr_ctx ctx;
-
- memset(&ctx, 0, sizeof(ctx));
-
- /* Start from the normalized address */
- ctx.ret_addr = norm_addr;
-
- ctx.nid = nid;
- ctx.inst_id = umc;
-
- /* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
- if (df_indirect_read_instance(nid, 0, 0x1B4, umc, &ctx.tmp))
- goto out_err;
-
- /* Remove HiAddrOffset from normalized address, if enabled: */
- if (ctx.tmp & BIT(0)) {
- u64 hi_addr_offset = (ctx.tmp & GENMASK_ULL(31, 20)) << 8;
-
- if (norm_addr >= hi_addr_offset) {
- ctx.ret_addr -= hi_addr_offset;
- base = 1;
- }
- }
-
- /* Read D18F0x110 (DramBaseAddress). */
- if (df_indirect_read_instance(nid, 0, 0x110 + (8 * base), umc, &ctx.tmp))
- goto out_err;
-
- /* Check if address range is valid. */
- if (!(ctx.tmp & BIT(0))) {
- pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
- __func__, ctx.tmp);
- goto out_err;
- }
-
- lgcy_mmio_hole_en = ctx.tmp & BIT(1);
- intlv_num_chan = (ctx.tmp >> 4) & 0xF;
- intlv_addr_sel = (ctx.tmp >> 8) & 0x7;
- dram_base_addr = (ctx.tmp & GENMASK_ULL(31, 12)) << 16;
-
- /* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
- if (intlv_addr_sel > 3) {
- pr_err("%s: Invalid interleave address select %d.\n",
- __func__, intlv_addr_sel);
- goto out_err;
- }
-
- /* Read D18F0x114 (DramLimitAddress). */
- if (df_indirect_read_instance(nid, 0, 0x114 + (8 * base), umc, &ctx.tmp))
- goto out_err;
-
- intlv_num_sockets = (ctx.tmp >> 8) & 0x1;
- intlv_num_dies = (ctx.tmp >> 10) & 0x3;
- dram_limit_addr = ((ctx.tmp & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
-
- intlv_addr_bit = intlv_addr_sel + 8;
-
- /* Re-use intlv_num_chan by setting it equal to log2(#channels) */
- switch (intlv_num_chan) {
- case 0: intlv_num_chan = 0; break;
- case 1: intlv_num_chan = 1; break;
- case 3: intlv_num_chan = 2; break;
- case 5: intlv_num_chan = 3; break;
- case 7: intlv_num_chan = 4; break;
-
- case 8: intlv_num_chan = 1;
- hash_enabled = true;
- break;
- default:
- pr_err("%s: Invalid number of interleaved channels %d.\n",
- __func__, intlv_num_chan);
- goto out_err;
- }
-
- num_intlv_bits = intlv_num_chan;
-
- if (intlv_num_dies > 2) {
- pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
- __func__, intlv_num_dies);
- goto out_err;
- }
-
- num_intlv_bits += intlv_num_dies;
-
- /* Add a bit if sockets are interleaved. */
- num_intlv_bits += intlv_num_sockets;
-
- /* Assert num_intlv_bits <= 4 */
- if (num_intlv_bits > 4) {
- pr_err("%s: Invalid interleave bits %d.\n",
- __func__, num_intlv_bits);
- goto out_err;
- }
-
- if (num_intlv_bits > 0) {
- u64 temp_addr_x, temp_addr_i, temp_addr_y;
- u8 die_id_bit, sock_id_bit, cs_fabric_id;
-
- /*
- * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
- * This is the fabric id for this coherent slave. Use
- * umc/channel# as instance id of the coherent slave
- * for FICAA.
- */
- if (df_indirect_read_instance(nid, 0, 0x50, umc, &ctx.tmp))
- goto out_err;
-
- cs_fabric_id = (ctx.tmp >> 8) & 0xFF;
- die_id_bit = 0;
-
- /* If interleaved over more than 1 channel: */
- if (intlv_num_chan) {
- die_id_bit = intlv_num_chan;
- cs_mask = (1 << die_id_bit) - 1;
- cs_id = cs_fabric_id & cs_mask;
- }
-
- sock_id_bit = die_id_bit;
-
- /* Read D18F1x208 (SystemFabricIdMask). */
- if (intlv_num_dies || intlv_num_sockets)
- if (df_indirect_read_broadcast(nid, 1, 0x208, &ctx.tmp))
- goto out_err;
-
- /* If interleaved over more than 1 die. */
- if (intlv_num_dies) {
- sock_id_bit = die_id_bit + intlv_num_dies;
- die_id_shift = (ctx.tmp >> 24) & 0xF;
- die_id_mask = (ctx.tmp >> 8) & 0xFF;
-
- cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
- }
-
- /* If interleaved over more than 1 socket. */
- if (intlv_num_sockets) {
- socket_id_shift = (ctx.tmp >> 28) & 0xF;
- socket_id_mask = (ctx.tmp >> 16) & 0xFF;
-
- cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
- }
-
- /*
- * The pre-interleaved address consists of XXXXXXIIIYYYYY
- * where III is the ID for this CS, and XXXXXXYYYYY are the
- * address bits from the post-interleaved address.
- * "num_intlv_bits" has been calculated to tell us how many "I"
- * bits there are. "intlv_addr_bit" tells us how many "Y" bits
- * there are (where "I" starts).
- */
- temp_addr_y = ctx.ret_addr & GENMASK_ULL(intlv_addr_bit - 1, 0);
- temp_addr_i = (cs_id << intlv_addr_bit);
- temp_addr_x = (ctx.ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
- ctx.ret_addr = temp_addr_x | temp_addr_i | temp_addr_y;
- }
-
- /* Add dram base address */
- ctx.ret_addr += dram_base_addr;
-
- /* If legacy MMIO hole enabled */
- if (lgcy_mmio_hole_en) {
- if (df_indirect_read_broadcast(nid, 0, 0x104, &ctx.tmp))
- goto out_err;
-
- dram_hole_base = ctx.tmp & GENMASK(31, 24);
- if (ctx.ret_addr >= dram_hole_base)
- ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
- }
-
- if (hash_enabled) {
- /* Save some parentheses and grab ls-bit at the end. */
- hashed_bit = (ctx.ret_addr >> 12) ^
- (ctx.ret_addr >> 18) ^
- (ctx.ret_addr >> 21) ^
- (ctx.ret_addr >> 30) ^
- cs_id;
-
- hashed_bit &= BIT(0);
-
- if (hashed_bit != ((ctx.ret_addr >> intlv_addr_bit) & BIT(0)))
- ctx.ret_addr ^= BIT(intlv_addr_bit);
- }
-
- /* Is calculated system address is above DRAM limit address? */
- if (ctx.ret_addr > dram_limit_addr)
- goto out_err;
-
- *sys_addr = ctx.ret_addr;
- return 0;
-
-out_err:
- return -EINVAL;
-}
-
static int get_channel_from_ecc_syndrome(struct mem_ctl_info *, u16);
/*
@@ -3098,7 +2824,7 @@ static void decode_umc_error(int node_id, struct mce *m)
pvt->ops->get_err_info(m, &err);
- if (umc_normaddr_to_sysaddr(m->addr, pvt->mc_node_id, err.channel, &sys_addr)) {
+ if (amd_convert_umc_mca_addr_to_sys_addr(m, &sys_addr)) {
err.err_code = ERR_NORM_ADDR;
goto log_error;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v3 3/3] Documentation: RAS: Add index and address translation section
2023-12-10 19:49 [PATCH v3 0/3] AMD Address Translation Library Yazen Ghannam
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
2023-12-10 19:49 ` [PATCH v3 2/3] EDAC/amd64: Use new " Yazen Ghannam
@ 2023-12-10 19:49 ` Yazen Ghannam
2 siblings, 0 replies; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-10 19:49 UTC (permalink / raw)
To: linux-edac
Cc: linux-kernel, tony.luck, x86, avadhut.naik, john.allen,
william.roche, muralidhara.mk, Yazen Ghannam
There are a lot of RAS topic to document, and there are a lot of details
for each topic.
Prep for this by adding an index for the RAS directory. This will
provide a top-level document and table of contents. It also provides the
option to build the RAS directory individually using "make SPHINXDIRS=".
Start a section on address translation. This will be expanded with
details for future translation methods and how they're used in the
kernel.
Move the error decoding topic to its own section. Links to other error
decoding kernel docs will be added.
Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lore.kernel.org/r/20231005173526.42831-2-yazen.ghannam@amd.com
v2->v3:
* New in v3.
* Move term glossary to Documentation.
v1->v2:
* N/A
Documentation/RAS/address-translation.rst | 24 +++++++++++++++++++
.../RAS/{ras.rst => error-decoding.rst} | 11 +++------
Documentation/RAS/index.rst | 14 +++++++++++
Documentation/index.rst | 2 +-
MAINTAINERS | 1 +
5 files changed, 43 insertions(+), 9 deletions(-)
create mode 100644 Documentation/RAS/address-translation.rst
rename Documentation/RAS/{ras.rst => error-decoding.rst} (73%)
create mode 100644 Documentation/RAS/index.rst
diff --git a/Documentation/RAS/address-translation.rst b/Documentation/RAS/address-translation.rst
new file mode 100644
index 000000000000..4b62ab007fce
--- /dev/null
+++ b/Documentation/RAS/address-translation.rst
@@ -0,0 +1,24 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Address translation
+===================
+
+x86 AMD
+-------
+
+Zen-based AMD systems include a Data Fabric that manages the layout of
+physical memory. Devices attached to the Fabric, like memory controllers,
+I/O, etc., may not have a complete view of the system physical memory map.
+These devices may provide a "normalized", i.e. device physical, address
+when reporting memory errors. Normalized addresses must be translated to
+a system physical address for the kernel to action on the memory.
+
+AMD Address Translation Library (CONFIG_AMD_ATL) provides translation for
+this case.
+
+Glossary of acronyms used in address translation for Zen-based systems
+
+* CCM = Cache Coherent Moderator
+* COD = Cluster-on-Die
+* CS = Coherent Station
+* DF = Data Fabric
diff --git a/Documentation/RAS/ras.rst b/Documentation/RAS/error-decoding.rst
similarity index 73%
rename from Documentation/RAS/ras.rst
rename to Documentation/RAS/error-decoding.rst
index 2556b397cd27..26a72f3fe5de 100644
--- a/Documentation/RAS/ras.rst
+++ b/Documentation/RAS/error-decoding.rst
@@ -1,15 +1,10 @@
.. SPDX-License-Identifier: GPL-2.0
-Reliability, Availability and Serviceability features
-=====================================================
-
-This documents different aspects of the RAS functionality present in the
-kernel.
-
Error decoding
----------------
+==============
-* x86
+x86
+---
Error decoding on AMD systems should be done using the rasdaemon tool:
https://github.com/mchehab/rasdaemon/
diff --git a/Documentation/RAS/index.rst b/Documentation/RAS/index.rst
new file mode 100644
index 000000000000..2794c1816e90
--- /dev/null
+++ b/Documentation/RAS/index.rst
@@ -0,0 +1,14 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===========================================================
+Reliability, Availability and Serviceability (RAS) features
+===========================================================
+
+This documents different aspects of the RAS functionality present in the
+kernel.
+
+.. toctree::
+ :maxdepth: 2
+
+ error-decoding
+ address-translation
diff --git a/Documentation/index.rst b/Documentation/index.rst
index 36e61783437c..07f2aa07f0fa 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -113,7 +113,7 @@ to ReStructured Text format, or are simply too old.
:maxdepth: 1
staging/index
- RAS/ras
+ RAS/index
Translations
diff --git a/MAINTAINERS b/MAINTAINERS
index b9b060917680..1e2d42cb7df3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18107,6 +18107,7 @@ M: Tony Luck <tony.luck@intel.com>
M: Borislav Petkov <bp@alien8.de>
L: linux-edac@vger.kernel.org
S: Maintained
+F: Documentation/RAS/
F: Documentation/admin-guide/ras.rst
F: drivers/ras/
F: include/linux/ras.h
--
2.34.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
@ 2023-12-11 14:20 ` Borislav Petkov
2023-12-11 15:28 ` Yazen Ghannam
2023-12-11 19:57 ` Borislav Petkov
` (3 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2023-12-11 14:20 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Sun, Dec 10, 2023 at 01:49:30PM -0600, Yazen Ghannam wrote:
> diff --git a/drivers/ras/amd/atl/Kconfig b/drivers/ras/amd/atl/Kconfig
> new file mode 100644
> index 000000000000..fcff387faa33
> --- /dev/null
> +++ b/drivers/ras/amd/atl/Kconfig
> @@ -0,0 +1,41 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# AMD Address Translation Library Kconfig
> +#
> +# Copyright (c) 2023, Advanced Micro Devices, Inc.
> +# All Rights Reserved.
> +#
> +# Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
> +
> +config AMD_ATL
> + tristate "AMD Address Translation Library"
> + depends on AMD_NB
> + imply AMD_ATL_MOD
> + select AMD_ATL_STUB
> + default N
> + help
> + This library includes support for implementation-specific
> + address translation procedures needed for various error
> + handling cases.
> +
> + Enable this option if using DRAM ECC on Zen-based systems
> + and OS-based error handling.
> +
> +config AMD_ATL_MOD
> + tristate "AMD Address Translation Library module"
> + depends on X86_64
> + default N
> + help
> + This option provides the module component of the AMD Address
> + Translation Library. It will be selected automatically if
> + building the library. Minimal dependencies are given to avoid
> + build errors for modules using the library.
> +
> +config AMD_ATL_STUB
> + bool "AMD Address Translation Library stub"
> + default N
> + help
> + This option provides minimal built-in code to support the
> + AMD Address Translation Library module. It will be selected
> + automatically if building the library. No dependencies are given
> + to avoid build errors for modules using the library.
So, provided this stub thing even makes sense (I need to continue
looking first since this is a huge diff), you don't need the help text
and the bool prompt: a stub is prompt-less and is selected
automatically.
IOW:
diff --git a/drivers/ras/amd/atl/Kconfig b/drivers/ras/amd/atl/Kconfig
index fcff387faa33..348498d49b01 100644
--- a/drivers/ras/amd/atl/Kconfig
+++ b/drivers/ras/amd/atl/Kconfig
@@ -32,10 +32,4 @@ config AMD_ATL_MOD
build errors for modules using the library.
config AMD_ATL_STUB
- bool "AMD Address Translation Library stub"
- default N
- help
- This option provides minimal built-in code to support the
- AMD Address Translation Library module. It will be selected
- automatically if building the library. No dependencies are given
- to avoid build errors for modules using the library.
+ def_bool N
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-11 14:20 ` Borislav Petkov
@ 2023-12-11 15:28 ` Yazen Ghannam
0 siblings, 0 replies; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-11 15:28 UTC (permalink / raw)
To: Borislav Petkov
Cc: yazen.ghannam, linux-edac, linux-kernel, tony.luck, x86,
avadhut.naik, john.allen, william.roche, muralidhara.mk
On 12/11/2023 9:20 AM, Borislav Petkov wrote:
> On Sun, Dec 10, 2023 at 01:49:30PM -0600, Yazen Ghannam wrote:
>> diff --git a/drivers/ras/amd/atl/Kconfig b/drivers/ras/amd/atl/Kconfig
>> new file mode 100644
>> index 000000000000..fcff387faa33
>> --- /dev/null
>> +++ b/drivers/ras/amd/atl/Kconfig
>> @@ -0,0 +1,41 @@
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +#
>> +# AMD Address Translation Library Kconfig
>> +#
>> +# Copyright (c) 2023, Advanced Micro Devices, Inc.
>> +# All Rights Reserved.
>> +#
>> +# Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
>> +
>> +config AMD_ATL
>> + tristate "AMD Address Translation Library"
>> + depends on AMD_NB
>> + imply AMD_ATL_MOD
>> + select AMD_ATL_STUB
>> + default N
>> + help
>> + This library includes support for implementation-specific
>> + address translation procedures needed for various error
>> + handling cases.
>> +
>> + Enable this option if using DRAM ECC on Zen-based systems
>> + and OS-based error handling.
>> +
>> +config AMD_ATL_MOD
>> + tristate "AMD Address Translation Library module"
>> + depends on X86_64
>> + default N
>> + help
>> + This option provides the module component of the AMD Address
>> + Translation Library. It will be selected automatically if
>> + building the library. Minimal dependencies are given to avoid
>> + build errors for modules using the library.
>> +
>> +config AMD_ATL_STUB
>> + bool "AMD Address Translation Library stub"
>> + default N
>> + help
>> + This option provides minimal built-in code to support the
>> + AMD Address Translation Library module. It will be selected
>> + automatically if building the library. No dependencies are given
>> + to avoid build errors for modules using the library.
>
> So, provided this stub thing even makes sense (I need to continue
> looking first since this is a huge diff), you don't need the help text
> and the bool prompt: a stub is prompt-less and is selected
> automatically.
>
> IOW:
>
> diff --git a/drivers/ras/amd/atl/Kconfig b/drivers/ras/amd/atl/Kconfig
> index fcff387faa33..348498d49b01 100644
> --- a/drivers/ras/amd/atl/Kconfig
> +++ b/drivers/ras/amd/atl/Kconfig
> @@ -32,10 +32,4 @@ config AMD_ATL_MOD
> build errors for modules using the library.
>
> config AMD_ATL_STUB
> - bool "AMD Address Translation Library stub"
> - default N
> - help
> - This option provides minimal built-in code to support the
> - AMD Address Translation Library module. It will be selected
> - automatically if building the library. No dependencies are given
> - to avoid build errors for modules using the library.
> + def_bool N
>
Ah good deal. Better than padding to meet a minimum length. :)
Thanks,
Yazen
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
2023-12-11 14:20 ` Borislav Petkov
@ 2023-12-11 19:57 ` Borislav Petkov
2023-12-12 14:23 ` Yazen Ghannam
2023-12-12 13:29 ` Borislav Petkov
` (2 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2023-12-11 19:57 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Sun, Dec 10, 2023 at 01:49:30PM -0600, Yazen Ghannam wrote:
> diff --git a/drivers/ras/amd/atl/core.c b/drivers/ras/amd/atl/core.c
> new file mode 100644
> index 000000000000..6a6220fef81f
> --- /dev/null
> +++ b/drivers/ras/amd/atl/core.c
> @@ -0,0 +1,217 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * AMD Address Translation Library
> + *
> + * core.c : Module init and base translation functions
> + *
> + * Copyright (c) 2023, Advanced Micro Devices, Inc.
> + * All Rights Reserved.
> + *
> + * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
> + */
> +
> +#include <linux/module.h>
> +#include <asm/cpu_device_id.h>
> +
> +#include "internal.h"
> +
> +struct df_config df_cfg __read_mostly;
> +
> +static int addr_over_limit(struct addr_ctx *ctx)
> +{
> + u64 dram_limit_addr;
> +
> + if (df_cfg.rev >= DF4)
> + dram_limit_addr = FIELD_GET(DF4_DRAM_LIMIT_ADDR, ctx->map.limit);
> + else
> + dram_limit_addr = FIELD_GET(DF2_DRAM_LIMIT_ADDR, ctx->map.limit);
One too many spaces before the '='.
> +
> + dram_limit_addr <<= DF_DRAM_BASE_LIMIT_LSB;
> + dram_limit_addr |= GENMASK(DF_DRAM_BASE_LIMIT_LSB - 1, 0);
> +
> + /* Is calculated system address above DRAM limit address? */
> + if (ctx->ret_addr > dram_limit_addr) {
> + warn_on_assert("Calculated address (0x%016llx) > DRAM limit (0x%016llx)",
Hmm, where is the "assert" aspect of that macro?
It looks to me more like atl_warn() type thing which you define for your
driver to do special stuff.
Also, are you sure you want to dump it here on every attempted SPA
conversion?
I guess yes. I'm just worried that it might become too noisy but we'll
fix it later if that turns out to be the case...
> + ctx->ret_addr, dram_limit_addr);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static bool legacy_hole_en(struct addr_ctx *ctx)
> +{
> + u32 reg = ctx->map.base;
> +
> + if (df_cfg.rev >= DF4)
> + reg = ctx->map.ctl;
> +
> + return FIELD_GET(DF_LEGACY_MMIO_HOLE_EN, reg);
> +}
> +
> +static int add_legacy_hole(struct addr_ctx *ctx)
> +{
> + u32 dram_hole_base;
> + u8 func = 0;
> +
> + if (!legacy_hole_en(ctx))
> + return 0;
> +
> + if (df_cfg.rev >= DF4)
> + func = 7;
> +
> + if (df_indirect_read_broadcast(ctx->node_id, func, 0x104, &dram_hole_base))
> + return -EINVAL;
> +
> + dram_hole_base &= DF_DRAM_HOLE_BASE_MASK;
> +
> + if (ctx->ret_addr >= dram_hole_base)
> + ctx->ret_addr += (BIT_ULL(32) - dram_hole_base);
> +
> + return 0;
> +}
> +
> +static u64 get_base_addr(struct addr_ctx *ctx)
> +{
> + u64 base_addr;
> +
> + if (df_cfg.rev >= DF4)
> + base_addr = FIELD_GET(DF4_BASE_ADDR, ctx->map.base);
> + else
> + base_addr = FIELD_GET(DF2_BASE_ADDR, ctx->map.base);
> +
> + return base_addr << DF_DRAM_BASE_LIMIT_LSB;
> +}
> +
> +static int add_base_and_hole(struct addr_ctx *ctx)
> +{
> + ctx->ret_addr += get_base_addr(ctx);
> +
> + if (add_legacy_hole(ctx))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static bool late_hole_remove(struct addr_ctx *ctx)
> +{
> + if (df_cfg.rev == DF3p5)
> + return true;
> +
> + if (df_cfg.rev == DF4)
> + return true;
> +
> + if (ctx->map.intlv_mode == DF3_6CHAN)
> + return true;
> +
> + return false;
> +}
> +
> +int norm_to_sys_addr(u8 socket_id, u8 die_id, u8 cs_inst_id, u64 *addr)
^^^^^^^^^
Can we not do that? Output function parameters.
Are all addr values valid or is there an invalid one - -1 for example
- which you can use as an error value?
And then you can turn this into:
unsigned long norm_to_sys_addr(u8 socket_id, u8 die_id, u8 cs_inst_id, unsigned long addr)
and callers can do IS_ERR_VALUE(ret) on the return value.
See include/linux/err.h
> +{
> + struct addr_ctx ctx;
> +
> + if (df_cfg.rev == UNKNOWN)
> + return -EINVAL;
> +
> + memset(&ctx, 0, sizeof(ctx));
> +
> + /* We start from the normalized address */
s/We start/Start/
> + ctx.ret_addr = *addr;
> + ctx.inst_id = cs_inst_id;
> +
> + ctx.inputs.norm_addr = *addr;
> + ctx.inputs.socket_id = socket_id;
> + ctx.inputs.die_id = die_id;
> + ctx.inputs.cs_inst_id = cs_inst_id;
> +
> + if (determine_node_id(&ctx, socket_id, die_id))
> + return -EINVAL;
> +
> + if (get_address_map(&ctx))
> + return -EINVAL;
> +
> + if (denormalize_address(&ctx))
> + return -EINVAL;
> +
> + if (!late_hole_remove(&ctx) && add_base_and_hole(&ctx))
> + return -EINVAL;
> +
> + if (dehash_address(&ctx))
> + return -EINVAL;
> +
> + if (late_hole_remove(&ctx) && add_base_and_hole(&ctx))
> + return -EINVAL;
> +
> + if (addr_over_limit(&ctx))
> + return -EINVAL;
> +
> + *addr = ctx.ret_addr;
> + return 0;
> +}
> +
> +static void check_for_legacy_df_access(void)
> +{
> + /*
> + * All Zen-based systems before Family 19h use the legacy
> + * DF Indirect Access (FICAA/FICAD) offsets.
> + */
> + if (boot_cpu_data.x86 < 0x19) {
> + df_cfg.flags.legacy_ficaa = true;
> + return;
> + }
> +
> + /* All systems after Family 19h use the current offsets. */
> + if (boot_cpu_data.x86 > 0x19)
> + return;
> +
> + /* Some Family 19h systems use the legacy offsets. */
> + switch (boot_cpu_data.x86_model) {
> + case 0x00 ... 0x0f:
> + case 0x20 ... 0x5f:
> + df_cfg.flags.legacy_ficaa = true;
> + }
> +}
> +
> +static const struct x86_cpu_id amd_atl_cpuids[] = {
> + X86_MATCH_FEATURE(X86_FEATURE_SMCA, NULL),
I'd expect for only this one to be needed, but not those below.
> + X86_MATCH_FEATURE(X86_FEATURE_ZEN, NULL),
> + X86_MATCH_FEATURE(X86_FEATURE_ZEN2, NULL),
> + X86_MATCH_FEATURE(X86_FEATURE_ZEN3, NULL),
> + X86_MATCH_FEATURE(X86_FEATURE_ZEN4, NULL),
> + { }
> +};
To be continued...
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
2023-12-11 14:20 ` Borislav Petkov
2023-12-11 19:57 ` Borislav Petkov
@ 2023-12-12 13:29 ` Borislav Petkov
2023-12-12 14:33 ` Yazen Ghannam
2023-12-14 10:54 ` Borislav Petkov
2023-12-14 14:30 ` Borislav Petkov
4 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2023-12-12 13:29 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Sun, Dec 10, 2023 at 01:49:30PM -0600, Yazen Ghannam wrote:
> diff --git a/drivers/ras/amd/atl/dehash.c b/drivers/ras/amd/atl/dehash.c
> new file mode 100644
> index 000000000000..84fe9793694e
> --- /dev/null
> +++ b/drivers/ras/amd/atl/dehash.c
> @@ -0,0 +1,446 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * AMD Address Translation Library
> + *
> + * dehash.c : Functions to account for hashing bits
> + *
> + * Copyright (c) 2023, Advanced Micro Devices, Inc.
> + * All Rights Reserved.
> + *
> + * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
> + */
> +
> +#include "internal.h"
> +
> +static inline bool assert_intlv_bit(struct addr_ctx *ctx, u8 bit1, u8 bit2)
> +{
> + if (ctx->map.intlv_bit_pos == bit1 || ctx->map.intlv_bit_pos == bit2)
> + return false;
> +
> + warn_on_assert("%s: Invalid interleave bit: %u",
> + __func__, ctx->map.intlv_bit_pos);
> +
> + return true;
> +}
> +
> +static inline bool assert_num_intlv_dies(struct addr_ctx *ctx, u8 num_intlv_dies)
> +{
> + if (ctx->map.num_intlv_dies <= num_intlv_dies)
> + return false;
> +
> + warn_on_assert("%s: Invalid number of interleave dies: %u",
> + __func__, ctx->map.num_intlv_dies);
> +
> + return true;
> +}
> +
> +static inline bool assert_num_intlv_sockets(struct addr_ctx *ctx, u8 num_intlv_sockets)
> +{
> + if (ctx->map.num_intlv_sockets <= num_intlv_sockets)
> + return false;
> +
> + warn_on_assert("%s: Invalid number of interleave sockets: %u",
> + __func__, ctx->map.num_intlv_sockets);
> +
> + return true;
> +}
> +
> +static int df2_dehash_addr(struct addr_ctx *ctx)
> +{
> + u8 hashed_bit, intlv_bit, intlv_bit_pos;
> +
> + /* Assert that interleave bit is 8 or 9. */
> + if (assert_intlv_bit(ctx, 8, 9))
> + return -EINVAL;
You don't need those homegrown assertions. Instead, you do this:
diff --git a/drivers/ras/amd/atl/dehash.c b/drivers/ras/amd/atl/dehash.c
index 84fe9793694e..11634001702e 100644
--- a/drivers/ras/amd/atl/dehash.c
+++ b/drivers/ras/amd/atl/dehash.c
@@ -47,10 +47,12 @@ static inline bool assert_num_intlv_sockets(struct addr_ctx *ctx, u8 num_intlv_s
static int df2_dehash_addr(struct addr_ctx *ctx)
{
- u8 hashed_bit, intlv_bit, intlv_bit_pos;
+ u8 hashed_bit, intlv_bit;
+ u8 intlv_bit_pos = ctx->map.intlv_bit_pos;
/* Assert that interleave bit is 8 or 9. */
- if (assert_intlv_bit(ctx, 8, 9))
+ if (WARN(intlv_bit_pos != 8 && intlv_bit_pos != 9,
+ "Invalid interleave bit: %u\n", intlv_bit_pos))
return -EINVAL;
/* Assert that die and socket interleaving are disabled. */
@@ -60,7 +62,6 @@ static int df2_dehash_addr(struct addr_ctx *ctx)
if (assert_num_intlv_sockets(ctx, 1))
return -EINVAL;
- intlv_bit_pos = ctx->map.intlv_bit_pos;
intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
hashed_bit = intlv_bit;
and so on for the other two.
> + /* Assert that die and socket interleaving are disabled. */
> + if (assert_num_intlv_dies(ctx, 1))
> + return -EINVAL;
> +
> + if (assert_num_intlv_sockets(ctx, 1))
> + return -EINVAL;
> +
> + intlv_bit_pos = ctx->map.intlv_bit_pos;
> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
Can we keep it simple please?
intlv_bit = !!(BIT_ULL(intlv_bit_pos) & ctx->ret_addr);
That atl_get_bit() is not necessary.
> + hashed_bit = intlv_bit;
> + hashed_bit ^= FIELD_GET(BIT_ULL(12), ctx->ret_addr);
> + hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr);
> + hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr);
> + hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr);
> +
> + if (hashed_bit != intlv_bit)
> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
> +
> + return 0;
> +}
<---
> +static int df3_dehash_addr(struct addr_ctx *ctx)
> +{
> + bool hash_ctl_64k, hash_ctl_2M, hash_ctl_1G;
> + u8 hashed_bit, intlv_bit, intlv_bit_pos;
> +
> + /* Assert that interleave bit is 8 or 9. */
> + if (assert_intlv_bit(ctx, 8, 9))
> + return -EINVAL;
> +
> + /* Assert that die and socket interleaving are disabled. */
> + if (assert_num_intlv_dies(ctx, 1))
> + return -EINVAL;
> +
> + if (assert_num_intlv_sockets(ctx, 1))
> + return -EINVAL;
Those assertions keep repeating. Extract them into a separate function
which you call from every *dehash_addr function?
> + hash_ctl_64k = FIELD_GET(DF3_HASH_CTL_64K, ctx->map.ctl);
> + hash_ctl_2M = FIELD_GET(DF3_HASH_CTL_2M, ctx->map.ctl);
> + hash_ctl_1G = FIELD_GET(DF3_HASH_CTL_1G, ctx->map.ctl);
I believe without the tabs looks good too:
hash_ctl_64k = FIELD_GET(DF3_HASH_CTL_64K, ctx->map.ctl);
hash_ctl_2M = FIELD_GET(DF3_HASH_CTL_2M, ctx->map.ctl);
hash_ctl_1G = FIELD_GET(DF3_HASH_CTL_1G, ctx->map.ctl);
> + intlv_bit_pos = ctx->map.intlv_bit_pos;
> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
> +
> + hashed_bit = intlv_bit;
> + hashed_bit ^= FIELD_GET(BIT_ULL(14), ctx->ret_addr);
> + hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr) & hash_ctl_64k;
> + hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
> + hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
> +
> + if (hashed_bit != intlv_bit)
> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
> +
> + /* Calculation complete for 2 channels. Continue for 4 and 8 channels. */
> + if (ctx->map.intlv_mode == DF3_COD4_2CHAN_HASH)
> + return 0;
> +
> + intlv_bit = FIELD_GET(BIT_ULL(12), ctx->ret_addr);
> +
> + hashed_bit = intlv_bit;
> + hashed_bit ^= FIELD_GET(BIT_ULL(16), ctx->ret_addr) & hash_ctl_64k;
> + hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
> + hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
> +
> + if (hashed_bit != intlv_bit)
> + ctx->ret_addr ^= BIT_ULL(12);
> +
> + /* Calculation complete for 4 channels. Continue for 8 channels. */
> + if (ctx->map.intlv_mode == DF3_COD2_4CHAN_HASH)
> + return 0;
> +
> + intlv_bit = FIELD_GET(BIT_ULL(13), ctx->ret_addr);
> +
> + hashed_bit = intlv_bit;
> + hashed_bit ^= FIELD_GET(BIT_ULL(17), ctx->ret_addr) & hash_ctl_64k;
> + hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
> + hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
> +
> + if (hashed_bit != intlv_bit)
> + ctx->ret_addr ^= BIT_ULL(13);
> +
> + return 0;
> +}
Also, same comments about this function as for df2_dehash_addr(). Below
too.
> +
> +static int df3_6chan_dehash_addr(struct addr_ctx *ctx)
> +{
> + u8 intlv_bit_pos = ctx->map.intlv_bit_pos;
> + u8 hashed_bit, intlv_bit, num_intlv_bits;
> + bool hash_ctl_2M, hash_ctl_1G;
> +
> + if (ctx->map.intlv_mode != DF3_6CHAN) {
> + warn_on_bad_intlv_mode(ctx);
> + return -EINVAL;
> + }
> +
> + num_intlv_bits = ilog2(ctx->map.num_intlv_chan) + 1;
> +
> + hash_ctl_2M = FIELD_GET(DF3_HASH_CTL_2M, ctx->map.ctl);
> + hash_ctl_1G = FIELD_GET(DF3_HASH_CTL_1G, ctx->map.ctl);
> +
> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
> +
> + hashed_bit = intlv_bit;
> + hashed_bit ^= atl_get_bit((intlv_bit_pos + num_intlv_bits), ctx->ret_addr);
> + hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
> + hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
> +
> + if (hashed_bit != intlv_bit)
> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
> +
> + intlv_bit_pos++;
> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
> +
> + hashed_bit = intlv_bit;
> + hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
> + hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
> +
> + if (hashed_bit != intlv_bit)
> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
> +
> + intlv_bit_pos++;
> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
> +
> + hashed_bit = intlv_bit;
> + hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
> + hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
> +
> + if (hashed_bit != intlv_bit)
> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
> +
> + return 0;
> +}
...
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-11 19:57 ` Borislav Petkov
@ 2023-12-12 14:23 ` Yazen Ghannam
2023-12-12 15:34 ` Borislav Petkov
0 siblings, 1 reply; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-12 14:23 UTC (permalink / raw)
To: Borislav Petkov
Cc: yazen.ghannam, linux-edac, linux-kernel, tony.luck, x86,
avadhut.naik, john.allen, william.roche, muralidhara.mk
On 12/11/2023 2:57 PM, Borislav Petkov wrote:
> On Sun, Dec 10, 2023 at 01:49:30PM -0600, Yazen Ghannam wrote:
>> diff --git a/drivers/ras/amd/atl/core.c b/drivers/ras/amd/atl/core.c
>> new file mode 100644
>> index 000000000000..6a6220fef81f
>> --- /dev/null
>> +++ b/drivers/ras/amd/atl/core.c
>> @@ -0,0 +1,217 @@
>> +// SPDX-License-Identifier: GPL-2.0-or-later
>> +/*
>> + * AMD Address Translation Library
>> + *
>> + * core.c : Module init and base translation functions
>> + *
>> + * Copyright (c) 2023, Advanced Micro Devices, Inc.
>> + * All Rights Reserved.
>> + *
>> + * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
>> + */
>> +
>> +#include <linux/module.h>
>> +#include <asm/cpu_device_id.h>
>> +
>> +#include "internal.h"
>> +
>> +struct df_config df_cfg __read_mostly;
>> +
>> +static int addr_over_limit(struct addr_ctx *ctx)
>> +{
>> + u64 dram_limit_addr;
>> +
>> + if (df_cfg.rev >= DF4)
>> + dram_limit_addr = FIELD_GET(DF4_DRAM_LIMIT_ADDR, ctx->map.limit);
>> + else
>> + dram_limit_addr = FIELD_GET(DF2_DRAM_LIMIT_ADDR, ctx->map.limit);
>
> One too many spaces before the '='.
>
Ack
>> +
>> + dram_limit_addr <<= DF_DRAM_BASE_LIMIT_LSB;
>> + dram_limit_addr |= GENMASK(DF_DRAM_BASE_LIMIT_LSB - 1, 0);
>> +
>> + /* Is calculated system address above DRAM limit address? */
>> + if (ctx->ret_addr > dram_limit_addr) {
>> + warn_on_assert("Calculated address (0x%016llx) > DRAM limit (0x%016llx)",
>
> Hmm, where is the "assert" aspect of that macro?
>
I'm thinking that the warning only happens if the "assert" condition
above is hit.
> It looks to me more like atl_warn() type thing which you define for your
> driver to do special stuff.
>
Right, that's what I was going for. I can rename/rework it.
> Also, are you sure you want to dump it here on every attempted SPA
> conversion?
>
> I guess yes. I'm just worried that it might become too noisy but we'll
> fix it later if that turns out to be the case...
>
In older revisions, I had all these messages as "debug" loglevel. I
don't think there's anything a user can do to fix these issues. They're
either coding bugs in the library or system configuration.
I'd rather go back to the debug messages if you don't mind. It's not
difficult to enable dynamic debug messages compared to DEBUG Kconfig
options. So I think it'd be okay to work with users on this if they
encounter an issue.
>> + ctx->ret_addr, dram_limit_addr);
>> + return -EINVAL;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static bool legacy_hole_en(struct addr_ctx *ctx)
>> +{
>> + u32 reg = ctx->map.base;
>> +
>> + if (df_cfg.rev >= DF4)
>> + reg = ctx->map.ctl;
>> +
>> + return FIELD_GET(DF_LEGACY_MMIO_HOLE_EN, reg);
>> +}
>> +
>> +static int add_legacy_hole(struct addr_ctx *ctx)
>> +{
>> + u32 dram_hole_base;
>> + u8 func = 0;
>> +
>> + if (!legacy_hole_en(ctx))
>> + return 0;
>> +
>> + if (df_cfg.rev >= DF4)
>> + func = 7;
>> +
>> + if (df_indirect_read_broadcast(ctx->node_id, func, 0x104, &dram_hole_base))
>> + return -EINVAL;
>> +
>> + dram_hole_base &= DF_DRAM_HOLE_BASE_MASK;
>> +
>> + if (ctx->ret_addr >= dram_hole_base)
>> + ctx->ret_addr += (BIT_ULL(32) - dram_hole_base);
>> +
>> + return 0;
>> +}
>> +
>> +static u64 get_base_addr(struct addr_ctx *ctx)
>> +{
>> + u64 base_addr;
>> +
>> + if (df_cfg.rev >= DF4)
>> + base_addr = FIELD_GET(DF4_BASE_ADDR, ctx->map.base);
>> + else
>> + base_addr = FIELD_GET(DF2_BASE_ADDR, ctx->map.base);
>> +
>> + return base_addr << DF_DRAM_BASE_LIMIT_LSB;
>> +}
>> +
>> +static int add_base_and_hole(struct addr_ctx *ctx)
>> +{
>> + ctx->ret_addr += get_base_addr(ctx);
>> +
>> + if (add_legacy_hole(ctx))
>> + return -EINVAL;
>> +
>> + return 0;
>> +}
>> +
>> +static bool late_hole_remove(struct addr_ctx *ctx)
>> +{
>> + if (df_cfg.rev == DF3p5)
>> + return true;
>> +
>> + if (df_cfg.rev == DF4)
>> + return true;
>> +
>> + if (ctx->map.intlv_mode == DF3_6CHAN)
>> + return true;
>> +
>> + return false;
>> +}
>> +
>> +int norm_to_sys_addr(u8 socket_id, u8 die_id, u8 cs_inst_id, u64 *addr)
> ^^^^^^^^^
>
> Can we not do that? Output function parameters.
>
> Are all addr values valid or is there an invalid one - -1 for example
> - which you can use as an error value?
>
> And then you can turn this into:
>
> unsigned long norm_to_sys_addr(u8 socket_id, u8 die_id, u8 cs_inst_id, unsigned long addr)
>
> and callers can do IS_ERR_VALUE(ret) on the return value.
>
> See include/linux/err.h
>
Good idea. I'll look into it.
>> +{
>> + struct addr_ctx ctx;
>> +
>> + if (df_cfg.rev == UNKNOWN)
>> + return -EINVAL;
>> +
>> + memset(&ctx, 0, sizeof(ctx));
>> +
>> + /* We start from the normalized address */
>
> s/We start/Start/
>
Ack
>> + ctx.ret_addr = *addr;
>> + ctx.inst_id = cs_inst_id;
>> +
>> + ctx.inputs.norm_addr = *addr;
>> + ctx.inputs.socket_id = socket_id;
>> + ctx.inputs.die_id = die_id;
>> + ctx.inputs.cs_inst_id = cs_inst_id;
>> +
>> + if (determine_node_id(&ctx, socket_id, die_id))
>> + return -EINVAL;
>> +
>> + if (get_address_map(&ctx))
>> + return -EINVAL;
>> +
>> + if (denormalize_address(&ctx))
>> + return -EINVAL;
>> +
>> + if (!late_hole_remove(&ctx) && add_base_and_hole(&ctx))
>> + return -EINVAL;
>> +
>> + if (dehash_address(&ctx))
>> + return -EINVAL;
>> +
>> + if (late_hole_remove(&ctx) && add_base_and_hole(&ctx))
>> + return -EINVAL;
>> +
>> + if (addr_over_limit(&ctx))
>> + return -EINVAL;
>> +
>> + *addr = ctx.ret_addr;
>> + return 0;
>> +}
>> +
>> +static void check_for_legacy_df_access(void)
>> +{
>> + /*
>> + * All Zen-based systems before Family 19h use the legacy
>> + * DF Indirect Access (FICAA/FICAD) offsets.
>> + */
>> + if (boot_cpu_data.x86 < 0x19) {
>> + df_cfg.flags.legacy_ficaa = true;
>> + return;
>> + }
>> +
>> + /* All systems after Family 19h use the current offsets. */
>> + if (boot_cpu_data.x86 > 0x19)
>> + return;
>> +
>> + /* Some Family 19h systems use the legacy offsets. */
>> + switch (boot_cpu_data.x86_model) {
>> + case 0x00 ... 0x0f:
>> + case 0x20 ... 0x5f:
>> + df_cfg.flags.legacy_ficaa = true;
>> + }
>> +}
>> +
>> +static const struct x86_cpu_id amd_atl_cpuids[] = {
>> + X86_MATCH_FEATURE(X86_FEATURE_SMCA, NULL),
>
> I'd expect for only this one to be needed, but not those below.
>
Me too. Those below are to workaround a current module loading issue.
I'll add a code comment for that.
>> + X86_MATCH_FEATURE(X86_FEATURE_ZEN, NULL),
>> + X86_MATCH_FEATURE(X86_FEATURE_ZEN2, NULL),
>> + X86_MATCH_FEATURE(X86_FEATURE_ZEN3, NULL),
>> + X86_MATCH_FEATURE(X86_FEATURE_ZEN4, NULL),
>> + { }
>> +};
>
> To be continued...
>
Thanks,
Yazen
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-12 13:29 ` Borislav Petkov
@ 2023-12-12 14:33 ` Yazen Ghannam
2023-12-12 16:07 ` Borislav Petkov
0 siblings, 1 reply; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-12 14:33 UTC (permalink / raw)
To: Borislav Petkov
Cc: yazen.ghannam, linux-edac, linux-kernel, tony.luck, x86,
avadhut.naik, john.allen, william.roche, muralidhara.mk
On 12/12/2023 8:29 AM, Borislav Petkov wrote:
> On Sun, Dec 10, 2023 at 01:49:30PM -0600, Yazen Ghannam wrote:
>> diff --git a/drivers/ras/amd/atl/dehash.c b/drivers/ras/amd/atl/dehash.c
>> new file mode 100644
>> index 000000000000..84fe9793694e
>> --- /dev/null
>> +++ b/drivers/ras/amd/atl/dehash.c
>> @@ -0,0 +1,446 @@
>> +// SPDX-License-Identifier: GPL-2.0-or-later
>> +/*
>> + * AMD Address Translation Library
>> + *
>> + * dehash.c : Functions to account for hashing bits
>> + *
>> + * Copyright (c) 2023, Advanced Micro Devices, Inc.
>> + * All Rights Reserved.
>> + *
>> + * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
>> + */
>> +
>> +#include "internal.h"
>> +
>> +static inline bool assert_intlv_bit(struct addr_ctx *ctx, u8 bit1, u8 bit2)
>> +{
>> + if (ctx->map.intlv_bit_pos == bit1 || ctx->map.intlv_bit_pos == bit2)
>> + return false;
>> +
>> + warn_on_assert("%s: Invalid interleave bit: %u",
>> + __func__, ctx->map.intlv_bit_pos);
>> +
>> + return true;
>> +}
>> +
>> +static inline bool assert_num_intlv_dies(struct addr_ctx *ctx, u8 num_intlv_dies)
>> +{
>> + if (ctx->map.num_intlv_dies <= num_intlv_dies)
>> + return false;
>> +
>> + warn_on_assert("%s: Invalid number of interleave dies: %u",
>> + __func__, ctx->map.num_intlv_dies);
>> +
>> + return true;
>> +}
>> +
>> +static inline bool assert_num_intlv_sockets(struct addr_ctx *ctx, u8 num_intlv_sockets)
>> +{
>> + if (ctx->map.num_intlv_sockets <= num_intlv_sockets)
>> + return false;
>> +
>> + warn_on_assert("%s: Invalid number of interleave sockets: %u",
>> + __func__, ctx->map.num_intlv_sockets);
>> +
>> + return true;
>> +}
>> +
>> +static int df2_dehash_addr(struct addr_ctx *ctx)
>> +{
>> + u8 hashed_bit, intlv_bit, intlv_bit_pos;
>> +
>> + /* Assert that interleave bit is 8 or 9. */
>> + if (assert_intlv_bit(ctx, 8, 9))
>> + return -EINVAL;
>
> You don't need those homegrown assertions. Instead, you do this:
>
> diff --git a/drivers/ras/amd/atl/dehash.c b/drivers/ras/amd/atl/dehash.c
> index 84fe9793694e..11634001702e 100644
> --- a/drivers/ras/amd/atl/dehash.c
> +++ b/drivers/ras/amd/atl/dehash.c
> @@ -47,10 +47,12 @@ static inline bool assert_num_intlv_sockets(struct addr_ctx *ctx, u8 num_intlv_s
>
> static int df2_dehash_addr(struct addr_ctx *ctx)
> {
> - u8 hashed_bit, intlv_bit, intlv_bit_pos;
> + u8 hashed_bit, intlv_bit;
> + u8 intlv_bit_pos = ctx->map.intlv_bit_pos;
>
> /* Assert that interleave bit is 8 or 9. */
> - if (assert_intlv_bit(ctx, 8, 9))
> + if (WARN(intlv_bit_pos != 8 && intlv_bit_pos != 9,
> + "Invalid interleave bit: %u\n", intlv_bit_pos))
> return -EINVAL;
>
> /* Assert that die and socket interleaving are disabled. */
> @@ -60,7 +62,6 @@ static int df2_dehash_addr(struct addr_ctx *ctx)
> if (assert_num_intlv_sockets(ctx, 1))
> return -EINVAL;
>
> - intlv_bit_pos = ctx->map.intlv_bit_pos;
> intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
>
> hashed_bit = intlv_bit;
>
> and so on for the other two.
That's mostly how it was in the previous revision. Should I go back to
that then?
>
>> + /* Assert that die and socket interleaving are disabled. */
>> + if (assert_num_intlv_dies(ctx, 1))
>> + return -EINVAL;
>> +
>> + if (assert_num_intlv_sockets(ctx, 1))
>> + return -EINVAL;
>> +
>> + intlv_bit_pos = ctx->map.intlv_bit_pos;
>> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
>
> Can we keep it simple please?
>
> intlv_bit = !!(BIT_ULL(intlv_bit_pos) & ctx->ret_addr);
>
> That atl_get_bit() is not necessary.
Okay, will change.
>
>> + hashed_bit = intlv_bit;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(12), ctx->ret_addr);
>> + hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr);
>> + hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr);
>> + hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr);
>> +
>> + if (hashed_bit != intlv_bit)
>> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
>> +
>> + return 0;
>> +}
>
> <---
>
Ack
>> +static int df3_dehash_addr(struct addr_ctx *ctx)
>> +{
>> + bool hash_ctl_64k, hash_ctl_2M, hash_ctl_1G;
>> + u8 hashed_bit, intlv_bit, intlv_bit_pos;
>> +
>> + /* Assert that interleave bit is 8 or 9. */
>> + if (assert_intlv_bit(ctx, 8, 9))
>> + return -EINVAL;
>> +
>> + /* Assert that die and socket interleaving are disabled. */
>> + if (assert_num_intlv_dies(ctx, 1))
>> + return -EINVAL;
>> +
>> + if (assert_num_intlv_sockets(ctx, 1))
>> + return -EINVAL;
>
> Those assertions keep repeating. Extract them into a separate function
> which you call from every *dehash_addr function?
>
Okay, can do. This can drop the assert_*() helpers like the comments above.
>> + hash_ctl_64k = FIELD_GET(DF3_HASH_CTL_64K, ctx->map.ctl);
>> + hash_ctl_2M = FIELD_GET(DF3_HASH_CTL_2M, ctx->map.ctl);
>> + hash_ctl_1G = FIELD_GET(DF3_HASH_CTL_1G, ctx->map.ctl);
>
> I believe without the tabs looks good too:
>
> hash_ctl_64k = FIELD_GET(DF3_HASH_CTL_64K, ctx->map.ctl);
> hash_ctl_2M = FIELD_GET(DF3_HASH_CTL_2M, ctx->map.ctl);
> hash_ctl_1G = FIELD_GET(DF3_HASH_CTL_1G, ctx->map.ctl);
>
Okay.
>> + intlv_bit_pos = ctx->map.intlv_bit_pos;
>> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
>> +
>> + hashed_bit = intlv_bit;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(14), ctx->ret_addr);
>> + hashed_bit ^= FIELD_GET(BIT_ULL(18), ctx->ret_addr) & hash_ctl_64k;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
>> +
>> + if (hashed_bit != intlv_bit)
>> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
>> +
>> + /* Calculation complete for 2 channels. Continue for 4 and 8 channels. */
>> + if (ctx->map.intlv_mode == DF3_COD4_2CHAN_HASH)
>> + return 0;
>> +
>> + intlv_bit = FIELD_GET(BIT_ULL(12), ctx->ret_addr);
>> +
>> + hashed_bit = intlv_bit;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(16), ctx->ret_addr) & hash_ctl_64k;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
>> +
>> + if (hashed_bit != intlv_bit)
>> + ctx->ret_addr ^= BIT_ULL(12);
>> +
>> + /* Calculation complete for 4 channels. Continue for 8 channels. */
>> + if (ctx->map.intlv_mode == DF3_COD2_4CHAN_HASH)
>> + return 0;
>> +
>> + intlv_bit = FIELD_GET(BIT_ULL(13), ctx->ret_addr);
>> +
>> + hashed_bit = intlv_bit;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(17), ctx->ret_addr) & hash_ctl_64k;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
>> +
>> + if (hashed_bit != intlv_bit)
>> + ctx->ret_addr ^= BIT_ULL(13);
>> +
>> + return 0;
>> +}
>
> Also, same comments about this function as for df2_dehash_addr(). Below
> too.
>
Okay.
>> +
>> +static int df3_6chan_dehash_addr(struct addr_ctx *ctx)
>> +{
>> + u8 intlv_bit_pos = ctx->map.intlv_bit_pos;
>> + u8 hashed_bit, intlv_bit, num_intlv_bits;
>> + bool hash_ctl_2M, hash_ctl_1G;
>> +
>> + if (ctx->map.intlv_mode != DF3_6CHAN) {
>> + warn_on_bad_intlv_mode(ctx);
>> + return -EINVAL;
>> + }
>> +
>> + num_intlv_bits = ilog2(ctx->map.num_intlv_chan) + 1;
>> +
>> + hash_ctl_2M = FIELD_GET(DF3_HASH_CTL_2M, ctx->map.ctl);
>> + hash_ctl_1G = FIELD_GET(DF3_HASH_CTL_1G, ctx->map.ctl);
>> +
>> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
>> +
>> + hashed_bit = intlv_bit;
>> + hashed_bit ^= atl_get_bit((intlv_bit_pos + num_intlv_bits), ctx->ret_addr);
>> + hashed_bit ^= FIELD_GET(BIT_ULL(23), ctx->ret_addr) & hash_ctl_2M;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(32), ctx->ret_addr) & hash_ctl_1G;
>> +
>> + if (hashed_bit != intlv_bit)
>> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
>> +
>> + intlv_bit_pos++;
>> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
>> +
>> + hashed_bit = intlv_bit;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(21), ctx->ret_addr) & hash_ctl_2M;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(30), ctx->ret_addr) & hash_ctl_1G;
>> +
>> + if (hashed_bit != intlv_bit)
>> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
>> +
>> + intlv_bit_pos++;
>> + intlv_bit = atl_get_bit(intlv_bit_pos, ctx->ret_addr);
>> +
>> + hashed_bit = intlv_bit;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(22), ctx->ret_addr) & hash_ctl_2M;
>> + hashed_bit ^= FIELD_GET(BIT_ULL(31), ctx->ret_addr) & hash_ctl_1G;
>> +
>> + if (hashed_bit != intlv_bit)
>> + ctx->ret_addr ^= BIT_ULL(intlv_bit_pos);
>> +
>> + return 0;
>> +}
>
> ...
>
Thanks,
Yazen
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-12 14:23 ` Yazen Ghannam
@ 2023-12-12 15:34 ` Borislav Petkov
2023-12-13 15:35 ` Yazen Ghannam
0 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2023-12-12 15:34 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Tue, Dec 12, 2023 at 09:23:44AM -0500, Yazen Ghannam wrote:
> I'm thinking that the warning only happens if the "assert" condition above
> is hit.
assert usually means "assert - abort the program if assertion is false"
- from assert(3).
> In older revisions, I had all these messages as "debug" loglevel. I don't
> think there's anything a user can do to fix these issues. They're either
> coding bugs in the library or system configuration.
>
> I'd rather go back to the debug messages if you don't mind. It's not
> difficult to enable dynamic debug messages compared to DEBUG Kconfig
> options. So I think it'd be okay to work with users on this if they
> encounter an issue.
Makes sense.
> > > +static const struct x86_cpu_id amd_atl_cpuids[] = {
> > > + X86_MATCH_FEATURE(X86_FEATURE_SMCA, NULL),
> >
> > I'd expect for only this one to be needed, but not those below.
> >
>
> Me too. Those below are to workaround a current module loading issue. I'll
> add a code comment for that.
You mean the systemdoofus crap?
Fget it - we don't fix the kernel because luserspace is nuts.
>
> > > + X86_MATCH_FEATURE(X86_FEATURE_ZEN, NULL),
...and those are influx - this is called X86_FEATURE_ZEN1 now and
X86_FEATURE_ZEN is set on all Zens. So you might as well match on
X86_FEATURE_ZEN only.
But you should not need it - if SMCA doesn't match then we have another
problem. ATL should load on SMCA systems only.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-12 14:33 ` Yazen Ghannam
@ 2023-12-12 16:07 ` Borislav Petkov
0 siblings, 0 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-12-12 16:07 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Tue, Dec 12, 2023 at 09:33:44AM -0500, Yazen Ghannam wrote:
> That's mostly how it was in the previous revision. Should I go back to that
> then?
Yes please. We should avoid doing homegrown stuff and use the kernel
facilities which are already there.
> Okay, can do. This can drop the assert_*() helpers like the comments above.
Yap.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-12 15:34 ` Borislav Petkov
@ 2023-12-13 15:35 ` Yazen Ghannam
2023-12-13 16:48 ` Borislav Petkov
0 siblings, 1 reply; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-13 15:35 UTC (permalink / raw)
To: Borislav Petkov
Cc: yazen.ghannam, linux-edac, linux-kernel, tony.luck, x86,
avadhut.naik, john.allen, william.roche, muralidhara.mk
On 12/12/2023 10:34 AM, Borislav Petkov wrote:
> On Tue, Dec 12, 2023 at 09:23:44AM -0500, Yazen Ghannam wrote:
>> I'm thinking that the warning only happens if the "assert" condition above
>> is hit.
>
> assert usually means "assert - abort the program if assertion is false"
> - from assert(3).
>
Right, agreed. In this context, the program is the translation method.
But yeah, it doesn't make much sense describing the kernel. I'll change
the wording if I don't drop the macros completely.
>> In older revisions, I had all these messages as "debug" loglevel. I don't
>> think there's anything a user can do to fix these issues. They're either
>> coding bugs in the library or system configuration.
>>
>> I'd rather go back to the debug messages if you don't mind. It's not
>> difficult to enable dynamic debug messages compared to DEBUG Kconfig
>> options. So I think it'd be okay to work with users on this if they
>> encounter an issue.
>
> Makes sense.
>
>>>> +static const struct x86_cpu_id amd_atl_cpuids[] = {
>>>> + X86_MATCH_FEATURE(X86_FEATURE_SMCA, NULL),
>>>
>>> I'd expect for only this one to be needed, but not those below.
>>>
>>
>> Me too. Those below are to workaround a current module loading issue. I'll
>> add a code comment for that.
>
> You mean the systemdoofus crap?
>
> Fget it - we don't fix the kernel because luserspace is nuts.
>
>>
>>>> + X86_MATCH_FEATURE(X86_FEATURE_ZEN, NULL),
>
> ...and those are influx - this is called X86_FEATURE_ZEN1 now and
> X86_FEATURE_ZEN is set on all Zens. So you might as well match on
> X86_FEATURE_ZEN only.
>
> But you should not need it - if SMCA doesn't match then we have another
> problem. ATL should load on SMCA systems only.
>
I agree in principle. But I don't think it hurts to include an
additional line to avoid the confusion when the module doesn't load.
Also, the SMCA feature is used here as a short-cut to match on systems
with a Data Fabric. We could use the Zen feature in the same way.
Thanks,
Yazen
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-13 15:35 ` Yazen Ghannam
@ 2023-12-13 16:48 ` Borislav Petkov
2023-12-13 17:04 ` Yazen Ghannam
0 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2023-12-13 16:48 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Wed, Dec 13, 2023 at 10:35:55AM -0500, Yazen Ghannam wrote:
> I agree in principle. But I don't think it hurts to include an additional
> line to avoid the confusion when the module doesn't load.
It does hurt because this turns into constant family updating the moment
a new family appears. This is one of the major reasons why we do CPUID
bits.
> Also, the SMCA feature is used here as a short-cut to match on systems with
> a Data Fabric. We could use the Zen feature in the same way.
We could.
What is the main description of the environment an ATL library belongs
into: a SMCA system or a Zen-based system?
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-13 16:48 ` Borislav Petkov
@ 2023-12-13 17:04 ` Yazen Ghannam
2023-12-13 17:07 ` Borislav Petkov
0 siblings, 1 reply; 18+ messages in thread
From: Yazen Ghannam @ 2023-12-13 17:04 UTC (permalink / raw)
To: Borislav Petkov
Cc: yazen.ghannam, linux-edac, linux-kernel, tony.luck, x86,
avadhut.naik, john.allen, william.roche, muralidhara.mk
On 12/13/2023 11:48 AM, Borislav Petkov wrote:
> On Wed, Dec 13, 2023 at 10:35:55AM -0500, Yazen Ghannam wrote:
>> I agree in principle. But I don't think it hurts to include an additional
>> line to avoid the confusion when the module doesn't load.
>
> It does hurt because this turns into constant family updating the moment
> a new family appears. This is one of the major reasons why we do CPUID
> bits.
>
>> Also, the SMCA feature is used here as a short-cut to match on systems with
>> a Data Fabric. We could use the Zen feature in the same way.
>
> We could.
>
> What is the main description of the environment an ATL library belongs
> into: a SMCA system or a Zen-based system?
>
Systems with an AMD Data Fabric.
I wrote up this comment for the amd_atl_cpuids[].
/*
This library provides functionality for AMD-based systems with a Data
Fabric. The set of systems with a Data Fabric is equivalent to the set
of Zen-based systems and the set of systems with the Scalable MCA
feature at this time. However, these are technically independent things.
It's possible to match on the PCI IDs of the Data Fabric devices, but
this will be an ever-expanding list. Instead match on the SMCA and Zen
features to cover all relevant systems.
*/
We could also introduce another software feature bit for Data Fabrics
"DF". And this could be set when we discover them, like in the AMD_NB code.
Thoughts?
The library init has three checks for system support. Comments added here.
// Load on systems with Data Fabrics (ZEN || SMCA).
// Filters out legacy systems.
if (!x86_match_cpu(amd_atl_cpuids))
return -ENODEV;
// Make sure the kernel recognizes this system's Data Fabric.
// Filters out new hardware.
if (!amd_nb_num())
return -ENODEV;
...
// Make sure the library supports this Data Fabric revision.
// Filters out totally new logic that requires library updates.
if (get_df_system_info())
return -ENODEV;
Thanks,
Yazen
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-13 17:04 ` Yazen Ghannam
@ 2023-12-13 17:07 ` Borislav Petkov
0 siblings, 0 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-12-13 17:07 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Wed, Dec 13, 2023 at 12:04:04PM -0500, Yazen Ghannam wrote:
> We could also introduce another software feature bit for Data Fabrics "DF".
> And this could be set when we discover them, like in the AMD_NB code.
And a DF system is a Zen system and a SMCA one. Currently, at least.
Ok, let's leave it with the two feature bits for now. We can always do
the DF bit later, if really needed.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
` (2 preceding siblings ...)
2023-12-12 13:29 ` Borislav Petkov
@ 2023-12-14 10:54 ` Borislav Petkov
2023-12-14 14:30 ` Borislav Petkov
4 siblings, 0 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-12-14 10:54 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Sun, Dec 10, 2023 at 01:49:30PM -0600, Yazen Ghannam wrote:
> diff --git a/drivers/ras/amd/atl/internal.h b/drivers/ras/amd/atl/internal.h
> new file mode 100644
> index 000000000000..08bc46f7cabf
> --- /dev/null
> +++ b/drivers/ras/amd/atl/internal.h
> @@ -0,0 +1,312 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * AMD Address Translation Library
> + *
> + * internal.h : Helper functions and common defines
> + *
> + * Copyright (c) 2023, Advanced Micro Devices, Inc.
> + * All Rights Reserved.
> + *
> + * Author: Yazen Ghannam <Yazen.Ghannam@amd.com>
> + */
> +
> +#ifndef __AMD_ATL_INTERNAL_H__
> +#define __AMD_ATL_INTERNAL_H__
> +
> +#include <linux/bitfield.h>
> +#include <linux/bitops.h>
> +
> +#include <asm/amd_nb.h>
> +
> +#include "reg_fields.h"
> +#include "stub.h"
> +
> +/* Maximum possible number of Coherent Stations within a single Data Fabric. */
> +#define MAX_CS_CHANNELS 32
Hmm, that's coherent stations and not chip select. Can we differentiate
between the two pls?
In my mind "cs" is chip select so can we call the other thing "coh_st"
or so?
...
> +/*
> + * Make a gap in 'data' that is 'num_bits' long starting at 'bit_num.
@data, @num_bits, @bit_num
This is the kernel-doc notation. You don't need make this function
kernel-doc yet but might as well get accustomed to the syntax. :)
> + * e.g. data = 11111111'b
> + * bit_num = 3
> + * num_bits = 2
> + * result = 1111100111'b
> + */
> +static inline u64 expand_bits(u8 bit_num, u8 num_bits, u64 data)
> +{
> + u64 temp1, temp2;
> +
> + /*
> + * Return the original data if the "space" needed is '0'.
> + * This helps avoid the need to check for '0' at each
> + * caller.
> + */
Yeah, you don't need that comment - it is kinda obvious that the
function should check for nonsensical inputs.
> + if (!num_bits)
> + return data;
> +
> + if (!bit_num)
> + return data << num_bits;
Like here: I was gonna say that num_bits cannot be more than
BITS_PER_LONG (approximating here on 64-bit) because it'll turn @data
into 0 but people get what they asked for.
Might do here at least a warn or so:
WARN_ON_ONCE(num_bits >= BITS_PER_LONG);
The same thing for the upper limit of bit_num.
> + temp1 = data & GENMASK_ULL(bit_num - 1, 0);
> +
> + temp2 = data & GENMASK_ULL(63, bit_num);
> + temp2 <<= num_bits;
> +
> + return temp1 | temp2;
> +}
> +
> +/*
> + * Remove bits in 'data' between low_bit and high_bit inclusive.
> + * e.g. data = XXXYYZZZ'b
> + * low_bit = 3
> + * high_bit = 4
> + * result = XXXZZZ'b
> + */
> +static inline u64 remove_bits(u8 low_bit, u8 high_bit, u64 data)
Ditto for this one.
...
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v3 1/3] RAS: Introduce AMD Address Translation Library
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
` (3 preceding siblings ...)
2023-12-14 10:54 ` Borislav Petkov
@ 2023-12-14 14:30 ` Borislav Petkov
4 siblings, 0 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-12-14 14:30 UTC (permalink / raw)
To: Yazen Ghannam
Cc: linux-edac, linux-kernel, tony.luck, x86, avadhut.naik,
john.allen, william.roche, muralidhara.mk
On Sun, Dec 10, 2023 at 01:49:30PM -0600, Yazen Ghannam wrote:
> +/*
> + * Some, but not all, cases have asserts.
> + * So use return values to indicate failure where needed.
> + */
No need for that comment.
> +static int get_intlv_mode(struct addr_ctx *ctx)
> +{
> + switch (df_cfg.rev) {
> + case DF2: return df2_get_intlv_mode(ctx);
> + case DF3: return df3_get_intlv_mode(ctx);
> + case DF3p5: return df3p5_get_intlv_mode(ctx);
> + case DF4: return df4_get_intlv_mode(ctx);
> + case DF4p5: return df4p5_get_intlv_mode(ctx);
> + default:
> + warn_on_bad_df_rev();
> + return -EINVAL;
> + }
You can warn once here instead of the callers:
int ret;
switch () {
... ret = ...get_intlv_mode();
...
default:
ret = -EINVAL;
}
if (ret)
warn_on_bad_df_rev();
return ret;
and save some text lines.
> +}
> +
> +static u64 get_hi_addr_offset(u32 reg_dram_offset)
> +{
> + u8 shift = DF_DRAM_BASE_LIMIT_LSB;
> + u64 hi_addr_offset = 0;
Move that assignment to 0...
> +
> + switch (df_cfg.rev) {
> + case DF2:
> + hi_addr_offset = FIELD_GET(DF2_HI_ADDR_OFFSET, reg_dram_offset);
> + break;
> + case DF3:
> + case DF3p5:
> + hi_addr_offset = FIELD_GET(DF3_HI_ADDR_OFFSET, reg_dram_offset);
> + break;
> + case DF4:
> + case DF4p5:
> + hi_addr_offset = FIELD_GET(DF4_HI_ADDR_OFFSET, reg_dram_offset);
> + break;
> + default:
... here.
<---
> + warn_on_bad_df_rev();
> + }
> +
> + return hi_addr_offset << shift;
> +}
> +
> +static int get_dram_offset(struct addr_ctx *ctx, bool *enabled, u64 *norm_offset)
> +{
You don't need *enabled. The retval can be:
< 0: fail
0: disabled
>0: enabled
and then you get rid of the IO param.
> + u32 reg_dram_offset;
> + u8 map_num;
> +
> + /* Should not be called for map 0. */
> + if (!ctx->map.num) {
> + warn_on_assert("Trying to find DRAM offset for map 0");
> + return -EINVAL;
> + }
> +
> + /*
> + * DramOffset registers don't exist for map 0, so the base register
> + * actually refers to map 1.
> + * Adjust the map_num for the register offsets.
> + */
> + map_num = ctx->map.num - 1;
> +
> + if (df_cfg.rev >= DF4) {
> + /* Read D18F7x140 (DramOffset) */
> + if (df_indirect_read_instance(ctx->node_id, 7, 0x140 + (4 * map_num),
> + ctx->inst_id, ®_dram_offset))
> + return -EINVAL;
> +
> + } else {
> + /* Read D18F0x1B4 (DramOffset) */
> + if (df_indirect_read_instance(ctx->node_id, 0, 0x1B4 + (4 * map_num),
> + ctx->inst_id, ®_dram_offset))
> + return -EINVAL;
> + }
> +
> + if (!FIELD_GET(DF_HI_ADDR_OFFSET_EN, reg_dram_offset))
> + return 0;
> +
> + *enabled = true;
> + *norm_offset = get_hi_addr_offset(reg_dram_offset);
> +
> + return 0;
> +}
...
> +static int get_cs_fabric_id(struct addr_ctx *ctx)
> +{
> + return lookup_cs_fabric_id(ctx);
> +}
Get rid of that silly helper.
> +
> +static bool valid_map(struct addr_ctx *ctx)
> +{
> + if (df_cfg.rev >= DF4)
> + return FIELD_GET(DF_ADDR_RANGE_VAL, ctx->map.ctl);
> +
> + return FIELD_GET(DF_ADDR_RANGE_VAL, ctx->map.base);
if (... )
return
else
return
Balanced.
> +int get_address_map(struct addr_ctx *ctx)
> +{
> + int ret = 0;
> +
> + ret = get_address_map_common(ctx);
> + if (ret)
> + return ret;
> +
> + if (get_global_map_data(ctx))
> + return -EINVAL;
Use ret here too.
> +
> + dump_address_map(&ctx->map);
> +
> + return ret;
> +}
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2023-12-14 14:30 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-12-10 19:49 [PATCH v3 0/3] AMD Address Translation Library Yazen Ghannam
2023-12-10 19:49 ` [PATCH v3 1/3] RAS: Introduce " Yazen Ghannam
2023-12-11 14:20 ` Borislav Petkov
2023-12-11 15:28 ` Yazen Ghannam
2023-12-11 19:57 ` Borislav Petkov
2023-12-12 14:23 ` Yazen Ghannam
2023-12-12 15:34 ` Borislav Petkov
2023-12-13 15:35 ` Yazen Ghannam
2023-12-13 16:48 ` Borislav Petkov
2023-12-13 17:04 ` Yazen Ghannam
2023-12-13 17:07 ` Borislav Petkov
2023-12-12 13:29 ` Borislav Petkov
2023-12-12 14:33 ` Yazen Ghannam
2023-12-12 16:07 ` Borislav Petkov
2023-12-14 10:54 ` Borislav Petkov
2023-12-14 14:30 ` Borislav Petkov
2023-12-10 19:49 ` [PATCH v3 2/3] EDAC/amd64: Use new " Yazen Ghannam
2023-12-10 19:49 ` [PATCH v3 3/3] Documentation: RAS: Add index and address translation section Yazen Ghannam
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).