[PATCH 00/25] AMD MCA Address Translation Updates

[PATCH 00/25] AMD MCA Address Translation Updates

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

This patchset refactors the AMD MCA Address Translation code and adds
support for newer systems. This patchset was written from scratch
compared to previous patchsets.

The reference code was recently refactored in preparation for updates
for future systems. These patches try to follow the reference code as
closely as possible. I also tried to address comments from previous
patchset reviews.

Patches 1-24 do the refactor without adding new system support. The goal
is to break down the translation algorithm into smaller chunks. There
are some simple wrapper functions defined. These will be filled in when
supporting newer systems. The intention is that new system support can
be added without any major refactor. I tried to make a patch for each
logical change. There's a bit of churn so as to not break the build with
each change. I think many of these patches can be squashed together, if
desired. The top level function was split first, then the next level of
functions, etc. in a somewhat breadth-first approach. 

Patch 25 adds support for systems with Data Fabric version 3 (Rome and
later).

Each patch was build tested individually. The entire set was
functionally tested with the following modes.

Naples:
  No interleaving
  Channel interleaving
  Die interleaving
  Socket interleaving

Rome:
  No interleaving
  Nodes-per-Socket 0 (NPS0)
  Nodes-per-Socket 1 (NPS1)
  Nodes-per-Socket 2 (NPS2)
  Nodes-per-Socket 4 (NPS4)
  NPS2 w/o hashing
  NPS4 w/o hashing

Thanks,
Yazen

Link:
https://lkml.kernel.org/r/20200903200144.310991-1-Yazen.Ghannam@amd.com

Yazen Ghannam (25):
  x86/MCE/AMD: Don't use naked values for DF registers
  x86/MCE/AMD: Add context struct
  x86/MCE/AMD: Define functions for DramOffset
  x86/MCE/AMD: Define function to read DRAM address map registers
  x86/MCE/AMD: Define function to find interleaving mode
  x86/MCE/AMD: Define function to denormalize address
  x86/MCE/AMD: Define function to add DRAM base and hole
  x86/MCE/AMD: Define function to dehash address
  x86/MCE/AMD: Define function to check DRAM limit address
  x86/MCE/AMD: Remove goto statements
  x86/MCE/AMD: Simplify function parameters
  x86/MCE/AMD: Define function to get Interleave Address Bit
  x86/MCE/AMD: Skip denormalization if no interleaving
  x86/MCE/AMD: Define function to get number of interleaved channels
  x86/MCE/AMD: Define function to get number of interleaved dies
  x86/MCE/AMD: Define function to get number of interleaved sockets
  x86/MCE/AMD: Remove unnecessary assert
  x86/MCE/AMD: Define function to make space for CS ID
  x86/MCE/AMD: Define function to calculate CS ID
  x86/MCE/AMD: Define function to insert CS ID into address
  x86/MCE/AMD: Define function to get CS Fabric ID
  x86/MCE/AMD: Define function to find shift and mask values
  x86/MCE/AMD: Update CS ID calculation to match reference code
  x86/MCE/AMD: Match hash function to reference code
  x86/MCE/AMD: Add support for address translation on DF3 systems

 arch/x86/include/asm/amd_nb.h |   7 +-
 arch/x86/include/asm/mce.h    |   5 +-
 arch/x86/kernel/amd_nb.c      |   6 +-
 arch/x86/kernel/cpu/mce/amd.c | 681 +++++++++++++++++++++++++++-------
 drivers/edac/amd64_edac.c     |   4 +-
 5 files changed, 555 insertions(+), 148 deletions(-)

-- 
2.25.1

[PATCH 01/25] x86/MCE/AMD: Don't use naked values for DF registers

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

AMD Data Fabric registers are defined using a combination of PCI
function number and offset. Define a struct to hold these values, and
update the DF Indirect Access function to accept a struct of this type.

Update the address translation code to include a list of the needed DF
registers using this new format. Define an enumeration to give the
registers more human-readable names.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/include/asm/amd_nb.h |  7 ++++-
 arch/x86/kernel/amd_nb.c      |  6 ++---
 arch/x86/kernel/cpu/mce/amd.c | 51 ++++++++++++++++++++++++++++-------
 3 files changed, 50 insertions(+), 14 deletions(-)

diff --git a/arch/x86/include/asm/amd_nb.h b/arch/x86/include/asm/amd_nb.h
index 455066a06f60..a5644c119317 100644
--- a/arch/x86/include/asm/amd_nb.h
+++ b/arch/x86/include/asm/amd_nb.h
@@ -22,9 +22,14 @@ extern int amd_numa_init(void);
 extern int amd_get_subcaches(int);
 extern int amd_set_subcaches(int, unsigned long);
 
+struct df_reg {
+	u8  func;
+	u16 offset;
+};
+
 extern int amd_smn_read(u16 node, u32 address, u32 *value);
 extern int amd_smn_write(u16 node, u32 address, u32 value);
-extern int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo);
+extern int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32 *lo);
 
 struct amd_l3_cache {
 	unsigned indices;
diff --git a/arch/x86/kernel/amd_nb.c b/arch/x86/kernel/amd_nb.c
index 09083094eb57..f06924093ae4 100644
--- a/arch/x86/kernel/amd_nb.c
+++ b/arch/x86/kernel/amd_nb.c
@@ -184,7 +184,7 @@ EXPORT_SYMBOL_GPL(amd_smn_write);
  * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
  * and FICAD HI registers but so far we only need the LO register.
  */
-int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
+int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32 *lo)
 {
 	struct pci_dev *F4;
 	u32 ficaa;
@@ -198,8 +198,8 @@ int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
 		goto out;
 
 	ficaa  = 1;
-	ficaa |= reg & 0x3FC;
-	ficaa |= (func & 0x7) << 11;
+	ficaa |= reg.offset & 0x3FC;
+	ficaa |= (reg.func & 0x7) << 11;
 	ficaa |= instance_id << 16;
 
 	mutex_lock(&smn_mutex);
diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index e486f96b3cb3..1b459c143886 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -675,6 +675,37 @@ void mce_amd_feature_init(struct cpuinfo_x86 *c)
 		deferred_error_interrupt_enable(c);
 }
 
+enum df_reg_names {
+	/* Function 0 */
+	FAB_BLK_INST_INFO_3,
+	DRAM_HOLE_CTL,
+	DRAM_BASE_ADDR_0,
+	DRAM_LIMIT_ADDR_0,
+	DRAM_BASE_ADDR_1,
+	DRAM_LIMIT_ADDR_1,
+	DRAM_OFFSET,
+
+	/* Function 1 */
+	SYS_FAB_ID_MASK,
+};
+
+static struct df_reg df_regs[] = {
+	/* D18F0x50 (FabricBlockInstanceInformation3_CS) */
+	[FAB_BLK_INST_INFO_3]	=	{0, 0x50},
+	/* D18F0x104 (DramHoleControl) */
+	[DRAM_HOLE_CTL]		=	{0, 0x104},
+	/* D18F0x110 (DramBaseAddress) */
+	[DRAM_BASE_ADDR_0]	=	{0, 0x110},
+	/* D18F0x114 (DramLimitAddress) */
+	[DRAM_LIMIT_ADDR_0]	=	{0, 0x114},
+	[DRAM_BASE_ADDR_1]	=	{0, 0x118},
+	[DRAM_LIMIT_ADDR_1]	=	{0, 0x11C},
+	/* D18F0x1B4 (DramOffset) */
+	[DRAM_OFFSET]		=	{0, 0x1B4},
+	/* D18F1x208 (SystemFabricIdMask) */
+	[SYS_FAB_ID_MASK]	=	{1, 0x208},
+};
+
 int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
@@ -691,8 +722,9 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	u8 cs_mask, cs_id = 0;
 	bool hash_enabled = false;
 
-	/* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
-	if (amd_df_indirect_read(nid, 0, 0x1B4, umc, &tmp))
+	struct df_reg reg;
+
+	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
 		goto out_err;
 
 	/* Remove HiAddrOffset from normalized address, if enabled: */
@@ -705,8 +737,8 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		}
 	}
 
-	/* Read D18F0x110 (DramBaseAddress). */
-	if (amd_df_indirect_read(nid, 0, 0x110 + (8 * base), umc, &tmp))
+	reg = base ? df_regs[DRAM_BASE_ADDR_1] : df_regs[DRAM_BASE_ADDR_0];
+	if (amd_df_indirect_read(nid, reg, umc, &tmp))
 		goto out_err;
 
 	/* Check if address range is valid. */
@@ -728,8 +760,8 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		goto out_err;
 	}
 
-	/* Read D18F0x114 (DramLimitAddress). */
-	if (amd_df_indirect_read(nid, 0, 0x114 + (8 * base), umc, &tmp))
+	reg = base ? df_regs[DRAM_LIMIT_ADDR_1] : df_regs[DRAM_LIMIT_ADDR_0];
+	if (amd_df_indirect_read(nid, reg, umc, &tmp))
 		goto out_err;
 
 	intlv_num_sockets = (tmp >> 8) & 0x1;
@@ -785,7 +817,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		 * umc/channel# as instance id of the coherent slave
 		 * for FICAA.
 		 */
-		if (amd_df_indirect_read(nid, 0, 0x50, umc, &tmp))
+		if (amd_df_indirect_read(nid, df_regs[FAB_BLK_INST_INFO_3], umc, &tmp))
 			goto out_err;
 
 		cs_fabric_id = (tmp >> 8) & 0xFF;
@@ -800,9 +832,8 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 
 		sock_id_bit = die_id_bit;
 
-		/* Read D18F1x208 (SystemFabricIdMask). */
 		if (intlv_num_dies || intlv_num_sockets)
-			if (amd_df_indirect_read(nid, 1, 0x208, umc, &tmp))
+			if (amd_df_indirect_read(nid, df_regs[SYS_FAB_ID_MASK], umc, &tmp))
 				goto out_err;
 
 		/* If interleaved over more than 1 die. */
@@ -841,7 +872,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 
 	/* If legacy MMIO hole enabled */
 	if (lgcy_mmio_hole_en) {
-		if (amd_df_indirect_read(nid, 0, 0x104, umc, &tmp))
+		if (amd_df_indirect_read(nid, df_regs[DRAM_HOLE_CTL], umc, &tmp))
 			goto out_err;
 
 		dram_hole_base = tmp & GENMASK(31, 24);
-- 
2.25.1

[PATCH 02/25] x86/MCE/AMD: Add context struct

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Define an address translation context struct. This will hold values that
will be passed between multiple functions.

Save DF type, return address, Node ID, and UMC number to start.

Add stub function to get DF type. Return DF2 type for now. This will be
updated with new DF types.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 61 ++++++++++++++++++++++++-----------
 1 file changed, 43 insertions(+), 18 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 1b459c143886..86df58b1051a 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -706,12 +706,25 @@ static struct df_reg df_regs[] = {
 	[SYS_FAB_ID_MASK]	=	{1, 0x208},
 };
 
+enum df_types {
+	DF2,
+};
+
+struct addr_ctx {
+	enum df_types df_type;
+	u64 ret_addr;
+	u16 nid;
+	u8 umc;
+};
+
+static enum df_types get_df_type(struct addr_ctx *ctx)
+{
+	return DF2;
+}
+
 int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
-	/* We start from the normalized address */
-	u64 ret_addr = norm_addr;
-
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
@@ -724,6 +737,18 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 
 	struct df_reg reg;
 
+	struct addr_ctx ctx;
+
+	memset(&ctx, 0, sizeof(ctx));
+
+	/* We start from the normalized address */
+	ctx.ret_addr = norm_addr;
+
+	ctx.nid = nid;
+	ctx.umc = umc;
+
+	ctx.df_type = get_df_type(&ctx);
+
 	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
 		goto out_err;
 
@@ -732,7 +757,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		u64 hi_addr_offset = (tmp & GENMASK_ULL(31, 20)) << 8;
 
 		if (norm_addr >= hi_addr_offset) {
-			ret_addr -= hi_addr_offset;
+			ctx.ret_addr -= hi_addr_offset;
 			base = 1;
 		}
 	}
@@ -861,14 +886,14 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
 		 * there are (where "I" starts).
 		 */
-		temp_addr_y = ret_addr & GENMASK_ULL(intlv_addr_bit-1, 0);
+		temp_addr_y = ctx.ret_addr & GENMASK_ULL(intlv_addr_bit - 1, 0);
 		temp_addr_i = (cs_id << intlv_addr_bit);
-		temp_addr_x = (ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
-		ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
+		temp_addr_x = (ctx.ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
+		ctx.ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
 	}
 
 	/* Add dram base address */
-	ret_addr += dram_base_addr;
+	ctx.ret_addr += dram_base_addr;
 
 	/* If legacy MMIO hole enabled */
 	if (lgcy_mmio_hole_en) {
@@ -876,29 +901,29 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 			goto out_err;
 
 		dram_hole_base = tmp & GENMASK(31, 24);
-		if (ret_addr >= dram_hole_base)
-			ret_addr += (BIT_ULL(32) - dram_hole_base);
+		if (ctx.ret_addr >= dram_hole_base)
+			ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
 	}
 
 	if (hash_enabled) {
 		/* Save some parentheses and grab ls-bit at the end. */
-		hashed_bit =	(ret_addr >> 12) ^
-				(ret_addr >> 18) ^
-				(ret_addr >> 21) ^
-				(ret_addr >> 30) ^
+		hashed_bit =	(ctx.ret_addr >> 12) ^
+				(ctx.ret_addr >> 18) ^
+				(ctx.ret_addr >> 21) ^
+				(ctx.ret_addr >> 30) ^
 				cs_id;
 
 		hashed_bit &= BIT(0);
 
-		if (hashed_bit != ((ret_addr >> intlv_addr_bit) & BIT(0)))
-			ret_addr ^= BIT(intlv_addr_bit);
+		if (hashed_bit != ((ctx.ret_addr >> intlv_addr_bit) & BIT(0)))
+			ctx.ret_addr ^= BIT(intlv_addr_bit);
 	}
 
 	/* Is calculated system address is above DRAM limit address? */
-	if (ret_addr > dram_limit_addr)
+	if (ctx.ret_addr > dram_limit_addr)
 		goto out_err;
 
-	*sys_addr = ret_addr;
+	*sys_addr = ctx.ret_addr;
 	return 0;
 
 out_err:
-- 
2.25.1

[PATCH 03/25] x86/MCE/AMD: Define functions for DramOffset

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Add helper functions to read the DramOffset register and to remove the
offset from the calculated address.

The helper functions will be expanded in future DF versions.

Rename the "base" variable to "map_num" to indicate that this is the
address map number. An address map is defined with a base and limit
value. The map_num variable is used to select the proper base and limit
registers to use for the address translation.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 54 ++++++++++++++++++++++++++---------
 1 file changed, 40 insertions(+), 14 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 86df58b1051a..20a6f34143f6 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -710,11 +710,14 @@ enum df_types {
 	DF2,
 };
 
+/* Use "reg_" prefix for raw register values. */
 struct addr_ctx {
 	enum df_types df_type;
 	u64 ret_addr;
+	u32 reg_dram_offset;
 	u16 nid;
 	u8 umc;
+	u8 map_num;
 };
 
 static enum df_types get_df_type(struct addr_ctx *ctx)
@@ -722,6 +725,39 @@ static enum df_types get_df_type(struct addr_ctx *ctx)
 	return DF2;
 }
 
+static int get_dram_offset_reg(struct addr_ctx *ctx)
+{
+	if (amd_df_indirect_read(ctx->nid, df_regs[DRAM_OFFSET], ctx->umc, &ctx->reg_dram_offset))
+		return -EINVAL;
+
+	return 0;
+}
+
+static u64 get_hi_addr_offset(struct addr_ctx *ctx)
+{
+	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
+}
+
+static int remove_dram_offset(struct addr_ctx *ctx)
+{
+	if (get_dram_offset_reg(ctx))
+		return -EINVAL;
+
+	ctx->map_num = 0;
+
+	/* Remove HiAddrOffset from normalized address, if enabled: */
+	if (ctx->reg_dram_offset & BIT(0)) {
+		u64 hi_addr_offset = get_hi_addr_offset(ctx);
+
+		if (ctx->ret_addr >= hi_addr_offset) {
+			ctx->ret_addr -= hi_addr_offset;
+			ctx->map_num = 1;
+		}
+	}
+
+	return 0;
+}
+
 int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
@@ -731,7 +767,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
 	u8 intlv_addr_sel, intlv_addr_bit;
 	u8 num_intlv_bits, hashed_bit;
-	u8 lgcy_mmio_hole_en, base = 0;
+	u8 lgcy_mmio_hole_en;
 	u8 cs_mask, cs_id = 0;
 	bool hash_enabled = false;
 
@@ -749,20 +785,10 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 
 	ctx.df_type = get_df_type(&ctx);
 
-	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
+	if (remove_dram_offset(&ctx))
 		goto out_err;
 
-	/* Remove HiAddrOffset from normalized address, if enabled: */
-	if (tmp & BIT(0)) {
-		u64 hi_addr_offset = (tmp & GENMASK_ULL(31, 20)) << 8;
-
-		if (norm_addr >= hi_addr_offset) {
-			ctx.ret_addr -= hi_addr_offset;
-			base = 1;
-		}
-	}
-
-	reg = base ? df_regs[DRAM_BASE_ADDR_1] : df_regs[DRAM_BASE_ADDR_0];
+	reg = ctx.map_num ? df_regs[DRAM_BASE_ADDR_1] : df_regs[DRAM_BASE_ADDR_0];
 	if (amd_df_indirect_read(nid, reg, umc, &tmp))
 		goto out_err;
 
@@ -785,7 +811,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		goto out_err;
 	}
 
-	reg = base ? df_regs[DRAM_LIMIT_ADDR_1] : df_regs[DRAM_LIMIT_ADDR_0];
+	reg = ctx.map_num ? df_regs[DRAM_LIMIT_ADDR_1] : df_regs[DRAM_LIMIT_ADDR_0];
 	if (amd_df_indirect_read(nid, reg, umc, &tmp))
 		goto out_err;
 
-- 
2.25.1

[PATCH 04/25] x86/MCE/AMD: Define function to read DRAM address map registers

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move the reading of the base and limit registers into a helper function.
Save the raw values in the context struct as they will be parsed later.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 53 ++++++++++++++++++++---------------
 1 file changed, 31 insertions(+), 22 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 20a6f34143f6..0957f11a4c84 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -715,6 +715,8 @@ struct addr_ctx {
 	enum df_types df_type;
 	u64 ret_addr;
 	u32 reg_dram_offset;
+	u32 reg_base_addr;
+	u32 reg_limit_addr;
 	u16 nid;
 	u8 umc;
 	u8 map_num;
@@ -758,6 +760,27 @@ static int remove_dram_offset(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int get_dram_addr_map(struct addr_ctx *ctx)
+{
+	struct df_reg reg = ctx->map_num ? df_regs[DRAM_BASE_ADDR_1] : df_regs[DRAM_BASE_ADDR_0];
+
+	if (amd_df_indirect_read(ctx->nid, reg, ctx->umc, &ctx->reg_base_addr))
+		return -EINVAL;
+
+	reg = ctx->map_num ? df_regs[DRAM_LIMIT_ADDR_1] : df_regs[DRAM_LIMIT_ADDR_0];
+	if (amd_df_indirect_read(ctx->nid, reg, ctx->umc, &ctx->reg_limit_addr))
+		return -EINVAL;
+
+	/* Check if address range is valid. */
+	if (!(ctx->reg_base_addr & BIT(0))) {
+		pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
+		       __func__, ctx->reg_base_addr);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
@@ -771,8 +794,6 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	u8 cs_mask, cs_id = 0;
 	bool hash_enabled = false;
 
-	struct df_reg reg;
-
 	struct addr_ctx ctx;
 
 	memset(&ctx, 0, sizeof(ctx));
@@ -788,21 +809,13 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	if (remove_dram_offset(&ctx))
 		goto out_err;
 
-	reg = ctx.map_num ? df_regs[DRAM_BASE_ADDR_1] : df_regs[DRAM_BASE_ADDR_0];
-	if (amd_df_indirect_read(nid, reg, umc, &tmp))
+	if (get_dram_addr_map(&ctx))
 		goto out_err;
 
-	/* Check if address range is valid. */
-	if (!(tmp & BIT(0))) {
-		pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
-			__func__, tmp);
-		goto out_err;
-	}
-
-	lgcy_mmio_hole_en = tmp & BIT(1);
-	intlv_num_chan	  = (tmp >> 4) & 0xF;
-	intlv_addr_sel	  = (tmp >> 8) & 0x7;
-	dram_base_addr	  = (tmp & GENMASK_ULL(31, 12)) << 16;
+	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
+	intlv_num_chan	  = (ctx.reg_base_addr >> 4) & 0xF;
+	intlv_addr_sel	  = (ctx.reg_base_addr >> 8) & 0x7;
+	dram_base_addr	  = (ctx.reg_base_addr & GENMASK_ULL(31, 12)) << 16;
 
 	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
 	if (intlv_addr_sel > 3) {
@@ -811,13 +824,9 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		goto out_err;
 	}
 
-	reg = ctx.map_num ? df_regs[DRAM_LIMIT_ADDR_1] : df_regs[DRAM_LIMIT_ADDR_0];
-	if (amd_df_indirect_read(nid, reg, umc, &tmp))
-		goto out_err;
-
-	intlv_num_sockets = (tmp >> 8) & 0x1;
-	intlv_num_dies	  = (tmp >> 10) & 0x3;
-	dram_limit_addr	  = ((tmp & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
+	intlv_num_sockets = (ctx.reg_limit_addr >> 8) & 0x1;
+	intlv_num_dies	  = (ctx.reg_limit_addr >> 10) & 0x3;
+	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
 
 	intlv_addr_bit = intlv_addr_sel + 8;
 
-- 
2.25.1

[PATCH 05/25] x86/MCE/AMD: Define function to find interleaving mode

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Define a helper function to find the interleaving mode. Define a
DF2-specific function now. Future DF versions will have their own
functions.

Use an enumeration for the interleaving modes to give a human-readable
value. Save the interleaving mode in the context struct, since this will
be used in multiple functions.

Multiple interleaving modes support hashing, so save a boolean in the
context struct to check if hashing is enabled.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 38 ++++++++++++++++++++++++++++++++---
 1 file changed, 35 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 0957f11a4c84..f1a467cb74e6 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -710,9 +710,17 @@ enum df_types {
 	DF2,
 };
 
+/* These are mapped 1:1 to the hardware values. Special cases are set at > 0x20. */
+enum intlv_modes {
+	NONE		= 0x00,
+	NOHASH_2CH	= 0x01,
+	DF2_HASH_2CH	= 0x21,
+};
+
 /* Use "reg_" prefix for raw register values. */
 struct addr_ctx {
 	enum df_types df_type;
+	enum intlv_modes intlv_mode;
 	u64 ret_addr;
 	u32 reg_dram_offset;
 	u32 reg_base_addr;
@@ -720,6 +728,7 @@ struct addr_ctx {
 	u16 nid;
 	u8 umc;
 	u8 map_num;
+	bool hash_enabled;
 };
 
 static enum df_types get_df_type(struct addr_ctx *ctx)
@@ -727,6 +736,28 @@ static enum df_types get_df_type(struct addr_ctx *ctx)
 	return DF2;
 }
 
+static int get_intlv_mode_df2(struct addr_ctx *ctx)
+{
+	ctx->intlv_mode = (ctx->reg_base_addr >> 4) & 0xF;
+
+	if (ctx->intlv_mode == 8) {
+		ctx->intlv_mode = DF2_HASH_2CH;
+		ctx->hash_enabled = true;
+	}
+
+	if (ctx->intlv_mode != NONE &&
+	    ctx->intlv_mode != NOHASH_2CH &&
+	    ctx->intlv_mode != DF2_HASH_2CH)
+		return -EINVAL;
+
+	return 0;
+}
+
+static int get_intlv_mode(struct addr_ctx *ctx)
+{
+	return get_intlv_mode_df2(ctx);
+}
+
 static int get_dram_offset_reg(struct addr_ctx *ctx)
 {
 	if (amd_df_indirect_read(ctx->nid, df_regs[DRAM_OFFSET], ctx->umc, &ctx->reg_dram_offset))
@@ -792,7 +823,6 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	u8 num_intlv_bits, hashed_bit;
 	u8 lgcy_mmio_hole_en;
 	u8 cs_mask, cs_id = 0;
-	bool hash_enabled = false;
 
 	struct addr_ctx ctx;
 
@@ -812,6 +842,9 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	if (get_dram_addr_map(&ctx))
 		goto out_err;
 
+	if (get_intlv_mode(&ctx))
+		goto out_err;
+
 	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
 	intlv_num_chan	  = (ctx.reg_base_addr >> 4) & 0xF;
 	intlv_addr_sel	  = (ctx.reg_base_addr >> 8) & 0x7;
@@ -839,7 +872,6 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	case 7:	intlv_num_chan = 4; break;
 
 	case 8: intlv_num_chan = 1;
-		hash_enabled = true;
 		break;
 	default:
 		pr_err("%s: Invalid number of interleaved channels %d.\n",
@@ -940,7 +972,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 			ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
 	}
 
-	if (hash_enabled) {
+	if (ctx.hash_enabled) {
 		/* Save some parentheses and grab ls-bit at the end. */
 		hashed_bit =	(ctx.ret_addr >> 12) ^
 				(ctx.ret_addr >> 18) ^
-- 
2.25.1

[PATCH 06/25] x86/MCE/AMD: Define function to denormalize address

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move the address denormalization into a separate helper function. This
will be further refactored in later patches.

Add the interleave address bit and the CS ID to the context struct.
These values will be used by multiple functions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 125 +++++++++++++++++++---------------
 1 file changed, 71 insertions(+), 54 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index f1a467cb74e6..263d419d4175 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -728,6 +728,8 @@ struct addr_ctx {
 	u16 nid;
 	u8 umc;
 	u8 map_num;
+	u8 intlv_addr_bit;
+	u8 cs_id;
 	bool hash_enabled;
 };
 
@@ -812,56 +814,26 @@ static int get_dram_addr_map(struct addr_ctx *ctx)
 	return 0;
 }
 
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+static int denormalize_addr(struct addr_ctx *ctx)
 {
-	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
 	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
-	u8 intlv_addr_sel, intlv_addr_bit;
-	u8 num_intlv_bits, hashed_bit;
-	u8 lgcy_mmio_hole_en;
-	u8 cs_mask, cs_id = 0;
-
-	struct addr_ctx ctx;
-
-	memset(&ctx, 0, sizeof(ctx));
-
-	/* We start from the normalized address */
-	ctx.ret_addr = norm_addr;
-
-	ctx.nid = nid;
-	ctx.umc = umc;
-
-	ctx.df_type = get_df_type(&ctx);
-
-	if (remove_dram_offset(&ctx))
-		goto out_err;
-
-	if (get_dram_addr_map(&ctx))
-		goto out_err;
-
-	if (get_intlv_mode(&ctx))
-		goto out_err;
-
-	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
-	intlv_num_chan	  = (ctx.reg_base_addr >> 4) & 0xF;
-	intlv_addr_sel	  = (ctx.reg_base_addr >> 8) & 0x7;
-	dram_base_addr	  = (ctx.reg_base_addr & GENMASK_ULL(31, 12)) << 16;
+	u8 intlv_addr_sel = (ctx->reg_base_addr >> 8) & 0x7;
+	u8 num_intlv_bits, cs_mask = 0;
 
 	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
 	if (intlv_addr_sel > 3) {
 		pr_err("%s: Invalid interleave address select %d.\n",
 			__func__, intlv_addr_sel);
-		goto out_err;
+		return -EINVAL;
 	}
 
-	intlv_num_sockets = (ctx.reg_limit_addr >> 8) & 0x1;
-	intlv_num_dies	  = (ctx.reg_limit_addr >> 10) & 0x3;
-	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
+	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
+	intlv_num_dies	  = (ctx->reg_limit_addr >> 10) & 0x3;
 
-	intlv_addr_bit = intlv_addr_sel + 8;
+	ctx->intlv_addr_bit = intlv_addr_sel + 8;
 
 	/* Re-use intlv_num_chan by setting it equal to log2(#channels) */
 	switch (intlv_num_chan) {
@@ -876,7 +848,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	default:
 		pr_err("%s: Invalid number of interleaved channels %d.\n",
 			__func__, intlv_num_chan);
-		goto out_err;
+		return -EINVAL;
 	}
 
 	num_intlv_bits = intlv_num_chan;
@@ -884,7 +856,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	if (intlv_num_dies > 2) {
 		pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
 			__func__, intlv_num_dies);
-		goto out_err;
+		return -EINVAL;
 	}
 
 	num_intlv_bits += intlv_num_dies;
@@ -896,7 +868,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	if (num_intlv_bits > 4) {
 		pr_err("%s: Invalid interleave bits %d.\n",
 			__func__, num_intlv_bits);
-		goto out_err;
+		return -EINVAL;
 	}
 
 	if (num_intlv_bits > 0) {
@@ -909,8 +881,8 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		 * umc/channel# as instance id of the coherent slave
 		 * for FICAA.
 		 */
-		if (amd_df_indirect_read(nid, df_regs[FAB_BLK_INST_INFO_3], umc, &tmp))
-			goto out_err;
+		if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3], ctx->umc, &tmp))
+			return -EINVAL;
 
 		cs_fabric_id = (tmp >> 8) & 0xFF;
 		die_id_bit   = 0;
@@ -919,14 +891,15 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		if (intlv_num_chan) {
 			die_id_bit = intlv_num_chan;
 			cs_mask	   = (1 << die_id_bit) - 1;
-			cs_id	   = cs_fabric_id & cs_mask;
+			ctx->cs_id = cs_fabric_id & cs_mask;
 		}
 
 		sock_id_bit = die_id_bit;
 
 		if (intlv_num_dies || intlv_num_sockets)
-			if (amd_df_indirect_read(nid, df_regs[SYS_FAB_ID_MASK], umc, &tmp))
-				goto out_err;
+			if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
+						 ctx->umc, &tmp))
+				return -EINVAL;
 
 		/* If interleaved over more than 1 die. */
 		if (intlv_num_dies) {
@@ -934,7 +907,8 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 			die_id_shift = (tmp >> 24) & 0xF;
 			die_id_mask  = (tmp >> 8) & 0xFF;
 
-			cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+			ctx->cs_id |= ((cs_fabric_id & die_id_mask)
+					>> die_id_shift) << die_id_bit;
 		}
 
 		/* If interleaved over more than 1 socket. */
@@ -942,7 +916,8 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 			socket_id_shift	= (tmp >> 28) & 0xF;
 			socket_id_mask	= (tmp >> 16) & 0xFF;
 
-			cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
+			ctx->cs_id |= ((cs_fabric_id & socket_id_mask)
+					>> socket_id_shift) << sock_id_bit;
 		}
 
 		/*
@@ -953,12 +928,54 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
 		 * there are (where "I" starts).
 		 */
-		temp_addr_y = ctx.ret_addr & GENMASK_ULL(intlv_addr_bit - 1, 0);
-		temp_addr_i = (cs_id << intlv_addr_bit);
-		temp_addr_x = (ctx.ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
-		ctx.ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
+		temp_addr_y = ctx->ret_addr & GENMASK_ULL(ctx->intlv_addr_bit - 1, 0);
+		temp_addr_i = (ctx->cs_id << ctx->intlv_addr_bit);
+		temp_addr_x = (ctx->ret_addr & GENMASK_ULL(63, ctx->intlv_addr_bit))
+			       << num_intlv_bits;
+		ctx->ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
 	}
 
+	return 0;
+}
+
+int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+{
+	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
+
+	u32 tmp;
+
+	u8 hashed_bit;
+	u8 lgcy_mmio_hole_en;
+
+	struct addr_ctx ctx;
+
+	memset(&ctx, 0, sizeof(ctx));
+
+	/* We start from the normalized address */
+	ctx.ret_addr = norm_addr;
+
+	ctx.nid = nid;
+	ctx.umc = umc;
+
+	ctx.df_type = get_df_type(&ctx);
+
+	if (remove_dram_offset(&ctx))
+		return -EINVAL;
+
+	if (get_dram_addr_map(&ctx))
+		goto out_err;
+
+	if (get_intlv_mode(&ctx))
+		goto out_err;
+
+	if (denormalize_addr(&ctx))
+		goto out_err;
+
+	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
+	dram_base_addr	  = (ctx.reg_base_addr & GENMASK_ULL(31, 12)) << 16;
+
+	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
+
 	/* Add dram base address */
 	ctx.ret_addr += dram_base_addr;
 
@@ -978,12 +995,12 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 				(ctx.ret_addr >> 18) ^
 				(ctx.ret_addr >> 21) ^
 				(ctx.ret_addr >> 30) ^
-				cs_id;
+				ctx.cs_id;
 
 		hashed_bit &= BIT(0);
 
-		if (hashed_bit != ((ctx.ret_addr >> intlv_addr_bit) & BIT(0)))
-			ctx.ret_addr ^= BIT(intlv_addr_bit);
+		if (hashed_bit != ((ctx.ret_addr >> ctx.intlv_addr_bit) & BIT(0)))
+			ctx.ret_addr ^= BIT(ctx.intlv_addr_bit);
 	}
 
 	/* Is calculated system address is above DRAM limit address? */
-- 
2.25.1

[PATCH 07/25] x86/MCE/AMD: Define function to add DRAM base and hole

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move adding of DRAM base and hole into a separate helper function.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 45 ++++++++++++++++++++---------------
 1 file changed, 26 insertions(+), 19 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 263d419d4175..d1a6405bb990 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -938,14 +938,34 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	return 0;
 }
 
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+static int add_base_and_hole(struct addr_ctx *ctx)
 {
-	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
+	u64 dram_base_addr = (ctx->reg_base_addr & GENMASK_ULL(31, 12)) << 16;
 
-	u32 tmp;
+	/* Add dram base address */
+	ctx->ret_addr += dram_base_addr;
+
+	/* If legacy MMIO hole enabled */
+	if (ctx->reg_base_addr & BIT(1)) {
+		u32 dram_hole_base;
+
+		if (amd_df_indirect_read(ctx->nid, df_regs[DRAM_HOLE_CTL],
+					 ctx->umc, &dram_hole_base))
+			return -EINVAL;
+
+		dram_hole_base &= GENMASK(31, 24);
+		if (ctx->ret_addr >= dram_hole_base)
+			ctx->ret_addr += (BIT_ULL(32) - dram_hole_base);
+	}
+
+	return 0;
+}
+
+int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+{
+	u64 dram_limit_addr;
 
 	u8 hashed_bit;
-	u8 lgcy_mmio_hole_en;
 
 	struct addr_ctx ctx;
 
@@ -971,23 +991,10 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	if (denormalize_addr(&ctx))
 		goto out_err;
 
-	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
-	dram_base_addr	  = (ctx.reg_base_addr & GENMASK_ULL(31, 12)) << 16;
-
 	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
 
-	/* Add dram base address */
-	ctx.ret_addr += dram_base_addr;
-
-	/* If legacy MMIO hole enabled */
-	if (lgcy_mmio_hole_en) {
-		if (amd_df_indirect_read(nid, df_regs[DRAM_HOLE_CTL], umc, &tmp))
-			goto out_err;
-
-		dram_hole_base = tmp & GENMASK(31, 24);
-		if (ctx.ret_addr >= dram_hole_base)
-			ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
-	}
+	if (add_base_and_hole(&ctx))
+		goto out_err;
 
 	if (ctx.hash_enabled) {
 		/* Save some parentheses and grab ls-bit at the end. */
-- 
2.25.1

[PATCH 08/25] x86/MCE/AMD: Define function to dehash address

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move the dehashing code into a separate helper function. Define a
DF2-specific function for the current code. Specific helper functions
will be added for future DF versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 41 ++++++++++++++++++++++-------------
 1 file changed, 26 insertions(+), 15 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index d1a6405bb990..e341737e03d4 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -961,12 +961,34 @@ static int add_base_and_hole(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int dehash_addr_df2(struct addr_ctx *ctx)
+{
+	u8 hashed_bit =	(ctx->ret_addr >> 12) ^
+			(ctx->ret_addr >> 18) ^
+			(ctx->ret_addr >> 21) ^
+			(ctx->ret_addr >> 30) ^
+			ctx->cs_id;
+
+	hashed_bit &= BIT(0);
+
+	if (hashed_bit != ((ctx->ret_addr >> ctx->intlv_addr_bit) & BIT(0)))
+		ctx->ret_addr ^= BIT(ctx->intlv_addr_bit);
+
+	return 0;
+}
+
+static int dehash_addr(struct addr_ctx *ctx)
+{
+	if (!ctx->hash_enabled)
+		return 0;
+
+	return dehash_addr_df2(ctx);
+}
+
 int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_limit_addr;
 
-	u8 hashed_bit;
-
 	struct addr_ctx ctx;
 
 	memset(&ctx, 0, sizeof(ctx));
@@ -996,19 +1018,8 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	if (add_base_and_hole(&ctx))
 		goto out_err;
 
-	if (ctx.hash_enabled) {
-		/* Save some parentheses and grab ls-bit at the end. */
-		hashed_bit =	(ctx.ret_addr >> 12) ^
-				(ctx.ret_addr >> 18) ^
-				(ctx.ret_addr >> 21) ^
-				(ctx.ret_addr >> 30) ^
-				ctx.cs_id;
-
-		hashed_bit &= BIT(0);
-
-		if (hashed_bit != ((ctx.ret_addr >> ctx.intlv_addr_bit) & BIT(0)))
-			ctx.ret_addr ^= BIT(ctx.intlv_addr_bit);
-	}
+	if (dehash_addr(&ctx))
+		goto out_err;
 
 	/* Is calculated system address is above DRAM limit address? */
 	if (ctx.ret_addr > dram_limit_addr)
-- 
2.25.1

[PATCH 09/25] x86/MCE/AMD: Define function to check DRAM limit address

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move the DRAM limit check into a separate helper function.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index e341737e03d4..7aaf8a1dce2b 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -985,10 +985,20 @@ static int dehash_addr(struct addr_ctx *ctx)
 	return dehash_addr_df2(ctx);
 }
 
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+static int addr_over_limit(struct addr_ctx *ctx)
 {
-	u64 dram_limit_addr;
+	u64 dram_limit_addr  = ((ctx->reg_limit_addr & GENMASK_ULL(31, 12)) << 16)
+					| GENMASK_ULL(27, 0);
+
+	/* Is calculated system address above DRAM limit address? */
+	if (ctx->ret_addr > dram_limit_addr)
+		return -EINVAL;
 
+	return 0;
+}
+
+int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+{
 	struct addr_ctx ctx;
 
 	memset(&ctx, 0, sizeof(ctx));
@@ -1013,16 +1023,13 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	if (denormalize_addr(&ctx))
 		goto out_err;
 
-	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
-
 	if (add_base_and_hole(&ctx))
 		goto out_err;
 
 	if (dehash_addr(&ctx))
 		goto out_err;
 
-	/* Is calculated system address is above DRAM limit address? */
-	if (ctx.ret_addr > dram_limit_addr)
+	if (addr_over_limit(&ctx))
 		goto out_err;
 
 	*sys_addr = ctx.ret_addr;
-- 
2.25.1

[PATCH 10/25] x86/MCE/AMD: Remove goto statements

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

...and just return error codes directly.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 7aaf8a1dce2b..d51ec5c2f319 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -1015,28 +1015,25 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 		return -EINVAL;
 
 	if (get_dram_addr_map(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (get_intlv_mode(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (denormalize_addr(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (add_base_and_hole(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (dehash_addr(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (addr_over_limit(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	*sys_addr = ctx.ret_addr;
 	return 0;
-
-out_err:
-	return -EINVAL;
 }
 EXPORT_SYMBOL_GPL(umc_normaddr_to_sysaddr);
 
-- 
2.25.1

[PATCH 11/25] x86/MCE/AMD: Simplify function parameters

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Use a single address parameter for input and result to reduce the number
of parameters.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/include/asm/mce.h    | 5 ++---
 arch/x86/kernel/cpu/mce/amd.c | 6 +++---
 drivers/edac/amd64_edac.c     | 4 ++--
 3 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/mce.h b/arch/x86/include/asm/mce.h
index ddfb3cad8dff..991ba48a4c0c 100644
--- a/arch/x86/include/asm/mce.h
+++ b/arch/x86/include/asm/mce.h
@@ -351,7 +351,7 @@ extern int mce_threshold_create_device(unsigned int cpu);
 extern int mce_threshold_remove_device(unsigned int cpu);
 
 void mce_amd_feature_init(struct cpuinfo_x86 *c);
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr);
+int umc_normaddr_to_sysaddr(u64 *addr, u16 nid, u8 umc);
 
 #else
 
@@ -359,8 +359,7 @@ static inline int mce_threshold_create_device(unsigned int cpu)		{ return 0; };
 static inline int mce_threshold_remove_device(unsigned int cpu)		{ return 0; };
 static inline bool amd_mce_is_memory_error(struct mce *m)		{ return false; };
 static inline void mce_amd_feature_init(struct cpuinfo_x86 *c)		{ }
-static inline int
-umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)	{ return -EINVAL; };
+static inline int umc_normaddr_to_sysaddr(u64 *addr, u16 nid, u8 umc)	{ return -EINVAL; };
 #endif
 
 static inline void mce_hygon_feature_init(struct cpuinfo_x86 *c)	{ return mce_amd_feature_init(c); }
diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index d51ec5c2f319..58899c57eb76 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -997,14 +997,14 @@ static int addr_over_limit(struct addr_ctx *ctx)
 	return 0;
 }
 
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+int umc_normaddr_to_sysaddr(u64 *addr, u16 nid, u8 umc)
 {
 	struct addr_ctx ctx;
 
 	memset(&ctx, 0, sizeof(ctx));
 
 	/* We start from the normalized address */
-	ctx.ret_addr = norm_addr;
+	ctx.ret_addr = *addr;
 
 	ctx.nid = nid;
 	ctx.umc = umc;
@@ -1032,7 +1032,7 @@ int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 	if (addr_over_limit(&ctx))
 		return -EINVAL;
 
-	*sys_addr = ctx.ret_addr;
+	*addr = ctx.ret_addr;
 	return 0;
 }
 EXPORT_SYMBOL_GPL(umc_normaddr_to_sysaddr);
diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 9fa4dfc6ebee..dc4a642d5d7c 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -2896,7 +2896,7 @@ static void decode_umc_error(int node_id, struct mce *m)
 	struct mem_ctl_info *mci;
 	struct amd64_pvt *pvt;
 	struct err_info err;
-	u64 sys_addr;
+	u64 sys_addr = m->addr;
 
 	mci = edac_mc_find(node_id);
 	if (!mci)
@@ -2927,7 +2927,7 @@ static void decode_umc_error(int node_id, struct mce *m)
 
 	err.csrow = m->synd & 0x7;
 
-	if (umc_normaddr_to_sysaddr(m->addr, pvt->mc_node_id, err.channel, &sys_addr)) {
+	if (umc_normaddr_to_sysaddr(&sys_addr, pvt->mc_node_id, err.channel)) {
 		err.err_code = ERR_NORM_ADDR;
 		goto log_error;
 	}
-- 
2.25.1

[PATCH 12/25] x86/MCE/AMD: Define function to get Interleave Address Bit

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move code to find the interleave address bit into a separate helper
function.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 58899c57eb76..a60d9d275d14 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -814,14 +814,9 @@ static int get_dram_addr_map(struct addr_ctx *ctx)
 	return 0;
 }
 
-static int denormalize_addr(struct addr_ctx *ctx)
+static int get_intlv_addr_bit(struct addr_ctx *ctx)
 {
-	u32 tmp;
-
-	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
 	u8 intlv_addr_sel = (ctx->reg_base_addr >> 8) & 0x7;
-	u8 num_intlv_bits, cs_mask = 0;
 
 	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
 	if (intlv_addr_sel > 3) {
@@ -830,11 +825,25 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		return -EINVAL;
 	}
 
+	ctx->intlv_addr_bit = intlv_addr_sel + 8;
+
+	return 0;
+}
+
+static int denormalize_addr(struct addr_ctx *ctx)
+{
+	u32 tmp;
+
+	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
+	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
+	u8 num_intlv_bits, cs_mask = 0;
+
+	if (get_intlv_addr_bit(ctx))
+		return -EINVAL;
+
 	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
 	intlv_num_dies	  = (ctx->reg_limit_addr >> 10) & 0x3;
 
-	ctx->intlv_addr_bit = intlv_addr_sel + 8;
-
 	/* Re-use intlv_num_chan by setting it equal to log2(#channels) */
 	switch (intlv_num_chan) {
 	case 0:	intlv_num_chan = 0; break;
-- 
2.25.1

[PATCH 13/25] x86/MCE/AMD: Skip denormalization if no interleaving

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Denormalization doesn't apply to the "no interleaving" mode, so return
early without error in this case.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index a60d9d275d14..605376351cc8 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -838,6 +838,10 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
 	u8 num_intlv_bits, cs_mask = 0;
 
+	/* Return early if no interleaving. */
+	if (ctx->intlv_mode == NONE)
+		return 0;
+
 	if (get_intlv_addr_bit(ctx))
 		return -EINVAL;
 
-- 
2.25.1

[PATCH 14/25] x86/MCE/AMD: Define function to get number of interleaved channels

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move number of interleaved channel calculation to a separate helper
function. Drop unused cases.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 42 +++++++++++++++++++----------------
 1 file changed, 23 insertions(+), 19 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 605376351cc8..0c24bdc346a3 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -729,6 +729,7 @@ struct addr_ctx {
 	u8 umc;
 	u8 map_num;
 	u8 intlv_addr_bit;
+	u8 intlv_num_chan;
 	u8 cs_id;
 	bool hash_enabled;
 };
@@ -830,12 +831,29 @@ static int get_intlv_addr_bit(struct addr_ctx *ctx)
 	return 0;
 }
 
+static void get_intlv_num_chan(struct addr_ctx *ctx)
+{
+	/* Save the log2(# of channels). */
+	switch (ctx->intlv_mode) {
+	case NONE:
+		ctx->intlv_num_chan = 0;
+		break;
+	case NOHASH_2CH:
+	case DF2_HASH_2CH:
+		ctx->intlv_num_chan = 1;
+		break;
+	default:
+		/* Valid interleaving modes where checked earlier. */
+		break;
+	}
+}
+
 static int denormalize_addr(struct addr_ctx *ctx)
 {
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
+	u8 intlv_num_dies, intlv_num_sockets;
 	u8 num_intlv_bits, cs_mask = 0;
 
 	/* Return early if no interleaving. */
@@ -848,23 +866,9 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
 	intlv_num_dies	  = (ctx->reg_limit_addr >> 10) & 0x3;
 
-	/* Re-use intlv_num_chan by setting it equal to log2(#channels) */
-	switch (intlv_num_chan) {
-	case 0:	intlv_num_chan = 0; break;
-	case 1: intlv_num_chan = 1; break;
-	case 3: intlv_num_chan = 2; break;
-	case 5:	intlv_num_chan = 3; break;
-	case 7:	intlv_num_chan = 4; break;
-
-	case 8: intlv_num_chan = 1;
-		break;
-	default:
-		pr_err("%s: Invalid number of interleaved channels %d.\n",
-			__func__, intlv_num_chan);
-		return -EINVAL;
-	}
+	get_intlv_num_chan(ctx);
 
-	num_intlv_bits = intlv_num_chan;
+	num_intlv_bits = ctx->intlv_num_chan;
 
 	if (intlv_num_dies > 2) {
 		pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
@@ -901,8 +905,8 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		die_id_bit   = 0;
 
 		/* If interleaved over more than 1 channel: */
-		if (intlv_num_chan) {
-			die_id_bit = intlv_num_chan;
+		if (ctx->intlv_num_chan) {
+			die_id_bit = ctx->intlv_num_chan;
 			cs_mask	   = (1 << die_id_bit) - 1;
 			ctx->cs_id = cs_fabric_id & cs_mask;
 		}
-- 
2.25.1

[PATCH 15/25] x86/MCE/AMD: Define function to get number of interleaved dies

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move parsing of the number of interleaved dies to a separate helper
function. This will be expanded for future DF versions. Also, drop an
unneeded assert to match the reference code.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 25 ++++++++++++-------------
 1 file changed, 12 insertions(+), 13 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 0c24bdc346a3..de59fa6cf540 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -730,6 +730,7 @@ struct addr_ctx {
 	u8 map_num;
 	u8 intlv_addr_bit;
 	u8 intlv_num_chan;
+	u8 intlv_num_dies;
 	u8 cs_id;
 	bool hash_enabled;
 };
@@ -848,12 +849,17 @@ static void get_intlv_num_chan(struct addr_ctx *ctx)
 	}
 }
 
+static void get_intlv_num_dies(struct addr_ctx *ctx)
+{
+	ctx->intlv_num_dies  = (ctx->reg_limit_addr >> 10) & 0x3;
+}
+
 static int denormalize_addr(struct addr_ctx *ctx)
 {
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 intlv_num_dies, intlv_num_sockets;
+	u8 intlv_num_sockets;
 	u8 num_intlv_bits, cs_mask = 0;
 
 	/* Return early if no interleaving. */
@@ -864,19 +870,12 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		return -EINVAL;
 
 	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
-	intlv_num_dies	  = (ctx->reg_limit_addr >> 10) & 0x3;
 
 	get_intlv_num_chan(ctx);
+	get_intlv_num_dies(ctx);
 
 	num_intlv_bits = ctx->intlv_num_chan;
-
-	if (intlv_num_dies > 2) {
-		pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
-			__func__, intlv_num_dies);
-		return -EINVAL;
-	}
-
-	num_intlv_bits += intlv_num_dies;
+	num_intlv_bits += ctx->intlv_num_dies;
 
 	/* Add a bit if sockets are interleaved. */
 	num_intlv_bits += intlv_num_sockets;
@@ -913,14 +912,14 @@ static int denormalize_addr(struct addr_ctx *ctx)
 
 		sock_id_bit = die_id_bit;
 
-		if (intlv_num_dies || intlv_num_sockets)
+		if (ctx->intlv_num_dies || intlv_num_sockets)
 			if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
 						 ctx->umc, &tmp))
 				return -EINVAL;
 
 		/* If interleaved over more than 1 die. */
-		if (intlv_num_dies) {
-			sock_id_bit  = die_id_bit + intlv_num_dies;
+		if (ctx->intlv_num_dies) {
+			sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
 			die_id_shift = (tmp >> 24) & 0xF;
 			die_id_mask  = (tmp >> 8) & 0xFF;
 
-- 
2.25.1

[PATCH 16/25] x86/MCE/AMD: Define function to get number of interleaved sockets

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move parsing of the number of interleaved sockets to a separate helper
function. This will be expanded for future DF versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index de59fa6cf540..670787be705d 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -731,6 +731,7 @@ struct addr_ctx {
 	u8 intlv_addr_bit;
 	u8 intlv_num_chan;
 	u8 intlv_num_dies;
+	u8 intlv_num_sockets;
 	u8 cs_id;
 	bool hash_enabled;
 };
@@ -854,12 +855,16 @@ static void get_intlv_num_dies(struct addr_ctx *ctx)
 	ctx->intlv_num_dies  = (ctx->reg_limit_addr >> 10) & 0x3;
 }
 
+static void get_intlv_num_sockets(struct addr_ctx *ctx)
+{
+	ctx->intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
+}
+
 static int denormalize_addr(struct addr_ctx *ctx)
 {
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 intlv_num_sockets;
 	u8 num_intlv_bits, cs_mask = 0;
 
 	/* Return early if no interleaving. */
@@ -869,16 +874,13 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	if (get_intlv_addr_bit(ctx))
 		return -EINVAL;
 
-	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
-
 	get_intlv_num_chan(ctx);
 	get_intlv_num_dies(ctx);
+	get_intlv_num_sockets(ctx);
 
 	num_intlv_bits = ctx->intlv_num_chan;
 	num_intlv_bits += ctx->intlv_num_dies;
-
-	/* Add a bit if sockets are interleaved. */
-	num_intlv_bits += intlv_num_sockets;
+	num_intlv_bits += ctx->intlv_num_sockets;
 
 	/* Assert num_intlv_bits <= 4 */
 	if (num_intlv_bits > 4) {
@@ -912,7 +914,7 @@ static int denormalize_addr(struct addr_ctx *ctx)
 
 		sock_id_bit = die_id_bit;
 
-		if (ctx->intlv_num_dies || intlv_num_sockets)
+		if (ctx->intlv_num_dies || ctx->intlv_num_sockets)
 			if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
 						 ctx->umc, &tmp))
 				return -EINVAL;
@@ -928,7 +930,7 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		}
 
 		/* If interleaved over more than 1 socket. */
-		if (intlv_num_sockets) {
+		if (ctx->intlv_num_sockets) {
 			socket_id_shift	= (tmp >> 28) & 0xF;
 			socket_id_mask	= (tmp >> 16) & 0xFF;
 
-- 
2.25.1

[PATCH 17/25] x86/MCE/AMD: Remove unnecessary assert

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

It was removed in the reference code, so remove it here.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 670787be705d..8f4838a3d509 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -882,13 +882,6 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	num_intlv_bits += ctx->intlv_num_dies;
 	num_intlv_bits += ctx->intlv_num_sockets;
 
-	/* Assert num_intlv_bits <= 4 */
-	if (num_intlv_bits > 4) {
-		pr_err("%s: Invalid interleave bits %d.\n",
-			__func__, num_intlv_bits);
-		return -EINVAL;
-	}
-
 	if (num_intlv_bits > 0) {
 		u64 temp_addr_x, temp_addr_i, temp_addr_y;
 		u8 die_id_bit, sock_id_bit, cs_fabric_id;
-- 
2.25.1

[PATCH 18/25] x86/MCE/AMD: Define function to make space for CS ID

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move code to makes a gap for the CS ID into a separate helper function.
The exact bits to use vary based on interleaving mode. New interleaving
modes in future DF versions will be added as new cases.

Also, introduce a helper function that does the bit manipulation to make
the gap.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 43 ++++++++++++++++++++++++++++-------
 1 file changed, 35 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 8f4838a3d509..ba365901e39d 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -860,6 +860,38 @@ static void get_intlv_num_sockets(struct addr_ctx *ctx)
 	ctx->intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
 }
 
+static void expand_bits(u8 start_bit, u8 num_bits, u64 *value)
+{
+	u64 temp1, temp2;
+
+	if (start_bit == 0) {
+		*value <<= num_bits;
+		return;
+	}
+
+	temp1 = *value & GENMASK_ULL(start_bit - 1, 0);
+	temp2 = (*value & GENMASK_ULL(63, start_bit)) << num_bits;
+	*value = temp1 | temp2;
+}
+
+static void make_space_for_cs_id(struct addr_ctx *ctx)
+{
+	u8 num_intlv_bits;
+
+	switch (ctx->intlv_mode) {
+	case NOHASH_2CH:
+	case DF2_HASH_2CH:
+		num_intlv_bits = ctx->intlv_num_chan;
+		num_intlv_bits += ctx->intlv_num_dies;
+		num_intlv_bits += ctx->intlv_num_sockets;
+		expand_bits(ctx->intlv_addr_bit, num_intlv_bits, &ctx->ret_addr);
+		break;
+	default:
+		/* Valid interleaving modes where checked earlier. */
+		break;
+	}
+}
+
 static int denormalize_addr(struct addr_ctx *ctx)
 {
 	u32 tmp;
@@ -878,13 +910,11 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	get_intlv_num_dies(ctx);
 	get_intlv_num_sockets(ctx);
 
-	num_intlv_bits = ctx->intlv_num_chan;
-	num_intlv_bits += ctx->intlv_num_dies;
-	num_intlv_bits += ctx->intlv_num_sockets;
+	make_space_for_cs_id(ctx);
 
 	if (num_intlv_bits > 0) {
-		u64 temp_addr_x, temp_addr_i, temp_addr_y;
 		u8 die_id_bit, sock_id_bit, cs_fabric_id;
+		u64 temp_addr_i;
 
 		/*
 		 * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
@@ -939,11 +969,8 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
 		 * there are (where "I" starts).
 		 */
-		temp_addr_y = ctx->ret_addr & GENMASK_ULL(ctx->intlv_addr_bit - 1, 0);
 		temp_addr_i = (ctx->cs_id << ctx->intlv_addr_bit);
-		temp_addr_x = (ctx->ret_addr & GENMASK_ULL(63, ctx->intlv_addr_bit))
-			       << num_intlv_bits;
-		ctx->ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
+		ctx->ret_addr |= temp_addr_i;
 	}
 
 	return 0;
-- 
2.25.1

[PATCH 19/25] x86/MCE/AMD: Define function to calculate CS ID

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move code used to calculate the CS ID into a separate helper function.

Drop redundant code comment about reading DF register.

The "num_intlv_bits" variable is left uninitialized as it will be removed
in a later patch.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 97 +++++++++++++++++------------------
 1 file changed, 48 insertions(+), 49 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index ba365901e39d..ae49cef4792f 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -892,12 +892,54 @@ static void make_space_for_cs_id(struct addr_ctx *ctx)
 	}
 }
 
-static int denormalize_addr(struct addr_ctx *ctx)
+static int calculate_cs_id(struct addr_ctx *ctx)
 {
+	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
+	u8 die_id_bit, sock_id_bit, cs_fabric_id, cs_mask = 0;
 	u32 tmp;
 
-	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 num_intlv_bits, cs_mask = 0;
+	if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3], ctx->umc, &tmp))
+		return -EINVAL;
+
+	cs_fabric_id = (tmp >> 8) & 0xFF;
+	die_id_bit   = 0;
+
+	/* If interleaved over more than 1 channel: */
+	if (ctx->intlv_num_chan) {
+		die_id_bit = ctx->intlv_num_chan;
+		cs_mask	   = (1 << die_id_bit) - 1;
+		ctx->cs_id = cs_fabric_id & cs_mask;
+	}
+
+	sock_id_bit = die_id_bit;
+
+	if (ctx->intlv_num_dies || ctx->intlv_num_sockets)
+		if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK], ctx->umc, &tmp))
+			return -EINVAL;
+
+	/* If interleaved over more than 1 die: */
+	if (ctx->intlv_num_dies) {
+		sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
+		die_id_shift = (tmp >> 24) & 0xF;
+		die_id_mask  = (tmp >> 8) & 0xFF;
+
+		ctx->cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+	}
+
+	/* If interleaved over more than 1 socket: */
+	if (ctx->intlv_num_sockets) {
+		socket_id_shift	= (tmp >> 28) & 0xF;
+		socket_id_mask	= (tmp >> 16) & 0xFF;
+
+		ctx->cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
+	}
+
+	return 0;
+}
+
+static int denormalize_addr(struct addr_ctx *ctx)
+{
+	u8 num_intlv_bits;
 
 	/* Return early if no interleaving. */
 	if (ctx->intlv_mode == NONE)
@@ -912,55 +954,12 @@ static int denormalize_addr(struct addr_ctx *ctx)
 
 	make_space_for_cs_id(ctx);
 
+	if (calculate_cs_id(ctx))
+		return -EINVAL;
+
 	if (num_intlv_bits > 0) {
-		u8 die_id_bit, sock_id_bit, cs_fabric_id;
 		u64 temp_addr_i;
 
-		/*
-		 * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
-		 * This is the fabric id for this coherent slave. Use
-		 * umc/channel# as instance id of the coherent slave
-		 * for FICAA.
-		 */
-		if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3], ctx->umc, &tmp))
-			return -EINVAL;
-
-		cs_fabric_id = (tmp >> 8) & 0xFF;
-		die_id_bit   = 0;
-
-		/* If interleaved over more than 1 channel: */
-		if (ctx->intlv_num_chan) {
-			die_id_bit = ctx->intlv_num_chan;
-			cs_mask	   = (1 << die_id_bit) - 1;
-			ctx->cs_id = cs_fabric_id & cs_mask;
-		}
-
-		sock_id_bit = die_id_bit;
-
-		if (ctx->intlv_num_dies || ctx->intlv_num_sockets)
-			if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
-						 ctx->umc, &tmp))
-				return -EINVAL;
-
-		/* If interleaved over more than 1 die. */
-		if (ctx->intlv_num_dies) {
-			sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
-			die_id_shift = (tmp >> 24) & 0xF;
-			die_id_mask  = (tmp >> 8) & 0xFF;
-
-			ctx->cs_id |= ((cs_fabric_id & die_id_mask)
-					>> die_id_shift) << die_id_bit;
-		}
-
-		/* If interleaved over more than 1 socket. */
-		if (ctx->intlv_num_sockets) {
-			socket_id_shift	= (tmp >> 28) & 0xF;
-			socket_id_mask	= (tmp >> 16) & 0xFF;
-
-			ctx->cs_id |= ((cs_fabric_id & socket_id_mask)
-					>> socket_id_shift) << sock_id_bit;
-		}
-
 		/*
 		 * The pre-interleaved address consists of XXXXXXIIIYYYYY
 		 * where III is the ID for this CS, and XXXXXXYYYYY are the
-- 
2.25.1

[PATCH 20/25] x86/MCE/AMD: Define function to insert CS ID into address

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move the code that inserts the CS ID into the address into a separate
helper function. This will be expanded for future DF versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 22 ++++++----------------
 1 file changed, 6 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index ae49cef4792f..b017e4613d8f 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -937,10 +937,13 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 	return 0;
 }
 
-static int denormalize_addr(struct addr_ctx *ctx)
+static void insert_cs_id(struct addr_ctx *ctx)
 {
-	u8 num_intlv_bits;
+	ctx->ret_addr |= (ctx->cs_id << ctx->intlv_addr_bit);
+}
 
+static int denormalize_addr(struct addr_ctx *ctx)
+{
 	/* Return early if no interleaving. */
 	if (ctx->intlv_mode == NONE)
 		return 0;
@@ -957,20 +960,7 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	if (calculate_cs_id(ctx))
 		return -EINVAL;
 
-	if (num_intlv_bits > 0) {
-		u64 temp_addr_i;
-
-		/*
-		 * The pre-interleaved address consists of XXXXXXIIIYYYYY
-		 * where III is the ID for this CS, and XXXXXXYYYYY are the
-		 * address bits from the post-interleaved address.
-		 * "num_intlv_bits" has been calculated to tell us how many "I"
-		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
-		 * there are (where "I" starts).
-		 */
-		temp_addr_i = (ctx->cs_id << ctx->intlv_addr_bit);
-		ctx->ret_addr |= temp_addr_i;
-	}
+	insert_cs_id(ctx);
 
 	return 0;
 }
-- 
2.25.1

[PATCH 21/25] x86/MCE/AMD: Define function to get CS Fabric ID

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move code that gets the CS Fabric ID into a separate helper function.
This will be expanded for future DF versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index b017e4613d8f..6980c5c62f49 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -733,6 +733,7 @@ struct addr_ctx {
 	u8 intlv_num_dies;
 	u8 intlv_num_sockets;
 	u8 cs_id;
+	u8 cs_fabric_id;
 	bool hash_enabled;
 };
 
@@ -892,23 +893,34 @@ static void make_space_for_cs_id(struct addr_ctx *ctx)
 	}
 }
 
+static int get_cs_fabric_id(struct addr_ctx *ctx)
+{
+	u32 tmp;
+
+	if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3], ctx->umc, &tmp))
+		return -EINVAL;
+
+	ctx->cs_fabric_id = (tmp >> 8) & 0xFF;
+
+	return 0;
+}
+
 static int calculate_cs_id(struct addr_ctx *ctx)
 {
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 die_id_bit, sock_id_bit, cs_fabric_id, cs_mask = 0;
+	u8 die_id_bit, sock_id_bit, cs_mask = 0;
 	u32 tmp;
 
-	if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3], ctx->umc, &tmp))
+	if (get_cs_fabric_id(ctx))
 		return -EINVAL;
 
-	cs_fabric_id = (tmp >> 8) & 0xFF;
 	die_id_bit   = 0;
 
 	/* If interleaved over more than 1 channel: */
 	if (ctx->intlv_num_chan) {
 		die_id_bit = ctx->intlv_num_chan;
 		cs_mask	   = (1 << die_id_bit) - 1;
-		ctx->cs_id = cs_fabric_id & cs_mask;
+		ctx->cs_id = ctx->cs_fabric_id & cs_mask;
 	}
 
 	sock_id_bit = die_id_bit;
@@ -923,7 +935,7 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 		die_id_shift = (tmp >> 24) & 0xF;
 		die_id_mask  = (tmp >> 8) & 0xFF;
 
-		ctx->cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+		ctx->cs_id |= ((ctx->cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
 	}
 
 	/* If interleaved over more than 1 socket: */
@@ -931,7 +943,8 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 		socket_id_shift	= (tmp >> 28) & 0xF;
 		socket_id_mask	= (tmp >> 16) & 0xFF;
 
-		ctx->cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
+		ctx->cs_id |= ((ctx->cs_fabric_id & socket_id_mask)
+				>> socket_id_shift) << sock_id_bit;
 	}
 
 	return 0;
-- 
2.25.1

[PATCH 22/25] x86/MCE/AMD: Define function to find shift and mask values

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Move code to find the shift and mask values used in die and socket
interleaving into separate helper functions. These will be expanded for
future DF versions.

Also, move reading of the System Fabric ID Mask register into a helper
function and call this early. This will be expanded for future DF
versions and will be used to determine DF version levels.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 56 +++++++++++++++++++++++++++--------
 1 file changed, 43 insertions(+), 13 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 6980c5c62f49..01931f98c6d9 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -725,6 +725,7 @@ struct addr_ctx {
 	u32 reg_dram_offset;
 	u32 reg_base_addr;
 	u32 reg_limit_addr;
+	u32 reg_fab_id_mask0;
 	u16 nid;
 	u8 umc;
 	u8 map_num;
@@ -734,6 +735,8 @@ struct addr_ctx {
 	u8 intlv_num_sockets;
 	u8 cs_id;
 	u8 cs_fabric_id;
+	u8 die_id_mask;
+	u8 socket_id_mask;
 	bool hash_enabled;
 };
 
@@ -905,15 +908,31 @@ static int get_cs_fabric_id(struct addr_ctx *ctx)
 	return 0;
 }
 
+static void get_masks(struct addr_ctx *ctx)
+{
+	ctx->die_id_mask    = (ctx->reg_fab_id_mask0 >> 8) & 0xFF;
+	ctx->socket_id_mask = (ctx->reg_fab_id_mask0 >> 16) & 0xFF;
+}
+
+static u8 get_die_id_shift(struct addr_ctx *ctx)
+{
+	return (ctx->reg_fab_id_mask0 >> 24) & 0xF;
+}
+
+static u8 get_socket_id_shift(struct addr_ctx *ctx)
+{
+	return (ctx->reg_fab_id_mask0 >> 28) & 0xF;
+}
+
 static int calculate_cs_id(struct addr_ctx *ctx)
 {
-	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
 	u8 die_id_bit, sock_id_bit, cs_mask = 0;
-	u32 tmp;
 
 	if (get_cs_fabric_id(ctx))
 		return -EINVAL;
 
+	get_masks(ctx);
+
 	die_id_bit   = 0;
 
 	/* If interleaved over more than 1 channel: */
@@ -923,27 +942,26 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 		ctx->cs_id = ctx->cs_fabric_id & cs_mask;
 	}
 
-	sock_id_bit = die_id_bit;
+	/* Return early if no die interleaving and no socket interleaving. */
+	if (!(ctx->intlv_num_dies || ctx->intlv_num_sockets))
+		return 0;
 
-	if (ctx->intlv_num_dies || ctx->intlv_num_sockets)
-		if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK], ctx->umc, &tmp))
-			return -EINVAL;
+	sock_id_bit = die_id_bit;
 
 	/* If interleaved over more than 1 die: */
 	if (ctx->intlv_num_dies) {
-		sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
-		die_id_shift = (tmp >> 24) & 0xF;
-		die_id_mask  = (tmp >> 8) & 0xFF;
+		u8 die_id_shift = get_die_id_shift(ctx);
 
-		ctx->cs_id |= ((ctx->cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+		sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
+		ctx->cs_id |= ((ctx->cs_fabric_id & ctx->die_id_mask)
+				>> die_id_shift) << die_id_bit;
 	}
 
 	/* If interleaved over more than 1 socket: */
 	if (ctx->intlv_num_sockets) {
-		socket_id_shift	= (tmp >> 28) & 0xF;
-		socket_id_mask	= (tmp >> 16) & 0xFF;
+		u8 socket_id_shift = get_socket_id_shift(ctx);
 
-		ctx->cs_id |= ((ctx->cs_fabric_id & socket_id_mask)
+		ctx->cs_id |= ((ctx->cs_fabric_id & ctx->socket_id_mask)
 				>> socket_id_shift) << sock_id_bit;
 	}
 
@@ -1037,6 +1055,15 @@ static int addr_over_limit(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int get_fabric_id_mask_reg(struct addr_ctx *ctx)
+{
+	if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
+				 ctx->umc, &ctx->reg_fab_id_mask0))
+		return -EINVAL;
+
+	return 0;
+}
+
 int umc_normaddr_to_sysaddr(u64 *addr, u16 nid, u8 umc)
 {
 	struct addr_ctx ctx;
@@ -1049,6 +1076,9 @@ int umc_normaddr_to_sysaddr(u64 *addr, u16 nid, u8 umc)
 	ctx.nid = nid;
 	ctx.umc = umc;
 
+	if (get_fabric_id_mask_reg(&ctx))
+		return -EINVAL;
+
 	ctx.df_type = get_df_type(&ctx);
 
 	if (remove_dram_offset(&ctx))
-- 
2.25.1

[PATCH 23/25] x86/MCE/AMD: Update CS ID calculation to match reference code

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

Redo the current CS ID calculations to match the reference code. Helper
functions are introduced that will be expanded for future DF versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 50 ++++++++++++++++++++++-------------
 1 file changed, 31 insertions(+), 19 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 01931f98c6d9..3340f8326681 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -924,45 +924,57 @@ static u8 get_socket_id_shift(struct addr_ctx *ctx)
 	return (ctx->reg_fab_id_mask0 >> 28) & 0xF;
 }
 
+static u8 get_dst_fabric_id(struct addr_ctx *ctx)
+{
+	return ctx->reg_limit_addr & 0xFF;
+}
+
+static u8 get_component_id_mask(struct addr_ctx *ctx)
+{
+	return (~(ctx->socket_id_mask | ctx->die_id_mask)) & 0xFF;
+}
+
+static u8 calc_level_bits(u8 id, u8 level_mask, u8 shift, u8 mask, u8 num_bits)
+{
+	return (((id & level_mask) >> shift) & mask) << num_bits;
+}
+
 static int calculate_cs_id(struct addr_ctx *ctx)
 {
-	u8 die_id_bit, sock_id_bit, cs_mask = 0;
+	u8 mask, num_intlv_bits = ctx->intlv_num_chan;
+	u8 dst_fabric_id = get_dst_fabric_id(ctx);
 
 	if (get_cs_fabric_id(ctx))
 		return -EINVAL;
 
 	get_masks(ctx);
 
-	die_id_bit   = 0;
+	mask = get_component_id_mask(ctx);
+	ctx->cs_id = (ctx->cs_fabric_id & mask) - (dst_fabric_id & mask);
 
-	/* If interleaved over more than 1 channel: */
-	if (ctx->intlv_num_chan) {
-		die_id_bit = ctx->intlv_num_chan;
-		cs_mask	   = (1 << die_id_bit) - 1;
-		ctx->cs_id = ctx->cs_fabric_id & cs_mask;
-	}
-
-	/* Return early if no die interleaving and no socket interleaving. */
-	if (!(ctx->intlv_num_dies || ctx->intlv_num_sockets))
-		return 0;
-
-	sock_id_bit = die_id_bit;
+	mask = (1 << num_intlv_bits) - 1;
+	ctx->cs_id &= mask;
 
 	/* If interleaved over more than 1 die: */
 	if (ctx->intlv_num_dies) {
 		u8 die_id_shift = get_die_id_shift(ctx);
 
-		sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
-		ctx->cs_id |= ((ctx->cs_fabric_id & ctx->die_id_mask)
-				>> die_id_shift) << die_id_bit;
+		mask = (1 << ctx->intlv_num_dies) - 1;
+
+		ctx->cs_id |= calc_level_bits(ctx->cs_fabric_id, ctx->die_id_mask,
+					      die_id_shift, mask, num_intlv_bits);
+
+		num_intlv_bits += ctx->intlv_num_dies;
 	}
 
 	/* If interleaved over more than 1 socket: */
 	if (ctx->intlv_num_sockets) {
 		u8 socket_id_shift = get_socket_id_shift(ctx);
 
-		ctx->cs_id |= ((ctx->cs_fabric_id & ctx->socket_id_mask)
-				>> socket_id_shift) << sock_id_bit;
+		mask = (1 << ctx->intlv_num_sockets) - 1;
+
+		ctx->cs_id |= calc_level_bits(ctx->cs_fabric_id, ctx->socket_id_mask,
+					      socket_id_shift, mask, num_intlv_bits);
 	}
 
 	return 0;
-- 
2.25.1

[PATCH 24/25] x86/MCE/AMD: Match hash function to reference code

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

The reference code for DF2 hashing was changed to XOR the interleave
address bit rather than the CS ID. Match that here.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 3340f8326681..b3bfdc42dcaa 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -1037,7 +1037,7 @@ static int dehash_addr_df2(struct addr_ctx *ctx)
 			(ctx->ret_addr >> 18) ^
 			(ctx->ret_addr >> 21) ^
 			(ctx->ret_addr >> 30) ^
-			ctx->cs_id;
+			(ctx->ret_addr >> ctx->intlv_addr_bit);
 
 	hashed_bit &= BIT(0);
 
-- 
2.25.1

[PATCH 25/25] x86/MCE/AMD: Add support for address translation on DF3 systems

[Yazen Ghannam]

From: Yazen Ghannam <yazen.ghannam@amd.com>

DF3-based systems (Rome and later) support new interleaving modes and a
number of bit fields have changed or moved entirely. Add support for
these new modes and fields.

Refactoring should be minimal due to earlier changes, and most updates
will be additions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 183 +++++++++++++++++++++++++++++++---
 1 file changed, 171 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index b3bfdc42dcaa..a165706c04f1 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -684,9 +684,11 @@ enum df_reg_names {
 	DRAM_BASE_ADDR_1,
 	DRAM_LIMIT_ADDR_1,
 	DRAM_OFFSET,
+	DF_GLOBAL_CTL,
 
 	/* Function 1 */
 	SYS_FAB_ID_MASK,
+	SYS_FAB_ID_MASK_1,
 };
 
 static struct df_reg df_regs[] = {
@@ -702,18 +704,28 @@ static struct df_reg df_regs[] = {
 	[DRAM_LIMIT_ADDR_1]	=	{0, 0x11C},
 	/* D18F0x1B4 (DramOffset) */
 	[DRAM_OFFSET]		=	{0, 0x1B4},
+	/* D18F0x3F8 (DfGlobalCtrl) */
+	[DF_GLOBAL_CTL]		=	{0, 0x3F8},
 	/* D18F1x208 (SystemFabricIdMask) */
 	[SYS_FAB_ID_MASK]	=	{1, 0x208},
+	/* D18F1x20C (SystemFabricIdMask1) */
+	[SYS_FAB_ID_MASK_1]	=	{1, 0x20C},
 };
 
 enum df_types {
 	DF2,
+	DF3,
 };
 
 /* These are mapped 1:1 to the hardware values. Special cases are set at > 0x20. */
 enum intlv_modes {
 	NONE		= 0x00,
 	NOHASH_2CH	= 0x01,
+	NOHASH_4CH	= 0x03,
+	NOHASH_8CH	= 0x05,
+	HASH_COD4_2CH	= 0x0C,
+	HASH_COD2_4CH	= 0x0D,
+	HASH_COD1_8CH	= 0x0E,
 	DF2_HASH_2CH	= 0x21,
 };
 
@@ -726,6 +738,7 @@ struct addr_ctx {
 	u32 reg_base_addr;
 	u32 reg_limit_addr;
 	u32 reg_fab_id_mask0;
+	u32 reg_fab_id_mask1;
 	u16 nid;
 	u8 umc;
 	u8 map_num;
@@ -737,11 +750,15 @@ struct addr_ctx {
 	u8 cs_fabric_id;
 	u8 die_id_mask;
 	u8 socket_id_mask;
+	u8 node_id_shift;
 	bool hash_enabled;
 };
 
 static enum df_types get_df_type(struct addr_ctx *ctx)
 {
+	if ((ctx->reg_fab_id_mask0 & 0xFF) != 0)
+		return DF3;
+
 	return DF2;
 }
 
@@ -762,8 +779,23 @@ static int get_intlv_mode_df2(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int get_intlv_mode_df3(struct addr_ctx *ctx)
+{
+	ctx->intlv_mode = (ctx->reg_base_addr >> 2) & 0xF;
+
+	if (ctx->intlv_mode == HASH_COD4_2CH ||
+	    ctx->intlv_mode == HASH_COD2_4CH ||
+	    ctx->intlv_mode == HASH_COD1_8CH)
+		ctx->hash_enabled = true;
+
+	return 0;
+}
+
 static int get_intlv_mode(struct addr_ctx *ctx)
 {
+	if (ctx->df_type == DF3)
+		return get_intlv_mode_df3(ctx);
+
 	return get_intlv_mode_df2(ctx);
 }
 
@@ -777,6 +809,9 @@ static int get_dram_offset_reg(struct addr_ctx *ctx)
 
 static u64 get_hi_addr_offset(struct addr_ctx *ctx)
 {
+	if (ctx->df_type == DF3)
+		return (ctx->reg_dram_offset & GENMASK_ULL(31, 12)) << 16;
+
 	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
 }
 
@@ -823,10 +858,15 @@ static int get_dram_addr_map(struct addr_ctx *ctx)
 
 static int get_intlv_addr_bit(struct addr_ctx *ctx)
 {
-	u8 intlv_addr_sel = (ctx->reg_base_addr >> 8) & 0x7;
+	u8 intlv_addr_sel;
+
+	if (ctx->df_type == DF3)
+		intlv_addr_sel = (ctx->reg_base_addr >> 9) & 0x7;
+	else
+		intlv_addr_sel = (ctx->reg_base_addr >> 8) & 0x7;
 
-	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
-	if (intlv_addr_sel > 3) {
+	/* {0, 1, 2, 3, 4} map to address bits {8, 9, 10, 11, 12} respectively */
+	if (intlv_addr_sel > 4) {
 		pr_err("%s: Invalid interleave address select %d.\n",
 			__func__, intlv_addr_sel);
 		return -EINVAL;
@@ -845,9 +885,18 @@ static void get_intlv_num_chan(struct addr_ctx *ctx)
 		ctx->intlv_num_chan = 0;
 		break;
 	case NOHASH_2CH:
+	case HASH_COD4_2CH:
 	case DF2_HASH_2CH:
 		ctx->intlv_num_chan = 1;
 		break;
+	case NOHASH_4CH:
+	case HASH_COD2_4CH:
+		ctx->intlv_num_chan = 2;
+		break;
+	case NOHASH_8CH:
+	case HASH_COD1_8CH:
+		ctx->intlv_num_chan = 3;
+		break;
 	default:
 		/* Valid interleaving modes where checked earlier. */
 		break;
@@ -856,12 +905,18 @@ static void get_intlv_num_chan(struct addr_ctx *ctx)
 
 static void get_intlv_num_dies(struct addr_ctx *ctx)
 {
-	ctx->intlv_num_dies  = (ctx->reg_limit_addr >> 10) & 0x3;
+	if (ctx->df_type == DF3)
+		ctx->intlv_num_dies  = (ctx->reg_base_addr >> 6) & 0x3;
+	else
+		ctx->intlv_num_dies  = (ctx->reg_limit_addr >> 10) & 0x3;
 }
 
 static void get_intlv_num_sockets(struct addr_ctx *ctx)
 {
-	ctx->intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
+	if (ctx->df_type == DF3)
+		ctx->intlv_num_sockets = (ctx->reg_base_addr >> 8) & 0x1;
+	else
+		ctx->intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
 }
 
 static void expand_bits(u8 start_bit, u8 num_bits, u64 *value)
@@ -884,12 +939,22 @@ static void make_space_for_cs_id(struct addr_ctx *ctx)
 
 	switch (ctx->intlv_mode) {
 	case NOHASH_2CH:
+	case NOHASH_4CH:
+	case NOHASH_8CH:
 	case DF2_HASH_2CH:
 		num_intlv_bits = ctx->intlv_num_chan;
 		num_intlv_bits += ctx->intlv_num_dies;
 		num_intlv_bits += ctx->intlv_num_sockets;
 		expand_bits(ctx->intlv_addr_bit, num_intlv_bits, &ctx->ret_addr);
 		break;
+	case HASH_COD4_2CH:
+	case HASH_COD2_4CH:
+	case HASH_COD1_8CH:
+		num_intlv_bits = ctx->intlv_num_chan;
+		num_intlv_bits += ctx->intlv_num_sockets;
+		expand_bits(ctx->intlv_addr_bit, 1, &ctx->ret_addr);
+		if (num_intlv_bits > 1)
+			expand_bits(12, num_intlv_bits - 1, &ctx->ret_addr);
 	default:
 		/* Valid interleaving modes where checked earlier. */
 		break;
@@ -908,29 +973,56 @@ static int get_cs_fabric_id(struct addr_ctx *ctx)
 	return 0;
 }
 
-static void get_masks(struct addr_ctx *ctx)
-{
-	ctx->die_id_mask    = (ctx->reg_fab_id_mask0 >> 8) & 0xFF;
-	ctx->socket_id_mask = (ctx->reg_fab_id_mask0 >> 16) & 0xFF;
-}
-
 static u8 get_die_id_shift(struct addr_ctx *ctx)
 {
+	if (ctx->df_type == DF3)
+		return ctx->node_id_shift;
+
 	return (ctx->reg_fab_id_mask0 >> 24) & 0xF;
 }
 
 static u8 get_socket_id_shift(struct addr_ctx *ctx)
 {
+	if (ctx->df_type == DF3)
+		return ((ctx->reg_fab_id_mask1 >> 8) & 0x3) + ctx->node_id_shift;
+
 	return (ctx->reg_fab_id_mask0 >> 28) & 0xF;
 }
 
+static void get_node_id_shift(struct addr_ctx *ctx)
+{
+	ctx->node_id_shift = ctx->reg_fab_id_mask1 & 0xF;
+}
+
+static void get_masks(struct addr_ctx *ctx)
+{
+	if (ctx->df_type == DF3) {
+		get_node_id_shift(ctx);
+
+		ctx->die_id_mask = (ctx->reg_fab_id_mask1 >> 16) & 0x7;
+		ctx->die_id_mask <<= ctx->node_id_shift;
+
+		ctx->socket_id_mask = (ctx->reg_fab_id_mask1 >> 24) & 0x7;
+		ctx->socket_id_mask <<= ctx->node_id_shift;
+	} else {
+		ctx->die_id_mask    = (ctx->reg_fab_id_mask0 >> 8) & 0xFF;
+		ctx->socket_id_mask = (ctx->reg_fab_id_mask0 >> 16) & 0xFF;
+	}
+}
+
 static u8 get_dst_fabric_id(struct addr_ctx *ctx)
 {
+	if (ctx->df_type == DF3)
+		return ctx->reg_limit_addr & 0x1FF;
+
 	return ctx->reg_limit_addr & 0xFF;
 }
 
 static u8 get_component_id_mask(struct addr_ctx *ctx)
 {
+	if (ctx->df_type == DF3)
+		return ctx->reg_fab_id_mask0 & 0x1FF;
+
 	return (~(ctx->socket_id_mask | ctx->die_id_mask)) & 0xFF;
 }
 
@@ -982,7 +1074,14 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 
 static void insert_cs_id(struct addr_ctx *ctx)
 {
-	ctx->ret_addr |= (ctx->cs_id << ctx->intlv_addr_bit);
+	if (ctx->intlv_mode == HASH_COD4_2CH ||
+	    ctx->intlv_mode == HASH_COD2_4CH ||
+	    ctx->intlv_mode == HASH_COD1_8CH) {
+		ctx->ret_addr |= ((ctx->cs_id & 0x1) << ctx->intlv_addr_bit);
+		ctx->ret_addr |= ((ctx->cs_id & 0xE) << 11);
+	} else {
+		ctx->ret_addr |= (ctx->cs_id << ctx->intlv_addr_bit);
+	}
 }
 
 static int denormalize_addr(struct addr_ctx *ctx)
@@ -1047,11 +1146,67 @@ static int dehash_addr_df2(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int dehash_addr_df3(struct addr_ctx *ctx)
+{
+	u8 hashed_bit, intlv_ctl_64k, intlv_ctl_2M, intlv_ctl_1G;
+	u32 tmp;
+
+	if (amd_df_indirect_read(ctx->nid, df_regs[DF_GLOBAL_CTL], ctx->umc, &tmp))
+		return -EINVAL;
+
+	intlv_ctl_64k = !!((tmp >> 20) & 0x1);
+	intlv_ctl_2M  = !!((tmp >> 21) & 0x1);
+	intlv_ctl_1G  = !!((tmp >> 22) & 0x1);
+
+	hashed_bit =	(ctx->ret_addr >> 14) ^
+			((ctx->ret_addr >> 18) & intlv_ctl_64k) ^
+			((ctx->ret_addr >> 23) & intlv_ctl_2M) ^
+			((ctx->ret_addr >> 32) & intlv_ctl_1G) ^
+			(ctx->ret_addr >> ctx->intlv_addr_bit);
+
+	hashed_bit &= BIT(0);
+
+	if (hashed_bit != ((ctx->ret_addr >> ctx->intlv_addr_bit) & BIT(0)))
+		ctx->ret_addr ^= BIT(ctx->intlv_addr_bit);
+
+	if (ctx->intlv_mode != HASH_COD2_4CH &&
+	    ctx->intlv_mode != HASH_COD1_8CH)
+		return 0;
+
+	hashed_bit =	(ctx->ret_addr >> 12) ^
+			((ctx->ret_addr >> 16) & intlv_ctl_64k) ^
+			((ctx->ret_addr >> 21) & intlv_ctl_2M) ^
+			((ctx->ret_addr >> 30) & intlv_ctl_1G);
+
+	hashed_bit &= BIT(0);
+
+	if (hashed_bit != ((ctx->ret_addr >> 12) & BIT(0)))
+		ctx->ret_addr ^= BIT(12);
+
+	if (ctx->intlv_mode != HASH_COD1_8CH)
+		return 0;
+
+	hashed_bit =	(ctx->ret_addr >> 13) ^
+			((ctx->ret_addr >> 17) & intlv_ctl_64k) ^
+			((ctx->ret_addr >> 22) & intlv_ctl_2M) ^
+			((ctx->ret_addr >> 31) & intlv_ctl_1G);
+
+	hashed_bit &= BIT(0);
+
+	if (hashed_bit != ((ctx->ret_addr >> 13) & BIT(0)))
+		ctx->ret_addr ^= BIT(13);
+
+	return 0;
+}
+
 static int dehash_addr(struct addr_ctx *ctx)
 {
 	if (!ctx->hash_enabled)
 		return 0;
 
+	if (ctx->df_type == DF3)
+		return dehash_addr_df3(ctx);
+
 	return dehash_addr_df2(ctx);
 }
 
@@ -1073,6 +1228,10 @@ static int get_fabric_id_mask_reg(struct addr_ctx *ctx)
 				 ctx->umc, &ctx->reg_fab_id_mask0))
 		return -EINVAL;
 
+	if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK_1],
+				 ctx->umc, &ctx->reg_fab_id_mask1))
+		return -EINVAL;
+
 	return 0;
 }
 
-- 
2.25.1

Re: [PATCH 00/25] AMD MCA Address Translation Updates

[Randy Dunlap]

On 5/7/21 12:01 PM, Yazen Ghannam wrote:
> From: Yazen Ghannam <yazen.ghannam@amd.com>
> 
...
> 
> Rome:
>   No interleaving
>   Nodes-per-Socket 0 (NPS0)
>   Nodes-per-Socket 1 (NPS1)
>   Nodes-per-Socket 2 (NPS2)
>   Nodes-per-Socket 4 (NPS4)
>   NPS2 w/o hashing
>   NPS4 w/o hashing
> 
> Thanks,
> Yazen

Hi Yazen,

It appears that you need to provide a glossary of acronyms, e.g.:

DF = (Data Fabric ?)
CS =
MCE =
NPS = Nodes per Socket
etc.


thanks.
-- 
~Randy

Re: [PATCH 00/25] AMD MCA Address Translation Updates

[Yazen Ghannam]

On Fri, May 07, 2021 at 01:32:28PM -0700, Randy Dunlap wrote:
> On 5/7/21 12:01 PM, Yazen Ghannam wrote:
> > From: Yazen Ghannam <yazen.ghannam@amd.com>
> > 
> ...
> > 
> > Rome:
> >   No interleaving
> >   Nodes-per-Socket 0 (NPS0)
> >   Nodes-per-Socket 1 (NPS1)
> >   Nodes-per-Socket 2 (NPS2)
> >   Nodes-per-Socket 4 (NPS4)
> >   NPS2 w/o hashing
> >   NPS4 w/o hashing
> > 
> > Thanks,
> > Yazen
> 
> Hi Yazen,
> 
> It appears that you need to provide a glossary of acronyms, e.g.:
> 
> DF = (Data Fabric ?)
> CS =
> MCE =
> NPS = Nodes per Socket
> etc.
> 
>

Hi Randy,

That's a good idea. Should it be in the cover letter, patches, or
somewhere else?

Thanks,
Yazen

Re: [PATCH 00/25] AMD MCA Address Translation Updates

[Randy Dunlap]

On 5/11/21 8:42 AM, Yazen Ghannam wrote:
> On Fri, May 07, 2021 at 01:32:28PM -0700, Randy Dunlap wrote:
>> On 5/7/21 12:01 PM, Yazen Ghannam wrote:
>>> From: Yazen Ghannam <yazen.ghannam@amd.com>
>>>
>> ...
>>>
>>> Rome:
>>>   No interleaving
>>>   Nodes-per-Socket 0 (NPS0)
>>>   Nodes-per-Socket 1 (NPS1)
>>>   Nodes-per-Socket 2 (NPS2)
>>>   Nodes-per-Socket 4 (NPS4)
>>>   NPS2 w/o hashing
>>>   NPS4 w/o hashing
>>>
>>> Thanks,
>>> Yazen
>>
>> Hi Yazen,
>>
>> It appears that you need to provide a glossary of acronyms, e.g.:
>>
>> DF = (Data Fabric ?)
>> CS =
>> MCE =
>> NPS = Nodes per Socket
>> etc.
>>
>>
> 
> Hi Randy,
> 
> That's a good idea. Should it be in the cover letter, patches, or
> somewhere else?

Cover letter seems good to me.
Thanks.

-- 
~Randy

Re: [PATCH 00/25] AMD MCA Address Translation Updates

[Borislav Petkov]

On Tue, May 11, 2021 at 09:13:56AM -0700, Randy Dunlap wrote:
> Cover letter seems good to me.

Cover letter gets lost and if we didn't have lore.kernel.org, it
would be a total waste of effort. I think if it is only a small,
acronym-mapping thing, the top of drivers/edac/mce_amd.c should be a
much better place.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH 00/25] AMD MCA Address Translation Updates

[Borislav Petkov]

On Fri, May 07, 2021 at 03:01:15PM -0400, Yazen Ghannam wrote:
> Patches 1-24 do the refactor without adding new system support. The goal
> is to break down the translation algorithm into smaller chunks. There
> are some simple wrapper functions defined. These will be filled in when
> supporting newer systems. The intention is that new system support can
> be added without any major refactor. I tried to make a patch for each
> logical change. There's a bit of churn so as to not break the build with
> each change. I think many of these patches can be squashed together, if
> desired. The top level function was split first, then the next level of
> functions, etc. in a somewhat breadth-first approach.

No, that's great what you did and keeping each logical change in a
single patch is a lot easier on everybody involved.

Now, looking at this - and I know we've talked about this before - but:

umc_normaddr_to_sysaddr() is used only in amd64_edac.c.
amd_df_indirect_read() is used only by this function, so how about
moving both to amd64_edac, where they're needed and then doing the
refactoring ontop?

You can simply reuse your current patches - just change the file they
patch from

arch/x86/kernel/cpu/mce/amd.c

to

drivers/edac/amd64_edac.c

I went through te umc_... function and AFAICT, it doesn't need any core
MCE facilities so it should be just fine in EDAC land.

Or?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH 00/25] AMD MCA Address Translation Updates

[Yazen Ghannam]

On Mon, May 17, 2021 at 02:57:04PM +0200, Borislav Petkov wrote:
> On Fri, May 07, 2021 at 03:01:15PM -0400, Yazen Ghannam wrote:
> > Patches 1-24 do the refactor without adding new system support. The goal
> > is to break down the translation algorithm into smaller chunks. There
> > are some simple wrapper functions defined. These will be filled in when
> > supporting newer systems. The intention is that new system support can
> > be added without any major refactor. I tried to make a patch for each
> > logical change. There's a bit of churn so as to not break the build with
> > each change. I think many of these patches can be squashed together, if
> > desired. The top level function was split first, then the next level of
> > functions, etc. in a somewhat breadth-first approach.
> 
> No, that's great what you did and keeping each logical change in a
> single patch is a lot easier on everybody involved.
> 
> Now, looking at this - and I know we've talked about this before - but:
> 
> umc_normaddr_to_sysaddr() is used only in amd64_edac.c.
> amd_df_indirect_read() is used only by this function, so how about
> moving both to amd64_edac, where they're needed and then doing the
> refactoring ontop?
> 
> You can simply reuse your current patches - just change the file they
> patch from
> 
> arch/x86/kernel/cpu/mce/amd.c
> 
> to
> 
> drivers/edac/amd64_edac.c
> 
> I went through te umc_... function and AFAICT, it doesn't need any core
> MCE facilities so it should be just fine in EDAC land.
> 
> Or?
>

I think this is a good idea. The only hang up is that we should be using
the output of this function, i.e. the systeme physical address, when
handling memory errors in the MCE notifier blocks. But I have an idea
where we can handle this. I can send that as a follow up series, if
that's okay.

One other issue is what if a user doesn't want to use amd64_edac_mod?
This is more of a user preference and/or configuration issue. Maybe the
module loads, but an uninterested user can tell EDAC to not log errors,
etc.? Or should the translation code live in its own module?

So for version 2, I have 1) Add a glossary of terms, and 2) Move
everything to EDAC. Any other comments?

Thanks,
Yazen

Re: [PATCH 00/25] AMD MCA Address Translation Updates

[Borislav Petkov]

On Tue, May 18, 2021 at 11:52:07PM -0400, Yazen Ghannam wrote:
> I think this is a good idea. The only hang up is that we should be using
> the output of this function, i.e. the systeme physical address, when
> handling memory errors in the MCE notifier blocks. But I have an idea
> where we can handle this. I can send that as a follow up series, if
> that's okay.

Yeah, so frankly I'm not happy with all this clumsy plugging we do
with notifiers and then amd_register_ecc_decoder() which is called in
mce_amd.c to be used in amd64_edac.c which finally logs it.

What I'd like to see is mce_amd.c still decoding all kinds of errors and
additionally amd64_edac continues processing only the DRAM errors.

> One other issue is what if a user doesn't want to use amd64_edac_mod?

Then she/he doesn't get DRAM errors mapped to a DIMM - simple.

> This is more of a user preference and/or configuration issue. Maybe the
> module loads, but an uninterested user can tell EDAC to not log errors,
> etc.? Or should the translation code live in its own module?

No need. Translation is part of EDAC so if you don't load it, you don't
get the functionality.

> So for version 2, I have 1) Add a glossary of terms, and 2) Move
> everything to EDAC. Any other comments?

None at the moment - I'll do a deeper review with v2.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette