linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 00/31] AMD MCA Address Translation Updates
@ 2021-06-23 19:19 Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 01/31] x86/MCE/AMD, EDAC/amd64: Move address translation to AMD64 EDAC Yazen Ghannam
                   ` (30 more replies)
  0 siblings, 31 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam, x86

This patchset refactors the AMD MCA Address Translation code and adds
support for newer systems.

The reference code was recently refactored in preparation for updates
for future systems. These patches try to follow the reference code as
closely as possible. I also tried to address comments from previous
patchset reviews.

Patch 1 moves the address translation code from arch/x86 to EDAC.

Patch 2 moves the df_indirect_read() function from arch/x86 to EDAC
also, since this is used only by the address translation code.

Patches 3-29 do the refactor without adding new system support. The goal
is to break down the translation algorithm into smaller chunks. Code
that changes between Data Fabric versions or interleaving modes is moved
to a set of function pointers. The intention is that new system support
can be added without any major refactor.

I tried to make a patch for each logical change. The top level function
was split first, then the next level of functions, etc. in a somewhat
breadth-first approach. 

Patch 30 adds support for systems with Data Fabric version 3 (Rome and
later).

Patch 31 adds a short glossary for acronyms used in the translation
code.

Each patch was build tested individually. The entire set was
functionally tested with the following modes.

Naples:
  No interleaving
  Channel interleaving
  Die interleaving
  Socket interleaving

Rome:
  No interleaving
  Nodes-per-Socket 0 (NPS0)
  Nodes-per-Socket 1 (NPS1)
  Nodes-per-Socket 2 (NPS2)
  Nodes-per-Socket 4 (NPS4)
  NPS2 w/o hashing
  NPS4 w/o hashing

I've copied x86@kernel.org for the cover letter and first two patches
that touch arch/x86. The EDAC maintainers are copied for the whole set.

Thanks,
Yazen

Cc: <x86@kernel.org>

Link:
https://lkml.kernel.org/r/20210507190140.18854-1-Yazen.Ghannam@amd.com

v1->v2:
* Move address translation code to EDAC.
* Use function pointers to handle code differences between DF versions.
* Add glossary of acronyms.

Yazen Ghannam (31):
  x86/MCE/AMD, EDAC/amd64: Move address translation to AMD64 EDAC
  x86/amd_nb, EDAC/amd64: Move DF Indirect Read to AMD64 EDAC
  EDAC/amd64: Don't use naked values for DF registers
  EDAC/amd64: Allow for DF Indirect Broadcast reads
  EDAC/amd64: Add context struct
  EDAC/amd64: Define Data Fabric operations
  EDAC/amd64: Define functions for DramOffset
  EDAC/amd64: Define function to read DRAM address map registers
  EDAC/amd64: Define function to find interleaving mode
  EDAC/amd64: Define function to denormalize address
  EDAC/amd64: Define function to add DRAM base and hole
  EDAC/amd64: Define function to dehash address
  EDAC/amd64: Define function to check DRAM limit address
  EDAC/amd64: Remove goto statements
  EDAC/amd64: Simplify function parameters
  EDAC/amd64: Define function to get Interleave Address Bit
  EDAC/amd64: Skip denormalization if no interleaving
  EDAC/amd64: Define function to get number of interleaved channels
  EDAC/amd64: Define function to get number of interleaved dies
  EDAC/amd64: Define function to get number of interleaved sockets
  EDAC/amd64: Remove unnecessary assert
  EDAC/amd64: Define function to make space for CS ID
  EDAC/amd64: Define function to calculate CS ID
  EDAC/amd64: Define function to insert CS ID into address
  EDAC/amd64: Define function to get CS Fabric ID
  EDAC/amd64: Define function to find shift and mask values
  EDAC/amd64: Update CS ID calculation to match reference code
  EDAC/amd64: Match hash function to reference code
  EDAC/amd64: Define helper function to get interleave address select
    bit
  EDAC/amd64: Add support for address translation on DF3 systems
  EDAC/amd64: Add glossary of acronyms for address translation

 arch/x86/include/asm/amd_nb.h |   1 -
 arch/x86/include/asm/mce.h    |   3 -
 arch/x86/kernel/amd_nb.c      |  50 +--
 arch/x86/kernel/cpu/mce/amd.c | 200 ----------
 drivers/edac/amd64_edac.c     | 710 +++++++++++++++++++++++++++++++++-
 5 files changed, 709 insertions(+), 255 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH v2 01/31] x86/MCE/AMD, EDAC/amd64: Move address translation to AMD64 EDAC
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 02/31] x86/amd_nb, EDAC/amd64: Move DF Indirect Read " Yazen Ghannam
                   ` (29 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam, x86

The address translation code used for current AMD systems is
non-architectural. So move it to EDAC.

Cc: <x86@kernel.org>
Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/YKJoICQzD/o7ZPBp@zn.tnic

v1->v2:
* New in v2.

 arch/x86/include/asm/mce.h    |   3 -
 arch/x86/kernel/cpu/mce/amd.c | 200 ----------------------------------
 drivers/edac/amd64_edac.c     | 199 +++++++++++++++++++++++++++++++++
 3 files changed, 199 insertions(+), 203 deletions(-)

diff --git a/arch/x86/include/asm/mce.h b/arch/x86/include/asm/mce.h
index 2ac265e832de..a547ab5b0532 100644
--- a/arch/x86/include/asm/mce.h
+++ b/arch/x86/include/asm/mce.h
@@ -357,7 +357,6 @@ extern int mce_threshold_create_device(unsigned int cpu);
 extern int mce_threshold_remove_device(unsigned int cpu);
 
 void mce_amd_feature_init(struct cpuinfo_x86 *c);
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr);
 
 #else
 
@@ -365,8 +364,6 @@ static inline int mce_threshold_create_device(unsigned int cpu)		{ return 0; };
 static inline int mce_threshold_remove_device(unsigned int cpu)		{ return 0; };
 static inline bool amd_mce_is_memory_error(struct mce *m)		{ return false; };
 static inline void mce_amd_feature_init(struct cpuinfo_x86 *c)		{ }
-static inline int
-umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)	{ return -EINVAL; };
 #endif
 
 static inline void mce_hygon_feature_init(struct cpuinfo_x86 *c)	{ return mce_amd_feature_init(c); }
diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index 1a2a90bd29e7..ce881f32ca41 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -689,206 +689,6 @@ void mce_amd_feature_init(struct cpuinfo_x86 *c)
 		deferred_error_interrupt_enable(c);
 }
 
-int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
-{
-	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
-	/* We start from the normalized address */
-	u64 ret_addr = norm_addr;
-
-	u32 tmp;
-
-	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
-	u8 intlv_addr_sel, intlv_addr_bit;
-	u8 num_intlv_bits, hashed_bit;
-	u8 lgcy_mmio_hole_en, base = 0;
-	u8 cs_mask, cs_id = 0;
-	bool hash_enabled = false;
-
-	/* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
-	if (amd_df_indirect_read(nid, 0, 0x1B4, umc, &tmp))
-		goto out_err;
-
-	/* Remove HiAddrOffset from normalized address, if enabled: */
-	if (tmp & BIT(0)) {
-		u64 hi_addr_offset = (tmp & GENMASK_ULL(31, 20)) << 8;
-
-		if (norm_addr >= hi_addr_offset) {
-			ret_addr -= hi_addr_offset;
-			base = 1;
-		}
-	}
-
-	/* Read D18F0x110 (DramBaseAddress). */
-	if (amd_df_indirect_read(nid, 0, 0x110 + (8 * base), umc, &tmp))
-		goto out_err;
-
-	/* Check if address range is valid. */
-	if (!(tmp & BIT(0))) {
-		pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
-			__func__, tmp);
-		goto out_err;
-	}
-
-	lgcy_mmio_hole_en = tmp & BIT(1);
-	intlv_num_chan	  = (tmp >> 4) & 0xF;
-	intlv_addr_sel	  = (tmp >> 8) & 0x7;
-	dram_base_addr	  = (tmp & GENMASK_ULL(31, 12)) << 16;
-
-	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
-	if (intlv_addr_sel > 3) {
-		pr_err("%s: Invalid interleave address select %d.\n",
-			__func__, intlv_addr_sel);
-		goto out_err;
-	}
-
-	/* Read D18F0x114 (DramLimitAddress). */
-	if (amd_df_indirect_read(nid, 0, 0x114 + (8 * base), umc, &tmp))
-		goto out_err;
-
-	intlv_num_sockets = (tmp >> 8) & 0x1;
-	intlv_num_dies	  = (tmp >> 10) & 0x3;
-	dram_limit_addr	  = ((tmp & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
-
-	intlv_addr_bit = intlv_addr_sel + 8;
-
-	/* Re-use intlv_num_chan by setting it equal to log2(#channels) */
-	switch (intlv_num_chan) {
-	case 0:	intlv_num_chan = 0; break;
-	case 1: intlv_num_chan = 1; break;
-	case 3: intlv_num_chan = 2; break;
-	case 5:	intlv_num_chan = 3; break;
-	case 7:	intlv_num_chan = 4; break;
-
-	case 8: intlv_num_chan = 1;
-		hash_enabled = true;
-		break;
-	default:
-		pr_err("%s: Invalid number of interleaved channels %d.\n",
-			__func__, intlv_num_chan);
-		goto out_err;
-	}
-
-	num_intlv_bits = intlv_num_chan;
-
-	if (intlv_num_dies > 2) {
-		pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
-			__func__, intlv_num_dies);
-		goto out_err;
-	}
-
-	num_intlv_bits += intlv_num_dies;
-
-	/* Add a bit if sockets are interleaved. */
-	num_intlv_bits += intlv_num_sockets;
-
-	/* Assert num_intlv_bits <= 4 */
-	if (num_intlv_bits > 4) {
-		pr_err("%s: Invalid interleave bits %d.\n",
-			__func__, num_intlv_bits);
-		goto out_err;
-	}
-
-	if (num_intlv_bits > 0) {
-		u64 temp_addr_x, temp_addr_i, temp_addr_y;
-		u8 die_id_bit, sock_id_bit, cs_fabric_id;
-
-		/*
-		 * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
-		 * This is the fabric id for this coherent slave. Use
-		 * umc/channel# as instance id of the coherent slave
-		 * for FICAA.
-		 */
-		if (amd_df_indirect_read(nid, 0, 0x50, umc, &tmp))
-			goto out_err;
-
-		cs_fabric_id = (tmp >> 8) & 0xFF;
-		die_id_bit   = 0;
-
-		/* If interleaved over more than 1 channel: */
-		if (intlv_num_chan) {
-			die_id_bit = intlv_num_chan;
-			cs_mask	   = (1 << die_id_bit) - 1;
-			cs_id	   = cs_fabric_id & cs_mask;
-		}
-
-		sock_id_bit = die_id_bit;
-
-		/* Read D18F1x208 (SystemFabricIdMask). */
-		if (intlv_num_dies || intlv_num_sockets)
-			if (amd_df_indirect_read(nid, 1, 0x208, umc, &tmp))
-				goto out_err;
-
-		/* If interleaved over more than 1 die. */
-		if (intlv_num_dies) {
-			sock_id_bit  = die_id_bit + intlv_num_dies;
-			die_id_shift = (tmp >> 24) & 0xF;
-			die_id_mask  = (tmp >> 8) & 0xFF;
-
-			cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
-		}
-
-		/* If interleaved over more than 1 socket. */
-		if (intlv_num_sockets) {
-			socket_id_shift	= (tmp >> 28) & 0xF;
-			socket_id_mask	= (tmp >> 16) & 0xFF;
-
-			cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
-		}
-
-		/*
-		 * The pre-interleaved address consists of XXXXXXIIIYYYYY
-		 * where III is the ID for this CS, and XXXXXXYYYYY are the
-		 * address bits from the post-interleaved address.
-		 * "num_intlv_bits" has been calculated to tell us how many "I"
-		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
-		 * there are (where "I" starts).
-		 */
-		temp_addr_y = ret_addr & GENMASK_ULL(intlv_addr_bit-1, 0);
-		temp_addr_i = (cs_id << intlv_addr_bit);
-		temp_addr_x = (ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
-		ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
-	}
-
-	/* Add dram base address */
-	ret_addr += dram_base_addr;
-
-	/* If legacy MMIO hole enabled */
-	if (lgcy_mmio_hole_en) {
-		if (amd_df_indirect_read(nid, 0, 0x104, umc, &tmp))
-			goto out_err;
-
-		dram_hole_base = tmp & GENMASK(31, 24);
-		if (ret_addr >= dram_hole_base)
-			ret_addr += (BIT_ULL(32) - dram_hole_base);
-	}
-
-	if (hash_enabled) {
-		/* Save some parentheses and grab ls-bit at the end. */
-		hashed_bit =	(ret_addr >> 12) ^
-				(ret_addr >> 18) ^
-				(ret_addr >> 21) ^
-				(ret_addr >> 30) ^
-				cs_id;
-
-		hashed_bit &= BIT(0);
-
-		if (hashed_bit != ((ret_addr >> intlv_addr_bit) & BIT(0)))
-			ret_addr ^= BIT(intlv_addr_bit);
-	}
-
-	/* Is calculated system address is above DRAM limit address? */
-	if (ret_addr > dram_limit_addr)
-		goto out_err;
-
-	*sys_addr = ret_addr;
-	return 0;
-
-out_err:
-	return -EINVAL;
-}
-EXPORT_SYMBOL_GPL(umc_normaddr_to_sysaddr);
-
 bool amd_mce_is_memory_error(struct mce *m)
 {
 	/* ErrCodeExt[20:16] */
diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index f0d8f60acee1..046b98fcc4fe 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -993,6 +993,205 @@ static int sys_addr_to_csrow(struct mem_ctl_info *mci, u64 sys_addr)
 	return csrow;
 }
 
+static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+{
+	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
+	/* We start from the normalized address */
+	u64 ret_addr = norm_addr;
+
+	u32 tmp;
+
+	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
+	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
+	u8 intlv_addr_sel, intlv_addr_bit;
+	u8 num_intlv_bits, hashed_bit;
+	u8 lgcy_mmio_hole_en, base = 0;
+	u8 cs_mask, cs_id = 0;
+	bool hash_enabled = false;
+
+	/* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
+	if (amd_df_indirect_read(nid, 0, 0x1B4, umc, &tmp))
+		goto out_err;
+
+	/* Remove HiAddrOffset from normalized address, if enabled: */
+	if (tmp & BIT(0)) {
+		u64 hi_addr_offset = (tmp & GENMASK_ULL(31, 20)) << 8;
+
+		if (norm_addr >= hi_addr_offset) {
+			ret_addr -= hi_addr_offset;
+			base = 1;
+		}
+	}
+
+	/* Read D18F0x110 (DramBaseAddress). */
+	if (amd_df_indirect_read(nid, 0, 0x110 + (8 * base), umc, &tmp))
+		goto out_err;
+
+	/* Check if address range is valid. */
+	if (!(tmp & BIT(0))) {
+		pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
+			__func__, tmp);
+		goto out_err;
+	}
+
+	lgcy_mmio_hole_en = tmp & BIT(1);
+	intlv_num_chan	  = (tmp >> 4) & 0xF;
+	intlv_addr_sel	  = (tmp >> 8) & 0x7;
+	dram_base_addr	  = (tmp & GENMASK_ULL(31, 12)) << 16;
+
+	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
+	if (intlv_addr_sel > 3) {
+		pr_err("%s: Invalid interleave address select %d.\n",
+			__func__, intlv_addr_sel);
+		goto out_err;
+	}
+
+	/* Read D18F0x114 (DramLimitAddress). */
+	if (amd_df_indirect_read(nid, 0, 0x114 + (8 * base), umc, &tmp))
+		goto out_err;
+
+	intlv_num_sockets = (tmp >> 8) & 0x1;
+	intlv_num_dies	  = (tmp >> 10) & 0x3;
+	dram_limit_addr	  = ((tmp & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
+
+	intlv_addr_bit = intlv_addr_sel + 8;
+
+	/* Re-use intlv_num_chan by setting it equal to log2(#channels) */
+	switch (intlv_num_chan) {
+	case 0:	intlv_num_chan = 0; break;
+	case 1: intlv_num_chan = 1; break;
+	case 3: intlv_num_chan = 2; break;
+	case 5:	intlv_num_chan = 3; break;
+	case 7:	intlv_num_chan = 4; break;
+
+	case 8: intlv_num_chan = 1;
+		hash_enabled = true;
+		break;
+	default:
+		pr_err("%s: Invalid number of interleaved channels %d.\n",
+			__func__, intlv_num_chan);
+		goto out_err;
+	}
+
+	num_intlv_bits = intlv_num_chan;
+
+	if (intlv_num_dies > 2) {
+		pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
+			__func__, intlv_num_dies);
+		goto out_err;
+	}
+
+	num_intlv_bits += intlv_num_dies;
+
+	/* Add a bit if sockets are interleaved. */
+	num_intlv_bits += intlv_num_sockets;
+
+	/* Assert num_intlv_bits <= 4 */
+	if (num_intlv_bits > 4) {
+		pr_err("%s: Invalid interleave bits %d.\n",
+			__func__, num_intlv_bits);
+		goto out_err;
+	}
+
+	if (num_intlv_bits > 0) {
+		u64 temp_addr_x, temp_addr_i, temp_addr_y;
+		u8 die_id_bit, sock_id_bit, cs_fabric_id;
+
+		/*
+		 * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
+		 * This is the fabric id for this coherent slave. Use
+		 * umc/channel# as instance id of the coherent slave
+		 * for FICAA.
+		 */
+		if (amd_df_indirect_read(nid, 0, 0x50, umc, &tmp))
+			goto out_err;
+
+		cs_fabric_id = (tmp >> 8) & 0xFF;
+		die_id_bit   = 0;
+
+		/* If interleaved over more than 1 channel: */
+		if (intlv_num_chan) {
+			die_id_bit = intlv_num_chan;
+			cs_mask	   = (1 << die_id_bit) - 1;
+			cs_id	   = cs_fabric_id & cs_mask;
+		}
+
+		sock_id_bit = die_id_bit;
+
+		/* Read D18F1x208 (SystemFabricIdMask). */
+		if (intlv_num_dies || intlv_num_sockets)
+			if (amd_df_indirect_read(nid, 1, 0x208, umc, &tmp))
+				goto out_err;
+
+		/* If interleaved over more than 1 die. */
+		if (intlv_num_dies) {
+			sock_id_bit  = die_id_bit + intlv_num_dies;
+			die_id_shift = (tmp >> 24) & 0xF;
+			die_id_mask  = (tmp >> 8) & 0xFF;
+
+			cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+		}
+
+		/* If interleaved over more than 1 socket. */
+		if (intlv_num_sockets) {
+			socket_id_shift	= (tmp >> 28) & 0xF;
+			socket_id_mask	= (tmp >> 16) & 0xFF;
+
+			cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
+		}
+
+		/*
+		 * The pre-interleaved address consists of XXXXXXIIIYYYYY
+		 * where III is the ID for this CS, and XXXXXXYYYYY are the
+		 * address bits from the post-interleaved address.
+		 * "num_intlv_bits" has been calculated to tell us how many "I"
+		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
+		 * there are (where "I" starts).
+		 */
+		temp_addr_y = ret_addr & GENMASK_ULL(intlv_addr_bit-1, 0);
+		temp_addr_i = (cs_id << intlv_addr_bit);
+		temp_addr_x = (ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
+		ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
+	}
+
+	/* Add dram base address */
+	ret_addr += dram_base_addr;
+
+	/* If legacy MMIO hole enabled */
+	if (lgcy_mmio_hole_en) {
+		if (amd_df_indirect_read(nid, 0, 0x104, umc, &tmp))
+			goto out_err;
+
+		dram_hole_base = tmp & GENMASK(31, 24);
+		if (ret_addr >= dram_hole_base)
+			ret_addr += (BIT_ULL(32) - dram_hole_base);
+	}
+
+	if (hash_enabled) {
+		/* Save some parentheses and grab ls-bit at the end. */
+		hashed_bit =	(ret_addr >> 12) ^
+				(ret_addr >> 18) ^
+				(ret_addr >> 21) ^
+				(ret_addr >> 30) ^
+				cs_id;
+
+		hashed_bit &= BIT(0);
+
+		if (hashed_bit != ((ret_addr >> intlv_addr_bit) & BIT(0)))
+			ret_addr ^= BIT(intlv_addr_bit);
+	}
+
+	/* Is calculated system address is above DRAM limit address? */
+	if (ret_addr > dram_limit_addr)
+		goto out_err;
+
+	*sys_addr = ret_addr;
+	return 0;
+
+out_err:
+	return -EINVAL;
+}
+
 static int get_channel_from_ecc_syndrome(struct mem_ctl_info *, u16);
 
 /*
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 02/31] x86/amd_nb, EDAC/amd64: Move DF Indirect Read to AMD64 EDAC
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 01/31] x86/MCE/AMD, EDAC/amd64: Move address translation to AMD64 EDAC Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 03/31] EDAC/amd64: Don't use naked values for DF registers Yazen Ghannam
                   ` (28 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam, x86

The df_indirect_read() function is only used for address translation.
Move this to EDAC along with the translation code.

Cc: <x86@kernel.org>
Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/YKJoICQzD/o7ZPBp@zn.tnic

v1->v2:
* New in v2.

 arch/x86/include/asm/amd_nb.h |  1 -
 arch/x86/kernel/amd_nb.c      | 50 +----------------------------------
 drivers/edac/amd64_edac.c     | 50 +++++++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 50 deletions(-)

diff --git a/arch/x86/include/asm/amd_nb.h b/arch/x86/include/asm/amd_nb.h
index 455066a06f60..00d1a400b7a1 100644
--- a/arch/x86/include/asm/amd_nb.h
+++ b/arch/x86/include/asm/amd_nb.h
@@ -24,7 +24,6 @@ extern int amd_set_subcaches(int, unsigned long);
 
 extern int amd_smn_read(u16 node, u32 address, u32 *value);
 extern int amd_smn_write(u16 node, u32 address, u32 value);
-extern int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo);
 
 struct amd_l3_cache {
 	unsigned indices;
diff --git a/arch/x86/kernel/amd_nb.c b/arch/x86/kernel/amd_nb.c
index 23dda362dc0f..5884dfa619ff 100644
--- a/arch/x86/kernel/amd_nb.c
+++ b/arch/x86/kernel/amd_nb.c
@@ -27,7 +27,7 @@
 #define PCI_DEVICE_ID_AMD_19H_DF_F4	0x1654
 #define PCI_DEVICE_ID_AMD_19H_M50H_DF_F4 0x166e
 
-/* Protect the PCI config register pairs used for SMN and DF indirect access. */
+/* Protect the PCI config register pairs used for SMN. */
 static DEFINE_MUTEX(smn_mutex);
 
 static u32 *flush_words;
@@ -177,54 +177,6 @@ int amd_smn_write(u16 node, u32 address, u32 value)
 }
 EXPORT_SYMBOL_GPL(amd_smn_write);
 
-/*
- * Data Fabric Indirect Access uses FICAA/FICAD.
- *
- * Fabric Indirect Configuration Access Address (FICAA): Constructed based
- * on the device's Instance Id and the PCI function and register offset of
- * the desired register.
- *
- * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
- * and FICAD HI registers but so far we only need the LO register.
- */
-int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
-{
-	struct pci_dev *F4;
-	u32 ficaa;
-	int err = -ENODEV;
-
-	if (node >= amd_northbridges.num)
-		goto out;
-
-	F4 = node_to_amd_nb(node)->link;
-	if (!F4)
-		goto out;
-
-	ficaa  = 1;
-	ficaa |= reg & 0x3FC;
-	ficaa |= (func & 0x7) << 11;
-	ficaa |= instance_id << 16;
-
-	mutex_lock(&smn_mutex);
-
-	err = pci_write_config_dword(F4, 0x5C, ficaa);
-	if (err) {
-		pr_warn("Error writing DF Indirect FICAA, FICAA=0x%x\n", ficaa);
-		goto out_unlock;
-	}
-
-	err = pci_read_config_dword(F4, 0x98, lo);
-	if (err)
-		pr_warn("Error reading DF Indirect FICAD LO, FICAA=0x%x.\n", ficaa);
-
-out_unlock:
-	mutex_unlock(&smn_mutex);
-
-out:
-	return err;
-}
-EXPORT_SYMBOL_GPL(amd_df_indirect_read);
-
 int amd_cache_northbridges(void)
 {
 	const struct pci_device_id *misc_ids = amd_nb_misc_ids;
diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 046b98fcc4fe..e9342d7d693f 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -993,6 +993,56 @@ static int sys_addr_to_csrow(struct mem_ctl_info *mci, u64 sys_addr)
 	return csrow;
 }
 
+/* Protect the PCI config register pairs used for DF indirect access. */
+static DEFINE_MUTEX(df_indirect_mutex);
+
+/*
+ * Data Fabric Indirect Access uses FICAA/FICAD.
+ *
+ * Fabric Indirect Configuration Access Address (FICAA): Constructed based
+ * on the device's Instance Id and the PCI function and register offset of
+ * the desired register.
+ *
+ * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
+ * and FICAD HI registers but so far we only need the LO register.
+ */
+static int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
+{
+	struct pci_dev *F4;
+	u32 ficaa;
+	int err = -ENODEV;
+
+	if (node >= amd_nb_num())
+		goto out;
+
+	F4 = node_to_amd_nb(node)->link;
+	if (!F4)
+		goto out;
+
+	ficaa  = 1;
+	ficaa |= reg & 0x3FC;
+	ficaa |= (func & 0x7) << 11;
+	ficaa |= instance_id << 16;
+
+	mutex_lock(&df_indirect_mutex);
+
+	err = pci_write_config_dword(F4, 0x5C, ficaa);
+	if (err) {
+		pr_warn("Error writing DF Indirect FICAA, FICAA=0x%x\n", ficaa);
+		goto out_unlock;
+	}
+
+	err = pci_read_config_dword(F4, 0x98, lo);
+	if (err)
+		pr_warn("Error reading DF Indirect FICAD LO, FICAA=0x%x.\n", ficaa);
+
+out_unlock:
+	mutex_unlock(&df_indirect_mutex);
+
+out:
+	return err;
+}
+
 static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 03/31] EDAC/amd64: Don't use naked values for DF registers
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 01/31] x86/MCE/AMD, EDAC/amd64: Move address translation to AMD64 EDAC Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 02/31] x86/amd_nb, EDAC/amd64: Move DF Indirect Read " Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-25 15:21   ` Borislav Petkov
  2021-06-23 19:19 ` [PATCH v2 04/31] EDAC/amd64: Allow for DF Indirect Broadcast reads Yazen Ghannam
                   ` (27 subsequent siblings)
  30 siblings, 1 reply; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

AMD Data Fabric registers are defined using a combination of PCI
function number and offset. Define a struct to hold these values, and
update the DF Indirect Access function to accept a struct of this type.

Update the address translation code to include a list of the needed DF
registers using this new format. Define an enumeration to give the
registers more human-readable names.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-2-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 60 ++++++++++++++++++++++++++++++---------
 1 file changed, 47 insertions(+), 13 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index e9342d7d693f..b94067e3952b 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -996,6 +996,11 @@ static int sys_addr_to_csrow(struct mem_ctl_info *mci, u64 sys_addr)
 /* Protect the PCI config register pairs used for DF indirect access. */
 static DEFINE_MUTEX(df_indirect_mutex);
 
+struct df_reg {
+	u8 func;
+	u16 offset;
+};
+
 /*
  * Data Fabric Indirect Access uses FICAA/FICAD.
  *
@@ -1006,7 +1011,7 @@ static DEFINE_MUTEX(df_indirect_mutex);
  * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
  * and FICAD HI registers but so far we only need the LO register.
  */
-static int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32 *lo)
+static int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32 *lo)
 {
 	struct pci_dev *F4;
 	u32 ficaa;
@@ -1020,8 +1025,8 @@ static int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32
 		goto out;
 
 	ficaa  = 1;
-	ficaa |= reg & 0x3FC;
-	ficaa |= (func & 0x7) << 11;
+	ficaa |= reg.offset & 0x3FC;
+	ficaa |= (reg.func & 0x7) << 11;
 	ficaa |= instance_id << 16;
 
 	mutex_lock(&df_indirect_mutex);
@@ -1043,6 +1048,33 @@ static int amd_df_indirect_read(u16 node, u8 func, u16 reg, u8 instance_id, u32
 	return err;
 }
 
+enum df_reg_names {
+	/* Function 0 */
+	FAB_BLK_INST_INFO_3,
+	DRAM_HOLE_CTL,
+	DRAM_BASE_ADDR,
+	DRAM_LIMIT_ADDR,
+	DRAM_OFFSET,
+
+	/* Function 1 */
+	SYS_FAB_ID_MASK,
+};
+
+static struct df_reg df_regs[] = {
+	/* D18F0x50 (FabricBlockInstanceInformation3_CS) */
+	[FAB_BLK_INST_INFO_3]	=	{0, 0x50},
+	/* D18F0x104 (DramHoleControl) */
+	[DRAM_HOLE_CTL]		=	{0, 0x104},
+	/* D18F0x110 (DramBaseAddress) */
+	[DRAM_BASE_ADDR]	=	{0, 0x110},
+	/* D18F0x114 (DramLimitAddress) */
+	[DRAM_LIMIT_ADDR]	=	{0, 0x114},
+	/* D18F0x1B4 (DramOffset) */
+	[DRAM_OFFSET]		=	{0, 0x1B4},
+	/* D18F1x208 (SystemFabricIdMask) */
+	[SYS_FAB_ID_MASK]	=	{1, 0x208},
+};
+
 static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
@@ -1059,8 +1091,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	u8 cs_mask, cs_id = 0;
 	bool hash_enabled = false;
 
-	/* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
-	if (amd_df_indirect_read(nid, 0, 0x1B4, umc, &tmp))
+	struct df_reg reg;
+
+	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
 		goto out_err;
 
 	/* Remove HiAddrOffset from normalized address, if enabled: */
@@ -1073,8 +1106,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		}
 	}
 
-	/* Read D18F0x110 (DramBaseAddress). */
-	if (amd_df_indirect_read(nid, 0, 0x110 + (8 * base), umc, &tmp))
+	reg = df_regs[DRAM_BASE_ADDR];
+	reg.offset += base * 8;
+	if (amd_df_indirect_read(nid, reg, umc, &tmp))
 		goto out_err;
 
 	/* Check if address range is valid. */
@@ -1096,8 +1130,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		goto out_err;
 	}
 
-	/* Read D18F0x114 (DramLimitAddress). */
-	if (amd_df_indirect_read(nid, 0, 0x114 + (8 * base), umc, &tmp))
+	reg = df_regs[DRAM_LIMIT_ADDR];
+	reg.offset += base * 8;
+	if (amd_df_indirect_read(nid, reg, umc, &tmp))
 		goto out_err;
 
 	intlv_num_sockets = (tmp >> 8) & 0x1;
@@ -1153,7 +1188,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		 * umc/channel# as instance id of the coherent slave
 		 * for FICAA.
 		 */
-		if (amd_df_indirect_read(nid, 0, 0x50, umc, &tmp))
+		if (amd_df_indirect_read(nid, df_regs[FAB_BLK_INST_INFO_3], umc, &tmp))
 			goto out_err;
 
 		cs_fabric_id = (tmp >> 8) & 0xFF;
@@ -1168,9 +1203,8 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 
 		sock_id_bit = die_id_bit;
 
-		/* Read D18F1x208 (SystemFabricIdMask). */
 		if (intlv_num_dies || intlv_num_sockets)
-			if (amd_df_indirect_read(nid, 1, 0x208, umc, &tmp))
+			if (amd_df_indirect_read(nid, df_regs[SYS_FAB_ID_MASK], umc, &tmp))
 				goto out_err;
 
 		/* If interleaved over more than 1 die. */
@@ -1209,7 +1243,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 
 	/* If legacy MMIO hole enabled */
 	if (lgcy_mmio_hole_en) {
-		if (amd_df_indirect_read(nid, 0, 0x104, umc, &tmp))
+		if (amd_df_indirect_read(nid, df_regs[DRAM_HOLE_CTL], umc, &tmp))
 			goto out_err;
 
 		dram_hole_base = tmp & GENMASK(31, 24);
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 04/31] EDAC/amd64: Allow for DF Indirect Broadcast reads
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (2 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 03/31] EDAC/amd64: Don't use naked values for DF registers Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-30 16:22   ` Borislav Petkov
  2021-06-23 19:19 ` [PATCH v2 05/31] EDAC/amd64: Add context struct Yazen Ghannam
                   ` (26 subsequent siblings)
  30 siblings, 1 reply; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

The DF Indirect Access method allows for "Broadcast" accesses in which
case no specific instance is targeted. Add support using a reserved
instance ID of 0xFF to indicate a broadcast access. Set the FICAA
register appropriately.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-1-Yazen.Ghannam@amd.com

v1->v2:
* New in v2.

 drivers/edac/amd64_edac.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index b94067e3952b..d67cd8f57b94 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1010,7 +1010,11 @@ struct df_reg {
  *
  * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
  * and FICAD HI registers but so far we only need the LO register.
+ *
+ * Use Instance Id 0xFF to indicate a broadcast read.
  */
+
+#define DF_BROADCAST	0xFF
 static int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32 *lo)
 {
 	struct pci_dev *F4;
@@ -1024,7 +1028,7 @@ static int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32
 	if (!F4)
 		goto out;
 
-	ficaa  = 1;
+	ficaa  = (instance_id == DF_BROADCAST) ? 0 : 1;
 	ficaa |= reg.offset & 0x3FC;
 	ficaa |= (reg.func & 0x7) << 11;
 	ficaa |= instance_id << 16;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 05/31] EDAC/amd64: Add context struct
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (3 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 04/31] EDAC/amd64: Allow for DF Indirect Broadcast reads Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-30 17:17   ` Borislav Petkov
  2021-06-23 19:19 ` [PATCH v2 06/31] EDAC/amd64: Define Data Fabric operations Yazen Ghannam
                   ` (25 subsequent siblings)
  30 siblings, 1 reply; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Define an address translation context struct. This will hold values that
will be passed between multiple functions.

Save return address, Node ID, and the Instance ID number to start.
Currently, we use the UMC number as the Instance ID, but future DF
versions may use another value.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-3-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Changed "umc" variable to "inst_id".
* Drop df_types enum.

 drivers/edac/amd64_edac.c | 49 +++++++++++++++++++++++++--------------
 1 file changed, 31 insertions(+), 18 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index d67cd8f57b94..62eca188458f 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1079,12 +1079,15 @@ static struct df_reg df_regs[] = {
 	[SYS_FAB_ID_MASK]	=	{1, 0x208},
 };
 
+struct addr_ctx {
+	u64 ret_addr;
+	u16 nid;
+	u8 inst_id;
+};
+
 static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
-	/* We start from the normalized address */
-	u64 ret_addr = norm_addr;
-
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
@@ -1097,6 +1100,16 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 
 	struct df_reg reg;
 
+	struct addr_ctx ctx;
+
+	memset(&ctx, 0, sizeof(ctx));
+
+	/* We start from the normalized address */
+	ctx.ret_addr = norm_addr;
+
+	ctx.nid = nid;
+	ctx.inst_id = umc;
+
 	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
 		goto out_err;
 
@@ -1105,7 +1118,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		u64 hi_addr_offset = (tmp & GENMASK_ULL(31, 20)) << 8;
 
 		if (norm_addr >= hi_addr_offset) {
-			ret_addr -= hi_addr_offset;
+			ctx.ret_addr -= hi_addr_offset;
 			base = 1;
 		}
 	}
@@ -1236,14 +1249,14 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
 		 * there are (where "I" starts).
 		 */
-		temp_addr_y = ret_addr & GENMASK_ULL(intlv_addr_bit-1, 0);
+		temp_addr_y = ctx.ret_addr & GENMASK_ULL(intlv_addr_bit - 1, 0);
 		temp_addr_i = (cs_id << intlv_addr_bit);
-		temp_addr_x = (ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
-		ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
+		temp_addr_x = (ctx.ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
+		ctx.ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
 	}
 
 	/* Add dram base address */
-	ret_addr += dram_base_addr;
+	ctx.ret_addr += dram_base_addr;
 
 	/* If legacy MMIO hole enabled */
 	if (lgcy_mmio_hole_en) {
@@ -1251,29 +1264,29 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 			goto out_err;
 
 		dram_hole_base = tmp & GENMASK(31, 24);
-		if (ret_addr >= dram_hole_base)
-			ret_addr += (BIT_ULL(32) - dram_hole_base);
+		if (ctx.ret_addr >= dram_hole_base)
+			ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
 	}
 
 	if (hash_enabled) {
 		/* Save some parentheses and grab ls-bit at the end. */
-		hashed_bit =	(ret_addr >> 12) ^
-				(ret_addr >> 18) ^
-				(ret_addr >> 21) ^
-				(ret_addr >> 30) ^
+		hashed_bit =	(ctx.ret_addr >> 12) ^
+				(ctx.ret_addr >> 18) ^
+				(ctx.ret_addr >> 21) ^
+				(ctx.ret_addr >> 30) ^
 				cs_id;
 
 		hashed_bit &= BIT(0);
 
-		if (hashed_bit != ((ret_addr >> intlv_addr_bit) & BIT(0)))
-			ret_addr ^= BIT(intlv_addr_bit);
+		if (hashed_bit != ((ctx.ret_addr >> intlv_addr_bit) & BIT(0)))
+			ctx.ret_addr ^= BIT(intlv_addr_bit);
 	}
 
 	/* Is calculated system address is above DRAM limit address? */
-	if (ret_addr > dram_limit_addr)
+	if (ctx.ret_addr > dram_limit_addr)
 		goto out_err;
 
-	*sys_addr = ret_addr;
+	*sys_addr = ctx.ret_addr;
 	return 0;
 
 out_err:
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 06/31] EDAC/amd64: Define Data Fabric operations
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (4 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 05/31] EDAC/amd64: Add context struct Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-30 17:19   ` Borislav Petkov
  2021-06-23 19:19 ` [PATCH v2 07/31] EDAC/amd64: Define functions for DramOffset Yazen Ghannam
                   ` (24 subsequent siblings)
  30 siblings, 1 reply; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Define a stub to hold operations for different Data Fabric versions.
This will be filled in following patches.

Define a function to set the appropriate operations. Use a return code
because future updates have checks that may fail.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-3-Yazen.Ghannam@amd.com

v1->v2:
* New in v2.

 drivers/edac/amd64_edac.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 62eca188458f..f769353ef7e4 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1085,6 +1085,21 @@ struct addr_ctx {
 	u8 inst_id;
 };
 
+struct data_fabric_ops {
+};
+
+struct data_fabric_ops df2_ops = {
+};
+
+struct data_fabric_ops *df_ops;
+
+static int set_df_ops(struct addr_ctx *ctx)
+{
+	df_ops = &df2_ops;
+
+	return 0;
+}
+
 static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
@@ -1110,6 +1125,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	ctx.nid = nid;
 	ctx.inst_id = umc;
 
+	if (set_df_ops(&ctx))
+		return -EINVAL;
+
 	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
 		goto out_err;
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 07/31] EDAC/amd64: Define functions for DramOffset
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (5 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 06/31] EDAC/amd64: Define Data Fabric operations Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-30 17:27   ` Borislav Petkov
  2021-06-23 19:19 ` [PATCH v2 08/31] EDAC/amd64: Define function to read DRAM address map registers Yazen Ghannam
                   ` (23 subsequent siblings)
  30 siblings, 1 reply; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Add helper functions to read the DramOffset register and to remove the
offset from the calculated address.

The helper functions will be expanded in future DF versions.

Rename the "base" variable to "map_num" to indicate that this is the
address map number. An address map is defined with a base and limit
value. The map_num variable is used to select the proper base and limit
registers to use for the address translation.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-4-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Add function to data_fabric_ops.

 drivers/edac/amd64_edac.c | 57 +++++++++++++++++++++++++++++----------
 1 file changed, 43 insertions(+), 14 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index f769353ef7e4..389f0621e885 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1079,16 +1079,26 @@ static struct df_reg df_regs[] = {
 	[SYS_FAB_ID_MASK]	=	{1, 0x208},
 };
 
+/* Use "reg_" prefix for raw register values. */
 struct addr_ctx {
 	u64 ret_addr;
+	u32 reg_dram_offset;
 	u16 nid;
 	u8 inst_id;
+	u8 map_num;
 };
 
 struct data_fabric_ops {
+	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
 };
 
+static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
+{
+	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
+}
+
 struct data_fabric_ops df2_ops = {
+	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
 };
 
 struct data_fabric_ops *df_ops;
@@ -1100,6 +1110,35 @@ static int set_df_ops(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int get_dram_offset_reg(struct addr_ctx *ctx)
+{
+	if (amd_df_indirect_read(ctx->nid, df_regs[DRAM_OFFSET],
+				 ctx->inst_id, &ctx->reg_dram_offset))
+		return -EINVAL;
+
+	return 0;
+}
+
+static int remove_dram_offset(struct addr_ctx *ctx)
+{
+	if (get_dram_offset_reg(ctx))
+		return -EINVAL;
+
+	ctx->map_num = 0;
+
+	/* Remove HiAddrOffset from normalized address, if enabled: */
+	if (ctx->reg_dram_offset & BIT(0)) {
+		u64 hi_addr_offset = df_ops->get_hi_addr_offset(ctx);
+
+		if (ctx->ret_addr >= hi_addr_offset) {
+			ctx->ret_addr -= hi_addr_offset;
+			ctx->map_num = 1;
+		}
+	}
+
+	return 0;
+}
+
 static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
@@ -1109,7 +1148,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
 	u8 intlv_addr_sel, intlv_addr_bit;
 	u8 num_intlv_bits, hashed_bit;
-	u8 lgcy_mmio_hole_en, base = 0;
+	u8 lgcy_mmio_hole_en;
 	u8 cs_mask, cs_id = 0;
 	bool hash_enabled = false;
 
@@ -1128,21 +1167,11 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (set_df_ops(&ctx))
 		return -EINVAL;
 
-	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
+	if (remove_dram_offset(&ctx))
 		goto out_err;
 
-	/* Remove HiAddrOffset from normalized address, if enabled: */
-	if (tmp & BIT(0)) {
-		u64 hi_addr_offset = (tmp & GENMASK_ULL(31, 20)) << 8;
-
-		if (norm_addr >= hi_addr_offset) {
-			ctx.ret_addr -= hi_addr_offset;
-			base = 1;
-		}
-	}
-
 	reg = df_regs[DRAM_BASE_ADDR];
-	reg.offset += base * 8;
+	reg.offset += ctx.map_num * 8;
 	if (amd_df_indirect_read(nid, reg, umc, &tmp))
 		goto out_err;
 
@@ -1166,7 +1195,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	}
 
 	reg = df_regs[DRAM_LIMIT_ADDR];
-	reg.offset += base * 8;
+	reg.offset += ctx.map_num * 8;
 	if (amd_df_indirect_read(nid, reg, umc, &tmp))
 		goto out_err;
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 08/31] EDAC/amd64: Define function to read DRAM address map registers
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (6 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 07/31] EDAC/amd64: Define functions for DramOffset Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-30 17:29   ` Borislav Petkov
  2021-06-23 19:19 ` [PATCH v2 09/31] EDAC/amd64: Define function to find interleaving mode Yazen Ghannam
                   ` (22 subsequent siblings)
  30 siblings, 1 reply; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move the reading of the base and limit registers into a helper function.
Save the raw values in the context struct as they will be parsed later.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-5-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 58 +++++++++++++++++++++++----------------
 1 file changed, 34 insertions(+), 24 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 389f0621e885..867ab32caab2 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1083,6 +1083,8 @@ static struct df_reg df_regs[] = {
 struct addr_ctx {
 	u64 ret_addr;
 	u32 reg_dram_offset;
+	u32 reg_base_addr;
+	u32 reg_limit_addr;
 	u16 nid;
 	u8 inst_id;
 	u8 map_num;
@@ -1139,6 +1141,30 @@ static int remove_dram_offset(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int get_dram_addr_map(struct addr_ctx *ctx)
+{
+	struct df_reg reg = df_regs[DRAM_BASE_ADDR];
+
+	reg.offset += ctx->map_num * 8;
+
+	if (amd_df_indirect_read(ctx->nid, reg, ctx->inst_id, &ctx->reg_base_addr))
+		return -EINVAL;
+
+	/* Check if address range is valid. */
+	if (!(ctx->reg_base_addr & BIT(0))) {
+		pr_debug("Invalid DramBaseAddress range: 0x%x.\n", ctx->reg_base_addr);
+		return -EINVAL;
+	}
+
+	reg = df_regs[DRAM_LIMIT_ADDR];
+	reg.offset += ctx->map_num * 8;
+
+	if (amd_df_indirect_read(ctx->nid, reg, ctx->inst_id, &ctx->reg_limit_addr))
+		return -EINVAL;
+
+	return 0;
+}
+
 static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
 {
 	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
@@ -1152,8 +1178,6 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	u8 cs_mask, cs_id = 0;
 	bool hash_enabled = false;
 
-	struct df_reg reg;
-
 	struct addr_ctx ctx;
 
 	memset(&ctx, 0, sizeof(ctx));
@@ -1170,22 +1194,13 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (remove_dram_offset(&ctx))
 		goto out_err;
 
-	reg = df_regs[DRAM_BASE_ADDR];
-	reg.offset += ctx.map_num * 8;
-	if (amd_df_indirect_read(nid, reg, umc, &tmp))
-		goto out_err;
-
-	/* Check if address range is valid. */
-	if (!(tmp & BIT(0))) {
-		pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
-			__func__, tmp);
+	if (get_dram_addr_map(&ctx))
 		goto out_err;
-	}
 
-	lgcy_mmio_hole_en = tmp & BIT(1);
-	intlv_num_chan	  = (tmp >> 4) & 0xF;
-	intlv_addr_sel	  = (tmp >> 8) & 0x7;
-	dram_base_addr	  = (tmp & GENMASK_ULL(31, 12)) << 16;
+	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
+	intlv_num_chan	  = (ctx.reg_base_addr >> 4) & 0xF;
+	intlv_addr_sel	  = (ctx.reg_base_addr >> 8) & 0x7;
+	dram_base_addr	  = (ctx.reg_base_addr & GENMASK_ULL(31, 12)) << 16;
 
 	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
 	if (intlv_addr_sel > 3) {
@@ -1194,14 +1209,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		goto out_err;
 	}
 
-	reg = df_regs[DRAM_LIMIT_ADDR];
-	reg.offset += ctx.map_num * 8;
-	if (amd_df_indirect_read(nid, reg, umc, &tmp))
-		goto out_err;
-
-	intlv_num_sockets = (tmp >> 8) & 0x1;
-	intlv_num_dies	  = (tmp >> 10) & 0x3;
-	dram_limit_addr	  = ((tmp & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
+	intlv_num_sockets = (ctx.reg_limit_addr >> 8) & 0x1;
+	intlv_num_dies	  = (ctx.reg_limit_addr >> 10) & 0x3;
+	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
 
 	intlv_addr_bit = intlv_addr_sel + 8;
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 09/31] EDAC/amd64: Define function to find interleaving mode
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (7 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 08/31] EDAC/amd64: Define function to read DRAM address map registers Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-30 17:33   ` Borislav Petkov
  2021-06-23 19:19 ` [PATCH v2 10/31] EDAC/amd64: Define function to denormalize address Yazen Ghannam
                   ` (21 subsequent siblings)
  30 siblings, 1 reply; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Define a helper function to find the interleaving mode. Define a
DF2-specific function now. Future DF versions will have their own
functions.

Use an enumeration for the interleaving modes to give a human-readable
value. Save the interleaving mode in the context struct, since this will
be used in multiple functions.

Multiple interleaving modes support hashing, so save a boolean in the
context struct to check if hashing is enabled.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-6-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Add new function to data_fabric_ops.

 drivers/edac/amd64_edac.c | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 867ab32caab2..93a4ce9eb735 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1079,8 +1079,16 @@ static struct df_reg df_regs[] = {
 	[SYS_FAB_ID_MASK]	=	{1, 0x208},
 };
 
+/* These are mapped 1:1 to the hardware values. Special cases are set at > 0x20. */
+enum intlv_modes {
+	NONE		= 0x00,
+	NOHASH_2CH	= 0x01,
+	DF2_HASH_2CH	= 0x21,
+};
+
 /* Use "reg_" prefix for raw register values. */
 struct addr_ctx {
+	enum intlv_modes intlv_mode;
 	u64 ret_addr;
 	u32 reg_dram_offset;
 	u32 reg_base_addr;
@@ -1088,10 +1096,12 @@ struct addr_ctx {
 	u16 nid;
 	u8 inst_id;
 	u8 map_num;
+	bool hash_enabled;
 };
 
 struct data_fabric_ops {
 	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
+	int (*get_intlv_mode)(struct addr_ctx *ctx);
 };
 
 static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
@@ -1099,8 +1109,26 @@ static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
 	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
 }
 
+static int get_intlv_mode_df2(struct addr_ctx *ctx)
+{
+	ctx->intlv_mode = (ctx->reg_base_addr >> 4) & 0xF;
+
+	if (ctx->intlv_mode == 8) {
+		ctx->intlv_mode = DF2_HASH_2CH;
+		ctx->hash_enabled = true;
+	}
+
+	if (ctx->intlv_mode != NONE &&
+	    ctx->intlv_mode != NOHASH_2CH &&
+	    ctx->intlv_mode != DF2_HASH_2CH)
+		return -EINVAL;
+
+	return 0;
+}
+
 struct data_fabric_ops df2_ops = {
 	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
+	.get_intlv_mode			=	&get_intlv_mode_df2,
 };
 
 struct data_fabric_ops *df_ops;
@@ -1176,7 +1204,6 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	u8 num_intlv_bits, hashed_bit;
 	u8 lgcy_mmio_hole_en;
 	u8 cs_mask, cs_id = 0;
-	bool hash_enabled = false;
 
 	struct addr_ctx ctx;
 
@@ -1197,6 +1224,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (get_dram_addr_map(&ctx))
 		goto out_err;
 
+	if (df_ops->get_intlv_mode(&ctx))
+		goto out_err;
+
 	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
 	intlv_num_chan	  = (ctx.reg_base_addr >> 4) & 0xF;
 	intlv_addr_sel	  = (ctx.reg_base_addr >> 8) & 0x7;
@@ -1224,7 +1254,6 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	case 7:	intlv_num_chan = 4; break;
 
 	case 8: intlv_num_chan = 1;
-		hash_enabled = true;
 		break;
 	default:
 		pr_err("%s: Invalid number of interleaved channels %d.\n",
@@ -1325,7 +1354,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 			ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
 	}
 
-	if (hash_enabled) {
+	if (ctx.hash_enabled) {
 		/* Save some parentheses and grab ls-bit at the end. */
 		hashed_bit =	(ctx.ret_addr >> 12) ^
 				(ctx.ret_addr >> 18) ^
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 10/31] EDAC/amd64: Define function to denormalize address
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (8 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 09/31] EDAC/amd64: Define function to find interleaving mode Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 11/31] EDAC/amd64: Define function to add DRAM base and hole Yazen Ghannam
                   ` (20 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move the address denormalization into a separate helper function. This
will be further refactored in later patches.

Add the interleave address bit and the CS ID to the context struct.
These values will be used by multiple functions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-7-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 128 ++++++++++++++++++++++----------------
 1 file changed, 73 insertions(+), 55 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 93a4ce9eb735..ebb82b5e155b 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1096,6 +1096,8 @@ struct addr_ctx {
 	u16 nid;
 	u8 inst_id;
 	u8 map_num;
+	u8 intlv_addr_bit;
+	u8 cs_id;
 	bool hash_enabled;
 };
 
@@ -1193,57 +1195,26 @@ static int get_dram_addr_map(struct addr_ctx *ctx)
 	return 0;
 }
 
-static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+static int denormalize_addr(struct addr_ctx *ctx)
 {
-	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
 	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
-	u8 intlv_addr_sel, intlv_addr_bit;
-	u8 num_intlv_bits, hashed_bit;
-	u8 lgcy_mmio_hole_en;
-	u8 cs_mask, cs_id = 0;
-
-	struct addr_ctx ctx;
-
-	memset(&ctx, 0, sizeof(ctx));
-
-	/* We start from the normalized address */
-	ctx.ret_addr = norm_addr;
-
-	ctx.nid = nid;
-	ctx.inst_id = umc;
-
-	if (set_df_ops(&ctx))
-		return -EINVAL;
-
-	if (remove_dram_offset(&ctx))
-		goto out_err;
-
-	if (get_dram_addr_map(&ctx))
-		goto out_err;
-
-	if (df_ops->get_intlv_mode(&ctx))
-		goto out_err;
-
-	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
-	intlv_num_chan	  = (ctx.reg_base_addr >> 4) & 0xF;
-	intlv_addr_sel	  = (ctx.reg_base_addr >> 8) & 0x7;
-	dram_base_addr	  = (ctx.reg_base_addr & GENMASK_ULL(31, 12)) << 16;
+	u8 intlv_addr_sel = (ctx->reg_base_addr >> 8) & 0x7;
+	u8 num_intlv_bits, cs_mask = 0;
 
 	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
 	if (intlv_addr_sel > 3) {
 		pr_err("%s: Invalid interleave address select %d.\n",
 			__func__, intlv_addr_sel);
-		goto out_err;
+		return -EINVAL;
 	}
 
-	intlv_num_sockets = (ctx.reg_limit_addr >> 8) & 0x1;
-	intlv_num_dies	  = (ctx.reg_limit_addr >> 10) & 0x3;
-	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
+	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
+	intlv_num_dies	  = (ctx->reg_limit_addr >> 10) & 0x3;
 
-	intlv_addr_bit = intlv_addr_sel + 8;
+	ctx->intlv_addr_bit = intlv_addr_sel + 8;
 
 	/* Re-use intlv_num_chan by setting it equal to log2(#channels) */
 	switch (intlv_num_chan) {
@@ -1258,7 +1229,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	default:
 		pr_err("%s: Invalid number of interleaved channels %d.\n",
 			__func__, intlv_num_chan);
-		goto out_err;
+		return -EINVAL;
 	}
 
 	num_intlv_bits = intlv_num_chan;
@@ -1266,7 +1237,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (intlv_num_dies > 2) {
 		pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
 			__func__, intlv_num_dies);
-		goto out_err;
+		return -EINVAL;
 	}
 
 	num_intlv_bits += intlv_num_dies;
@@ -1278,7 +1249,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (num_intlv_bits > 4) {
 		pr_err("%s: Invalid interleave bits %d.\n",
 			__func__, num_intlv_bits);
-		goto out_err;
+		return -EINVAL;
 	}
 
 	if (num_intlv_bits > 0) {
@@ -1291,8 +1262,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		 * umc/channel# as instance id of the coherent slave
 		 * for FICAA.
 		 */
-		if (amd_df_indirect_read(nid, df_regs[FAB_BLK_INST_INFO_3], umc, &tmp))
-			goto out_err;
+		if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3],
+					 ctx->inst_id, &tmp))
+			return -EINVAL;
 
 		cs_fabric_id = (tmp >> 8) & 0xFF;
 		die_id_bit   = 0;
@@ -1301,14 +1273,15 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		if (intlv_num_chan) {
 			die_id_bit = intlv_num_chan;
 			cs_mask	   = (1 << die_id_bit) - 1;
-			cs_id	   = cs_fabric_id & cs_mask;
+			ctx->cs_id = cs_fabric_id & cs_mask;
 		}
 
 		sock_id_bit = die_id_bit;
 
 		if (intlv_num_dies || intlv_num_sockets)
-			if (amd_df_indirect_read(nid, df_regs[SYS_FAB_ID_MASK], umc, &tmp))
-				goto out_err;
+			if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
+						 ctx->inst_id, &tmp))
+				return -EINVAL;
 
 		/* If interleaved over more than 1 die. */
 		if (intlv_num_dies) {
@@ -1316,7 +1289,8 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 			die_id_shift = (tmp >> 24) & 0xF;
 			die_id_mask  = (tmp >> 8) & 0xFF;
 
-			cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+			ctx->cs_id |= ((cs_fabric_id & die_id_mask)
+					>> die_id_shift) << die_id_bit;
 		}
 
 		/* If interleaved over more than 1 socket. */
@@ -1324,7 +1298,8 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 			socket_id_shift	= (tmp >> 28) & 0xF;
 			socket_id_mask	= (tmp >> 16) & 0xFF;
 
-			cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
+			ctx->cs_id |= ((cs_fabric_id & socket_id_mask)
+					>> socket_id_shift) << sock_id_bit;
 		}
 
 		/*
@@ -1335,12 +1310,55 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
 		 * there are (where "I" starts).
 		 */
-		temp_addr_y = ctx.ret_addr & GENMASK_ULL(intlv_addr_bit - 1, 0);
-		temp_addr_i = (cs_id << intlv_addr_bit);
-		temp_addr_x = (ctx.ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
-		ctx.ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
+		temp_addr_y = ctx->ret_addr & GENMASK_ULL(ctx->intlv_addr_bit - 1, 0);
+		temp_addr_i = (ctx->cs_id << ctx->intlv_addr_bit);
+		temp_addr_x = (ctx->ret_addr & GENMASK_ULL(63, ctx->intlv_addr_bit))
+			       << num_intlv_bits;
+		ctx->ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
 	}
 
+	return 0;
+}
+
+static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+{
+	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
+
+	u32 tmp;
+
+	u8 hashed_bit;
+	u8 lgcy_mmio_hole_en;
+
+	struct addr_ctx ctx;
+
+	memset(&ctx, 0, sizeof(ctx));
+
+	/* We start from the normalized address */
+	ctx.ret_addr = norm_addr;
+
+	ctx.nid = nid;
+	ctx.inst_id = umc;
+
+	if (set_df_ops(&ctx))
+		return -EINVAL;
+
+	if (remove_dram_offset(&ctx))
+		return -EINVAL;
+
+	if (get_dram_addr_map(&ctx))
+		goto out_err;
+
+	if (df_ops->get_intlv_mode(&ctx))
+		goto out_err;
+
+	if (denormalize_addr(&ctx))
+		goto out_err;
+
+	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
+	dram_base_addr	  = (ctx.reg_base_addr & GENMASK_ULL(31, 12)) << 16;
+
+	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
+
 	/* Add dram base address */
 	ctx.ret_addr += dram_base_addr;
 
@@ -1360,12 +1378,12 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 				(ctx.ret_addr >> 18) ^
 				(ctx.ret_addr >> 21) ^
 				(ctx.ret_addr >> 30) ^
-				cs_id;
+				ctx.cs_id;
 
 		hashed_bit &= BIT(0);
 
-		if (hashed_bit != ((ctx.ret_addr >> intlv_addr_bit) & BIT(0)))
-			ctx.ret_addr ^= BIT(intlv_addr_bit);
+		if (hashed_bit != ((ctx.ret_addr >> ctx.intlv_addr_bit) & BIT(0)))
+			ctx.ret_addr ^= BIT(ctx.intlv_addr_bit);
 	}
 
 	/* Is calculated system address is above DRAM limit address? */
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 11/31] EDAC/amd64: Define function to add DRAM base and hole
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (9 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 10/31] EDAC/amd64: Define function to denormalize address Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 12/31] EDAC/amd64: Define function to dehash address Yazen Ghannam
                   ` (19 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move adding of DRAM base and hole into a separate helper function.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-8-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 45 ++++++++++++++++++++++-----------------
 1 file changed, 26 insertions(+), 19 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index ebb82b5e155b..68c4aecd5e7a 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1320,14 +1320,34 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	return 0;
 }
 
-static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+static int add_base_and_hole(struct addr_ctx *ctx)
 {
-	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
+	u64 dram_base_addr = (ctx->reg_base_addr & GENMASK_ULL(31, 12)) << 16;
 
-	u32 tmp;
+	/* Add dram base address */
+	ctx->ret_addr += dram_base_addr;
+
+	/* If legacy MMIO hole enabled */
+	if (ctx->reg_base_addr & BIT(1)) {
+		u32 dram_hole_base;
+
+		if (amd_df_indirect_read(0, df_regs[DRAM_HOLE_CTL],
+					 DF_BROADCAST, &dram_hole_base))
+			return -EINVAL;
+
+		dram_hole_base &= GENMASK(31, 24);
+		if (ctx->ret_addr >= dram_hole_base)
+			ctx->ret_addr += (BIT_ULL(32) - dram_hole_base);
+	}
+
+	return 0;
+}
+
+static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+{
+	u64 dram_limit_addr;
 
 	u8 hashed_bit;
-	u8 lgcy_mmio_hole_en;
 
 	struct addr_ctx ctx;
 
@@ -1354,23 +1374,10 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (denormalize_addr(&ctx))
 		goto out_err;
 
-	lgcy_mmio_hole_en = ctx.reg_base_addr & BIT(1);
-	dram_base_addr	  = (ctx.reg_base_addr & GENMASK_ULL(31, 12)) << 16;
-
 	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
 
-	/* Add dram base address */
-	ctx.ret_addr += dram_base_addr;
-
-	/* If legacy MMIO hole enabled */
-	if (lgcy_mmio_hole_en) {
-		if (amd_df_indirect_read(nid, df_regs[DRAM_HOLE_CTL], umc, &tmp))
-			goto out_err;
-
-		dram_hole_base = tmp & GENMASK(31, 24);
-		if (ctx.ret_addr >= dram_hole_base)
-			ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
-	}
+	if (add_base_and_hole(&ctx))
+		goto out_err;
 
 	if (ctx.hash_enabled) {
 		/* Save some parentheses and grab ls-bit at the end. */
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 12/31] EDAC/amd64: Define function to dehash address
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (10 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 11/31] EDAC/amd64: Define function to add DRAM base and hole Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 13/31] EDAC/amd64: Define function to check DRAM limit address Yazen Ghannam
                   ` (18 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move the dehashing code into a separate helper function. Define a
DF2-specific function for the current code. Specific helper functions
will be added for future DF versions.

The dehashing code is tied to interleaving mode rather than Data Fabric
version, so save the function pointer in the ctx struct. The name
include "df2" because this mode only exists on DF2.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-9-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Add new function pointer in ctx struct.

 drivers/edac/amd64_edac.c | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 68c4aecd5e7a..46b0279a2e12 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1098,7 +1098,7 @@ struct addr_ctx {
 	u8 map_num;
 	u8 intlv_addr_bit;
 	u8 cs_id;
-	bool hash_enabled;
+	int (*dehash_addr)(struct addr_ctx *ctx);
 };
 
 struct data_fabric_ops {
@@ -1111,13 +1111,29 @@ static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
 	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
 }
 
+static int dehash_addr_df2(struct addr_ctx *ctx)
+{
+	u8 hashed_bit =	(ctx->ret_addr >> 12) ^
+			(ctx->ret_addr >> 18) ^
+			(ctx->ret_addr >> 21) ^
+			(ctx->ret_addr >> 30) ^
+			ctx->cs_id;
+
+	hashed_bit &= BIT(0);
+
+	if (hashed_bit != ((ctx->ret_addr >> ctx->intlv_addr_bit) & BIT(0)))
+		ctx->ret_addr ^= BIT(ctx->intlv_addr_bit);
+
+	return 0;
+}
+
 static int get_intlv_mode_df2(struct addr_ctx *ctx)
 {
 	ctx->intlv_mode = (ctx->reg_base_addr >> 4) & 0xF;
 
 	if (ctx->intlv_mode == 8) {
 		ctx->intlv_mode = DF2_HASH_2CH;
-		ctx->hash_enabled = true;
+		ctx->dehash_addr = &dehash_addr_df2;
 	}
 
 	if (ctx->intlv_mode != NONE &&
@@ -1347,8 +1363,6 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 {
 	u64 dram_limit_addr;
 
-	u8 hashed_bit;
-
 	struct addr_ctx ctx;
 
 	memset(&ctx, 0, sizeof(ctx));
@@ -1379,19 +1393,8 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (add_base_and_hole(&ctx))
 		goto out_err;
 
-	if (ctx.hash_enabled) {
-		/* Save some parentheses and grab ls-bit at the end. */
-		hashed_bit =	(ctx.ret_addr >> 12) ^
-				(ctx.ret_addr >> 18) ^
-				(ctx.ret_addr >> 21) ^
-				(ctx.ret_addr >> 30) ^
-				ctx.cs_id;
-
-		hashed_bit &= BIT(0);
-
-		if (hashed_bit != ((ctx.ret_addr >> ctx.intlv_addr_bit) & BIT(0)))
-			ctx.ret_addr ^= BIT(ctx.intlv_addr_bit);
-	}
+	if (ctx.dehash_addr && ctx.dehash_addr(&ctx))
+		goto out_err;
 
 	/* Is calculated system address is above DRAM limit address? */
 	if (ctx.ret_addr > dram_limit_addr)
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 13/31] EDAC/amd64: Define function to check DRAM limit address
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (11 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 12/31] EDAC/amd64: Define function to dehash address Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 14/31] EDAC/amd64: Remove goto statements Yazen Ghannam
                   ` (17 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move the DRAM limit check into a separate helper function.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-10-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 46b0279a2e12..de80a224ef12 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1359,10 +1359,20 @@ static int add_base_and_hole(struct addr_ctx *ctx)
 	return 0;
 }
 
-static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+static int addr_over_limit(struct addr_ctx *ctx)
 {
-	u64 dram_limit_addr;
+	u64 dram_limit_addr  = ((ctx->reg_limit_addr & GENMASK_ULL(31, 12)) << 16)
+					| GENMASK_ULL(27, 0);
+
+	/* Is calculated system address above DRAM limit address? */
+	if (ctx->ret_addr > dram_limit_addr)
+		return -EINVAL;
 
+	return 0;
+}
+
+static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+{
 	struct addr_ctx ctx;
 
 	memset(&ctx, 0, sizeof(ctx));
@@ -1388,16 +1398,13 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (denormalize_addr(&ctx))
 		goto out_err;
 
-	dram_limit_addr	  = ((ctx.reg_limit_addr & GENMASK_ULL(31, 12)) << 16) | GENMASK_ULL(27, 0);
-
 	if (add_base_and_hole(&ctx))
 		goto out_err;
 
 	if (ctx.dehash_addr && ctx.dehash_addr(&ctx))
 		goto out_err;
 
-	/* Is calculated system address is above DRAM limit address? */
-	if (ctx.ret_addr > dram_limit_addr)
+	if (addr_over_limit(&ctx))
 		goto out_err;
 
 	*sys_addr = ctx.ret_addr;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 14/31] EDAC/amd64: Remove goto statements
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (12 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 13/31] EDAC/amd64: Define function to check DRAM limit address Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 15/31] EDAC/amd64: Simplify function parameters Yazen Ghannam
                   ` (16 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

...and just return error codes directly.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-11-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index de80a224ef12..3f1ff5adce4a 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1390,28 +1390,25 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 		return -EINVAL;
 
 	if (get_dram_addr_map(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (df_ops->get_intlv_mode(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (denormalize_addr(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (add_base_and_hole(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (ctx.dehash_addr && ctx.dehash_addr(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	if (addr_over_limit(&ctx))
-		goto out_err;
+		return -EINVAL;
 
 	*sys_addr = ctx.ret_addr;
 	return 0;
-
-out_err:
-	return -EINVAL;
 }
 
 static int get_channel_from_ecc_syndrome(struct mem_ctl_info *, u16);
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 15/31] EDAC/amd64: Simplify function parameters
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (13 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 14/31] EDAC/amd64: Remove goto statements Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 16/31] EDAC/amd64: Define function to get Interleave Address Bit Yazen Ghannam
                   ` (15 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Use a single address parameter for input and result to reduce the number
of parameters.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-12-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 3f1ff5adce4a..496ca863f45a 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1371,14 +1371,14 @@ static int addr_over_limit(struct addr_ctx *ctx)
 	return 0;
 }
 
-static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
+static int umc_normaddr_to_sysaddr(u64 *addr, u16 nid, u8 umc)
 {
 	struct addr_ctx ctx;
 
 	memset(&ctx, 0, sizeof(ctx));
 
 	/* We start from the normalized address */
-	ctx.ret_addr = norm_addr;
+	ctx.ret_addr = *addr;
 
 	ctx.nid = nid;
 	ctx.inst_id = umc;
@@ -1407,7 +1407,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
 	if (addr_over_limit(&ctx))
 		return -EINVAL;
 
-	*sys_addr = ctx.ret_addr;
+	*addr = ctx.ret_addr;
 	return 0;
 }
 
@@ -3314,7 +3314,7 @@ static void decode_umc_error(int node_id, struct mce *m)
 	struct mem_ctl_info *mci;
 	struct amd64_pvt *pvt;
 	struct err_info err;
-	u64 sys_addr;
+	u64 sys_addr = m->addr;
 
 	mci = edac_mc_find(node_id);
 	if (!mci)
@@ -3345,7 +3345,7 @@ static void decode_umc_error(int node_id, struct mce *m)
 
 	err.csrow = m->synd & 0x7;
 
-	if (umc_normaddr_to_sysaddr(m->addr, pvt->mc_node_id, err.channel, &sys_addr)) {
+	if (umc_normaddr_to_sysaddr(&sys_addr, pvt->mc_node_id, err.channel)) {
 		err.err_code = ERR_NORM_ADDR;
 		goto log_error;
 	}
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 16/31] EDAC/amd64: Define function to get Interleave Address Bit
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (14 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 15/31] EDAC/amd64: Simplify function parameters Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 17/31] EDAC/amd64: Skip denormalization if no interleaving Yazen Ghannam
                   ` (14 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move code to find the interleave address bit into a separate helper
function.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-13-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 496ca863f45a..233f8cec4fe8 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1211,27 +1211,35 @@ static int get_dram_addr_map(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int get_intlv_addr_bit(struct addr_ctx *ctx)
+{
+	u8 intlv_addr_sel = (ctx->reg_base_addr >> 8) & 0x7;
+
+	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
+	if (intlv_addr_sel > 3) {
+		pr_debug("Invalid interleave address select %d.\n", intlv_addr_sel);
+		return -EINVAL;
+	}
+
+	ctx->intlv_addr_bit = intlv_addr_sel + 8;
+
+	return 0;
+}
+
 static int denormalize_addr(struct addr_ctx *ctx)
 {
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
 	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
-	u8 intlv_addr_sel = (ctx->reg_base_addr >> 8) & 0x7;
 	u8 num_intlv_bits, cs_mask = 0;
 
-	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
-	if (intlv_addr_sel > 3) {
-		pr_err("%s: Invalid interleave address select %d.\n",
-			__func__, intlv_addr_sel);
+	if (get_intlv_addr_bit(ctx))
 		return -EINVAL;
-	}
 
 	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
 	intlv_num_dies	  = (ctx->reg_limit_addr >> 10) & 0x3;
 
-	ctx->intlv_addr_bit = intlv_addr_sel + 8;
-
 	/* Re-use intlv_num_chan by setting it equal to log2(#channels) */
 	switch (intlv_num_chan) {
 	case 0:	intlv_num_chan = 0; break;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 17/31] EDAC/amd64: Skip denormalization if no interleaving
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (15 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 16/31] EDAC/amd64: Define function to get Interleave Address Bit Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 18/31] EDAC/amd64: Define function to get number of interleaved channels Yazen Ghannam
                   ` (13 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Denormalization doesn't apply to the "no interleaving" mode, so return
early without error in this case.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-14-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 233f8cec4fe8..02dc34c13d65 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1234,6 +1234,10 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
 	u8 num_intlv_bits, cs_mask = 0;
 
+	/* Return early if no interleaving. */
+	if (ctx->intlv_mode == NONE)
+		return 0;
+
 	if (get_intlv_addr_bit(ctx))
 		return -EINVAL;
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 18/31] EDAC/amd64: Define function to get number of interleaved channels
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (16 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 17/31] EDAC/amd64: Skip denormalization if no interleaving Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 19/31] EDAC/amd64: Define function to get number of interleaved dies Yazen Ghannam
                   ` (12 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move number of interleaved channel calculation to a separate helper
function. Drop unused cases.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-15-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 42 +++++++++++++++++++++------------------
 1 file changed, 23 insertions(+), 19 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 02dc34c13d65..80192a05bb93 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1097,6 +1097,7 @@ struct addr_ctx {
 	u8 inst_id;
 	u8 map_num;
 	u8 intlv_addr_bit;
+	u8 intlv_num_chan;
 	u8 cs_id;
 	int (*dehash_addr)(struct addr_ctx *ctx);
 };
@@ -1226,12 +1227,29 @@ static int get_intlv_addr_bit(struct addr_ctx *ctx)
 	return 0;
 }
 
+static void get_intlv_num_chan(struct addr_ctx *ctx)
+{
+	/* Save the log2(# of channels). */
+	switch (ctx->intlv_mode) {
+	case NONE:
+		ctx->intlv_num_chan = 0;
+		break;
+	case NOHASH_2CH:
+	case DF2_HASH_2CH:
+		ctx->intlv_num_chan = 1;
+		break;
+	default:
+		/* Valid interleaving modes where checked earlier. */
+		break;
+	}
+}
+
 static int denormalize_addr(struct addr_ctx *ctx)
 {
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 intlv_num_dies, intlv_num_chan, intlv_num_sockets;
+	u8 intlv_num_dies, intlv_num_sockets;
 	u8 num_intlv_bits, cs_mask = 0;
 
 	/* Return early if no interleaving. */
@@ -1244,23 +1262,9 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
 	intlv_num_dies	  = (ctx->reg_limit_addr >> 10) & 0x3;
 
-	/* Re-use intlv_num_chan by setting it equal to log2(#channels) */
-	switch (intlv_num_chan) {
-	case 0:	intlv_num_chan = 0; break;
-	case 1: intlv_num_chan = 1; break;
-	case 3: intlv_num_chan = 2; break;
-	case 5:	intlv_num_chan = 3; break;
-	case 7:	intlv_num_chan = 4; break;
-
-	case 8: intlv_num_chan = 1;
-		break;
-	default:
-		pr_err("%s: Invalid number of interleaved channels %d.\n",
-			__func__, intlv_num_chan);
-		return -EINVAL;
-	}
+	get_intlv_num_chan(ctx);
 
-	num_intlv_bits = intlv_num_chan;
+	num_intlv_bits = ctx->intlv_num_chan;
 
 	if (intlv_num_dies > 2) {
 		pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
@@ -1298,8 +1302,8 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		die_id_bit   = 0;
 
 		/* If interleaved over more than 1 channel: */
-		if (intlv_num_chan) {
-			die_id_bit = intlv_num_chan;
+		if (ctx->intlv_num_chan) {
+			die_id_bit = ctx->intlv_num_chan;
 			cs_mask	   = (1 << die_id_bit) - 1;
 			ctx->cs_id = cs_fabric_id & cs_mask;
 		}
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 19/31] EDAC/amd64: Define function to get number of interleaved dies
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (17 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 18/31] EDAC/amd64: Define function to get number of interleaved channels Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 20/31] EDAC/amd64: Define function to get number of interleaved sockets Yazen Ghannam
                   ` (11 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move parsing of the number of interleaved dies to a separate helper
function. This will be expanded for future DF versions. Also, drop an
unneeded assert to match the reference code.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-16-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Add new function to data_fabric_ops.

 drivers/edac/amd64_edac.c | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 80192a05bb93..360fbecf0d8b 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1098,6 +1098,7 @@ struct addr_ctx {
 	u8 map_num;
 	u8 intlv_addr_bit;
 	u8 intlv_num_chan;
+	u8 intlv_num_dies;
 	u8 cs_id;
 	int (*dehash_addr)(struct addr_ctx *ctx);
 };
@@ -1105,6 +1106,7 @@ struct addr_ctx {
 struct data_fabric_ops {
 	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
 	int (*get_intlv_mode)(struct addr_ctx *ctx);
+	void (*get_intlv_num_dies)(struct addr_ctx *ctx);
 };
 
 static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
@@ -1145,9 +1147,15 @@ static int get_intlv_mode_df2(struct addr_ctx *ctx)
 	return 0;
 }
 
+static void get_intlv_num_dies_df2(struct addr_ctx *ctx)
+{
+	ctx->intlv_num_dies  = (ctx->reg_limit_addr >> 10) & 0x3;
+}
+
 struct data_fabric_ops df2_ops = {
 	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
 	.get_intlv_mode			=	&get_intlv_mode_df2,
+	.get_intlv_num_dies		=	&get_intlv_num_dies_df2,
 };
 
 struct data_fabric_ops *df_ops;
@@ -1249,7 +1257,7 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 intlv_num_dies, intlv_num_sockets;
+	u8 intlv_num_sockets;
 	u8 num_intlv_bits, cs_mask = 0;
 
 	/* Return early if no interleaving. */
@@ -1260,19 +1268,12 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		return -EINVAL;
 
 	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
-	intlv_num_dies	  = (ctx->reg_limit_addr >> 10) & 0x3;
 
 	get_intlv_num_chan(ctx);
+	df_ops->get_intlv_num_dies(ctx);
 
 	num_intlv_bits = ctx->intlv_num_chan;
-
-	if (intlv_num_dies > 2) {
-		pr_err("%s: Invalid number of interleaved nodes/dies %d.\n",
-			__func__, intlv_num_dies);
-		return -EINVAL;
-	}
-
-	num_intlv_bits += intlv_num_dies;
+	num_intlv_bits += ctx->intlv_num_dies;
 
 	/* Add a bit if sockets are interleaved. */
 	num_intlv_bits += intlv_num_sockets;
@@ -1310,14 +1311,14 @@ static int denormalize_addr(struct addr_ctx *ctx)
 
 		sock_id_bit = die_id_bit;
 
-		if (intlv_num_dies || intlv_num_sockets)
+		if (ctx->intlv_num_dies || intlv_num_sockets)
 			if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
 						 ctx->inst_id, &tmp))
 				return -EINVAL;
 
 		/* If interleaved over more than 1 die. */
-		if (intlv_num_dies) {
-			sock_id_bit  = die_id_bit + intlv_num_dies;
+		if (ctx->intlv_num_dies) {
+			sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
 			die_id_shift = (tmp >> 24) & 0xF;
 			die_id_mask  = (tmp >> 8) & 0xFF;
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 20/31] EDAC/amd64: Define function to get number of interleaved sockets
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (18 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 19/31] EDAC/amd64: Define function to get number of interleaved dies Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 21/31] EDAC/amd64: Remove unnecessary assert Yazen Ghannam
                   ` (10 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move parsing of the number of interleaved sockets to a separate helper
function. This will be expanded for future DF versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-17-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Add new function to data_fabric_ops.

 drivers/edac/amd64_edac.c | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 360fbecf0d8b..09cdc0466ae2 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1099,6 +1099,7 @@ struct addr_ctx {
 	u8 intlv_addr_bit;
 	u8 intlv_num_chan;
 	u8 intlv_num_dies;
+	u8 intlv_num_sockets;
 	u8 cs_id;
 	int (*dehash_addr)(struct addr_ctx *ctx);
 };
@@ -1107,6 +1108,7 @@ struct data_fabric_ops {
 	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
 	int (*get_intlv_mode)(struct addr_ctx *ctx);
 	void (*get_intlv_num_dies)(struct addr_ctx *ctx);
+	void (*get_intlv_num_sockets)(struct addr_ctx *ctx);
 };
 
 static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
@@ -1152,10 +1154,16 @@ static void get_intlv_num_dies_df2(struct addr_ctx *ctx)
 	ctx->intlv_num_dies  = (ctx->reg_limit_addr >> 10) & 0x3;
 }
 
+static void get_intlv_num_sockets_df2(struct addr_ctx *ctx)
+{
+	ctx->intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
+}
+
 struct data_fabric_ops df2_ops = {
 	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
 	.get_intlv_mode			=	&get_intlv_mode_df2,
 	.get_intlv_num_dies		=	&get_intlv_num_dies_df2,
+	.get_intlv_num_sockets		=	&get_intlv_num_sockets_df2,
 };
 
 struct data_fabric_ops *df_ops;
@@ -1257,7 +1265,6 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	u32 tmp;
 
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 intlv_num_sockets;
 	u8 num_intlv_bits, cs_mask = 0;
 
 	/* Return early if no interleaving. */
@@ -1267,16 +1274,13 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	if (get_intlv_addr_bit(ctx))
 		return -EINVAL;
 
-	intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
-
 	get_intlv_num_chan(ctx);
 	df_ops->get_intlv_num_dies(ctx);
+	df_ops->get_intlv_num_sockets(ctx);
 
 	num_intlv_bits = ctx->intlv_num_chan;
 	num_intlv_bits += ctx->intlv_num_dies;
-
-	/* Add a bit if sockets are interleaved. */
-	num_intlv_bits += intlv_num_sockets;
+	num_intlv_bits += ctx->intlv_num_sockets;
 
 	/* Assert num_intlv_bits <= 4 */
 	if (num_intlv_bits > 4) {
@@ -1311,7 +1315,7 @@ static int denormalize_addr(struct addr_ctx *ctx)
 
 		sock_id_bit = die_id_bit;
 
-		if (ctx->intlv_num_dies || intlv_num_sockets)
+		if (ctx->intlv_num_dies || ctx->intlv_num_sockets)
 			if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
 						 ctx->inst_id, &tmp))
 				return -EINVAL;
@@ -1327,7 +1331,7 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		}
 
 		/* If interleaved over more than 1 socket. */
-		if (intlv_num_sockets) {
+		if (ctx->intlv_num_sockets) {
 			socket_id_shift	= (tmp >> 28) & 0xF;
 			socket_id_mask	= (tmp >> 16) & 0xFF;
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 21/31] EDAC/amd64: Remove unnecessary assert
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (19 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 20/31] EDAC/amd64: Define function to get number of interleaved sockets Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 22/31] EDAC/amd64: Define function to make space for CS ID Yazen Ghannam
                   ` (9 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

It was removed in the reference code, so remove it here.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-18-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 09cdc0466ae2..e5c296b00f5c 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1282,13 +1282,6 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	num_intlv_bits += ctx->intlv_num_dies;
 	num_intlv_bits += ctx->intlv_num_sockets;
 
-	/* Assert num_intlv_bits <= 4 */
-	if (num_intlv_bits > 4) {
-		pr_err("%s: Invalid interleave bits %d.\n",
-			__func__, num_intlv_bits);
-		return -EINVAL;
-	}
-
 	if (num_intlv_bits > 0) {
 		u64 temp_addr_x, temp_addr_i, temp_addr_y;
 		u8 die_id_bit, sock_id_bit, cs_fabric_id;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 22/31] EDAC/amd64: Define function to make space for CS ID
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (20 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 21/31] EDAC/amd64: Remove unnecessary assert Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 23/31] EDAC/amd64: Define function to calculate " Yazen Ghannam
                   ` (8 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move code that makes a gap for the CS ID into a separate helper function.
The exact bits to use vary based on interleaving mode. New interleaving
modes in future DF versions will be added as new cases.

Also, introduce a helper function that does the bit manipulation to make
the gap. The current version of this function is "simple", and future
interleaving modes may reuse this or use a more advanced function.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-19-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Added new function pointer to ctx struct.

 drivers/edac/amd64_edac.c | 37 +++++++++++++++++++++++++++++--------
 1 file changed, 29 insertions(+), 8 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index e5c296b00f5c..b497af7c3561 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1102,6 +1102,7 @@ struct addr_ctx {
 	u8 intlv_num_sockets;
 	u8 cs_id;
 	int (*dehash_addr)(struct addr_ctx *ctx);
+	void (*make_space_for_cs_id)(struct addr_ctx *ctx);
 };
 
 struct data_fabric_ops {
@@ -1111,6 +1112,29 @@ struct data_fabric_ops {
 	void (*get_intlv_num_sockets)(struct addr_ctx *ctx);
 };
 
+static void expand_bits(u8 start_bit, u8 num_bits, u64 *value)
+{
+	u64 temp1, temp2;
+
+	if (start_bit == 0) {
+		*value <<= num_bits;
+		return;
+	}
+
+	temp1 = *value & GENMASK_ULL(start_bit - 1, 0);
+	temp2 = (*value & GENMASK_ULL(63, start_bit)) << num_bits;
+	*value = temp1 | temp2;
+}
+
+static void make_space_for_cs_id_simple(struct addr_ctx *ctx)
+{
+	u8 num_intlv_bits = ctx->intlv_num_chan;
+
+	num_intlv_bits += ctx->intlv_num_dies;
+	num_intlv_bits += ctx->intlv_num_sockets;
+	expand_bits(ctx->intlv_addr_bit, num_intlv_bits, &ctx->ret_addr);
+}
+
 static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
 {
 	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
@@ -1141,6 +1165,8 @@ static int get_intlv_mode_df2(struct addr_ctx *ctx)
 		ctx->dehash_addr = &dehash_addr_df2;
 	}
 
+	ctx->make_space_for_cs_id = &make_space_for_cs_id_simple;
+
 	if (ctx->intlv_mode != NONE &&
 	    ctx->intlv_mode != NOHASH_2CH &&
 	    ctx->intlv_mode != DF2_HASH_2CH)
@@ -1278,13 +1304,11 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	df_ops->get_intlv_num_dies(ctx);
 	df_ops->get_intlv_num_sockets(ctx);
 
-	num_intlv_bits = ctx->intlv_num_chan;
-	num_intlv_bits += ctx->intlv_num_dies;
-	num_intlv_bits += ctx->intlv_num_sockets;
+	ctx->make_space_for_cs_id(ctx);
 
 	if (num_intlv_bits > 0) {
-		u64 temp_addr_x, temp_addr_i, temp_addr_y;
 		u8 die_id_bit, sock_id_bit, cs_fabric_id;
+		u64 temp_addr_i;
 
 		/*
 		 * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
@@ -1340,11 +1364,8 @@ static int denormalize_addr(struct addr_ctx *ctx)
 		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
 		 * there are (where "I" starts).
 		 */
-		temp_addr_y = ctx->ret_addr & GENMASK_ULL(ctx->intlv_addr_bit - 1, 0);
 		temp_addr_i = (ctx->cs_id << ctx->intlv_addr_bit);
-		temp_addr_x = (ctx->ret_addr & GENMASK_ULL(63, ctx->intlv_addr_bit))
-			       << num_intlv_bits;
-		ctx->ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
+		ctx->ret_addr |= temp_addr_i;
 	}
 
 	return 0;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 23/31] EDAC/amd64: Define function to calculate CS ID
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (21 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 22/31] EDAC/amd64: Define function to make space for CS ID Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 24/31] EDAC/amd64: Define function to insert CS ID into address Yazen Ghannam
                   ` (7 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move code used to calculate the CS ID into a separate helper function.

Drop redundant code comment about reading DF register.

The "num_intlv_bits" variable is left uninitialized as it will be removed
in a later patch.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-20-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 98 +++++++++++++++++++--------------------
 1 file changed, 48 insertions(+), 50 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index b497af7c3561..0270bf4f1f90 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1286,12 +1286,54 @@ static void get_intlv_num_chan(struct addr_ctx *ctx)
 	}
 }
 
-static int denormalize_addr(struct addr_ctx *ctx)
+static int calculate_cs_id(struct addr_ctx *ctx)
 {
+	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
+	u8 die_id_bit, sock_id_bit, cs_fabric_id, cs_mask = 0;
 	u32 tmp;
 
-	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 num_intlv_bits, cs_mask = 0;
+	if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3], ctx->inst_id, &tmp))
+		return -EINVAL;
+
+	cs_fabric_id = (tmp >> 8) & 0xFF;
+	die_id_bit   = 0;
+
+	/* If interleaved over more than 1 channel: */
+	if (ctx->intlv_num_chan) {
+		die_id_bit = ctx->intlv_num_chan;
+		cs_mask	   = (1 << die_id_bit) - 1;
+		ctx->cs_id = cs_fabric_id & cs_mask;
+	}
+
+	sock_id_bit = die_id_bit;
+
+	if (ctx->intlv_num_dies || ctx->intlv_num_sockets)
+		if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK], ctx->inst_id, &tmp))
+			return -EINVAL;
+
+	/* If interleaved over more than 1 die: */
+	if (ctx->intlv_num_dies) {
+		sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
+		die_id_shift = (tmp >> 24) & 0xF;
+		die_id_mask  = (tmp >> 8) & 0xFF;
+
+		ctx->cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+	}
+
+	/* If interleaved over more than 1 socket: */
+	if (ctx->intlv_num_sockets) {
+		socket_id_shift	= (tmp >> 28) & 0xF;
+		socket_id_mask	= (tmp >> 16) & 0xFF;
+
+		ctx->cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
+	}
+
+	return 0;
+}
+
+static int denormalize_addr(struct addr_ctx *ctx)
+{
+	u8 num_intlv_bits;
 
 	/* Return early if no interleaving. */
 	if (ctx->intlv_mode == NONE)
@@ -1306,56 +1348,12 @@ static int denormalize_addr(struct addr_ctx *ctx)
 
 	ctx->make_space_for_cs_id(ctx);
 
+	if (calculate_cs_id(ctx))
+		return -EINVAL;
+
 	if (num_intlv_bits > 0) {
-		u8 die_id_bit, sock_id_bit, cs_fabric_id;
 		u64 temp_addr_i;
 
-		/*
-		 * Read FabricBlockInstanceInformation3_CS[BlockFabricID].
-		 * This is the fabric id for this coherent slave. Use
-		 * umc/channel# as instance id of the coherent slave
-		 * for FICAA.
-		 */
-		if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3],
-					 ctx->inst_id, &tmp))
-			return -EINVAL;
-
-		cs_fabric_id = (tmp >> 8) & 0xFF;
-		die_id_bit   = 0;
-
-		/* If interleaved over more than 1 channel: */
-		if (ctx->intlv_num_chan) {
-			die_id_bit = ctx->intlv_num_chan;
-			cs_mask	   = (1 << die_id_bit) - 1;
-			ctx->cs_id = cs_fabric_id & cs_mask;
-		}
-
-		sock_id_bit = die_id_bit;
-
-		if (ctx->intlv_num_dies || ctx->intlv_num_sockets)
-			if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK],
-						 ctx->inst_id, &tmp))
-				return -EINVAL;
-
-		/* If interleaved over more than 1 die. */
-		if (ctx->intlv_num_dies) {
-			sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
-			die_id_shift = (tmp >> 24) & 0xF;
-			die_id_mask  = (tmp >> 8) & 0xFF;
-
-			ctx->cs_id |= ((cs_fabric_id & die_id_mask)
-					>> die_id_shift) << die_id_bit;
-		}
-
-		/* If interleaved over more than 1 socket. */
-		if (ctx->intlv_num_sockets) {
-			socket_id_shift	= (tmp >> 28) & 0xF;
-			socket_id_mask	= (tmp >> 16) & 0xFF;
-
-			ctx->cs_id |= ((cs_fabric_id & socket_id_mask)
-					>> socket_id_shift) << sock_id_bit;
-		}
-
 		/*
 		 * The pre-interleaved address consists of XXXXXXIIIYYYYY
 		 * where III is the ID for this CS, and XXXXXXYYYYY are the
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 24/31] EDAC/amd64: Define function to insert CS ID into address
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (22 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 23/31] EDAC/amd64: Define function to calculate " Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 25/31] EDAC/amd64: Define function to get CS Fabric ID Yazen Ghannam
                   ` (6 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move the code that inserts the CS ID into the address into a separate
helper function. This will be expanded for future DF versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-21-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Added function pointer to ctx struct.

 drivers/edac/amd64_edac.c | 24 ++++++++----------------
 1 file changed, 8 insertions(+), 16 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 0270bf4f1f90..d09a17747abd 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1103,6 +1103,7 @@ struct addr_ctx {
 	u8 cs_id;
 	int (*dehash_addr)(struct addr_ctx *ctx);
 	void (*make_space_for_cs_id)(struct addr_ctx *ctx);
+	void (*insert_cs_id)(struct addr_ctx *ctx);
 };
 
 struct data_fabric_ops {
@@ -1135,6 +1136,11 @@ static void make_space_for_cs_id_simple(struct addr_ctx *ctx)
 	expand_bits(ctx->intlv_addr_bit, num_intlv_bits, &ctx->ret_addr);
 }
 
+static void insert_cs_id_simple(struct addr_ctx *ctx)
+{
+	ctx->ret_addr |= (ctx->cs_id << ctx->intlv_addr_bit);
+}
+
 static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
 {
 	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
@@ -1166,6 +1172,7 @@ static int get_intlv_mode_df2(struct addr_ctx *ctx)
 	}
 
 	ctx->make_space_for_cs_id = &make_space_for_cs_id_simple;
+	ctx->insert_cs_id = &insert_cs_id_simple;
 
 	if (ctx->intlv_mode != NONE &&
 	    ctx->intlv_mode != NOHASH_2CH &&
@@ -1333,8 +1340,6 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 
 static int denormalize_addr(struct addr_ctx *ctx)
 {
-	u8 num_intlv_bits;
-
 	/* Return early if no interleaving. */
 	if (ctx->intlv_mode == NONE)
 		return 0;
@@ -1351,20 +1356,7 @@ static int denormalize_addr(struct addr_ctx *ctx)
 	if (calculate_cs_id(ctx))
 		return -EINVAL;
 
-	if (num_intlv_bits > 0) {
-		u64 temp_addr_i;
-
-		/*
-		 * The pre-interleaved address consists of XXXXXXIIIYYYYY
-		 * where III is the ID for this CS, and XXXXXXYYYYY are the
-		 * address bits from the post-interleaved address.
-		 * "num_intlv_bits" has been calculated to tell us how many "I"
-		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
-		 * there are (where "I" starts).
-		 */
-		temp_addr_i = (ctx->cs_id << ctx->intlv_addr_bit);
-		ctx->ret_addr |= temp_addr_i;
-	}
+	ctx->insert_cs_id(ctx);
 
 	return 0;
 }
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 25/31] EDAC/amd64: Define function to get CS Fabric ID
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (23 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 24/31] EDAC/amd64: Define function to insert CS ID into address Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 26/31] EDAC/amd64: Define function to find shift and mask values Yazen Ghannam
                   ` (5 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move code that gets the CS Fabric ID into a separate helper function.
This will be expanded for future DF versions.

The bitfield used for this value may be larger than the 8 bits currently
used. So make it a u16 type which is large enough to hold all known
sizes of this bitfield across DF versions.

Also, call this function early as future DF versions may need the value
early.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-22-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Added function to data_fabric_ops.

 drivers/edac/amd64_edac.c | 31 +++++++++++++++++++++++--------
 1 file changed, 23 insertions(+), 8 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index d09a17747abd..f5ac3b63376d 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1093,6 +1093,7 @@ struct addr_ctx {
 	u32 reg_dram_offset;
 	u32 reg_base_addr;
 	u32 reg_limit_addr;
+	u16 cs_fabric_id;
 	u16 nid;
 	u8 inst_id;
 	u8 map_num;
@@ -1109,6 +1110,7 @@ struct addr_ctx {
 struct data_fabric_ops {
 	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
 	int (*get_intlv_mode)(struct addr_ctx *ctx);
+	int (*get_cs_fabric_id)(struct addr_ctx *ctx);
 	void (*get_intlv_num_dies)(struct addr_ctx *ctx);
 	void (*get_intlv_num_sockets)(struct addr_ctx *ctx);
 };
@@ -1192,11 +1194,24 @@ static void get_intlv_num_sockets_df2(struct addr_ctx *ctx)
 	ctx->intlv_num_sockets = (ctx->reg_limit_addr >> 8) & 0x1;
 }
 
+static int get_cs_fabric_id_df2(struct addr_ctx *ctx)
+{
+	u32 tmp;
+
+	if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3], ctx->inst_id, &tmp))
+		return -EINVAL;
+
+	ctx->cs_fabric_id = (tmp >> 8) & 0xFF;
+
+	return 0;
+}
+
 struct data_fabric_ops df2_ops = {
 	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
 	.get_intlv_mode			=	&get_intlv_mode_df2,
 	.get_intlv_num_dies		=	&get_intlv_num_dies_df2,
 	.get_intlv_num_sockets		=	&get_intlv_num_sockets_df2,
+	.get_cs_fabric_id		=	&get_cs_fabric_id_df2,
 };
 
 struct data_fabric_ops *df_ops;
@@ -1296,20 +1311,16 @@ static void get_intlv_num_chan(struct addr_ctx *ctx)
 static int calculate_cs_id(struct addr_ctx *ctx)
 {
 	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 die_id_bit, sock_id_bit, cs_fabric_id, cs_mask = 0;
+	u8 die_id_bit, sock_id_bit, cs_mask = 0;
 	u32 tmp;
 
-	if (amd_df_indirect_read(ctx->nid, df_regs[FAB_BLK_INST_INFO_3], ctx->inst_id, &tmp))
-		return -EINVAL;
-
-	cs_fabric_id = (tmp >> 8) & 0xFF;
 	die_id_bit   = 0;
 
 	/* If interleaved over more than 1 channel: */
 	if (ctx->intlv_num_chan) {
 		die_id_bit = ctx->intlv_num_chan;
 		cs_mask	   = (1 << die_id_bit) - 1;
-		ctx->cs_id = cs_fabric_id & cs_mask;
+		ctx->cs_id = ctx->cs_fabric_id & cs_mask;
 	}
 
 	sock_id_bit = die_id_bit;
@@ -1324,7 +1335,7 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 		die_id_shift = (tmp >> 24) & 0xF;
 		die_id_mask  = (tmp >> 8) & 0xFF;
 
-		ctx->cs_id |= ((cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+		ctx->cs_id |= ((ctx->cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
 	}
 
 	/* If interleaved over more than 1 socket: */
@@ -1332,7 +1343,8 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 		socket_id_shift	= (tmp >> 28) & 0xF;
 		socket_id_mask	= (tmp >> 16) & 0xFF;
 
-		ctx->cs_id |= ((cs_fabric_id & socket_id_mask) >> socket_id_shift) << sock_id_bit;
+		ctx->cs_id |= ((ctx->cs_fabric_id & socket_id_mask)
+				>> socket_id_shift) << sock_id_bit;
 	}
 
 	return 0;
@@ -1411,6 +1423,9 @@ static int umc_normaddr_to_sysaddr(u64 *addr, u16 nid, u8 umc)
 	if (set_df_ops(&ctx))
 		return -EINVAL;
 
+	if (df_ops->get_cs_fabric_id(&ctx))
+		return -EINVAL;
+
 	if (remove_dram_offset(&ctx))
 		return -EINVAL;
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 26/31] EDAC/amd64: Define function to find shift and mask values
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (24 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 25/31] EDAC/amd64: Define function to get CS Fabric ID Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 27/31] EDAC/amd64: Update CS ID calculation to match reference code Yazen Ghannam
                   ` (4 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Move code to find the shift and mask values used in die and socket
interleaving into separate helper functions. These will be expanded for
future DF versions. Make the die_id_mask and socket_id_mask values u16
type to accommodate larger bitfields in future DF versions.

Also, move reading of the System Fabric ID Mask register into
set_df_ops(). This will be expanded for future DF versions and will be
used to determine DF version levels.

And call this function early since future DF versions may need these
values early.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-23-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Added functions to data_fabric_ops.

 drivers/edac/amd64_edac.c | 61 +++++++++++++++++++++++++++++----------
 1 file changed, 45 insertions(+), 16 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index f5ac3b63376d..fa8046e2b055 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1093,7 +1093,10 @@ struct addr_ctx {
 	u32 reg_dram_offset;
 	u32 reg_base_addr;
 	u32 reg_limit_addr;
+	u32 reg_fab_id_mask0;
 	u16 cs_fabric_id;
+	u16 die_id_mask;
+	u16 socket_id_mask;
 	u16 nid;
 	u8 inst_id;
 	u8 map_num;
@@ -1109,8 +1112,11 @@ struct addr_ctx {
 
 struct data_fabric_ops {
 	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
+	u8 (*get_die_id_shift)(struct addr_ctx *ctx);
+	u8 (*get_socket_id_shift)(struct addr_ctx *ctx);
 	int (*get_intlv_mode)(struct addr_ctx *ctx);
 	int (*get_cs_fabric_id)(struct addr_ctx *ctx);
+	int (*get_masks)(struct addr_ctx *ctx);
 	void (*get_intlv_num_dies)(struct addr_ctx *ctx);
 	void (*get_intlv_num_sockets)(struct addr_ctx *ctx);
 };
@@ -1206,18 +1212,43 @@ static int get_cs_fabric_id_df2(struct addr_ctx *ctx)
 	return 0;
 }
 
+static int get_masks_df2(struct addr_ctx *ctx)
+{
+	ctx->die_id_mask    = (ctx->reg_fab_id_mask0 >> 8) & 0xFF;
+	ctx->socket_id_mask = (ctx->reg_fab_id_mask0 >> 16) & 0xFF;
+
+	return 0;
+}
+
+static u8 get_die_id_shift_df2(struct addr_ctx *ctx)
+{
+	return (ctx->reg_fab_id_mask0 >> 24) & 0xF;
+}
+
+static u8 get_socket_id_shift_df2(struct addr_ctx *ctx)
+{
+	return (ctx->reg_fab_id_mask0 >> 28) & 0xF;
+}
+
 struct data_fabric_ops df2_ops = {
 	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
 	.get_intlv_mode			=	&get_intlv_mode_df2,
 	.get_intlv_num_dies		=	&get_intlv_num_dies_df2,
 	.get_intlv_num_sockets		=	&get_intlv_num_sockets_df2,
 	.get_cs_fabric_id		=	&get_cs_fabric_id_df2,
+	.get_masks			=	&get_masks_df2,
+	.get_die_id_shift		=	&get_die_id_shift_df2,
+	.get_socket_id_shift		=	&get_socket_id_shift_df2,
 };
 
 struct data_fabric_ops *df_ops;
 
 static int set_df_ops(struct addr_ctx *ctx)
 {
+	if (amd_df_indirect_read(0, df_regs[SYS_FAB_ID_MASK],
+				 DF_BROADCAST, &ctx->reg_fab_id_mask0))
+		return -EINVAL;
+
 	df_ops = &df2_ops;
 
 	return 0;
@@ -1310,11 +1341,7 @@ static void get_intlv_num_chan(struct addr_ctx *ctx)
 
 static int calculate_cs_id(struct addr_ctx *ctx)
 {
-	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
-	u8 die_id_bit, sock_id_bit, cs_mask = 0;
-	u32 tmp;
-
-	die_id_bit   = 0;
+	u8 die_id_bit = 0, sock_id_bit, cs_mask = 0;
 
 	/* If interleaved over more than 1 channel: */
 	if (ctx->intlv_num_chan) {
@@ -1323,27 +1350,26 @@ static int calculate_cs_id(struct addr_ctx *ctx)
 		ctx->cs_id = ctx->cs_fabric_id & cs_mask;
 	}
 
-	sock_id_bit = die_id_bit;
+	/* Return early if no die interleaving and no socket interleaving. */
+	if (!(ctx->intlv_num_dies || ctx->intlv_num_sockets))
+		return 0;
 
-	if (ctx->intlv_num_dies || ctx->intlv_num_sockets)
-		if (amd_df_indirect_read(ctx->nid, df_regs[SYS_FAB_ID_MASK], ctx->inst_id, &tmp))
-			return -EINVAL;
+	sock_id_bit = die_id_bit;
 
 	/* If interleaved over more than 1 die: */
 	if (ctx->intlv_num_dies) {
-		sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
-		die_id_shift = (tmp >> 24) & 0xF;
-		die_id_mask  = (tmp >> 8) & 0xFF;
+		u8 die_id_shift = df_ops->get_die_id_shift(ctx);
 
-		ctx->cs_id |= ((ctx->cs_fabric_id & die_id_mask) >> die_id_shift) << die_id_bit;
+		sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
+		ctx->cs_id |= ((ctx->cs_fabric_id & ctx->die_id_mask)
+				>> die_id_shift) << die_id_bit;
 	}
 
 	/* If interleaved over more than 1 socket: */
 	if (ctx->intlv_num_sockets) {
-		socket_id_shift	= (tmp >> 28) & 0xF;
-		socket_id_mask	= (tmp >> 16) & 0xFF;
+		u8 socket_id_shift = df_ops->get_socket_id_shift(ctx);
 
-		ctx->cs_id |= ((ctx->cs_fabric_id & socket_id_mask)
+		ctx->cs_id |= ((ctx->cs_fabric_id & ctx->socket_id_mask)
 				>> socket_id_shift) << sock_id_bit;
 	}
 
@@ -1423,6 +1449,9 @@ static int umc_normaddr_to_sysaddr(u64 *addr, u16 nid, u8 umc)
 	if (set_df_ops(&ctx))
 		return -EINVAL;
 
+	if (df_ops->get_masks(&ctx))
+		return -EINVAL;
+
 	if (df_ops->get_cs_fabric_id(&ctx))
 		return -EINVAL;
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 27/31] EDAC/amd64: Update CS ID calculation to match reference code
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (25 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 26/31] EDAC/amd64: Define function to find shift and mask values Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:19 ` [PATCH v2 28/31] EDAC/amd64: Match hash function to " Yazen Ghannam
                   ` (3 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Redo the current CS ID calculations to match the reference code. Helper
functions are introduced that will be expanded for future DF versions.

Use u16 type for dst_fabric_id and component_id_mask values to
accommodate larger bitfields in future DF versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-24-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Added functions to data_fabric_ops.

 drivers/edac/amd64_edac.c | 52 ++++++++++++++++++++++++++-------------
 1 file changed, 35 insertions(+), 17 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index fa8046e2b055..fc2cd288df0f 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1112,6 +1112,8 @@ struct addr_ctx {
 
 struct data_fabric_ops {
 	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
+	u16 (*get_dst_fabric_id)(struct addr_ctx *ctx);
+	u16 (*get_component_id_mask)(struct addr_ctx *ctx);
 	u8 (*get_die_id_shift)(struct addr_ctx *ctx);
 	u8 (*get_socket_id_shift)(struct addr_ctx *ctx);
 	int (*get_intlv_mode)(struct addr_ctx *ctx);
@@ -1230,6 +1232,16 @@ static u8 get_socket_id_shift_df2(struct addr_ctx *ctx)
 	return (ctx->reg_fab_id_mask0 >> 28) & 0xF;
 }
 
+static u16 get_dst_fabric_id_df2(struct addr_ctx *ctx)
+{
+	return ctx->reg_limit_addr & 0xFF;
+}
+
+static u16 get_component_id_mask_df2(struct addr_ctx *ctx)
+{
+	return (~(ctx->socket_id_mask | ctx->die_id_mask)) & 0xFF;
+}
+
 struct data_fabric_ops df2_ops = {
 	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
 	.get_intlv_mode			=	&get_intlv_mode_df2,
@@ -1239,6 +1251,8 @@ struct data_fabric_ops df2_ops = {
 	.get_masks			=	&get_masks_df2,
 	.get_die_id_shift		=	&get_die_id_shift_df2,
 	.get_socket_id_shift		=	&get_socket_id_shift_df2,
+	.get_dst_fabric_id		=	&get_dst_fabric_id_df2,
+	.get_component_id_mask		=	&get_component_id_mask_df2,
 };
 
 struct data_fabric_ops *df_ops;
@@ -1339,38 +1353,42 @@ static void get_intlv_num_chan(struct addr_ctx *ctx)
 	}
 }
 
-static int calculate_cs_id(struct addr_ctx *ctx)
+static u8 calc_level_bits(u8 id, u8 level_mask, u8 shift, u8 mask, u8 num_bits)
 {
-	u8 die_id_bit = 0, sock_id_bit, cs_mask = 0;
+	return (((id & level_mask) >> shift) & mask) << num_bits;
+}
 
-	/* If interleaved over more than 1 channel: */
-	if (ctx->intlv_num_chan) {
-		die_id_bit = ctx->intlv_num_chan;
-		cs_mask	   = (1 << die_id_bit) - 1;
-		ctx->cs_id = ctx->cs_fabric_id & cs_mask;
-	}
+static int calculate_cs_id(struct addr_ctx *ctx)
+{
+	u16 dst_fabric_id = df_ops->get_dst_fabric_id(ctx);
+	u16 mask, num_intlv_bits = ctx->intlv_num_chan;
 
-	/* Return early if no die interleaving and no socket interleaving. */
-	if (!(ctx->intlv_num_dies || ctx->intlv_num_sockets))
-		return 0;
+	mask = df_ops->get_component_id_mask(ctx);
+	ctx->cs_id = (ctx->cs_fabric_id & mask) - (dst_fabric_id & mask);
 
-	sock_id_bit = die_id_bit;
+	mask = (1 << num_intlv_bits) - 1;
+	ctx->cs_id &= mask;
 
 	/* If interleaved over more than 1 die: */
 	if (ctx->intlv_num_dies) {
 		u8 die_id_shift = df_ops->get_die_id_shift(ctx);
 
-		sock_id_bit  = die_id_bit + ctx->intlv_num_dies;
-		ctx->cs_id |= ((ctx->cs_fabric_id & ctx->die_id_mask)
-				>> die_id_shift) << die_id_bit;
+		mask = (1 << ctx->intlv_num_dies) - 1;
+
+		ctx->cs_id |= calc_level_bits(ctx->cs_fabric_id, ctx->die_id_mask,
+					      die_id_shift, mask, num_intlv_bits);
+
+		num_intlv_bits += ctx->intlv_num_dies;
 	}
 
 	/* If interleaved over more than 1 socket: */
 	if (ctx->intlv_num_sockets) {
 		u8 socket_id_shift = df_ops->get_socket_id_shift(ctx);
 
-		ctx->cs_id |= ((ctx->cs_fabric_id & ctx->socket_id_mask)
-				>> socket_id_shift) << sock_id_bit;
+		mask = (1 << ctx->intlv_num_sockets) - 1;
+
+		ctx->cs_id |= calc_level_bits(ctx->cs_fabric_id, ctx->socket_id_mask,
+					      socket_id_shift, mask, num_intlv_bits);
 	}
 
 	return 0;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 28/31] EDAC/amd64: Match hash function to reference code
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (26 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 27/31] EDAC/amd64: Update CS ID calculation to match reference code Yazen Ghannam
@ 2021-06-23 19:19 ` Yazen Ghannam
  2021-06-23 19:20 ` [PATCH v2 29/31] EDAC/amd64: Define helper function to get interleave address select bit Yazen Ghannam
                   ` (2 subsequent siblings)
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:19 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

The reference code for DF2 hashing was changed to XOR the interleave
address bit rather than the CS ID. Match that here.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-25-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.

 drivers/edac/amd64_edac.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index fc2cd288df0f..7ba6aa97e80d 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1162,7 +1162,7 @@ static int dehash_addr_df2(struct addr_ctx *ctx)
 			(ctx->ret_addr >> 18) ^
 			(ctx->ret_addr >> 21) ^
 			(ctx->ret_addr >> 30) ^
-			ctx->cs_id;
+			(ctx->ret_addr >> ctx->intlv_addr_bit);
 
 	hashed_bit &= BIT(0);
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 29/31] EDAC/amd64: Define helper function to get interleave address select bit
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (27 preceding siblings ...)
  2021-06-23 19:19 ` [PATCH v2 28/31] EDAC/amd64: Match hash function to " Yazen Ghannam
@ 2021-06-23 19:20 ` Yazen Ghannam
  2021-06-23 19:20 ` [PATCH v2 30/31] EDAC/amd64: Add support for address translation on DF3 systems Yazen Ghannam
  2021-06-23 19:20 ` [PATCH v2 31/31] EDAC/amd64: Add glossary of acronyms for address translation Yazen Ghannam
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:20 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

...this will be expanded for future Data Fabric versions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-1-Yazen.Ghannam@amd.com

v1->v2:
* New in v2.

 drivers/edac/amd64_edac.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 7ba6aa97e80d..aa8b8d0b319d 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1116,6 +1116,7 @@ struct data_fabric_ops {
 	u16 (*get_component_id_mask)(struct addr_ctx *ctx);
 	u8 (*get_die_id_shift)(struct addr_ctx *ctx);
 	u8 (*get_socket_id_shift)(struct addr_ctx *ctx);
+	u8 (*get_intlv_addr_sel)(struct addr_ctx *ctx);
 	int (*get_intlv_mode)(struct addr_ctx *ctx);
 	int (*get_cs_fabric_id)(struct addr_ctx *ctx);
 	int (*get_masks)(struct addr_ctx *ctx);
@@ -1192,6 +1193,11 @@ static int get_intlv_mode_df2(struct addr_ctx *ctx)
 	return 0;
 }
 
+static u8 get_intlv_addr_sel_df2(struct addr_ctx *ctx)
+{
+	return (ctx->reg_base_addr >> 8) & 0x7;
+}
+
 static void get_intlv_num_dies_df2(struct addr_ctx *ctx)
 {
 	ctx->intlv_num_dies  = (ctx->reg_limit_addr >> 10) & 0x3;
@@ -1245,6 +1251,7 @@ static u16 get_component_id_mask_df2(struct addr_ctx *ctx)
 struct data_fabric_ops df2_ops = {
 	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
 	.get_intlv_mode			=	&get_intlv_mode_df2,
+	.get_intlv_addr_sel		=	&get_intlv_addr_sel_df2,
 	.get_intlv_num_dies		=	&get_intlv_num_dies_df2,
 	.get_intlv_num_sockets		=	&get_intlv_num_sockets_df2,
 	.get_cs_fabric_id		=	&get_cs_fabric_id_df2,
@@ -1323,7 +1330,7 @@ static int get_dram_addr_map(struct addr_ctx *ctx)
 
 static int get_intlv_addr_bit(struct addr_ctx *ctx)
 {
-	u8 intlv_addr_sel = (ctx->reg_base_addr >> 8) & 0x7;
+	u8 intlv_addr_sel = df_ops->get_intlv_addr_sel(ctx);
 
 	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
 	if (intlv_addr_sel > 3) {
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 30/31] EDAC/amd64: Add support for address translation on DF3 systems
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (28 preceding siblings ...)
  2021-06-23 19:20 ` [PATCH v2 29/31] EDAC/amd64: Define helper function to get interleave address select bit Yazen Ghannam
@ 2021-06-23 19:20 ` Yazen Ghannam
  2021-06-23 19:20 ` [PATCH v2 31/31] EDAC/amd64: Add glossary of acronyms for address translation Yazen Ghannam
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:20 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

DF3-based systems (Rome and later) support new interleaving modes and a
number of bit fields have changed or moved entirely. Add support for
these new modes and fields.

Refactoring should be minimal due to earlier changes, and most updates
will be additions.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/20210507190140.18854-26-Yazen.Ghannam@amd.com

v1->v2:
* Moved from arch/x86 to EDAC.
* Use function pointers as needed.

 drivers/edac/amd64_edac.c | 189 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 187 insertions(+), 2 deletions(-)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index aa8b8d0b319d..882ac3fbc832 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -1059,9 +1059,11 @@ enum df_reg_names {
 	DRAM_BASE_ADDR,
 	DRAM_LIMIT_ADDR,
 	DRAM_OFFSET,
+	DF_GLOBAL_CTL,
 
 	/* Function 1 */
 	SYS_FAB_ID_MASK,
+	SYS_FAB_ID_MASK_1,
 };
 
 static struct df_reg df_regs[] = {
@@ -1075,14 +1077,23 @@ static struct df_reg df_regs[] = {
 	[DRAM_LIMIT_ADDR]	=	{0, 0x114},
 	/* D18F0x1B4 (DramOffset) */
 	[DRAM_OFFSET]		=	{0, 0x1B4},
+	/* D18F0x3F8 (DfGlobalCtrl) */
+	[DF_GLOBAL_CTL]		=	{0, 0x3F8},
 	/* D18F1x208 (SystemFabricIdMask) */
 	[SYS_FAB_ID_MASK]	=	{1, 0x208},
+	/* D18F1x20C (SystemFabricIdMask1) */
+	[SYS_FAB_ID_MASK_1]	=	{1, 0x20C},
 };
 
 /* These are mapped 1:1 to the hardware values. Special cases are set at > 0x20. */
 enum intlv_modes {
 	NONE		= 0x00,
 	NOHASH_2CH	= 0x01,
+	NOHASH_4CH	= 0x03,
+	NOHASH_8CH	= 0x05,
+	HASH_COD4_2CH	= 0x0C,
+	HASH_COD2_4CH	= 0x0D,
+	HASH_COD1_8CH	= 0x0E,
 	DF2_HASH_2CH	= 0x21,
 };
 
@@ -1094,6 +1105,7 @@ struct addr_ctx {
 	u32 reg_base_addr;
 	u32 reg_limit_addr;
 	u32 reg_fab_id_mask0;
+	u32 reg_fab_id_mask1;
 	u16 cs_fabric_id;
 	u16 die_id_mask;
 	u16 socket_id_mask;
@@ -1105,6 +1117,7 @@ struct addr_ctx {
 	u8 intlv_num_dies;
 	u8 intlv_num_sockets;
 	u8 cs_id;
+	u8 node_id_shift;
 	int (*dehash_addr)(struct addr_ctx *ctx);
 	void (*make_space_for_cs_id)(struct addr_ctx *ctx);
 	void (*insert_cs_id)(struct addr_ctx *ctx);
@@ -1262,6 +1275,164 @@ struct data_fabric_ops df2_ops = {
 	.get_component_id_mask		=	&get_component_id_mask_df2,
 };
 
+static u64 get_hi_addr_offset_df3(struct addr_ctx *ctx)
+{
+	return (ctx->reg_dram_offset & GENMASK_ULL(31, 12)) << 16;
+}
+
+static void make_space_for_cs_id_cod_hash(struct addr_ctx *ctx)
+{
+	u8 num_intlv_bits = ctx->intlv_num_chan;
+
+	num_intlv_bits += ctx->intlv_num_sockets;
+	expand_bits(ctx->intlv_addr_bit, 1, &ctx->ret_addr);
+	if (num_intlv_bits > 1)
+		expand_bits(12, num_intlv_bits - 1, &ctx->ret_addr);
+}
+
+static void insert_cs_id_cod_hash(struct addr_ctx *ctx)
+{
+	ctx->ret_addr |= ((ctx->cs_id & 0x1) << ctx->intlv_addr_bit);
+	ctx->ret_addr |= ((ctx->cs_id & 0xE) << 11);
+}
+
+static int dehash_addr_df3(struct addr_ctx *ctx)
+{
+	u8 hashed_bit, intlv_ctl_64k, intlv_ctl_2M, intlv_ctl_1G;
+	u32 tmp;
+
+	if (amd_df_indirect_read(0, df_regs[DF_GLOBAL_CTL], DF_BROADCAST, &tmp))
+		return -EINVAL;
+
+	intlv_ctl_64k = !!((tmp >> 20) & 0x1);
+	intlv_ctl_2M  = !!((tmp >> 21) & 0x1);
+	intlv_ctl_1G  = !!((tmp >> 22) & 0x1);
+
+	hashed_bit =	(ctx->ret_addr >> 14) ^
+			((ctx->ret_addr >> 18) & intlv_ctl_64k) ^
+			((ctx->ret_addr >> 23) & intlv_ctl_2M) ^
+			((ctx->ret_addr >> 32) & intlv_ctl_1G) ^
+			(ctx->ret_addr >> ctx->intlv_addr_bit);
+
+	hashed_bit &= BIT(0);
+
+	if (hashed_bit != ((ctx->ret_addr >> ctx->intlv_addr_bit) & BIT(0)))
+		ctx->ret_addr ^= BIT(ctx->intlv_addr_bit);
+
+	if (ctx->intlv_mode != HASH_COD2_4CH &&
+	    ctx->intlv_mode != HASH_COD1_8CH)
+		return 0;
+
+	hashed_bit =	(ctx->ret_addr >> 12) ^
+			((ctx->ret_addr >> 16) & intlv_ctl_64k) ^
+			((ctx->ret_addr >> 21) & intlv_ctl_2M) ^
+			((ctx->ret_addr >> 30) & intlv_ctl_1G);
+
+	hashed_bit &= BIT(0);
+
+	if (hashed_bit != ((ctx->ret_addr >> 12) & BIT(0)))
+		ctx->ret_addr ^= BIT(12);
+
+	if (ctx->intlv_mode != HASH_COD1_8CH)
+		return 0;
+
+	hashed_bit =	(ctx->ret_addr >> 13) ^
+			((ctx->ret_addr >> 17) & intlv_ctl_64k) ^
+			((ctx->ret_addr >> 22) & intlv_ctl_2M) ^
+			((ctx->ret_addr >> 31) & intlv_ctl_1G);
+
+	hashed_bit &= BIT(0);
+
+	if (hashed_bit != ((ctx->ret_addr >> 13) & BIT(0)))
+		ctx->ret_addr ^= BIT(13);
+
+	return 0;
+}
+
+static int get_intlv_mode_df3(struct addr_ctx *ctx)
+{
+	ctx->intlv_mode = (ctx->reg_base_addr >> 2) & 0xF;
+
+	if (ctx->intlv_mode == HASH_COD4_2CH ||
+	    ctx->intlv_mode == HASH_COD2_4CH ||
+	    ctx->intlv_mode == HASH_COD1_8CH) {
+		ctx->make_space_for_cs_id = &make_space_for_cs_id_cod_hash;
+		ctx->insert_cs_id = &insert_cs_id_cod_hash;
+		ctx->dehash_addr = &dehash_addr_df3;
+	} else {
+		ctx->make_space_for_cs_id = &make_space_for_cs_id_simple;
+		ctx->insert_cs_id = &insert_cs_id_simple;
+	}
+
+	return 0;
+}
+
+static u8 get_intlv_addr_sel_df3(struct addr_ctx *ctx)
+{
+	return (ctx->reg_base_addr >> 9) & 0x7;
+}
+
+static void get_intlv_num_dies_df3(struct addr_ctx *ctx)
+{
+	ctx->intlv_num_dies  = (ctx->reg_base_addr >> 6) & 0x3;
+}
+
+static void get_intlv_num_sockets_df3(struct addr_ctx *ctx)
+{
+	ctx->intlv_num_sockets = (ctx->reg_base_addr >> 8) & 0x1;
+}
+
+static u8 get_die_id_shift_df3(struct addr_ctx *ctx)
+{
+	return ctx->node_id_shift;
+}
+
+static u8 get_socket_id_shift_df3(struct addr_ctx *ctx)
+{
+	return ((ctx->reg_fab_id_mask1 >> 8) & 0x3) + ctx->node_id_shift;
+}
+
+static int get_masks_df3(struct addr_ctx *ctx)
+{
+	if (amd_df_indirect_read(0, df_regs[SYS_FAB_ID_MASK_1],
+				 DF_BROADCAST, &ctx->reg_fab_id_mask1))
+		return -EINVAL;
+
+	ctx->node_id_shift = ctx->reg_fab_id_mask1 & 0xF;
+
+	ctx->die_id_mask = (ctx->reg_fab_id_mask1 >> 16) & 0x7;
+	ctx->die_id_mask <<= ctx->node_id_shift;
+
+	ctx->socket_id_mask = (ctx->reg_fab_id_mask1 >> 24) & 0x7;
+	ctx->socket_id_mask <<= ctx->node_id_shift;
+
+	return 0;
+}
+
+static u16 get_dst_fabric_id_df3(struct addr_ctx *ctx)
+{
+	return ctx->reg_limit_addr & 0x3FF;
+}
+
+static u16 get_component_id_mask_df3(struct addr_ctx *ctx)
+{
+	return ctx->reg_fab_id_mask0 & 0x3FF;
+}
+
+struct data_fabric_ops df3_ops = {
+	.get_hi_addr_offset		=	&get_hi_addr_offset_df3,
+	.get_intlv_mode			=	&get_intlv_mode_df3,
+	.get_intlv_addr_sel		=	&get_intlv_addr_sel_df3,
+	.get_intlv_num_dies		=	&get_intlv_num_dies_df3,
+	.get_intlv_num_sockets		=	&get_intlv_num_sockets_df3,
+	.get_cs_fabric_id		=	&get_cs_fabric_id_df2,
+	.get_masks			=	&get_masks_df3,
+	.get_die_id_shift		=	&get_die_id_shift_df3,
+	.get_socket_id_shift		=	&get_socket_id_shift_df3,
+	.get_dst_fabric_id		=	&get_dst_fabric_id_df3,
+	.get_component_id_mask		=	&get_component_id_mask_df3,
+};
+
 struct data_fabric_ops *df_ops;
 
 static int set_df_ops(struct addr_ctx *ctx)
@@ -1270,6 +1441,11 @@ static int set_df_ops(struct addr_ctx *ctx)
 				 DF_BROADCAST, &ctx->reg_fab_id_mask0))
 		return -EINVAL;
 
+	if ((ctx->reg_fab_id_mask0 & 0xFF) != 0) {
+		df_ops = &df3_ops;
+		return 0;
+	}
+
 	df_ops = &df2_ops;
 
 	return 0;
@@ -1332,8 +1508,8 @@ static int get_intlv_addr_bit(struct addr_ctx *ctx)
 {
 	u8 intlv_addr_sel = df_ops->get_intlv_addr_sel(ctx);
 
-	/* {0, 1, 2, 3} map to address bits {8, 9, 10, 11} respectively */
-	if (intlv_addr_sel > 3) {
+	/* {0, 1, 2, 3, 4} map to address bits {8, 9, 10, 11, 12} respectively */
+	if (intlv_addr_sel > 4) {
 		pr_debug("Invalid interleave address select %d.\n", intlv_addr_sel);
 		return -EINVAL;
 	}
@@ -1351,9 +1527,18 @@ static void get_intlv_num_chan(struct addr_ctx *ctx)
 		ctx->intlv_num_chan = 0;
 		break;
 	case NOHASH_2CH:
+	case HASH_COD4_2CH:
 	case DF2_HASH_2CH:
 		ctx->intlv_num_chan = 1;
 		break;
+	case NOHASH_4CH:
+	case HASH_COD2_4CH:
+		ctx->intlv_num_chan = 2;
+		break;
+	case NOHASH_8CH:
+	case HASH_COD1_8CH:
+		ctx->intlv_num_chan = 3;
+		break;
 	default:
 		/* Valid interleaving modes where checked earlier. */
 		break;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* [PATCH v2 31/31] EDAC/amd64: Add glossary of acronyms for address translation
  2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
                   ` (29 preceding siblings ...)
  2021-06-23 19:20 ` [PATCH v2 30/31] EDAC/amd64: Add support for address translation on DF3 systems Yazen Ghannam
@ 2021-06-23 19:20 ` Yazen Ghannam
  30 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-06-23 19:20 UTC (permalink / raw)
  To: linux-edac
  Cc: linux-kernel, bp, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa, Yazen Ghannam

Add a short glossary of the acronyms used for address translation. Some
terms may be seen with other AMD code, and some are only used within the
translation code.

Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
Link:
https://lkml.kernel.org/r/YJqwpKBsKL1QKR5z@zn.tnic

v1->v2:
* New in v2.

 drivers/edac/amd64_edac.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 882ac3fbc832..d8557716d65e 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -993,6 +993,15 @@ static int sys_addr_to_csrow(struct mem_ctl_info *mci, u64 sys_addr)
 	return csrow;
 }
 
+/*
+ * Glossary of acronyms used in address translation for Zen-based systems
+ *
+ * COD		=	Cluster-on-Die
+ * CS		=	Coherent Slave
+ * DF		=	Data Fabric
+ * UMC		=	Unified Memory Controller
+ */
+
 /* Protect the PCI config register pairs used for DF indirect access. */
 static DEFINE_MUTEX(df_indirect_mutex);
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 03/31] EDAC/amd64: Don't use naked values for DF registers
  2021-06-23 19:19 ` [PATCH v2 03/31] EDAC/amd64: Don't use naked values for DF registers Yazen Ghannam
@ 2021-06-25 15:21   ` Borislav Petkov
  2021-07-08 19:35     ` Yazen Ghannam
  0 siblings, 1 reply; 45+ messages in thread
From: Borislav Petkov @ 2021-06-25 15:21 UTC (permalink / raw)
  To: Yazen Ghannam
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 23, 2021 at 07:19:34PM +0000, Yazen Ghannam wrote:
> +static struct df_reg df_regs[] = {
> +	/* D18F0x50 (FabricBlockInstanceInformation3_CS) */
> +	[FAB_BLK_INST_INFO_3]	=	{0, 0x50},
> +	/* D18F0x104 (DramHoleControl) */
> +	[DRAM_HOLE_CTL]		=	{0, 0x104},
> +	/* D18F0x110 (DramBaseAddress) */
> +	[DRAM_BASE_ADDR]	=	{0, 0x110},
> +	/* D18F0x114 (DramLimitAddress) */
> +	[DRAM_LIMIT_ADDR]	=	{0, 0x114},
> +	/* D18F0x1B4 (DramOffset) */
> +	[DRAM_OFFSET]		=	{0, 0x1B4},
> +	/* D18F1x208 (SystemFabricIdMask) */
> +	[SYS_FAB_ID_MASK]	=	{1, 0x208},
> +};
> +
>  static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
>  {
>  	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
> @@ -1059,8 +1091,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
>  	u8 cs_mask, cs_id = 0;
>  	bool hash_enabled = false;
>  
> -	/* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
> -	if (amd_df_indirect_read(nid, 0, 0x1B4, umc, &tmp))
> +	struct df_reg reg;
> +
> +	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
>  		goto out_err;
>  
>  	/* Remove HiAddrOffset from normalized address, if enabled: */
> @@ -1073,8 +1106,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
>  		}
>  	}
>  
> -	/* Read D18F0x110 (DramBaseAddress). */
> -	if (amd_df_indirect_read(nid, 0, 0x110 + (8 * base), umc, &tmp))
> +	reg = df_regs[DRAM_BASE_ADDR];
> +	reg.offset += base * 8;

So this looks weird: you have a df_regs[] array of all those different
DF registers which I'd assume is a read-only thing because, well, those
func and offset things are immutable, i.e., hw registers offsets etc.

But then here you go and and modify the offset.

And that df_regs array is globally visible in the driver and if some
later functionality decides to use it, it'll see the modified offset.

IOW, I'd make that array read only (const) and use local vars instead to
pass down to amd_df_indirect_read().

And I'm also questioning what the point is for that df_reg thing?

You have them defined but then you have to change them.

I.e., you can just as well pass in func and offset separately and be
done with it.

But maybe there's something else happening in the patches which comes
later and which will make me go, ahaa.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 04/31] EDAC/amd64: Allow for DF Indirect Broadcast reads
  2021-06-23 19:19 ` [PATCH v2 04/31] EDAC/amd64: Allow for DF Indirect Broadcast reads Yazen Ghannam
@ 2021-06-30 16:22   ` Borislav Petkov
  2021-07-08 19:44     ` Yazen Ghannam
  0 siblings, 1 reply; 45+ messages in thread
From: Borislav Petkov @ 2021-06-30 16:22 UTC (permalink / raw)
  To: Yazen Ghannam
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 23, 2021 at 07:19:35PM +0000, Yazen Ghannam wrote:
> The DF Indirect Access method allows for "Broadcast" accesses in which
> case no specific instance is targeted. Add support using a reserved
> instance ID of 0xFF to indicate a broadcast access. Set the FICAA
> register appropriately.
> 
> Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
> ---
> Link:
> https://lkml.kernel.org/r/20210507190140.18854-1-Yazen.Ghannam@amd.com
> 
> v1->v2:
> * New in v2.
> 
>  drivers/edac/amd64_edac.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
> index b94067e3952b..d67cd8f57b94 100644
> --- a/drivers/edac/amd64_edac.c
> +++ b/drivers/edac/amd64_edac.c
> @@ -1010,7 +1010,11 @@ struct df_reg {
>   *
>   * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
>   * and FICAD HI registers but so far we only need the LO register.
> + *
> + * Use Instance Id 0xFF to indicate a broadcast read.
>   */
> +
> +#define DF_BROADCAST	0xFF
>  static int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32 *lo)
>  {
>  	struct pci_dev *F4;
> @@ -1024,7 +1028,7 @@ static int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32
>  	if (!F4)
>  		goto out;
>  
> -	ficaa  = 1;
> +	ficaa  = (instance_id == DF_BROADCAST) ? 0 : 1;

Or, you can define two functions:

df_indirect_read_broadcast()
df_indirect_read_umc()

(no need for the "amd_" prefix either - this is a static function now)

which both call a low-level helper:

static int __df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32 *lo)

and there you either pass the instance_id or 0xff depending on which
outer function is calling it...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 05/31] EDAC/amd64: Add context struct
  2021-06-23 19:19 ` [PATCH v2 05/31] EDAC/amd64: Add context struct Yazen Ghannam
@ 2021-06-30 17:17   ` Borislav Petkov
  2021-07-08 19:53     ` Yazen Ghannam
  0 siblings, 1 reply; 45+ messages in thread
From: Borislav Petkov @ 2021-06-30 17:17 UTC (permalink / raw)
  To: Yazen Ghannam
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 23, 2021 at 07:19:36PM +0000, Yazen Ghannam wrote:
> Define an address translation context struct. This will hold values that
> will be passed between multiple functions.
> 
> Save return address, Node ID, and the Instance ID number to start.
> Currently, we use the UMC number as the Instance ID, but future DF

Please use passive voice in your commit message: no "we" or "I", etc.

And this here is a perfect example: it sounds here like "we" is "AMD"
but we use "we" mostly for the kernel. And there's the confusion.

So please teach yourself to formulate those commit messages properly -
the future you will thank you, trust me!

:-)

> versions may use another value.
> 
> Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
> ---
> Link:
> https://lkml.kernel.org/r/20210507190140.18854-3-Yazen.Ghannam@amd.com
> 
> v1->v2:
> * Moved from arch/x86 to EDAC.
> * Changed "umc" variable to "inst_id".
> * Drop df_types enum.
> 
>  drivers/edac/amd64_edac.c | 49 +++++++++++++++++++++++++--------------
>  1 file changed, 31 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
> index d67cd8f57b94..62eca188458f 100644
> --- a/drivers/edac/amd64_edac.c
> +++ b/drivers/edac/amd64_edac.c
> @@ -1079,12 +1079,15 @@ static struct df_reg df_regs[] = {
>  	[SYS_FAB_ID_MASK]	=	{1, 0x208},
>  };
>  
> +struct addr_ctx {
> +	u64 ret_addr;
> +	u16 nid;
> +	u8 inst_id;
> +};
> +
>  static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
>  {
>  	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
> -	/* We start from the normalized address */
> -	u64 ret_addr = norm_addr;
> -
>  	u32 tmp;
>  
>  	u8 die_id_shift, die_id_mask, socket_id_shift, socket_id_mask;
> @@ -1097,6 +1100,16 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
>  
>  	struct df_reg reg;
>  
> +	struct addr_ctx ctx;

Those empty-lines spaced-out local vars look weird.

> +	memset(&ctx, 0, sizeof(ctx));
> +
> +	/* We start from the normalized address */
> +	ctx.ret_addr = norm_addr;
> +
> +	ctx.nid = nid;
> +	ctx.inst_id = umc;
> +
>  	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
>  		goto out_err;
>  
> @@ -1105,7 +1118,7 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
>  		u64 hi_addr_offset = (tmp & GENMASK_ULL(31, 20)) << 8;
>  
>  		if (norm_addr >= hi_addr_offset) {
> -			ret_addr -= hi_addr_offset;
> +			ctx.ret_addr -= hi_addr_offset;
>  			base = 1;
>  		}
>  	}
> @@ -1236,14 +1249,14 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
>  		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
>  		 * there are (where "I" starts).
>  		 */
> -		temp_addr_y = ret_addr & GENMASK_ULL(intlv_addr_bit-1, 0);
> +		temp_addr_y = ctx.ret_addr & GENMASK_ULL(intlv_addr_bit - 1, 0);
>  		temp_addr_i = (cs_id << intlv_addr_bit);
> -		temp_addr_x = (ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
> -		ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
> +		temp_addr_x = (ctx.ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
> +		ctx.ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;

You want to align those vertically on the "=" for better readability.

>  	}
>  
>  	/* Add dram base address */
> -	ret_addr += dram_base_addr;
> +	ctx.ret_addr += dram_base_addr;
>  
>  	/* If legacy MMIO hole enabled */
>  	if (lgcy_mmio_hole_en) {
> @@ -1251,29 +1264,29 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
>  			goto out_err;
>  
>  		dram_hole_base = tmp & GENMASK(31, 24);
> -		if (ret_addr >= dram_hole_base)
> -			ret_addr += (BIT_ULL(32) - dram_hole_base);
> +		if (ctx.ret_addr >= dram_hole_base)
> +			ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
>  	}
>  
>  	if (hash_enabled) {
>  		/* Save some parentheses and grab ls-bit at the end. */
> -		hashed_bit =	(ret_addr >> 12) ^
> -				(ret_addr >> 18) ^
> -				(ret_addr >> 21) ^
> -				(ret_addr >> 30) ^
> +		hashed_bit =	(ctx.ret_addr >> 12) ^
> +				(ctx.ret_addr >> 18) ^
> +				(ctx.ret_addr >> 21) ^
> +				(ctx.ret_addr >> 30) ^
>  				cs_id;
>  
>  		hashed_bit &= BIT(0);
>  
> -		if (hashed_bit != ((ret_addr >> intlv_addr_bit) & BIT(0)))
> -			ret_addr ^= BIT(intlv_addr_bit);
> +		if (hashed_bit != ((ctx.ret_addr >> intlv_addr_bit) & BIT(0)))
> +			ctx.ret_addr ^= BIT(intlv_addr_bit);
>  	}
>  
>  	/* Is calculated system address is above DRAM limit address? */
> -	if (ret_addr > dram_limit_addr)
> +	if (ctx.ret_addr > dram_limit_addr)
>  		goto out_err;
>  
> -	*sys_addr = ret_addr;
> +	*sys_addr = ctx.ret_addr;

So adding ctx to exchange data between functions - that was a good idea.

But what this patch does is pointless because you simply replace those
variables with a local struct.

I guess the aha moment will come with the later patches when you start
passing it around to functions.

/me waits to see.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 06/31] EDAC/amd64: Define Data Fabric operations
  2021-06-23 19:19 ` [PATCH v2 06/31] EDAC/amd64: Define Data Fabric operations Yazen Ghannam
@ 2021-06-30 17:19   ` Borislav Petkov
  2021-07-08 19:55     ` Yazen Ghannam
  0 siblings, 1 reply; 45+ messages in thread
From: Borislav Petkov @ 2021-06-30 17:19 UTC (permalink / raw)
  To: Yazen Ghannam
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 23, 2021 at 07:19:37PM +0000, Yazen Ghannam wrote:
> Define a stub to hold operations for different Data Fabric versions.
> This will be filled in following patches.
> 
> Define a function to set the appropriate operations. Use a return code
> because future updates have checks that may fail.
> 
> Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
> ---
> Link:
> https://lkml.kernel.org/r/20210507190140.18854-3-Yazen.Ghannam@amd.com
> 
> v1->v2:
> * New in v2.
> 
>  drivers/edac/amd64_edac.c | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
> index 62eca188458f..f769353ef7e4 100644
> --- a/drivers/edac/amd64_edac.c
> +++ b/drivers/edac/amd64_edac.c
> @@ -1085,6 +1085,21 @@ struct addr_ctx {
>  	u8 inst_id;
>  };
>  
> +struct data_fabric_ops {
> +};
> +
> +struct data_fabric_ops df2_ops = {
> +};
> +
> +struct data_fabric_ops *df_ops;
> +
> +static int set_df_ops(struct addr_ctx *ctx)
> +{
> +	df_ops = &df2_ops;
> +
> +	return 0;
> +}
> +
>  static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
>  {
>  	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
> @@ -1110,6 +1125,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
>  	ctx.nid = nid;
>  	ctx.inst_id = umc;
>  
> +	if (set_df_ops(&ctx))
> +		return -EINVAL;
> +
>  	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
>  		goto out_err;
>  
> -- 

Why?

You can simply assign to df_ops as this is all driver-static data.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 07/31] EDAC/amd64: Define functions for DramOffset
  2021-06-23 19:19 ` [PATCH v2 07/31] EDAC/amd64: Define functions for DramOffset Yazen Ghannam
@ 2021-06-30 17:27   ` Borislav Petkov
  2021-07-08 20:08     ` Yazen Ghannam
  0 siblings, 1 reply; 45+ messages in thread
From: Borislav Petkov @ 2021-06-30 17:27 UTC (permalink / raw)
  To: Yazen Ghannam
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 23, 2021 at 07:19:38PM +0000, Yazen Ghannam wrote:
> Add helper functions to read the DramOffset register and to remove the
> offset from the calculated address.
> 
> The helper functions will be expanded in future DF versions.
> 
> Rename the "base" variable to "map_num" to indicate that this is the
> address map number. An address map is defined with a base and limit
> value. The map_num variable is used to select the proper base and limit
> registers to use for the address translation.
> 
> Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
> ---
> Link:
> https://lkml.kernel.org/r/20210507190140.18854-4-Yazen.Ghannam@amd.com
> 
> v1->v2:
> * Moved from arch/x86 to EDAC.
> * Add function to data_fabric_ops.
> 
>  drivers/edac/amd64_edac.c | 57 +++++++++++++++++++++++++++++----------
>  1 file changed, 43 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
> index f769353ef7e4..389f0621e885 100644
> --- a/drivers/edac/amd64_edac.c
> +++ b/drivers/edac/amd64_edac.c
> @@ -1079,16 +1079,26 @@ static struct df_reg df_regs[] = {
>  	[SYS_FAB_ID_MASK]	=	{1, 0x208},
>  };
>  
> +/* Use "reg_" prefix for raw register values. */
>  struct addr_ctx {
>  	u64 ret_addr;
> +	u32 reg_dram_offset;
>  	u16 nid;
>  	u8 inst_id;
> +	u8 map_num;
>  };
>  
>  struct data_fabric_ops {
> +	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
>  };
>  
> +static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
> +{
> +	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
> +}
> +
>  struct data_fabric_ops df2_ops = {

Those "df2" ops are for UMC v2? 

> +	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
						^

No need for the "&".

Also, if you define a member ->get_hi_addr_offset in df2_ops, then you
need to define that member in *every* df_ops structure you have so that
there's no NULL pointers.

I guess that's coming...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 08/31] EDAC/amd64: Define function to read DRAM address map registers
  2021-06-23 19:19 ` [PATCH v2 08/31] EDAC/amd64: Define function to read DRAM address map registers Yazen Ghannam
@ 2021-06-30 17:29   ` Borislav Petkov
  0 siblings, 0 replies; 45+ messages in thread
From: Borislav Petkov @ 2021-06-30 17:29 UTC (permalink / raw)
  To: Yazen Ghannam
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 23, 2021 at 07:19:39PM +0000, Yazen Ghannam wrote:
> @@ -1170,22 +1194,13 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
>  	if (remove_dram_offset(&ctx))
>  		goto out_err;
>  
> -	reg = df_regs[DRAM_BASE_ADDR];
> -	reg.offset += ctx.map_num * 8;
> -	if (amd_df_indirect_read(nid, reg, umc, &tmp))
> -		goto out_err;
> -
> -	/* Check if address range is valid. */
> -	if (!(tmp & BIT(0))) {
> -		pr_err("%s: Invalid DramBaseAddress range: 0x%x.\n",
> -			__func__, tmp);
> +	if (get_dram_addr_map(&ctx))
>  		goto out_err;
> -	}

Aha, I see where you're going with the context struct. Yap, ok.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 09/31] EDAC/amd64: Define function to find interleaving mode
  2021-06-23 19:19 ` [PATCH v2 09/31] EDAC/amd64: Define function to find interleaving mode Yazen Ghannam
@ 2021-06-30 17:33   ` Borislav Petkov
  2021-07-08 20:09     ` Yazen Ghannam
  0 siblings, 1 reply; 45+ messages in thread
From: Borislav Petkov @ 2021-06-30 17:33 UTC (permalink / raw)
  To: Yazen Ghannam
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 23, 2021 at 07:19:40PM +0000, Yazen Ghannam wrote:
> @@ -1088,10 +1096,12 @@ struct addr_ctx {
>  	u16 nid;
>  	u8 inst_id;
>  	u8 map_num;
> +	bool hash_enabled;

For such single bits info you can use bitfields, like struct mca_config
does it, for example.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 03/31] EDAC/amd64: Don't use naked values for DF registers
  2021-06-25 15:21   ` Borislav Petkov
@ 2021-07-08 19:35     ` Yazen Ghannam
  0 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-07-08 19:35 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Fri, Jun 25, 2021 at 05:21:08PM +0200, Borislav Petkov wrote:
> On Wed, Jun 23, 2021 at 07:19:34PM +0000, Yazen Ghannam wrote:
> > +static struct df_reg df_regs[] = {
> > +	/* D18F0x50 (FabricBlockInstanceInformation3_CS) */
> > +	[FAB_BLK_INST_INFO_3]	=	{0, 0x50},
> > +	/* D18F0x104 (DramHoleControl) */
> > +	[DRAM_HOLE_CTL]		=	{0, 0x104},
> > +	/* D18F0x110 (DramBaseAddress) */
> > +	[DRAM_BASE_ADDR]	=	{0, 0x110},
> > +	/* D18F0x114 (DramLimitAddress) */
> > +	[DRAM_LIMIT_ADDR]	=	{0, 0x114},
> > +	/* D18F0x1B4 (DramOffset) */
> > +	[DRAM_OFFSET]		=	{0, 0x1B4},
> > +	/* D18F1x208 (SystemFabricIdMask) */
> > +	[SYS_FAB_ID_MASK]	=	{1, 0x208},
> > +};
> > +
> >  static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr)
> >  {
> >  	u64 dram_base_addr, dram_limit_addr, dram_hole_base;
> > @@ -1059,8 +1091,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
> >  	u8 cs_mask, cs_id = 0;
> >  	bool hash_enabled = false;
> >  
> > -	/* Read D18F0x1B4 (DramOffset), check if base 1 is used. */
> > -	if (amd_df_indirect_read(nid, 0, 0x1B4, umc, &tmp))
> > +	struct df_reg reg;
> > +
> > +	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
> >  		goto out_err;
> >  
> >  	/* Remove HiAddrOffset from normalized address, if enabled: */
> > @@ -1073,8 +1106,9 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
> >  		}
> >  	}
> >  
> > -	/* Read D18F0x110 (DramBaseAddress). */
> > -	if (amd_df_indirect_read(nid, 0, 0x110 + (8 * base), umc, &tmp))
> > +	reg = df_regs[DRAM_BASE_ADDR];
> > +	reg.offset += base * 8;
> 
> So this looks weird: you have a df_regs[] array of all those different
> DF registers which I'd assume is a read-only thing because, well, those
> func and offset things are immutable, i.e., hw registers offsets etc.
> 
> But then here you go and and modify the offset.
> 
> And that df_regs array is globally visible in the driver and if some
> later functionality decides to use it, it'll see the modified offset.
> 
> IOW, I'd make that array read only (const) and use local vars instead to
> pass down to amd_df_indirect_read().
> 
> And I'm also questioning what the point is for that df_reg thing?
> 
> You have them defined but then you have to change them.
> 
> I.e., you can just as well pass in func and offset separately and be
> done with it.
> 
> But maybe there's something else happening in the patches which comes
> later and which will make me go, ahaa.
>

You're right that the values should be immutable. The changes done here
are only for this pair of base/limit registers. Most of the time we'll
only use 2 pairs (4 registers). But some systems will need to look at 16
pairs, and so this current approach seemed nicer than writing out 32
registers with mostly redundant information.

I was trying to make the code more "self-documenting" and move away from
magic numbers, etc. But it all looks okay to me, so I'm not sure which
way to go (magic numbers + code comments, something else, etc.).

So I'm inclined to stick with passing in the func/offset values and
dropping the df_regs thing.

Thanks,
Yazen


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 04/31] EDAC/amd64: Allow for DF Indirect Broadcast reads
  2021-06-30 16:22   ` Borislav Petkov
@ 2021-07-08 19:44     ` Yazen Ghannam
  0 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-07-08 19:44 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 30, 2021 at 06:22:54PM +0200, Borislav Petkov wrote:
> On Wed, Jun 23, 2021 at 07:19:35PM +0000, Yazen Ghannam wrote:
> > The DF Indirect Access method allows for "Broadcast" accesses in which
> > case no specific instance is targeted. Add support using a reserved
> > instance ID of 0xFF to indicate a broadcast access. Set the FICAA
> > register appropriately.
> > 
> > Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
> > ---
> > Link:
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flkml.kernel.org%2Fr%2F20210507190140.18854-1-Yazen.Ghannam%40amd.com&amp;data=04%7C01%7Cyazen.ghannam%40amd.com%7C83af21ab3dcc4b529ff008d93be354cb%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637606669851041887%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=DVzJaZy8OOETbvcISwzhg7%2FP3iHjOiP%2BOgMFd8rrrQk%3D&amp;reserved=0
> > 
> > v1->v2:
> > * New in v2.
> > 
> >  drivers/edac/amd64_edac.c | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
> > index b94067e3952b..d67cd8f57b94 100644
> > --- a/drivers/edac/amd64_edac.c
> > +++ b/drivers/edac/amd64_edac.c
> > @@ -1010,7 +1010,11 @@ struct df_reg {
> >   *
> >   * Fabric Indirect Configuration Access Data (FICAD): There are FICAD LO
> >   * and FICAD HI registers but so far we only need the LO register.
> > + *
> > + * Use Instance Id 0xFF to indicate a broadcast read.
> >   */
> > +
> > +#define DF_BROADCAST	0xFF
> >  static int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32 *lo)
> >  {
> >  	struct pci_dev *F4;
> > @@ -1024,7 +1028,7 @@ static int amd_df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32
> >  	if (!F4)
> >  		goto out;
> >  
> > -	ficaa  = 1;
> > +	ficaa  = (instance_id == DF_BROADCAST) ? 0 : 1;
> 
> Or, you can define two functions:
> 
> df_indirect_read_broadcast()
> df_indirect_read_umc()
> 
> (no need for the "amd_" prefix either - this is a static function now)
> 
> which both call a low-level helper:
> 
> static int __df_indirect_read(u16 node, struct df_reg reg, u8 instance_id, u32 *lo)
> 
> and there you either pass the instance_id or 0xff depending on which
> outer function is calling it...
>

Yeah, I like it. I'll see what I can do.

BTW, I'm thinking to include a "tmp" or "scratch" u32 value in the
context used to hold raw register values that don't need to be saved
long term. There are a few places where a value is read and some fields
are extracted, so a few functions have a u32 tmp variable declared. What
do you think?

Thanks,
Yazen

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 05/31] EDAC/amd64: Add context struct
  2021-06-30 17:17   ` Borislav Petkov
@ 2021-07-08 19:53     ` Yazen Ghannam
  0 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-07-08 19:53 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 30, 2021 at 07:17:41PM +0200, Borislav Petkov wrote:
> On Wed, Jun 23, 2021 at 07:19:36PM +0000, Yazen Ghannam wrote:
> > Define an address translation context struct. This will hold values that
> > will be passed between multiple functions.
> > 
> > Save return address, Node ID, and the Instance ID number to start.
> > Currently, we use the UMC number as the Instance ID, but future DF
> 
> Please use passive voice in your commit message: no "we" or "I", etc.
> 
> And this here is a perfect example: it sounds here like "we" is "AMD"
> but we use "we" mostly for the kernel. And there's the confusion.
> 
> So please teach yourself to formulate those commit messages properly -
> the future you will thank you, trust me!
> 
> :-)
>

Yeah, sorry I always seem to let it slip. :/

...

> > @@ -1097,6 +1100,16 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
> >  
> >  	struct df_reg reg;
> >  
> > +	struct addr_ctx ctx;
> 
> Those empty-lines spaced-out local vars look weird.
> 

Okay, I'll fix that.

...

> > @@ -1236,14 +1249,14 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
> >  		 * bits there are. "intlv_addr_bit" tells us how many "Y" bits
> >  		 * there are (where "I" starts).
> >  		 */
> > -		temp_addr_y = ret_addr & GENMASK_ULL(intlv_addr_bit-1, 0);
> > +		temp_addr_y = ctx.ret_addr & GENMASK_ULL(intlv_addr_bit - 1, 0);
> >  		temp_addr_i = (cs_id << intlv_addr_bit);
> > -		temp_addr_x = (ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
> > -		ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
> > +		temp_addr_x = (ctx.ret_addr & GENMASK_ULL(63, intlv_addr_bit)) << num_intlv_bits;
> > +		ctx.ret_addr    = temp_addr_x | temp_addr_i | temp_addr_y;
> 
> You want to align those vertically on the "=" for better readability.
> 

Most of this will be removed in later patches, so I didn't think to
change any more than is necessary to make sure the patch builds.

I'll check the end result to make sure things are aligned nicely.

> >  	}
> >  
> >  	/* Add dram base address */
> > -	ret_addr += dram_base_addr;
> > +	ctx.ret_addr += dram_base_addr;
> >  
> >  	/* If legacy MMIO hole enabled */
> >  	if (lgcy_mmio_hole_en) {
> > @@ -1251,29 +1264,29 @@ static int umc_normaddr_to_sysaddr(u64 norm_addr, u16 nid, u8 umc, u64 *sys_addr
> >  			goto out_err;
> >  
> >  		dram_hole_base = tmp & GENMASK(31, 24);
> > -		if (ret_addr >= dram_hole_base)
> > -			ret_addr += (BIT_ULL(32) - dram_hole_base);
> > +		if (ctx.ret_addr >= dram_hole_base)
> > +			ctx.ret_addr += (BIT_ULL(32) - dram_hole_base);
> >  	}
> >  
> >  	if (hash_enabled) {
> >  		/* Save some parentheses and grab ls-bit at the end. */
> > -		hashed_bit =	(ret_addr >> 12) ^
> > -				(ret_addr >> 18) ^
> > -				(ret_addr >> 21) ^
> > -				(ret_addr >> 30) ^
> > +		hashed_bit =	(ctx.ret_addr >> 12) ^
> > +				(ctx.ret_addr >> 18) ^
> > +				(ctx.ret_addr >> 21) ^
> > +				(ctx.ret_addr >> 30) ^
> >  				cs_id;
> >  
> >  		hashed_bit &= BIT(0);
> >  
> > -		if (hashed_bit != ((ret_addr >> intlv_addr_bit) & BIT(0)))
> > -			ret_addr ^= BIT(intlv_addr_bit);
> > +		if (hashed_bit != ((ctx.ret_addr >> intlv_addr_bit) & BIT(0)))
> > +			ctx.ret_addr ^= BIT(intlv_addr_bit);
> >  	}
> >  
> >  	/* Is calculated system address is above DRAM limit address? */
> > -	if (ret_addr > dram_limit_addr)
> > +	if (ctx.ret_addr > dram_limit_addr)
> >  		goto out_err;
> >  
> > -	*sys_addr = ret_addr;
> > +	*sys_addr = ctx.ret_addr;
> 
> So adding ctx to exchange data between functions - that was a good idea.
> 
> But what this patch does is pointless because you simply replace those
> variables with a local struct.
> 
> I guess the aha moment will come with the later patches when you start
> passing it around to functions.
> 
> /me waits to see.
>

That's true. I was trying to make each patch a single logical change, so
there are a few that have a lot of churn. I can squash some of the
patches together if that's okay.

Thanks,
Yazen

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 06/31] EDAC/amd64: Define Data Fabric operations
  2021-06-30 17:19   ` Borislav Petkov
@ 2021-07-08 19:55     ` Yazen Ghannam
  0 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-07-08 19:55 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 30, 2021 at 07:19:45PM +0200, Borislav Petkov wrote:
> On Wed, Jun 23, 2021 at 07:19:37PM +0000, Yazen Ghannam wrote:

...

> >  
> > +	if (set_df_ops(&ctx))
> > +		return -EINVAL;
> > +
> >  	if (amd_df_indirect_read(nid, df_regs[DRAM_OFFSET], umc, &tmp))
> >  		goto out_err;
> >  
> > -- 
> 
> Why?
> 
> You can simply assign to df_ops as this is all driver-static data.
>

Good point. We can set this in per_family_init().

Thanks,
Yazen

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 07/31] EDAC/amd64: Define functions for DramOffset
  2021-06-30 17:27   ` Borislav Petkov
@ 2021-07-08 20:08     ` Yazen Ghannam
  0 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-07-08 20:08 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 30, 2021 at 07:27:07PM +0200, Borislav Petkov wrote:
> On Wed, Jun 23, 2021 at 07:19:38PM +0000, Yazen Ghannam wrote:

...

> >  struct data_fabric_ops {
> > +	u64 (*get_hi_addr_offset)(struct addr_ctx *ctx);
> >  };
> >  
> > +static u64 get_hi_addr_offset_df2(struct addr_ctx *ctx)
> > +{
> > +	return (ctx->reg_dram_offset & GENMASK_ULL(31, 20)) << 8;
> > +}
> > +
> >  struct data_fabric_ops df2_ops = {
> 
> Those "df2" ops are for UMC v2? 
>

No, Data Fabric 2 is used in the first Zen products (Ryzen, Naples,
etc.)

> > +	.get_hi_addr_offset		=	&get_hi_addr_offset_df2,
> 						^
> 
> No need for the "&".
>

Okay.

> Also, if you define a member ->get_hi_addr_offset in df2_ops, then you
> need to define that member in *every* df_ops structure you have so that
> there's no NULL pointers.
> 
> I guess that's coming...
>

Yes, a function goes into df_ops only when details change between Data
Fabric versions, i.e. different bitfields, registers, etc., but the flow
is otherwise the same.

I'll give the NULL pointer concern more thought. I've tried to cover all
code paths and use cases testing on real hardware. But it's possible
something can be missed with how configurable our systems can be.

Thanks,
Yazen

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH v2 09/31] EDAC/amd64: Define function to find interleaving mode
  2021-06-30 17:33   ` Borislav Petkov
@ 2021-07-08 20:09     ` Yazen Ghannam
  0 siblings, 0 replies; 45+ messages in thread
From: Yazen Ghannam @ 2021-07-08 20:09 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: linux-edac, linux-kernel, mchehab, tony.luck,
	Smita.KoralahalliChannabasappa

On Wed, Jun 30, 2021 at 07:33:43PM +0200, Borislav Petkov wrote:
> On Wed, Jun 23, 2021 at 07:19:40PM +0000, Yazen Ghannam wrote:
> > @@ -1088,10 +1096,12 @@ struct addr_ctx {
> >  	u16 nid;
> >  	u8 inst_id;
> >  	u8 map_num;
> > +	bool hash_enabled;
> 
> For such single bits info you can use bitfields, like struct mca_config
> does it, for example.
>

Good idea. This particular field gets replaced later with a function
pointer. But I'll keep the bitfield idea in mind.

Thanks,
Yazen

^ permalink raw reply	[flat|nested] 45+ messages in thread

end of thread, other threads:[~2021-07-08 20:10 UTC | newest]

Thread overview: 45+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-23 19:19 [PATCH v2 00/31] AMD MCA Address Translation Updates Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 01/31] x86/MCE/AMD, EDAC/amd64: Move address translation to AMD64 EDAC Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 02/31] x86/amd_nb, EDAC/amd64: Move DF Indirect Read " Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 03/31] EDAC/amd64: Don't use naked values for DF registers Yazen Ghannam
2021-06-25 15:21   ` Borislav Petkov
2021-07-08 19:35     ` Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 04/31] EDAC/amd64: Allow for DF Indirect Broadcast reads Yazen Ghannam
2021-06-30 16:22   ` Borislav Petkov
2021-07-08 19:44     ` Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 05/31] EDAC/amd64: Add context struct Yazen Ghannam
2021-06-30 17:17   ` Borislav Petkov
2021-07-08 19:53     ` Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 06/31] EDAC/amd64: Define Data Fabric operations Yazen Ghannam
2021-06-30 17:19   ` Borislav Petkov
2021-07-08 19:55     ` Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 07/31] EDAC/amd64: Define functions for DramOffset Yazen Ghannam
2021-06-30 17:27   ` Borislav Petkov
2021-07-08 20:08     ` Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 08/31] EDAC/amd64: Define function to read DRAM address map registers Yazen Ghannam
2021-06-30 17:29   ` Borislav Petkov
2021-06-23 19:19 ` [PATCH v2 09/31] EDAC/amd64: Define function to find interleaving mode Yazen Ghannam
2021-06-30 17:33   ` Borislav Petkov
2021-07-08 20:09     ` Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 10/31] EDAC/amd64: Define function to denormalize address Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 11/31] EDAC/amd64: Define function to add DRAM base and hole Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 12/31] EDAC/amd64: Define function to dehash address Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 13/31] EDAC/amd64: Define function to check DRAM limit address Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 14/31] EDAC/amd64: Remove goto statements Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 15/31] EDAC/amd64: Simplify function parameters Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 16/31] EDAC/amd64: Define function to get Interleave Address Bit Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 17/31] EDAC/amd64: Skip denormalization if no interleaving Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 18/31] EDAC/amd64: Define function to get number of interleaved channels Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 19/31] EDAC/amd64: Define function to get number of interleaved dies Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 20/31] EDAC/amd64: Define function to get number of interleaved sockets Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 21/31] EDAC/amd64: Remove unnecessary assert Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 22/31] EDAC/amd64: Define function to make space for CS ID Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 23/31] EDAC/amd64: Define function to calculate " Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 24/31] EDAC/amd64: Define function to insert CS ID into address Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 25/31] EDAC/amd64: Define function to get CS Fabric ID Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 26/31] EDAC/amd64: Define function to find shift and mask values Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 27/31] EDAC/amd64: Update CS ID calculation to match reference code Yazen Ghannam
2021-06-23 19:19 ` [PATCH v2 28/31] EDAC/amd64: Match hash function to " Yazen Ghannam
2021-06-23 19:20 ` [PATCH v2 29/31] EDAC/amd64: Define helper function to get interleave address select bit Yazen Ghannam
2021-06-23 19:20 ` [PATCH v2 30/31] EDAC/amd64: Add support for address translation on DF3 systems Yazen Ghannam
2021-06-23 19:20 ` [PATCH v2 31/31] EDAC/amd64: Add glossary of acronyms for address translation Yazen Ghannam

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).