[PATCH v10 00/12] riscv: support for Svpbmt and D1 memory types

[PATCH v10 00/12] riscv: support for Svpbmt and D1 memory types

[Heiko Stuebner]

Svpbmt is an extension defining "Supervisor-mode: page-based memory types"
for things like non-cacheable pages or I/O memory pages.


So this is my 2nd try at implementing Svpbmt (and the diverging D1 memory
types) using the alternatives framework.

This includes a number of changes to the alternatives mechanism itself.
The biggest one being the move to a more central location, as I expect
in the future, nearly every chip needing some sort of patching, be it
either for erratas or for optional features (svpbmt or others).

Detection of the svpbmt functionality is done via Atish's isa extension
handling series [0] and thus does not need any dt-parsing of its own
anymore.

The series also introduces support for the memory types of the D1
which are implemented differently to svpbmt. But when patching anyway
it's pretty clean to add the D1 variant via ALTERNATIVE_2 to the same
location.

The only slightly bigger difference is that the "normal" type is not 0
as with svpbmt, so kernel patches for this PMA type need to be applied
even before the MMU is brought up, so the series introduces a separate
stage for that.


In theory this series is 2 parts:
- alternatives improvements
- svpbmt+d1

I picked the recipient list from the previous versions, hopefully
I didn't forget anybody.

I tested the series on:
- qemu-rv32 + buildroot rootfs
- qemu-rv64 + debian roots
- Allwinner D1-Nezha
- BeagleV - it at least reached the same point as without the series

I also ran Palmers CI environment on 5.18-rc6 + this series and
it passed with all testcases now.


changes in v10:
- add received review-tags
- put early patching behind a kconfig symbol
- adapt compiler flags of sources in use by early patching
  similar to other riscv arch-parts.
  This fixes the medlow cmodel issue on rv32 and also issues
  with Kasan.

changes in v9:
- rebase onto 5.18-rc1
- drop the sbi null-ptr patch
  While I still think this to be non-ideal as is, it isn't really
  necessary for svpbmt support anymore
- merge cpufeature + svpbmt patch, as otherwise some empty shells
  cause build warnings when a bisection stops between these two
  patches
- address review comments from Christoph Hellwig:
  - keep alternatives optional, they now get selected by its
    users (erratas and also the newly introduced svpbmt kconfig)
  - wrap long lines and keep things below 80 characters
  - restyle svpbmt + thead errata assembly
  - introduce a helper for the repeated calls to
    (val & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT

changes in v8:
- rebase onto 5.17-final + isa extension series
  We're halfway through the merge-window, so this series
  should be merge after that
- adapt to fix limiting alternatives to non-xip-kernels
- add .norelax option for alternatives
- fix unused cpu_apply_errata in thead errata
- don't use static globals to store cpu-manufacturer data
  as it makes machines hang if done too early

changes in v7:
- fix typo in patch1 (Atish)
- moved to Atish's isa-extension framework
- and therefore move regular boot-alternatives directly behind fill_hwcaps
- change T-Head errata Kconfig text (Atish)

changes in v6:
- rebase onto 5.17-rc1
- handle sbi null-ptr differently
- improve commit messages
- use riscv,mmu as property name

changes in v5:
- move to use alternatives for runtime-patching
- add D1 variant


[0] https://lore.kernel.org/r/20220222204811.2281949-2-atishp@rivosinc.com

Heiko Stuebner (12):
  riscv: integrate alternatives better into the main architecture
  riscv: allow different stages with alternatives
  riscv: implement module alternatives
  riscv: implement ALTERNATIVE_2 macro
  riscv: extend concatenated alternatives-lines to the same length
  riscv: prevent compressed instructions in alternatives
  riscv: move boot alternatives to after fill_hwcap
  riscv: Fix accessing pfn bits in PTEs for non-32bit variants
  riscv: add RISC-V Svpbmt extension support
  riscv: remove FIXMAP_PAGE_IO and fall back to its default value
  riscv: don't use global static vars to store alternative data
  riscv: add memory-type errata for T-Head

 arch/riscv/Kconfig                          |  28 +++++
 arch/riscv/Kconfig.erratas                  |  34 ++++--
 arch/riscv/Kconfig.socs                     |   1 -
 arch/riscv/Makefile                         |   2 +-
 arch/riscv/errata/Makefile                  |   2 +-
 arch/riscv/errata/alternative.c             |  75 ------------
 arch/riscv/errata/sifive/errata.c           |  20 ++-
 arch/riscv/errata/thead/Makefile            |  11 ++
 arch/riscv/errata/thead/errata.c            |  82 +++++++++++++
 arch/riscv/include/asm/alternative-macros.h | 129 +++++++++++++++-----
 arch/riscv/include/asm/alternative.h        |  25 +++-
 arch/riscv/include/asm/errata_list.h        |  59 +++++++++
 arch/riscv/include/asm/fixmap.h             |   2 -
 arch/riscv/include/asm/hwcap.h              |   1 +
 arch/riscv/include/asm/pgtable-32.h         |  17 +++
 arch/riscv/include/asm/pgtable-64.h         |  79 +++++++++++-
 arch/riscv/include/asm/pgtable-bits.h       |  10 --
 arch/riscv/include/asm/pgtable.h            |  55 +++++++--
 arch/riscv/include/asm/vendorid_list.h      |   1 +
 arch/riscv/kernel/Makefile                  |  15 +++
 arch/riscv/kernel/alternative.c             | 118 ++++++++++++++++++
 arch/riscv/kernel/cpu.c                     |   1 +
 arch/riscv/kernel/cpufeature.c              |  80 +++++++++++-
 arch/riscv/kernel/module.c                  |  29 +++++
 arch/riscv/kernel/setup.c                   |   2 +
 arch/riscv/kernel/smpboot.c                 |   4 -
 arch/riscv/kernel/traps.c                   |   2 +-
 arch/riscv/mm/init.c                        |   1 +
 28 files changed, 724 insertions(+), 161 deletions(-)
 delete mode 100644 arch/riscv/errata/alternative.c
 create mode 100644 arch/riscv/errata/thead/Makefile
 create mode 100644 arch/riscv/errata/thead/errata.c
 create mode 100644 arch/riscv/kernel/alternative.c

-- 
2.35.1

[PATCH 01/12] riscv: integrate alternatives better into the main architecture

[Heiko Stuebner]

Right now the alternatives need to be explicitly enabled and
erratas are limited to SiFive ones.

We want to use alternatives not only for patching soc erratas,
but in the future also for handling different behaviour depending
on the existence of future extensions.

So move the core alternatives over to the kernel subdirectory
and move the CONFIG_RISCV_ALTERNATIVE to be a hidden symbol
which we expect relevant erratas and extensions to just select
if needed.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
---
 arch/riscv/Kconfig                          |  9 +++++++++
 arch/riscv/Kconfig.erratas                  | 13 ++-----------
 arch/riscv/Kconfig.socs                     |  1 -
 arch/riscv/Makefile                         |  2 +-
 arch/riscv/errata/Makefile                  |  1 -
 arch/riscv/include/asm/alternative-macros.h |  7 ++++---
 arch/riscv/include/asm/alternative.h        |  8 ++++++++
 arch/riscv/kernel/Makefile                  |  1 +
 arch/riscv/{errata => kernel}/alternative.c |  0
 arch/riscv/kernel/smpboot.c                 |  2 --
 arch/riscv/kernel/traps.c                   |  2 +-
 11 files changed, 26 insertions(+), 20 deletions(-)
 rename arch/riscv/{errata => kernel}/alternative.c (100%)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index f8a55d94016d..1ec07aa582a3 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -325,6 +325,15 @@ config NODES_SHIFT
 	  Specify the maximum number of NUMA Nodes available on the target
 	  system.  Increases memory reserved to accommodate various tables.
 
+config RISCV_ALTERNATIVE
+	bool
+	depends on !XIP_KERNEL
+	help
+	  This Kconfig allows the kernel to automatically patch the
+	  errata required by the execution platform at run time. The
+	  code patching is performed once in the boot stages. It means
+	  that the overhead from this mechanism is just taken once.
+
 config RISCV_ISA_C
 	bool "Emit compressed instructions when building Linux"
 	default y
diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
index 0aacd7052585..c521c2ae2de2 100644
--- a/arch/riscv/Kconfig.erratas
+++ b/arch/riscv/Kconfig.erratas
@@ -1,18 +1,9 @@
 menu "CPU errata selection"
 
-config RISCV_ERRATA_ALTERNATIVE
-	bool "RISC-V alternative scheme"
-	depends on !XIP_KERNEL
-	default y
-	help
-	  This Kconfig allows the kernel to automatically patch the
-	  errata required by the execution platform at run time. The
-	  code patching is performed once in the boot stages. It means
-	  that the overhead from this mechanism is just taken once.
-
 config ERRATA_SIFIVE
 	bool "SiFive errata"
-	depends on RISCV_ERRATA_ALTERNATIVE
+	depends on !XIP_KERNEL
+	select RISCV_ALTERNATIVE
 	help
 	  All SiFive errata Kconfig depend on this Kconfig. Disabling
 	  this Kconfig will disable all SiFive errata. Please say "Y"
diff --git a/arch/riscv/Kconfig.socs b/arch/riscv/Kconfig.socs
index f6ef358d8a2c..85670dc9fe95 100644
--- a/arch/riscv/Kconfig.socs
+++ b/arch/riscv/Kconfig.socs
@@ -14,7 +14,6 @@ config SOC_SIFIVE
 	select CLK_SIFIVE
 	select CLK_SIFIVE_PRCI
 	select SIFIVE_PLIC
-	select RISCV_ERRATA_ALTERNATIVE if !XIP_KERNEL
 	select ERRATA_SIFIVE if !XIP_KERNEL
 	help
 	  This enables support for SiFive SoC platform hardware.
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 7d81102cffd4..a7ed47ce9311 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -103,7 +103,7 @@ endif
 
 head-y := arch/riscv/kernel/head.o
 
-core-$(CONFIG_RISCV_ERRATA_ALTERNATIVE) += arch/riscv/errata/
+core-y += arch/riscv/errata/
 core-$(CONFIG_KVM) += arch/riscv/kvm/
 
 libs-y += arch/riscv/lib/
diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
index b8f8740a3e44..0ca1c5281a2d 100644
--- a/arch/riscv/errata/Makefile
+++ b/arch/riscv/errata/Makefile
@@ -1,2 +1 @@
-obj-y	+= alternative.o
 obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index 67406c376389..5dd8d03a13da 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -2,7 +2,7 @@
 #ifndef __ASM_ALTERNATIVE_MACROS_H
 #define __ASM_ALTERNATIVE_MACROS_H
 
-#ifdef CONFIG_RISCV_ERRATA_ALTERNATIVE
+#ifdef CONFIG_RISCV_ALTERNATIVE
 
 #ifdef __ASSEMBLY__
 
@@ -76,7 +76,7 @@
 
 #endif /* __ASSEMBLY__ */
 
-#else /* !CONFIG_RISCV_ERRATA_ALTERNATIVE*/
+#else /* CONFIG_RISCV_ALTERNATIVE */
 #ifdef __ASSEMBLY__
 
 .macro __ALTERNATIVE_CFG old_c
@@ -95,7 +95,8 @@
 	__ALTERNATIVE_CFG(old_c)
 
 #endif /* __ASSEMBLY__ */
-#endif /* CONFIG_RISCV_ERRATA_ALTERNATIVE */
+#endif /* CONFIG_RISCV_ALTERNATIVE */
+
 /*
  * Usage:
  *   ALTERNATIVE(old_content, new_content, vendor_id, errata_id, CONFIG_k)
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index e625d3cafbed..7b42bcef0ecf 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -12,6 +12,8 @@
 
 #ifndef __ASSEMBLY__
 
+#ifdef CONFIG_RISCV_ALTERNATIVE
+
 #include <linux/init.h>
 #include <linux/types.h>
 #include <linux/stddef.h>
@@ -35,5 +37,11 @@ struct errata_checkfunc_id {
 void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
 			      unsigned long archid, unsigned long impid);
 
+#else /* CONFIG_RISCV_ALTERNATIVE */
+
+static inline void apply_boot_alternatives(void) { }
+
+#endif /* CONFIG_RISCV_ALTERNATIVE */
+
 #endif
 #endif
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 87adbe47bc15..0f8348ac30f1 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -18,6 +18,7 @@ extra-y += head.o
 extra-y += vmlinux.lds
 
 obj-y	+= soc.o
+obj-$(CONFIG_RISCV_ALTERNATIVE) += alternative.o
 obj-y	+= cpu.o
 obj-y	+= cpufeature.o
 obj-y	+= entry.o
diff --git a/arch/riscv/errata/alternative.c b/arch/riscv/kernel/alternative.c
similarity index 100%
rename from arch/riscv/errata/alternative.c
rename to arch/riscv/kernel/alternative.c
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index 622f226454d5..a6d13dca1403 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -41,9 +41,7 @@ static DECLARE_COMPLETION(cpu_running);
 void __init smp_prepare_boot_cpu(void)
 {
 	init_cpu_topology();
-#ifdef CONFIG_RISCV_ERRATA_ALTERNATIVE
 	apply_boot_alternatives();
-#endif
 }
 
 void __init smp_prepare_cpus(unsigned int max_cpus)
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index fe92e119e6a3..efa693b325a1 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -86,7 +86,7 @@ static void do_trap_error(struct pt_regs *regs, int signo, int code,
 	}
 }
 
-#if defined (CONFIG_XIP_KERNEL) && defined (CONFIG_RISCV_ERRATA_ALTERNATIVE)
+#if defined (CONFIG_XIP_KERNEL) && defined (CONFIG_RISCV_ALTERNATIVE)
 #define __trap_section		__section(".xip.traps")
 #else
 #define __trap_section
-- 
2.35.1

[PATCH 02/12] riscv: allow different stages with alternatives

[Heiko Stuebner]

Future features may need to be applied at a different
time during boot, so allow defining stages for alternatives
and handling them differently depending on the stage.

Also make the alternatives-location more flexible so that
future stages may provide their own location.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
---
 arch/riscv/errata/sifive/errata.c    |  3 ++-
 arch/riscv/include/asm/alternative.h |  5 ++++-
 arch/riscv/kernel/alternative.c      | 26 +++++++++++++++++---------
 3 files changed, 23 insertions(+), 11 deletions(-)

diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
index f5e5ae70e829..4fe03ac41fd7 100644
--- a/arch/riscv/errata/sifive/errata.c
+++ b/arch/riscv/errata/sifive/errata.c
@@ -80,7 +80,8 @@ static void __init warn_miss_errata(u32 miss_errata)
 }
 
 void __init sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
-				     unsigned long archid, unsigned long impid)
+				     unsigned long archid, unsigned long impid,
+				     unsigned int stage)
 {
 	struct alt_entry *alt;
 	u32 cpu_req_errata = sifive_errata_probe(archid, impid);
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index 7b42bcef0ecf..0ff550667e94 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -19,6 +19,8 @@
 #include <linux/stddef.h>
 #include <asm/hwcap.h>
 
+#define RISCV_ALTERNATIVES_BOOT		0 /* alternatives applied during regular boot */
+
 void __init apply_boot_alternatives(void);
 
 struct alt_entry {
@@ -35,7 +37,8 @@ struct errata_checkfunc_id {
 };
 
 void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
-			      unsigned long archid, unsigned long impid);
+			      unsigned long archid, unsigned long impid,
+			      unsigned int stage);
 
 #else /* CONFIG_RISCV_ALTERNATIVE */
 
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index e8b4a0fe488c..02db62f55bac 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -22,8 +22,8 @@ static struct cpu_manufacturer_info_t {
 } cpu_mfr_info;
 
 static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
-				 unsigned long archid,
-				 unsigned long impid) __initdata;
+				 unsigned long archid, unsigned long impid,
+				 unsigned int stage) __initdata;
 
 static inline void __init riscv_fill_cpu_mfr_info(void)
 {
@@ -58,6 +58,18 @@ static void __init init_alternative(void)
  * a feature detect on the boot CPU). No need to worry about other CPUs
  * here.
  */
+static void __init _apply_alternatives(struct alt_entry *begin,
+				       struct alt_entry *end,
+				       unsigned int stage)
+{
+	if (!vendor_patch_func)
+		return;
+
+	vendor_patch_func(begin, end,
+			  cpu_mfr_info.arch_id, cpu_mfr_info.imp_id,
+			  stage);
+}
+
 void __init apply_boot_alternatives(void)
 {
 	/* If called on non-boot cpu things could go wrong */
@@ -65,11 +77,7 @@ void __init apply_boot_alternatives(void)
 
 	init_alternative();
 
-	if (!vendor_patch_func)
-		return;
-
-	vendor_patch_func((struct alt_entry *)__alt_start,
-			  (struct alt_entry *)__alt_end,
-			  cpu_mfr_info.arch_id, cpu_mfr_info.imp_id);
+	_apply_alternatives((struct alt_entry *)__alt_start,
+			    (struct alt_entry *)__alt_end,
+			    RISCV_ALTERNATIVES_BOOT);
 }
-
-- 
2.35.1

[PATCH 03/12] riscv: implement module alternatives

[Heiko Stuebner]

This allows alternatives to also be applied when loading modules
and follows the implementation of other architectures (e.g. arm64).

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
---
 arch/riscv/errata/sifive/errata.c    | 14 +++++++++-----
 arch/riscv/include/asm/alternative.h |  3 +++
 arch/riscv/kernel/alternative.c      | 18 +++++++++++++----
 arch/riscv/kernel/module.c           | 29 ++++++++++++++++++++++++++++
 4 files changed, 55 insertions(+), 9 deletions(-)

diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
index 4fe03ac41fd7..3e39587a49dc 100644
--- a/arch/riscv/errata/sifive/errata.c
+++ b/arch/riscv/errata/sifive/errata.c
@@ -4,6 +4,7 @@
  */
 
 #include <linux/kernel.h>
+#include <linux/module.h>
 #include <linux/string.h>
 #include <linux/bug.h>
 #include <asm/patch.h>
@@ -54,7 +55,8 @@ static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
 	},
 };
 
-static u32 __init sifive_errata_probe(unsigned long archid, unsigned long impid)
+static u32 __init_or_module sifive_errata_probe(unsigned long archid,
+						unsigned long impid)
 {
 	int idx;
 	u32 cpu_req_errata = 0;
@@ -66,7 +68,7 @@ static u32 __init sifive_errata_probe(unsigned long archid, unsigned long impid)
 	return cpu_req_errata;
 }
 
-static void __init warn_miss_errata(u32 miss_errata)
+static void __init_or_module warn_miss_errata(u32 miss_errata)
 {
 	int i;
 
@@ -79,9 +81,11 @@ static void __init warn_miss_errata(u32 miss_errata)
 	pr_warn("----------------------------------------------------------------\n");
 }
 
-void __init sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
-				     unsigned long archid, unsigned long impid,
-				     unsigned int stage)
+void __init_or_module sifive_errata_patch_func(struct alt_entry *begin,
+					       struct alt_entry *end,
+					       unsigned long archid,
+					       unsigned long impid,
+					       unsigned int stage)
 {
 	struct alt_entry *alt;
 	u32 cpu_req_errata = sifive_errata_probe(archid, impid);
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index 0ff550667e94..0859529ff08e 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -20,8 +20,10 @@
 #include <asm/hwcap.h>
 
 #define RISCV_ALTERNATIVES_BOOT		0 /* alternatives applied during regular boot */
+#define RISCV_ALTERNATIVES_MODULE	1 /* alternatives applied during module-init */
 
 void __init apply_boot_alternatives(void);
+void apply_module_alternatives(void *start, size_t length);
 
 struct alt_entry {
 	void *old_ptr;		 /* address of original instruciton or data  */
@@ -43,6 +45,7 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
 #else /* CONFIG_RISCV_ALTERNATIVE */
 
 static inline void apply_boot_alternatives(void) { }
+static inline void apply_module_alternatives(void *start, size_t length) { }
 
 #endif /* CONFIG_RISCV_ALTERNATIVE */
 
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index 02db62f55bac..223770b3945c 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -7,6 +7,7 @@
  */
 
 #include <linux/init.h>
+#include <linux/module.h>
 #include <linux/cpu.h>
 #include <linux/uaccess.h>
 #include <asm/alternative.h>
@@ -23,7 +24,7 @@ static struct cpu_manufacturer_info_t {
 
 static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
 				 unsigned long archid, unsigned long impid,
-				 unsigned int stage) __initdata;
+				 unsigned int stage) __initdata_or_module;
 
 static inline void __init riscv_fill_cpu_mfr_info(void)
 {
@@ -58,9 +59,9 @@ static void __init init_alternative(void)
  * a feature detect on the boot CPU). No need to worry about other CPUs
  * here.
  */
-static void __init _apply_alternatives(struct alt_entry *begin,
-				       struct alt_entry *end,
-				       unsigned int stage)
+static void __init_or_module _apply_alternatives(struct alt_entry *begin,
+						 struct alt_entry *end,
+						 unsigned int stage)
 {
 	if (!vendor_patch_func)
 		return;
@@ -81,3 +82,12 @@ void __init apply_boot_alternatives(void)
 			    (struct alt_entry *)__alt_end,
 			    RISCV_ALTERNATIVES_BOOT);
 }
+
+#ifdef CONFIG_MODULES
+void apply_module_alternatives(void *start, size_t length)
+{
+	_apply_alternatives((struct alt_entry *)start,
+			    (struct alt_entry *)(start + length),
+			    RISCV_ALTERNATIVES_MODULE);
+}
+#endif
diff --git a/arch/riscv/kernel/module.c b/arch/riscv/kernel/module.c
index c29cef90d1dd..91fe16bfaa07 100644
--- a/arch/riscv/kernel/module.c
+++ b/arch/riscv/kernel/module.c
@@ -11,6 +11,7 @@
 #include <linux/vmalloc.h>
 #include <linux/sizes.h>
 #include <linux/pgtable.h>
+#include <asm/alternative.h>
 #include <asm/sections.h>
 
 /*
@@ -427,3 +428,31 @@ void *module_alloc(unsigned long size)
 				    __builtin_return_address(0));
 }
 #endif
+
+static const Elf_Shdr *find_section(const Elf_Ehdr *hdr,
+				    const Elf_Shdr *sechdrs,
+				    const char *name)
+{
+	const Elf_Shdr *s, *se;
+	const char *secstrs = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
+
+	for (s = sechdrs, se = sechdrs + hdr->e_shnum; s < se; s++) {
+		if (strcmp(name, secstrs + s->sh_name) == 0)
+			return s;
+	}
+
+	return NULL;
+}
+
+int module_finalize(const Elf_Ehdr *hdr,
+		    const Elf_Shdr *sechdrs,
+		    struct module *me)
+{
+	const Elf_Shdr *s;
+
+	s = find_section(hdr, sechdrs, ".alternative");
+	if (s)
+		apply_module_alternatives((void *)s->sh_addr, s->sh_size);
+
+	return 0;
+}
-- 
2.35.1

[PATCH 04/12] riscv: implement ALTERNATIVE_2 macro

[Heiko Stuebner]

When the alternatives were added the commit already provided a template
on how to implement 2 different alternatives for one piece of code.

Make this usable.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
---
 arch/riscv/include/asm/alternative-macros.h | 78 +++++++++++++++------
 1 file changed, 58 insertions(+), 20 deletions(-)

diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index 5dd8d03a13da..9e04cd53afc8 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -39,6 +39,24 @@
 #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
 	__ALTERNATIVE_CFG old_c, new_c, vendor_id, errata_id, IS_ENABLED(CONFIG_k)
 
+.macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
+				  new_c_2, vendor_id_2, errata_id_2, enable_2
+886 :
+	\old_c
+887 :
+	ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
+	ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
+.endm
+
+#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,	\
+					CONFIG_k_1,			\
+				  new_c_2, vendor_id_2, errata_id_2,	\
+					CONFIG_k_2)			\
+	__ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1,	\
+					IS_ENABLED(CONFIG_k_1),		\
+				   new_c_2, vendor_id_2, errata_id_2,	\
+					IS_ENABLED(CONFIG_k_2)
+
 #else /* !__ASSEMBLY__ */
 
 #include <asm/asm.h>
@@ -74,6 +92,25 @@
 #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k)	\
 	__ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, IS_ENABLED(CONFIG_k))
 
+#define __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,	\
+					enable_1,			\
+				   new_c_2, vendor_id_2, errata_id_2,	\
+					enable_2)			\
+	"886 :\n"							\
+	old_c "\n"							\
+	"887 :\n"							\
+	ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1)	\
+	ALT_NEW_CONTENT(vendor_id_2, errata_id_2, enable_2, new_c_2)
+
+#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,	\
+					CONFIG_k_1,			\
+				  new_c_2, vendor_id_2, errata_id_2,	\
+					CONFIG_k_2)			\
+	__ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,	\
+					IS_ENABLED(CONFIG_k_1),		\
+				   new_c_2, vendor_id_2, errata_id_2,	\
+					IS_ENABLED(CONFIG_k_2))
+
 #endif /* __ASSEMBLY__ */
 
 #else /* CONFIG_RISCV_ALTERNATIVE */
@@ -86,6 +123,12 @@
 #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
 	__ALTERNATIVE_CFG old_c
 
+#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,	\
+					CONFIG_k_1,			\
+				  new_c_2, vendor_id_2, errata_id_2,	\
+					CONFIG_k_2)			\
+       __ALTERNATIVE_CFG old_c
+
 #else /* !__ASSEMBLY__ */
 
 #define __ALTERNATIVE_CFG(old_c)  \
@@ -94,6 +137,12 @@
 #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
 	__ALTERNATIVE_CFG(old_c)
 
+#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,	\
+					CONFIG_k_1,			\
+				  new_c_2, vendor_id_2, errata_id_2,	\
+					CONFIG_k_2) \
+       __ALTERNATIVE_CFG(old_c)
+
 #endif /* __ASSEMBLY__ */
 #endif /* CONFIG_RISCV_ALTERNATIVE */
 
@@ -119,25 +168,14 @@
  * this case, this vendor can create a new macro ALTERNATIVE_2() based
  * on the following sample code and then replace ALTERNATIVE() with
  * ALTERNATIVE_2() to append its customized content.
- *
- * .macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
- *                                   new_c_2, vendor_id_2, errata_id_2, enable_2
- * 886 :
- *      \old_c
- * 887 :
- *      ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
- *      ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
- * .endm
- *
- * #define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
- *                                   new_c_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
- *        __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, IS_ENABLED(CONFIG_k_1), \
- *                                   new_c_2, vendor_id_2, errata_id_2, IS_ENABLED(CONFIG_k_2) \
- *
- * #define ALTERNATIVE_2(old_content, new_content_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
- *                                    new_content_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
- *         _ALTERNATIVE_CFG_2(old_content, new_content_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
- *                                         new_content_2, vendor_id_2, errata_id_2, CONFIG_k_2)
- *
  */
+#define ALTERNATIVE_2(old_content, new_content_1, vendor_id_1,		\
+					errata_id_1, CONFIG_k_1,	\
+				   new_content_2, vendor_id_2,		\
+					errata_id_2, CONFIG_k_2)	\
+	_ALTERNATIVE_CFG_2(old_content, new_content_1, vendor_id_1,	\
+					    errata_id_1, CONFIG_k_1,	\
+					new_content_2, vendor_id_2,	\
+					    errata_id_2, CONFIG_k_2)
+
 #endif
-- 
2.35.1

[PATCH 05/12] riscv: extend concatenated alternatives-lines to the same length

[Heiko Stuebner]

ALT_NEW_CONTENT already uses same-length assembler lines, so
extend this to the other elements as well.

This makes it more readable when these elements need to be extended
in the future.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
---
 arch/riscv/include/asm/alternative-macros.h | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index 9e04cd53afc8..8c2bbc7bbe50 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -62,14 +62,14 @@
 #include <asm/asm.h>
 #include <linux/stringify.h>
 
-#define ALT_ENTRY(oldptr, newptr, vendor_id, errata_id, newlen) \
-	RISCV_PTR " " oldptr "\n" \
-	RISCV_PTR " " newptr "\n" \
-	REG_ASM " " vendor_id "\n" \
-	REG_ASM " " newlen "\n" \
+#define ALT_ENTRY(oldptr, newptr, vendor_id, errata_id, newlen)		\
+	RISCV_PTR " " oldptr "\n"					\
+	RISCV_PTR " " newptr "\n"					\
+	REG_ASM " " vendor_id "\n"					\
+	REG_ASM " " newlen "\n"						\
 	".word " errata_id "\n"
 
-#define ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c) \
+#define ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)		\
 	".if " __stringify(enable) " == 1\n"				\
 	".pushsection .alternative, \"a\"\n"				\
 	ALT_ENTRY("886b", "888f", __stringify(vendor_id), __stringify(errata_id), "889f - 888f") \
@@ -83,10 +83,10 @@
 	".org	. - (889b - 888b) + (887b - 886b)\n"			\
 	".endif\n"
 
-#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable) \
-	"886 :\n"	\
-	old_c "\n"	\
-	"887 :\n"	\
+#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable)	\
+	"886 :\n"							\
+	old_c "\n"							\
+	"887 :\n"							\
 	ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)
 
 #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k)	\
-- 
2.35.1

[PATCH 06/12] riscv: prevent compressed instructions in alternatives

[Heiko Stuebner]

Instructions are opportunistically compressed by the RISC-V assembler
when possible, but in alternatives-blocks both the old and new content
need to be the same size, so having the toolchain do somewhat random
optimizations will cause strange side-effects like
"attempt to move .org backwards" compile-time errors.

Already a simple "and" used in alternatives assembly will cause these
mismatched code sizes.

So prevent compressed instructions to be generated in alternatives-
code and use option-push and -pop to only limit this to the relevant
code blocks

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
---
 arch/riscv/include/asm/alternative-macros.h | 24 +++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
index 8c2bbc7bbe50..e13b1f6bb400 100644
--- a/arch/riscv/include/asm/alternative-macros.h
+++ b/arch/riscv/include/asm/alternative-macros.h
@@ -21,7 +21,11 @@
 	.popsection
 	.subsection 1
 888 :
+	.option push
+	.option norvc
+	.option norelax
 	\new_c
+	.option pop
 889 :
 	.previous
 	.org    . - (889b - 888b) + (887b - 886b)
@@ -31,7 +35,11 @@
 
 .macro __ALTERNATIVE_CFG old_c, new_c, vendor_id, errata_id, enable
 886 :
+	.option push
+	.option norvc
+	.option norelax
 	\old_c
+	.option pop
 887 :
 	ALT_NEW_CONTENT \vendor_id, \errata_id, \enable, \new_c
 .endm
@@ -42,7 +50,11 @@
 .macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
 				  new_c_2, vendor_id_2, errata_id_2, enable_2
 886 :
+	.option push
+	.option norvc
+	.option norelax
 	\old_c
+	.option pop
 887 :
 	ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
 	ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
@@ -76,7 +88,11 @@
 	".popsection\n"							\
 	".subsection 1\n"						\
 	"888 :\n"							\
+	".option push\n"						\
+	".option norvc\n"						\
+	".option norelax\n"						\
 	new_c "\n"							\
+	".option pop\n"							\
 	"889 :\n"							\
 	".previous\n"							\
 	".org	. - (887b - 886b) + (889b - 888b)\n"			\
@@ -85,7 +101,11 @@
 
 #define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable)	\
 	"886 :\n"							\
+	".option push\n"						\
+	".option norvc\n"						\
+	".option norelax\n"						\
 	old_c "\n"							\
+	".option pop\n"							\
 	"887 :\n"							\
 	ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)
 
@@ -97,7 +117,11 @@
 				   new_c_2, vendor_id_2, errata_id_2,	\
 					enable_2)			\
 	"886 :\n"							\
+	".option push\n"						\
+	".option norvc\n"						\
+	".option norelax\n"						\
 	old_c "\n"							\
+	".option pop\n"							\
 	"887 :\n"							\
 	ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1)	\
 	ALT_NEW_CONTENT(vendor_id_2, errata_id_2, enable_2, new_c_2)
-- 
2.35.1

[PATCH 07/12] riscv: move boot alternatives to after fill_hwcap

[Heiko Stuebner]

Move the application of boot alternatives to after the hw-capabilities
are populated. This allows to check for available extensions when
determining which alternatives to apply and also makes it actually
work if CONFIG_SMP is disabled for whatever reason.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Reviewed-by: Guo Ren <guoren@kernel.org>
---
 arch/riscv/kernel/setup.c   | 2 ++
 arch/riscv/kernel/smpboot.c | 2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index f4f4a64bc3a0..9162e9a824d2 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -21,6 +21,7 @@
 #include <linux/efi.h>
 #include <linux/crash_dump.h>
 
+#include <asm/alternative.h>
 #include <asm/cpu_ops.h>
 #include <asm/early_ioremap.h>
 #include <asm/pgtable.h>
@@ -295,6 +296,7 @@ void __init setup_arch(char **cmdline_p)
 #endif
 
 	riscv_fill_hwcap();
+	apply_boot_alternatives();
 }
 
 static int __init topology_init(void)
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index a6d13dca1403..f1e4948a4b52 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -32,7 +32,6 @@
 #include <asm/sections.h>
 #include <asm/sbi.h>
 #include <asm/smp.h>
-#include <asm/alternative.h>
 
 #include "head.h"
 
@@ -41,7 +40,6 @@ static DECLARE_COMPLETION(cpu_running);
 void __init smp_prepare_boot_cpu(void)
 {
 	init_cpu_topology();
-	apply_boot_alternatives();
 }
 
 void __init smp_prepare_cpus(unsigned int max_cpus)
-- 
2.35.1

[PATCH 08/12] riscv: Fix accessing pfn bits in PTEs for non-32bit variants

[Heiko Stuebner]

On rv32 the PFN part of PTEs is defined to use bits [xlen-1:10]
while on rv64 it is defined to use bits [53:10], leaving [63:54]
as reserved.

With upcoming optional extensions like svpbmt these previously
reserved bits will get used so simply right-shifting the PTE
to get the PFN won't be enough.

So introduce a _PAGE_PFN_MASK constant to mask the correct bits
for both rv32 and rv64 before shifting.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
---
 arch/riscv/include/asm/pgtable-32.h   |  8 ++++++++
 arch/riscv/include/asm/pgtable-64.h   | 14 +++++++++++---
 arch/riscv/include/asm/pgtable-bits.h |  6 ------
 arch/riscv/include/asm/pgtable.h      |  8 +++++---
 4 files changed, 24 insertions(+), 12 deletions(-)

diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
index 5b2e79e5bfa5..e266a4fe7f43 100644
--- a/arch/riscv/include/asm/pgtable-32.h
+++ b/arch/riscv/include/asm/pgtable-32.h
@@ -7,6 +7,7 @@
 #define _ASM_RISCV_PGTABLE_32_H
 
 #include <asm-generic/pgtable-nopmd.h>
+#include <linux/bits.h>
 #include <linux/const.h>
 
 /* Size of region mapped by a page global directory */
@@ -16,4 +17,11 @@
 
 #define MAX_POSSIBLE_PHYSMEM_BITS 34
 
+/*
+ * rv32 PTE format:
+ * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
+ *       PFN      reserved for SW   D   A   G   U   X   W   R   V
+ */
+#define _PAGE_PFN_MASK  GENMASK(31, 10)
+
 #endif /* _ASM_RISCV_PGTABLE_32_H */
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 7e246e9f8d70..15f3ad5aee4f 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -6,6 +6,7 @@
 #ifndef _ASM_RISCV_PGTABLE_64_H
 #define _ASM_RISCV_PGTABLE_64_H
 
+#include <linux/bits.h>
 #include <linux/const.h>
 
 extern bool pgtable_l4_enabled;
@@ -65,6 +66,13 @@ typedef struct {
 
 #define PTRS_PER_PMD    (PAGE_SIZE / sizeof(pmd_t))
 
+/*
+ * rv64 PTE format:
+ * | 63 | 62 61 | 60 54 | 53  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
+ *   N      MT     RSV    PFN      reserved for SW   D   A   G   U   X   W   R   V
+ */
+#define _PAGE_PFN_MASK  GENMASK(53, 10)
+
 static inline int pud_present(pud_t pud)
 {
 	return (pud_val(pud) & _PAGE_PRESENT);
@@ -108,12 +116,12 @@ static inline unsigned long _pud_pfn(pud_t pud)
 
 static inline pmd_t *pud_pgtable(pud_t pud)
 {
-	return (pmd_t *)pfn_to_virt(pud_val(pud) >> _PAGE_PFN_SHIFT);
+	return (pmd_t *)pfn_to_virt(__page_val_to_pfn(pud_val(pud)));
 }
 
 static inline struct page *pud_page(pud_t pud)
 {
-	return pfn_to_page(pud_val(pud) >> _PAGE_PFN_SHIFT);
+	return pfn_to_page(__page_val_to_pfn(pud_val(pud)));
 }
 
 #define mm_p4d_folded  mm_p4d_folded
@@ -143,7 +151,7 @@ static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
 
 static inline unsigned long _pmd_pfn(pmd_t pmd)
 {
-	return pmd_val(pmd) >> _PAGE_PFN_SHIFT;
+	return __page_val_to_pfn(pmd_val(pmd));
 }
 
 #define mk_pmd(page, prot)    pfn_pmd(page_to_pfn(page), prot)
diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
index a6b0c89824c2..e571fa954afc 100644
--- a/arch/riscv/include/asm/pgtable-bits.h
+++ b/arch/riscv/include/asm/pgtable-bits.h
@@ -6,12 +6,6 @@
 #ifndef _ASM_RISCV_PGTABLE_BITS_H
 #define _ASM_RISCV_PGTABLE_BITS_H
 
-/*
- * PTE format:
- * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
- *       PFN      reserved for SW   D   A   G   U   X   W   R   V
- */
-
 #define _PAGE_ACCESSED_OFFSET 6
 
 #define _PAGE_PRESENT   (1 << 0)
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index 046b44225623..faba543e2b08 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -108,6 +108,8 @@
 #include <asm/tlbflush.h>
 #include <linux/mm_types.h>
 
+#define __page_val_to_pfn(_val)  (((_val) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT)
+
 #ifdef CONFIG_64BIT
 #include <asm/pgtable-64.h>
 #else
@@ -261,12 +263,12 @@ static inline unsigned long _pgd_pfn(pgd_t pgd)
 
 static inline struct page *pmd_page(pmd_t pmd)
 {
-	return pfn_to_page(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
+	return pfn_to_page(__page_val_to_pfn(pmd_val(pmd)));
 }
 
 static inline unsigned long pmd_page_vaddr(pmd_t pmd)
 {
-	return (unsigned long)pfn_to_virt(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
+	return (unsigned long)pfn_to_virt(__page_val_to_pfn(pmd_val(pmd)));
 }
 
 static inline pte_t pmd_pte(pmd_t pmd)
@@ -282,7 +284,7 @@ static inline pte_t pud_pte(pud_t pud)
 /* Yields the page frame number (PFN) of a page table entry */
 static inline unsigned long pte_pfn(pte_t pte)
 {
-	return (pte_val(pte) >> _PAGE_PFN_SHIFT);
+	return __page_val_to_pfn(pte_val(pte));
 }
 
 #define pte_page(x)     pfn_to_page(pte_pfn(x))
-- 
2.35.1

[PATCH 09/12] riscv: add RISC-V Svpbmt extension support

[Heiko Stuebner]

Svpbmt (the S should be capitalized) is the
"Supervisor-mode: page-based memory types" extension
that specifies attributes for cacheability, idempotency
and ordering.

The relevant settings are done in special bits in PTEs:

Here is the svpbmt PTE format:
| 63 | 62-61 | 60-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
  N     MT     RSW    D   A   G   U   X   W   R   V
        ^

Of the Reserved bits [63:54] in a leaf PTE, the high bit is already
allocated (as the N bit), so bits [62:61] are used as the MT (aka
MemType) field. This field specifies one of three memory types that
are close equivalents (or equivalent in effect) to the three main x86
and ARMv8 memory types - as shown in the following table.

RISC-V
Encoding &
MemType     RISC-V Description
----------  ------------------------------------------------
00 - PMA    Normal Cacheable, No change to implied PMA memory type
01 - NC     Non-cacheable, idempotent, weakly-ordered Main Memory
10 - IO     Non-cacheable, non-idempotent, strongly-ordered I/O memory
11 - Rsvd   Reserved for future standard use

As the extension will not be present on all implementations,
implement a method to handle cpufeatures via alternatives
to not incur runtime penalties on cpu variants not supporting
specific extensions and patch relevant code parts at runtime.

Co-developed-by: Wei Fu <wefu@redhat.com>
Signed-off-by: Wei Fu <wefu@redhat.com>
Co-developed-by: Liu Shaohua <liush@allwinnertech.com>
Signed-off-by: Liu Shaohua <liush@allwinnertech.com>
Co-developed-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Guo Ren <guoren@kernel.org>
[moved to use the alternatives mechanism]
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Drew Fustini <drew@beagleboard.org>
Cc: Wei Fu <wefu@redhat.com>
Cc: Wei Wu <lazyparser@gmail.com>
Cc: Chen-Yu Tsai <wens@csie.org>
Cc: Maxime Ripard <maxime@cerno.tech>
Cc: Daniel Lustig <dlustig@nvidia.com>
Cc: Greg Favor <gfavor@ventanamicro.com>
Cc: Andrea Mondelli <andrea.mondelli@huawei.com>
Cc: Jonathan Behrens <behrensj@mit.edu>
Cc: Xinhaoqu (Freddie) <xinhaoqu@huawei.com>
Cc: Bill Huffman <huffman@cadence.com>
Cc: Nick Kossifidis <mick@ics.forth.gr>
Cc: Allen Baum <allen.baum@esperantotech.com>
Cc: Josh Scheid <jscheid@ventanamicro.com>
Cc: Richard Trauben <rtrauben@gmail.com>
---
 arch/riscv/Kconfig                    | 13 +++++
 arch/riscv/include/asm/alternative.h  |  3 ++
 arch/riscv/include/asm/errata_list.h  | 15 ++++++
 arch/riscv/include/asm/hwcap.h        |  1 +
 arch/riscv/include/asm/pgtable-32.h   |  9 ++++
 arch/riscv/include/asm/pgtable-64.h   | 47 +++++++++++++++++
 arch/riscv/include/asm/pgtable-bits.h |  4 --
 arch/riscv/include/asm/pgtable.h      | 29 +++++++++--
 arch/riscv/kernel/alternative.c       |  2 +
 arch/riscv/kernel/cpu.c               |  1 +
 arch/riscv/kernel/cpufeature.c        | 75 ++++++++++++++++++++++++++-
 11 files changed, 189 insertions(+), 10 deletions(-)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 1ec07aa582a3..90317a3d40a8 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -344,6 +344,19 @@ config RISCV_ISA_C
 
 	   If you don't know what to do here, say Y.
 
+config RISCV_ISA_SVPBMT
+	bool "SVPBMT extension support"
+	depends on 64BIT && MMU
+	select RISCV_ALTERNATIVE
+	default y
+	help
+	   Adds support to dynamically detect the presence of the SVPBMT extension
+	   (Supervisor-mode: page-based memory types) and enable its usage.
+
+	   The SVPBMT extension is only available on 64Bit cpus.
+
+	   If you don't know what to do here, say Y.
+
 config FPU
 	bool "FPU support"
 	default y
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index 0859529ff08e..64936356c37c 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -42,6 +42,9 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
 			      unsigned long archid, unsigned long impid,
 			      unsigned int stage);
 
+void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
+				 unsigned int stage);
+
 #else /* CONFIG_RISCV_ALTERNATIVE */
 
 static inline void apply_boot_alternatives(void) { }
diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index 5f1046e82d9f..dbfcd9b72bd8 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -14,6 +14,9 @@
 #define	ERRATA_SIFIVE_NUMBER 2
 #endif
 
+#define	CPUFEATURE_SVPBMT 0
+#define	CPUFEATURE_NUMBER 1
+
 #ifdef __ASSEMBLY__
 
 #define ALT_INSN_FAULT(x)						\
@@ -34,6 +37,18 @@ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID,	\
 		ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200)	\
 		: : "r" (addr) : "memory")
 
+/*
+ * _val is marked as "will be overwritten", so need to set it to 0
+ * in the default case.
+ */
+#define ALT_SVPBMT_SHIFT 61
+#define ALT_SVPBMT(_val, prot)						\
+asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0,	\
+		CPUFEATURE_SVPBMT, CONFIG_RISCV_ISA_SVPBMT)		\
+		: "=r"(_val)						\
+		: "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT),		\
+		  "I"(ALT_SVPBMT_SHIFT))
+
 #endif /* __ASSEMBLY__ */
 
 #endif
diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
index 0734e42f74f2..4e2486881840 100644
--- a/arch/riscv/include/asm/hwcap.h
+++ b/arch/riscv/include/asm/hwcap.h
@@ -52,6 +52,7 @@ extern unsigned long elf_hwcap;
  */
 enum riscv_isa_ext_id {
 	RISCV_ISA_EXT_SSCOFPMF = RISCV_ISA_EXT_BASE,
+	RISCV_ISA_EXT_SVPBMT,
 	RISCV_ISA_EXT_ID_MAX = RISCV_ISA_EXT_MAX,
 };
 
diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
index e266a4fe7f43..59ba1fbaf784 100644
--- a/arch/riscv/include/asm/pgtable-32.h
+++ b/arch/riscv/include/asm/pgtable-32.h
@@ -24,4 +24,13 @@
  */
 #define _PAGE_PFN_MASK  GENMASK(31, 10)
 
+#define _PAGE_NOCACHE		0
+#define _PAGE_IO		0
+#define _PAGE_MTMASK		0
+
+/* Set of bits to preserve across pte_modify() */
+#define _PAGE_CHG_MASK  (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ |	\
+					  _PAGE_WRITE | _PAGE_EXEC |	\
+					  _PAGE_USER | _PAGE_GLOBAL))
+
 #endif /* _ASM_RISCV_PGTABLE_32_H */
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 15f3ad5aee4f..2354501f0203 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -8,6 +8,7 @@
 
 #include <linux/bits.h>
 #include <linux/const.h>
+#include <asm/errata_list.h>
 
 extern bool pgtable_l4_enabled;
 extern bool pgtable_l5_enabled;
@@ -73,6 +74,52 @@ typedef struct {
  */
 #define _PAGE_PFN_MASK  GENMASK(53, 10)
 
+/*
+ * [62:61] Svpbmt Memory Type definitions:
+ *
+ *  00 - PMA    Normal Cacheable, No change to implied PMA memory type
+ *  01 - NC     Non-cacheable, idempotent, weakly-ordered Main Memory
+ *  10 - IO     Non-cacheable, non-idempotent, strongly-ordered I/O memory
+ *  11 - Rsvd   Reserved for future standard use
+ */
+#define _PAGE_NOCACHE_SVPBMT	(1UL << 61)
+#define _PAGE_IO_SVPBMT		(1UL << 62)
+#define _PAGE_MTMASK_SVPBMT	(_PAGE_NOCACHE_SVPBMT | _PAGE_IO_SVPBMT)
+
+static inline u64 riscv_page_mtmask(void)
+{
+	u64 val;
+
+	ALT_SVPBMT(val, _PAGE_MTMASK);
+	return val;
+}
+
+static inline u64 riscv_page_nocache(void)
+{
+	u64 val;
+
+	ALT_SVPBMT(val, _PAGE_NOCACHE);
+	return val;
+}
+
+static inline u64 riscv_page_io(void)
+{
+	u64 val;
+
+	ALT_SVPBMT(val, _PAGE_IO);
+	return val;
+}
+
+#define _PAGE_NOCACHE		riscv_page_nocache()
+#define _PAGE_IO		riscv_page_io()
+#define _PAGE_MTMASK		riscv_page_mtmask()
+
+/* Set of bits to preserve across pte_modify() */
+#define _PAGE_CHG_MASK  (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ |	\
+					  _PAGE_WRITE | _PAGE_EXEC |	\
+					  _PAGE_USER | _PAGE_GLOBAL |	\
+					  _PAGE_MTMASK))
+
 static inline int pud_present(pud_t pud)
 {
 	return (pud_val(pud) & _PAGE_PRESENT);
diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
index e571fa954afc..b9e13a8fe2b7 100644
--- a/arch/riscv/include/asm/pgtable-bits.h
+++ b/arch/riscv/include/asm/pgtable-bits.h
@@ -29,10 +29,6 @@
 
 #define _PAGE_PFN_SHIFT 10
 
-/* Set of bits to preserve across pte_modify() */
-#define _PAGE_CHG_MASK  (~(unsigned long)(_PAGE_PRESENT | _PAGE_READ |	\
-					  _PAGE_WRITE | _PAGE_EXEC |	\
-					  _PAGE_USER | _PAGE_GLOBAL))
 /*
  * when all of R/W/X are zero, the PTE is a pointer to the next level
  * of the page table; otherwise, it is a leaf PTE.
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index faba543e2b08..c55341b72de1 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -179,11 +179,8 @@ extern struct pt_alloc_ops pt_ops __initdata;
 
 #define PAGE_TABLE		__pgprot(_PAGE_TABLE)
 
-/*
- * The RISC-V ISA doesn't yet specify how to query or modify PMAs, so we can't
- * change the properties of memory regions.
- */
-#define _PAGE_IOREMAP _PAGE_KERNEL
+#define _PAGE_IOREMAP	((_PAGE_KERNEL & ~_PAGE_MTMASK) | _PAGE_IO)
+#define PAGE_KERNEL_IO		__pgprot(_PAGE_IOREMAP)
 
 extern pgd_t swapper_pg_dir[];
 
@@ -523,6 +520,28 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma,
 	return ptep_test_and_clear_young(vma, address, ptep);
 }
 
+#define pgprot_noncached pgprot_noncached
+static inline pgprot_t pgprot_noncached(pgprot_t _prot)
+{
+	unsigned long prot = pgprot_val(_prot);
+
+	prot &= ~_PAGE_MTMASK;
+	prot |= _PAGE_IO;
+
+	return __pgprot(prot);
+}
+
+#define pgprot_writecombine pgprot_writecombine
+static inline pgprot_t pgprot_writecombine(pgprot_t _prot)
+{
+	unsigned long prot = pgprot_val(_prot);
+
+	prot &= ~_PAGE_MTMASK;
+	prot |= _PAGE_NOCACHE;
+
+	return __pgprot(prot);
+}
+
 /*
  * THP functions
  */
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index 223770b3945c..e6c9de9f9ba6 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -63,6 +63,8 @@ static void __init_or_module _apply_alternatives(struct alt_entry *begin,
 						 struct alt_entry *end,
 						 unsigned int stage)
 {
+	riscv_cpufeature_patch_func(begin, end, stage);
+
 	if (!vendor_patch_func)
 		return;
 
diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
index ccb617791e56..40c8776aec12 100644
--- a/arch/riscv/kernel/cpu.c
+++ b/arch/riscv/kernel/cpu.c
@@ -88,6 +88,7 @@ int riscv_of_parent_hartid(struct device_node *node)
  */
 static struct riscv_isa_ext_data isa_ext_arr[] = {
 	__RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF),
+	__RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT),
 	__RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX),
 };
 
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index 1b2d42d7f589..f514b949c6a7 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -8,9 +8,15 @@
 
 #include <linux/bitmap.h>
 #include <linux/ctype.h>
+#include <linux/libfdt.h>
+#include <linux/module.h>
 #include <linux/of.h>
-#include <asm/processor.h>
+#include <asm/alternative.h>
+#include <asm/errata_list.h>
 #include <asm/hwcap.h>
+#include <asm/patch.h>
+#include <asm/pgtable.h>
+#include <asm/processor.h>
 #include <asm/smp.h>
 #include <asm/switch_to.h>
 
@@ -192,6 +198,7 @@ void __init riscv_fill_hwcap(void)
 				set_bit(*ext - 'a', this_isa);
 			} else {
 				SET_ISA_EXT_MAP("sscofpmf", RISCV_ISA_EXT_SSCOFPMF);
+				SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT);
 			}
 #undef SET_ISA_EXT_MAP
 		}
@@ -237,3 +244,69 @@ void __init riscv_fill_hwcap(void)
 		static_branch_enable(&cpu_hwcap_fpu);
 #endif
 }
+
+#ifdef CONFIG_RISCV_ALTERNATIVE
+struct cpufeature_info {
+	char name[ERRATA_STRING_LENGTH_MAX];
+	bool (*check_func)(unsigned int stage);
+};
+
+static bool __init_or_module cpufeature_svpbmt_check_func(unsigned int stage)
+{
+#ifdef CONFIG_RISCV_ISA_SVPBMT
+	return riscv_isa_extension_available(NULL, SVPBMT);
+#endif
+
+	return false;
+}
+
+static const struct cpufeature_info __initdata_or_module
+cpufeature_list[CPUFEATURE_NUMBER] = {
+	{
+		.name = "svpbmt",
+		.check_func = cpufeature_svpbmt_check_func
+	},
+};
+
+static u32 __init_or_module cpufeature_probe(unsigned int stage)
+{
+	const struct cpufeature_info *info;
+	u32 cpu_req_feature = 0;
+	int idx;
+
+	for (idx = 0; idx < CPUFEATURE_NUMBER; idx++) {
+		info = &cpufeature_list[idx];
+
+		if (info->check_func(stage))
+			cpu_req_feature |= (1U << idx);
+	}
+
+	return cpu_req_feature;
+}
+
+void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin,
+						  struct alt_entry *end,
+						  unsigned int stage)
+{
+	u32 cpu_req_feature = cpufeature_probe(stage);
+	u32 cpu_apply_feature = 0;
+	struct alt_entry *alt;
+	u32 tmp;
+
+	for (alt = begin; alt < end; alt++) {
+		if (alt->vendor_id != 0)
+			continue;
+		if (alt->errata_id >= CPUFEATURE_NUMBER) {
+			WARN(1, "This feature id:%d is not in kernel cpufeature list",
+				alt->errata_id);
+			continue;
+		}
+
+		tmp = (1U << alt->errata_id);
+		if (cpu_req_feature & tmp) {
+			patch_text_nosync(alt->old_ptr, alt->alt_ptr, alt->alt_len);
+			cpu_apply_feature |= tmp;
+		}
+	}
+}
+#endif
-- 
2.35.1

[PATCH 10/12] riscv: remove FIXMAP_PAGE_IO and fall back to its default value

[Heiko Stuebner]

If not defined in the arch, FIXMAP_PAGE_IO defaults to PAGE_KERNEL_IO,
which we defined when adding the svpbmt implementation.

So drop the FIXMAP_PAGE_IO riscv define.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Reviewed-by: Guo Ren <guoren@kernel.org>
---
 arch/riscv/include/asm/fixmap.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/riscv/include/asm/fixmap.h b/arch/riscv/include/asm/fixmap.h
index 3cfece8b6568..5c3e7b97fcc6 100644
--- a/arch/riscv/include/asm/fixmap.h
+++ b/arch/riscv/include/asm/fixmap.h
@@ -45,8 +45,6 @@ enum fixed_addresses {
 	__end_of_fixed_addresses
 };
 
-#define FIXMAP_PAGE_IO		PAGE_KERNEL
-
 #define __early_set_fixmap	__set_fixmap
 
 #define __late_set_fixmap	__set_fixmap
-- 
2.35.1

[PATCH 11/12] riscv: don't use global static vars to store alternative data

[Heiko Stuebner]

Right now the code uses a global struct to store vendor-ids
and another global variable to store the vendor-patch-function.

There exist specific cases where we'll need to patch the kernel
at an even earlier stage, where trying to write to a static
variable might actually result in hangs.

Also collecting the vendor-information consists of 3 sbi-ecalls
(or csr-reads) which is pretty negligible in the context of
booting a kernel.

So rework the code to not rely on static variables and instead
collect the vendor-information when a round of alternatives is
to be applied.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Guo Ren <guoren@kernel.org>
Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
---
 arch/riscv/kernel/alternative.c | 51 ++++++++++++++++-----------------
 1 file changed, 24 insertions(+), 27 deletions(-)

diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index e6c9de9f9ba6..27f722ae452b 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -16,41 +16,35 @@
 #include <asm/sbi.h>
 #include <asm/csr.h>
 
-static struct cpu_manufacturer_info_t {
+struct cpu_manufacturer_info_t {
 	unsigned long vendor_id;
 	unsigned long arch_id;
 	unsigned long imp_id;
-} cpu_mfr_info;
+	void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
+				  unsigned long archid, unsigned long impid,
+				  unsigned int stage);
+};
 
-static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
-				 unsigned long archid, unsigned long impid,
-				 unsigned int stage) __initdata_or_module;
-
-static inline void __init riscv_fill_cpu_mfr_info(void)
+static void __init_or_module riscv_fill_cpu_mfr_info(struct cpu_manufacturer_info_t *cpu_mfr_info)
 {
 #ifdef CONFIG_RISCV_M_MODE
-	cpu_mfr_info.vendor_id = csr_read(CSR_MVENDORID);
-	cpu_mfr_info.arch_id = csr_read(CSR_MARCHID);
-	cpu_mfr_info.imp_id = csr_read(CSR_MIMPID);
+	cpu_mfr_info->vendor_id = csr_read(CSR_MVENDORID);
+	cpu_mfr_info->arch_id = csr_read(CSR_MARCHID);
+	cpu_mfr_info->imp_id = csr_read(CSR_MIMPID);
 #else
-	cpu_mfr_info.vendor_id = sbi_get_mvendorid();
-	cpu_mfr_info.arch_id = sbi_get_marchid();
-	cpu_mfr_info.imp_id = sbi_get_mimpid();
+	cpu_mfr_info->vendor_id = sbi_get_mvendorid();
+	cpu_mfr_info->arch_id = sbi_get_marchid();
+	cpu_mfr_info->imp_id = sbi_get_mimpid();
 #endif
-}
-
-static void __init init_alternative(void)
-{
-	riscv_fill_cpu_mfr_info();
 
-	switch (cpu_mfr_info.vendor_id) {
+	switch (cpu_mfr_info->vendor_id) {
 #ifdef CONFIG_ERRATA_SIFIVE
 	case SIFIVE_VENDOR_ID:
-		vendor_patch_func = sifive_errata_patch_func;
+		cpu_mfr_info->vendor_patch_func = sifive_errata_patch_func;
 		break;
 #endif
 	default:
-		vendor_patch_func = NULL;
+		cpu_mfr_info->vendor_patch_func = NULL;
 	}
 }
 
@@ -63,14 +57,19 @@ static void __init_or_module _apply_alternatives(struct alt_entry *begin,
 						 struct alt_entry *end,
 						 unsigned int stage)
 {
+	struct cpu_manufacturer_info_t cpu_mfr_info;
+
+	riscv_fill_cpu_mfr_info(&cpu_mfr_info);
+
 	riscv_cpufeature_patch_func(begin, end, stage);
 
-	if (!vendor_patch_func)
+	if (!cpu_mfr_info.vendor_patch_func)
 		return;
 
-	vendor_patch_func(begin, end,
-			  cpu_mfr_info.arch_id, cpu_mfr_info.imp_id,
-			  stage);
+	cpu_mfr_info.vendor_patch_func(begin, end,
+				   cpu_mfr_info.arch_id,
+				   cpu_mfr_info.imp_id,
+				   stage);
 }
 
 void __init apply_boot_alternatives(void)
@@ -78,8 +77,6 @@ void __init apply_boot_alternatives(void)
 	/* If called on non-boot cpu things could go wrong */
 	WARN_ON(smp_processor_id() != 0);
 
-	init_alternative();
-
 	_apply_alternatives((struct alt_entry *)__alt_start,
 			    (struct alt_entry *)__alt_end,
 			    RISCV_ALTERNATIVES_BOOT);
-- 
2.35.1

[PATCH 12/12] riscv: add memory-type errata for T-Head

[Heiko Stuebner]

Some current cpus based on T-Head cores implement memory-types
way different than described in the svpbmt spec even going
so far as using PTE bits marked as reserved.

Add the T-Head vendor-id and necessary errata code to
replace the affected instructions.

Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Tested-by: Samuel Holland <samuel@sholland.org>
---
 arch/riscv/Kconfig                     |  6 ++
 arch/riscv/Kconfig.erratas             | 21 +++++++
 arch/riscv/errata/Makefile             |  1 +
 arch/riscv/errata/sifive/errata.c      |  7 ++-
 arch/riscv/errata/thead/Makefile       | 11 ++++
 arch/riscv/errata/thead/errata.c       | 82 ++++++++++++++++++++++++++
 arch/riscv/include/asm/alternative.h   |  6 ++
 arch/riscv/include/asm/errata_list.h   | 50 +++++++++++++++-
 arch/riscv/include/asm/pgtable-64.h    | 18 +++++-
 arch/riscv/include/asm/pgtable.h       | 18 +++++-
 arch/riscv/include/asm/vendorid_list.h |  1 +
 arch/riscv/kernel/Makefile             | 14 +++++
 arch/riscv/kernel/alternative.c        | 26 ++++++++
 arch/riscv/kernel/cpufeature.c         |  7 ++-
 arch/riscv/mm/init.c                   |  1 +
 15 files changed, 260 insertions(+), 9 deletions(-)
 create mode 100644 arch/riscv/errata/thead/Makefile
 create mode 100644 arch/riscv/errata/thead/errata.c

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 90317a3d40a8..65285b980134 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -334,6 +334,12 @@ config RISCV_ALTERNATIVE
 	  code patching is performed once in the boot stages. It means
 	  that the overhead from this mechanism is just taken once.
 
+config RISCV_ALTERNATIVE_EARLY
+	bool
+	depends on RISCV_ALTERNATIVE
+	help
+	  Allows early patching of the kernel for special errata
+
 config RISCV_ISA_C
 	bool "Emit compressed instructions when building Linux"
 	default y
diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
index c521c2ae2de2..ebfcd5cc6eaf 100644
--- a/arch/riscv/Kconfig.erratas
+++ b/arch/riscv/Kconfig.erratas
@@ -33,4 +33,25 @@ config ERRATA_SIFIVE_CIP_1200
 
 	  If you don't know what to do here, say "Y".
 
+config ERRATA_THEAD
+	bool "T-HEAD errata"
+	select RISCV_ALTERNATIVE
+	help
+	  All T-HEAD errata Kconfig depend on this Kconfig. Disabling
+	  this Kconfig will disable all T-HEAD errata. Please say "Y"
+	  here if your platform uses T-HEAD CPU cores.
+
+	  Otherwise, please say "N" here to avoid unnecessary overhead.
+
+config ERRATA_THEAD_PBMT
+	bool "Apply T-Head memory type errata"
+	depends on ERRATA_THEAD && 64BIT
+	select RISCV_ALTERNATIVE_EARLY
+	default y
+	help
+	  This will apply the memory type errata to handle the non-standard
+	  memory type bits in page-table-entries on T-Head SoCs.
+
+	  If you don't know what to do here, say "Y".
+
 endmenu
diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
index 0ca1c5281a2d..a1055965fbee 100644
--- a/arch/riscv/errata/Makefile
+++ b/arch/riscv/errata/Makefile
@@ -1 +1,2 @@
 obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
+obj-$(CONFIG_ERRATA_THEAD) += thead/
diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
index 3e39587a49dc..672f02b21ce0 100644
--- a/arch/riscv/errata/sifive/errata.c
+++ b/arch/riscv/errata/sifive/errata.c
@@ -88,10 +88,15 @@ void __init_or_module sifive_errata_patch_func(struct alt_entry *begin,
 					       unsigned int stage)
 {
 	struct alt_entry *alt;
-	u32 cpu_req_errata = sifive_errata_probe(archid, impid);
+	u32 cpu_req_errata;
 	u32 cpu_apply_errata = 0;
 	u32 tmp;
 
+	if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+		return;
+
+	cpu_req_errata = sifive_errata_probe(archid, impid);
+
 	for (alt = begin; alt < end; alt++) {
 		if (alt->vendor_id != SIFIVE_VENDOR_ID)
 			continue;
diff --git a/arch/riscv/errata/thead/Makefile b/arch/riscv/errata/thead/Makefile
new file mode 100644
index 000000000000..137e700d9d3f
--- /dev/null
+++ b/arch/riscv/errata/thead/Makefile
@@ -0,0 +1,11 @@
+ifdef CONFIG_RISCV_ALTERNATIVE_EARLY
+CFLAGS_errata.o := -mcmodel=medany
+ifdef CONFIG_FTRACE
+CFLAGS_REMOVE_errata.o = $(CC_FLAGS_FTRACE)
+endif
+ifdef CONFIG_KASAN
+KASAN_SANITIZE_errata.o := n
+endif
+endif
+
+obj-y += errata.o
diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
new file mode 100644
index 000000000000..e5d75270b99c
--- /dev/null
+++ b/arch/riscv/errata/thead/errata.c
@@ -0,0 +1,82 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2021 Heiko Stuebner <heiko@sntech.de>
+ */
+
+#include <linux/bug.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+#include <asm/alternative.h>
+#include <asm/cacheflush.h>
+#include <asm/errata_list.h>
+#include <asm/patch.h>
+#include <asm/vendorid_list.h>
+
+struct errata_info {
+	char name[ERRATA_STRING_LENGTH_MAX];
+	bool (*check_func)(unsigned long arch_id, unsigned long impid);
+	unsigned int stage;
+};
+
+static bool errata_mt_check_func(unsigned long  arch_id, unsigned long impid)
+{
+	if (arch_id != 0 || impid != 0)
+		return false;
+	return true;
+}
+
+static const struct errata_info errata_list[ERRATA_THEAD_NUMBER] = {
+	{
+		.name = "memory-types",
+		.stage = RISCV_ALTERNATIVES_EARLY_BOOT,
+		.check_func = errata_mt_check_func
+	},
+};
+
+static u32 thead_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid)
+{
+	const struct errata_info *info;
+	u32 cpu_req_errata = 0;
+	int idx;
+
+	for (idx = 0; idx < ERRATA_THEAD_NUMBER; idx++) {
+		info = &errata_list[idx];
+
+		if ((stage == RISCV_ALTERNATIVES_MODULE ||
+		     info->stage == stage) && info->check_func(archid, impid))
+			cpu_req_errata |= (1U << idx);
+	}
+
+	return cpu_req_errata;
+}
+
+void __init_or_module thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
+					      unsigned long archid, unsigned long impid,
+					      unsigned int stage)
+{
+	struct alt_entry *alt;
+	u32 cpu_req_errata = thead_errata_probe(stage, archid, impid);
+	u32 tmp;
+
+	for (alt = begin; alt < end; alt++) {
+		if (alt->vendor_id != THEAD_VENDOR_ID)
+			continue;
+		if (alt->errata_id >= ERRATA_THEAD_NUMBER)
+			continue;
+
+		tmp = (1U << alt->errata_id);
+		if (cpu_req_errata & tmp) {
+			/* On vm-alternatives, the mmu isn't running yet */
+			if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+				memcpy((void *)__pa_symbol(alt->old_ptr),
+				       (void *)__pa_symbol(alt->alt_ptr), alt->alt_len);
+			else
+				patch_text_nosync(alt->old_ptr, alt->alt_ptr, alt->alt_len);
+		}
+	}
+
+	if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
+		local_flush_icache_all();
+}
diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
index 64936356c37c..6511dd73e812 100644
--- a/arch/riscv/include/asm/alternative.h
+++ b/arch/riscv/include/asm/alternative.h
@@ -21,8 +21,10 @@
 
 #define RISCV_ALTERNATIVES_BOOT		0 /* alternatives applied during regular boot */
 #define RISCV_ALTERNATIVES_MODULE	1 /* alternatives applied during module-init */
+#define RISCV_ALTERNATIVES_EARLY_BOOT	2 /* alternatives applied before mmu start */
 
 void __init apply_boot_alternatives(void);
+void __init apply_early_boot_alternatives(void);
 void apply_module_alternatives(void *start, size_t length);
 
 struct alt_entry {
@@ -41,6 +43,9 @@ struct errata_checkfunc_id {
 void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
 			      unsigned long archid, unsigned long impid,
 			      unsigned int stage);
+void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
+			     unsigned long archid, unsigned long impid,
+			     unsigned int stage);
 
 void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
 				 unsigned int stage);
@@ -48,6 +53,7 @@ void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
 #else /* CONFIG_RISCV_ALTERNATIVE */
 
 static inline void apply_boot_alternatives(void) { }
+static inline void apply_early_boot_alternatives(void) { }
 static inline void apply_module_alternatives(void *start, size_t length) { }
 
 #endif /* CONFIG_RISCV_ALTERNATIVE */
diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
index dbfcd9b72bd8..9e2888dbb5b1 100644
--- a/arch/riscv/include/asm/errata_list.h
+++ b/arch/riscv/include/asm/errata_list.h
@@ -14,6 +14,11 @@
 #define	ERRATA_SIFIVE_NUMBER 2
 #endif
 
+#ifdef CONFIG_ERRATA_THEAD
+#define	ERRATA_THEAD_PBMT 0
+#define	ERRATA_THEAD_NUMBER 1
+#endif
+
 #define	CPUFEATURE_SVPBMT 0
 #define	CPUFEATURE_NUMBER 1
 
@@ -42,12 +47,51 @@ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID,	\
  * in the default case.
  */
 #define ALT_SVPBMT_SHIFT 61
+#define ALT_THEAD_PBMT_SHIFT 59
 #define ALT_SVPBMT(_val, prot)						\
-asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0,	\
-		CPUFEATURE_SVPBMT, CONFIG_RISCV_ISA_SVPBMT)		\
+asm(ALTERNATIVE_2("li %0, 0\t\nnop",					\
+		  "li %0, %1\t\nslli %0,%0,%3", 0,			\
+			CPUFEATURE_SVPBMT, CONFIG_RISCV_ISA_SVPBMT,	\
+		  "li %0, %2\t\nslli %0,%0,%4", THEAD_VENDOR_ID,	\
+			ERRATA_THEAD_PBMT, CONFIG_ERRATA_THEAD_PBMT)	\
 		: "=r"(_val)						\
 		: "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT),		\
-		  "I"(ALT_SVPBMT_SHIFT))
+		  "I"(prot##_THEAD >> ALT_THEAD_PBMT_SHIFT),		\
+		  "I"(ALT_SVPBMT_SHIFT),				\
+		  "I"(ALT_THEAD_PBMT_SHIFT))
+
+#ifdef CONFIG_ERRATA_THEAD_PBMT
+/*
+ * IO/NOCACHE memory types are handled together with svpbmt,
+ * so on T-Head chips, check if no other memory type is set,
+ * and set the non-0 PMA type if applicable.
+ */
+#define ALT_THEAD_PMA(_val)						\
+asm volatile(ALTERNATIVE(						\
+	"nop\n\t"							\
+	"nop\n\t"							\
+	"nop\n\t"							\
+	"nop\n\t"							\
+	"nop\n\t"							\
+	"nop\n\t"							\
+	"nop",								\
+	"li      t3, %2\n\t"						\
+	"slli    t3, t3, %4\n\t"					\
+	"and     t3, %0, t3\n\t"					\
+	"bne     t3, zero, 2f\n\t"					\
+	"li      t3, %3\n\t"						\
+	"slli    t3, t3, %4\n\t"					\
+	"or      %0, %0, t3\n\t"					\
+	"2:",  THEAD_VENDOR_ID,						\
+		ERRATA_THEAD_PBMT, CONFIG_ERRATA_THEAD_PBMT)		\
+	: "+r"(_val)							\
+	: "0"(_val),							\
+	  "I"(_PAGE_MTMASK_THEAD >> ALT_THEAD_PBMT_SHIFT),		\
+	  "I"(_PAGE_PMA_THEAD >> ALT_THEAD_PBMT_SHIFT),			\
+	  "I"(ALT_THEAD_PBMT_SHIFT))
+#else
+#define ALT_THEAD_PMA(_val)
+#endif
 
 #endif /* __ASSEMBLY__ */
 
diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
index 2354501f0203..e4ff3e0ab887 100644
--- a/arch/riscv/include/asm/pgtable-64.h
+++ b/arch/riscv/include/asm/pgtable-64.h
@@ -86,6 +86,18 @@ typedef struct {
 #define _PAGE_IO_SVPBMT		(1UL << 62)
 #define _PAGE_MTMASK_SVPBMT	(_PAGE_NOCACHE_SVPBMT | _PAGE_IO_SVPBMT)
 
+/*
+ * [63:59] T-Head Memory Type definitions:
+ *
+ * 00000 - NC   Weakly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
+ * 01110 - PMA  Weakly-ordered, Cacheable, Bufferable, Shareable, Non-trustable
+ * 10000 - IO   Strongly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
+ */
+#define _PAGE_PMA_THEAD		((1UL << 62) | (1UL << 61) | (1UL << 60))
+#define _PAGE_NOCACHE_THEAD	0UL
+#define _PAGE_IO_THEAD		(1UL << 63)
+#define _PAGE_MTMASK_THEAD	(_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1UL << 59))
+
 static inline u64 riscv_page_mtmask(void)
 {
 	u64 val;
@@ -193,7 +205,11 @@ static inline bool mm_pud_folded(struct mm_struct *mm)
 
 static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
 {
-	return __pmd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+	unsigned long prot_val = pgprot_val(prot);
+
+	ALT_THEAD_PMA(prot_val);
+
+	return __pmd((pfn << _PAGE_PFN_SHIFT) | prot_val);
 }
 
 static inline unsigned long _pmd_pfn(pmd_t pmd)
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index c55341b72de1..6f0a260d3f2c 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -250,7 +250,11 @@ static inline void pmd_clear(pmd_t *pmdp)
 
 static inline pgd_t pfn_pgd(unsigned long pfn, pgprot_t prot)
 {
-	return __pgd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+	unsigned long prot_val = pgprot_val(prot);
+
+	ALT_THEAD_PMA(prot_val);
+
+	return __pgd((pfn << _PAGE_PFN_SHIFT) | prot_val);
 }
 
 static inline unsigned long _pgd_pfn(pgd_t pgd)
@@ -289,7 +293,11 @@ static inline unsigned long pte_pfn(pte_t pte)
 /* Constructs a page table entry */
 static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
 {
-	return __pte((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
+	unsigned long prot_val = pgprot_val(prot);
+
+	ALT_THEAD_PMA(prot_val);
+
+	return __pte((pfn << _PAGE_PFN_SHIFT) | prot_val);
 }
 
 #define mk_pte(page, prot)       pfn_pte(page_to_pfn(page), prot)
@@ -398,7 +406,11 @@ static inline int pmd_protnone(pmd_t pmd)
 /* Modify page protection bits */
 static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
 {
-	return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
+	unsigned long newprot_val = pgprot_val(newprot);
+
+	ALT_THEAD_PMA(newprot_val);
+
+	return __pte((pte_val(pte) & _PAGE_CHG_MASK) | newprot_val);
 }
 
 #define pgd_ERROR(e) \
diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
index 9d934215b3c8..cb89af3f0704 100644
--- a/arch/riscv/include/asm/vendorid_list.h
+++ b/arch/riscv/include/asm/vendorid_list.h
@@ -6,5 +6,6 @@
 #define ASM_VENDOR_LIST_H
 
 #define SIFIVE_VENDOR_ID	0x489
+#define THEAD_VENDOR_ID		0x5b7
 
 #endif
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 0f8348ac30f1..bf3876a77ed7 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -14,6 +14,20 @@ ifdef CONFIG_KEXEC
 AFLAGS_kexec_relocate.o := -mcmodel=medany $(call cc-option,-mno-relax)
 endif
 
+# cmodel=medany and notrace when patching early
+ifdef CONFIG_RISCV_ALTERNATIVE_EARLY
+CFLAGS_alternative.o := -mcmodel=medany
+CFLAGS_cpufeature.o := -mcmodel=medany
+ifdef CONFIG_FTRACE
+CFLAGS_REMOVE_alternative.o = $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_cpufeature.o = $(CC_FLAGS_FTRACE)
+endif
+ifdef CONFIG_KASAN
+KASAN_SANITIZE_alternative.o := n
+KASAN_SANITIZE_cpufeature.o := n
+endif
+endif
+
 extra-y += head.o
 extra-y += vmlinux.lds
 
diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
index 27f722ae452b..c9d0d3c53223 100644
--- a/arch/riscv/kernel/alternative.c
+++ b/arch/riscv/kernel/alternative.c
@@ -42,6 +42,11 @@ static void __init_or_module riscv_fill_cpu_mfr_info(struct cpu_manufacturer_inf
 	case SIFIVE_VENDOR_ID:
 		cpu_mfr_info->vendor_patch_func = sifive_errata_patch_func;
 		break;
+#endif
+#ifdef CONFIG_ERRATA_THEAD
+	case THEAD_VENDOR_ID:
+		cpu_mfr_info->vendor_patch_func = thead_errata_patch_func;
+		break;
 #endif
 	default:
 		cpu_mfr_info->vendor_patch_func = NULL;
@@ -82,6 +87,27 @@ void __init apply_boot_alternatives(void)
 			    RISCV_ALTERNATIVES_BOOT);
 }
 
+/*
+ * apply_early_boot_alternatives() is called from setup_vm() with MMU-off.
+ *
+ * Following requirements should be honoured for it to work correctly:
+ * 1) It should use PC-relative addressing for accessing kernel symbols.
+ *    To achieve this we always use GCC cmodel=medany.
+ * 2) The compiler instrumentation for FTRACE will not work for setup_vm()
+ *    so disable compiler instrumentation when FTRACE is enabled.
+ *
+ * Currently, the above requirements are honoured by using custom CFLAGS
+ * for alternative.o in kernel/Makefile.
+ */
+void __init apply_early_boot_alternatives(void)
+{
+#ifdef CONFIG_RISCV_ALTERNATIVE_EARLY
+	_apply_alternatives((struct alt_entry *)__alt_start,
+			    (struct alt_entry *)__alt_end,
+			    RISCV_ALTERNATIVES_EARLY_BOOT);
+#endif
+}
+
 #ifdef CONFIG_MODULES
 void apply_module_alternatives(void *start, size_t length)
 {
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index f514b949c6a7..dea3ea19deee 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -254,7 +254,12 @@ struct cpufeature_info {
 static bool __init_or_module cpufeature_svpbmt_check_func(unsigned int stage)
 {
 #ifdef CONFIG_RISCV_ISA_SVPBMT
-	return riscv_isa_extension_available(NULL, SVPBMT);
+	switch (stage) {
+	case RISCV_ALTERNATIVES_EARLY_BOOT:
+		return false;
+	default:
+		return riscv_isa_extension_available(NULL, SVPBMT);
+	}
 #endif
 
 	return false;
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index b0793dc0c291..45a4bf107bd2 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -936,6 +936,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)
 	BUG_ON((kernel_map.virt_addr + kernel_map.size) > ADDRESS_SPACE_END - SZ_4K);
 #endif
 
+	apply_early_boot_alternatives();
 	pt_ops_set_early();
 
 	/* Setup early PGD for fixmap */
-- 
2.35.1

Re: [PATCH v10 00/12] riscv: support for Svpbmt and D1 memory types

[Palmer Dabbelt]

On Wed, 11 May 2022 12:29:09 PDT (-0700), heiko@sntech.de wrote:
> Svpbmt is an extension defining "Supervisor-mode: page-based memory types"
> for things like non-cacheable pages or I/O memory pages.
>
>
> So this is my 2nd try at implementing Svpbmt (and the diverging D1 memory
> types) using the alternatives framework.
>
> This includes a number of changes to the alternatives mechanism itself.
> The biggest one being the move to a more central location, as I expect
> in the future, nearly every chip needing some sort of patching, be it
> either for erratas or for optional features (svpbmt or others).
>
> Detection of the svpbmt functionality is done via Atish's isa extension
> handling series [0] and thus does not need any dt-parsing of its own
> anymore.
>
> The series also introduces support for the memory types of the D1
> which are implemented differently to svpbmt. But when patching anyway
> it's pretty clean to add the D1 variant via ALTERNATIVE_2 to the same
> location.
>
> The only slightly bigger difference is that the "normal" type is not 0
> as with svpbmt, so kernel patches for this PMA type need to be applied
> even before the MMU is brought up, so the series introduces a separate
> stage for that.
>
>
> In theory this series is 2 parts:
> - alternatives improvements
> - svpbmt+d1
>
> I picked the recipient list from the previous versions, hopefully
> I didn't forget anybody.
>
> I tested the series on:
> - qemu-rv32 + buildroot rootfs
> - qemu-rv64 + debian roots
> - Allwinner D1-Nezha
> - BeagleV - it at least reached the same point as without the series

IMO that's fine, it's also broken due to issues around non-coherence but 
it has an entirely different way of handling things than.

> I also ran Palmers CI environment on 5.18-rc6 + this series and
> it passed with all testcases now.

Thanks, I know that's a bit of a mess.  If I ever get some time I'll try 
and clean it up, but it keeps finding issues so I'm sort of stuck with 
it for now.  As expected it now passes locally, so I've put this on 
for-next.  I hadn't noticed your testing was on rc6, I put this on top 
of rc1 -- that's what I usually do for merge window stuff, but if 
there's something specific between rc1 and rc6 this depends on then LMK 
and I'll sort it out.

> changes in v10:
> - add received review-tags
> - put early patching behind a kconfig symbol
> - adapt compiler flags of sources in use by early patching
>   similar to other riscv arch-parts.
>   This fixes the medlow cmodel issue on rv32 and also issues
>   with Kasan.
>
> changes in v9:
> - rebase onto 5.18-rc1
> - drop the sbi null-ptr patch
>   While I still think this to be non-ideal as is, it isn't really
>   necessary for svpbmt support anymore
> - merge cpufeature + svpbmt patch, as otherwise some empty shells
>   cause build warnings when a bisection stops between these two
>   patches
> - address review comments from Christoph Hellwig:
>   - keep alternatives optional, they now get selected by its
>     users (erratas and also the newly introduced svpbmt kconfig)
>   - wrap long lines and keep things below 80 characters
>   - restyle svpbmt + thead errata assembly
>   - introduce a helper for the repeated calls to
>     (val & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT
>
> changes in v8:
> - rebase onto 5.17-final + isa extension series
>   We're halfway through the merge-window, so this series
>   should be merge after that
> - adapt to fix limiting alternatives to non-xip-kernels
> - add .norelax option for alternatives
> - fix unused cpu_apply_errata in thead errata
> - don't use static globals to store cpu-manufacturer data
>   as it makes machines hang if done too early
>
> changes in v7:
> - fix typo in patch1 (Atish)
> - moved to Atish's isa-extension framework
> - and therefore move regular boot-alternatives directly behind fill_hwcaps
> - change T-Head errata Kconfig text (Atish)
>
> changes in v6:
> - rebase onto 5.17-rc1
> - handle sbi null-ptr differently
> - improve commit messages
> - use riscv,mmu as property name
>
> changes in v5:
> - move to use alternatives for runtime-patching
> - add D1 variant
>
>
> [0] https://lore.kernel.org/r/20220222204811.2281949-2-atishp@rivosinc.com
>
> Heiko Stuebner (12):
>   riscv: integrate alternatives better into the main architecture
>   riscv: allow different stages with alternatives
>   riscv: implement module alternatives
>   riscv: implement ALTERNATIVE_2 macro
>   riscv: extend concatenated alternatives-lines to the same length
>   riscv: prevent compressed instructions in alternatives
>   riscv: move boot alternatives to after fill_hwcap
>   riscv: Fix accessing pfn bits in PTEs for non-32bit variants
>   riscv: add RISC-V Svpbmt extension support
>   riscv: remove FIXMAP_PAGE_IO and fall back to its default value
>   riscv: don't use global static vars to store alternative data
>   riscv: add memory-type errata for T-Head
>
>  arch/riscv/Kconfig                          |  28 +++++
>  arch/riscv/Kconfig.erratas                  |  34 ++++--
>  arch/riscv/Kconfig.socs                     |   1 -
>  arch/riscv/Makefile                         |   2 +-
>  arch/riscv/errata/Makefile                  |   2 +-
>  arch/riscv/errata/alternative.c             |  75 ------------
>  arch/riscv/errata/sifive/errata.c           |  20 ++-
>  arch/riscv/errata/thead/Makefile            |  11 ++
>  arch/riscv/errata/thead/errata.c            |  82 +++++++++++++
>  arch/riscv/include/asm/alternative-macros.h | 129 +++++++++++++++-----
>  arch/riscv/include/asm/alternative.h        |  25 +++-
>  arch/riscv/include/asm/errata_list.h        |  59 +++++++++
>  arch/riscv/include/asm/fixmap.h             |   2 -
>  arch/riscv/include/asm/hwcap.h              |   1 +
>  arch/riscv/include/asm/pgtable-32.h         |  17 +++
>  arch/riscv/include/asm/pgtable-64.h         |  79 +++++++++++-
>  arch/riscv/include/asm/pgtable-bits.h       |  10 --
>  arch/riscv/include/asm/pgtable.h            |  55 +++++++--
>  arch/riscv/include/asm/vendorid_list.h      |   1 +
>  arch/riscv/kernel/Makefile                  |  15 +++
>  arch/riscv/kernel/alternative.c             | 118 ++++++++++++++++++
>  arch/riscv/kernel/cpu.c                     |   1 +
>  arch/riscv/kernel/cpufeature.c              |  80 +++++++++++-
>  arch/riscv/kernel/module.c                  |  29 +++++
>  arch/riscv/kernel/setup.c                   |   2 +
>  arch/riscv/kernel/smpboot.c                 |   4 -
>  arch/riscv/kernel/traps.c                   |   2 +-
>  arch/riscv/mm/init.c                        |   1 +
>  28 files changed, 724 insertions(+), 161 deletions(-)
>  delete mode 100644 arch/riscv/errata/alternative.c
>  create mode 100644 arch/riscv/errata/thead/Makefile
>  create mode 100644 arch/riscv/errata/thead/errata.c
>  create mode 100644 arch/riscv/kernel/alternative.c

Re: [PATCH 12/12] riscv: add memory-type errata for T-Head

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, May 12, 2022 at 3:33 AM Heiko Stuebner <heiko@sntech.de> wrote:
>
> Some current cpus based on T-Head cores implement memory-types
> way different than described in the svpbmt spec even going
> so far as using PTE bits marked as reserved.
>
> Add the T-Head vendor-id and necessary errata code to
> replace the affected instructions.
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Tested-by: Samuel Holland <samuel@sholland.org>
> ---
>  arch/riscv/Kconfig                     |  6 ++
>  arch/riscv/Kconfig.erratas             | 21 +++++++
>  arch/riscv/errata/Makefile             |  1 +
>  arch/riscv/errata/sifive/errata.c      |  7 ++-
>  arch/riscv/errata/thead/Makefile       | 11 ++++
>  arch/riscv/errata/thead/errata.c       | 82 ++++++++++++++++++++++++++
>  arch/riscv/include/asm/alternative.h   |  6 ++
>  arch/riscv/include/asm/errata_list.h   | 50 +++++++++++++++-
>  arch/riscv/include/asm/pgtable-64.h    | 18 +++++-
>  arch/riscv/include/asm/pgtable.h       | 18 +++++-
>  arch/riscv/include/asm/vendorid_list.h |  1 +
>  arch/riscv/kernel/Makefile             | 14 +++++
>  arch/riscv/kernel/alternative.c        | 26 ++++++++
>  arch/riscv/kernel/cpufeature.c         |  7 ++-
>  arch/riscv/mm/init.c                   |  1 +
>  15 files changed, 260 insertions(+), 9 deletions(-)
>  create mode 100644 arch/riscv/errata/thead/Makefile
>  create mode 100644 arch/riscv/errata/thead/errata.c
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 90317a3d40a8..65285b980134 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -334,6 +334,12 @@ config RISCV_ALTERNATIVE
>           code patching is performed once in the boot stages. It means
>           that the overhead from this mechanism is just taken once.
>
> +config RISCV_ALTERNATIVE_EARLY
> +       bool
> +       depends on RISCV_ALTERNATIVE
> +       help
> +         Allows early patching of the kernel for special errata
> +
>  config RISCV_ISA_C
>         bool "Emit compressed instructions when building Linux"
>         default y
> diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
> index c521c2ae2de2..ebfcd5cc6eaf 100644
> --- a/arch/riscv/Kconfig.erratas
> +++ b/arch/riscv/Kconfig.erratas
> @@ -33,4 +33,25 @@ config ERRATA_SIFIVE_CIP_1200
>
>           If you don't know what to do here, say "Y".
>
> +config ERRATA_THEAD
> +       bool "T-HEAD errata"
> +       select RISCV_ALTERNATIVE
> +       help
> +         All T-HEAD errata Kconfig depend on this Kconfig. Disabling
> +         this Kconfig will disable all T-HEAD errata. Please say "Y"
> +         here if your platform uses T-HEAD CPU cores.
> +
> +         Otherwise, please say "N" here to avoid unnecessary overhead.
> +
> +config ERRATA_THEAD_PBMT
> +       bool "Apply T-Head memory type errata"
> +       depends on ERRATA_THEAD && 64BIT
> +       select RISCV_ALTERNATIVE_EARLY
> +       default y
> +       help
> +         This will apply the memory type errata to handle the non-standard
> +         memory type bits in page-table-entries on T-Head SoCs.
> +
> +         If you don't know what to do here, say "Y".
> +
>  endmenu
> diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
> index 0ca1c5281a2d..a1055965fbee 100644
> --- a/arch/riscv/errata/Makefile
> +++ b/arch/riscv/errata/Makefile
> @@ -1 +1,2 @@
>  obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
> +obj-$(CONFIG_ERRATA_THEAD) += thead/
> diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
> index 3e39587a49dc..672f02b21ce0 100644
> --- a/arch/riscv/errata/sifive/errata.c
> +++ b/arch/riscv/errata/sifive/errata.c
> @@ -88,10 +88,15 @@ void __init_or_module sifive_errata_patch_func(struct alt_entry *begin,
>                                                unsigned int stage)
>  {
>         struct alt_entry *alt;
> -       u32 cpu_req_errata = sifive_errata_probe(archid, impid);
> +       u32 cpu_req_errata;
>         u32 cpu_apply_errata = 0;
>         u32 tmp;
>
> +       if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
> +               return;
> +
> +       cpu_req_errata = sifive_errata_probe(archid, impid);
> +
>         for (alt = begin; alt < end; alt++) {
>                 if (alt->vendor_id != SIFIVE_VENDOR_ID)
>                         continue;
> diff --git a/arch/riscv/errata/thead/Makefile b/arch/riscv/errata/thead/Makefile
> new file mode 100644
> index 000000000000..137e700d9d3f
> --- /dev/null
> +++ b/arch/riscv/errata/thead/Makefile
> @@ -0,0 +1,11 @@
> +ifdef CONFIG_RISCV_ALTERNATIVE_EARLY
> +CFLAGS_errata.o := -mcmodel=medany
> +ifdef CONFIG_FTRACE
> +CFLAGS_REMOVE_errata.o = $(CC_FLAGS_FTRACE)
> +endif
> +ifdef CONFIG_KASAN
> +KASAN_SANITIZE_errata.o := n
> +endif
> +endif
> +
> +obj-y += errata.o
> diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c
> new file mode 100644
> index 000000000000..e5d75270b99c
> --- /dev/null
> +++ b/arch/riscv/errata/thead/errata.c
> @@ -0,0 +1,82 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (C) 2021 Heiko Stuebner <heiko@sntech.de>
> + */
> +
> +#include <linux/bug.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/string.h>
> +#include <linux/uaccess.h>
> +#include <asm/alternative.h>
> +#include <asm/cacheflush.h>
> +#include <asm/errata_list.h>
> +#include <asm/patch.h>
> +#include <asm/vendorid_list.h>
> +
> +struct errata_info {
> +       char name[ERRATA_STRING_LENGTH_MAX];
> +       bool (*check_func)(unsigned long arch_id, unsigned long impid);
> +       unsigned int stage;
> +};
> +
> +static bool errata_mt_check_func(unsigned long  arch_id, unsigned long impid)
> +{
> +       if (arch_id != 0 || impid != 0)
> +               return false;
> +       return true;
> +}
> +
> +static const struct errata_info errata_list[ERRATA_THEAD_NUMBER] = {
> +       {
> +               .name = "memory-types",
> +               .stage = RISCV_ALTERNATIVES_EARLY_BOOT,
> +               .check_func = errata_mt_check_func
> +       },
> +};
> +
> +static u32 thead_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid)
> +{
> +       const struct errata_info *info;
> +       u32 cpu_req_errata = 0;
> +       int idx;
> +
> +       for (idx = 0; idx < ERRATA_THEAD_NUMBER; idx++) {
> +               info = &errata_list[idx];
> +
> +               if ((stage == RISCV_ALTERNATIVES_MODULE ||
> +                    info->stage == stage) && info->check_func(archid, impid))
> +                       cpu_req_errata |= (1U << idx);
> +       }
> +
> +       return cpu_req_errata;
> +}
> +
> +void __init_or_module thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
> +                                             unsigned long archid, unsigned long impid,
> +                                             unsigned int stage)
> +{
> +       struct alt_entry *alt;
> +       u32 cpu_req_errata = thead_errata_probe(stage, archid, impid);
> +       u32 tmp;
> +
> +       for (alt = begin; alt < end; alt++) {
> +               if (alt->vendor_id != THEAD_VENDOR_ID)
> +                       continue;
> +               if (alt->errata_id >= ERRATA_THEAD_NUMBER)
> +                       continue;
> +
> +               tmp = (1U << alt->errata_id);
> +               if (cpu_req_errata & tmp) {
> +                       /* On vm-alternatives, the mmu isn't running yet */
> +                       if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
> +                               memcpy((void *)__pa_symbol(alt->old_ptr),
> +                                      (void *)__pa_symbol(alt->alt_ptr), alt->alt_len);
> +                       else
> +                               patch_text_nosync(alt->old_ptr, alt->alt_ptr, alt->alt_len);
> +               }
> +       }
> +
> +       if (stage == RISCV_ALTERNATIVES_EARLY_BOOT)
> +               local_flush_icache_all();
> +}
> diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
> index 64936356c37c..6511dd73e812 100644
> --- a/arch/riscv/include/asm/alternative.h
> +++ b/arch/riscv/include/asm/alternative.h
> @@ -21,8 +21,10 @@
>
>  #define RISCV_ALTERNATIVES_BOOT                0 /* alternatives applied during regular boot */
>  #define RISCV_ALTERNATIVES_MODULE      1 /* alternatives applied during module-init */
> +#define RISCV_ALTERNATIVES_EARLY_BOOT  2 /* alternatives applied before mmu start */
>
>  void __init apply_boot_alternatives(void);
> +void __init apply_early_boot_alternatives(void);
>  void apply_module_alternatives(void *start, size_t length);
>
>  struct alt_entry {
> @@ -41,6 +43,9 @@ struct errata_checkfunc_id {
>  void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
>                               unsigned long archid, unsigned long impid,
>                               unsigned int stage);
> +void thead_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
> +                            unsigned long archid, unsigned long impid,
> +                            unsigned int stage);
>
>  void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
>                                  unsigned int stage);
> @@ -48,6 +53,7 @@ void riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end,
>  #else /* CONFIG_RISCV_ALTERNATIVE */
>
>  static inline void apply_boot_alternatives(void) { }
> +static inline void apply_early_boot_alternatives(void) { }
>  static inline void apply_module_alternatives(void *start, size_t length) { }
>
>  #endif /* CONFIG_RISCV_ALTERNATIVE */
> diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h
> index dbfcd9b72bd8..9e2888dbb5b1 100644
> --- a/arch/riscv/include/asm/errata_list.h
> +++ b/arch/riscv/include/asm/errata_list.h
> @@ -14,6 +14,11 @@
>  #define        ERRATA_SIFIVE_NUMBER 2
>  #endif
>
> +#ifdef CONFIG_ERRATA_THEAD
> +#define        ERRATA_THEAD_PBMT 0
> +#define        ERRATA_THEAD_NUMBER 1
> +#endif
> +
>  #define        CPUFEATURE_SVPBMT 0
>  #define        CPUFEATURE_NUMBER 1
>
> @@ -42,12 +47,51 @@ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID,    \
>   * in the default case.
>   */
>  #define ALT_SVPBMT_SHIFT 61
> +#define ALT_THEAD_PBMT_SHIFT 59
>  #define ALT_SVPBMT(_val, prot)                                         \
> -asm(ALTERNATIVE("li %0, 0\t\nnop", "li %0, %1\t\nslli %0,%0,%2", 0,    \
> -               CPUFEATURE_SVPBMT, CONFIG_RISCV_ISA_SVPBMT)             \
> +asm(ALTERNATIVE_2("li %0, 0\t\nnop",                                   \
> +                 "li %0, %1\t\nslli %0,%0,%3", 0,                      \
> +                       CPUFEATURE_SVPBMT, CONFIG_RISCV_ISA_SVPBMT,     \
> +                 "li %0, %2\t\nslli %0,%0,%4", THEAD_VENDOR_ID,        \
> +                       ERRATA_THEAD_PBMT, CONFIG_ERRATA_THEAD_PBMT)    \
>                 : "=r"(_val)                                            \
>                 : "I"(prot##_SVPBMT >> ALT_SVPBMT_SHIFT),               \
> -                 "I"(ALT_SVPBMT_SHIFT))
> +                 "I"(prot##_THEAD >> ALT_THEAD_PBMT_SHIFT),            \
> +                 "I"(ALT_SVPBMT_SHIFT),                                \
> +                 "I"(ALT_THEAD_PBMT_SHIFT))
> +
> +#ifdef CONFIG_ERRATA_THEAD_PBMT
> +/*
> + * IO/NOCACHE memory types are handled together with svpbmt,
> + * so on T-Head chips, check if no other memory type is set,
> + * and set the non-0 PMA type if applicable.
> + */
> +#define ALT_THEAD_PMA(_val)                                            \
> +asm volatile(ALTERNATIVE(                                              \
> +       "nop\n\t"                                                       \
> +       "nop\n\t"                                                       \
> +       "nop\n\t"                                                       \
> +       "nop\n\t"                                                       \
> +       "nop\n\t"                                                       \
> +       "nop\n\t"                                                       \
> +       "nop",                                                          \
> +       "li      t3, %2\n\t"                                            \
> +       "slli    t3, t3, %4\n\t"                                        \
> +       "and     t3, %0, t3\n\t"                                        \
> +       "bne     t3, zero, 2f\n\t"                                      \
> +       "li      t3, %3\n\t"                                            \
> +       "slli    t3, t3, %4\n\t"                                        \
> +       "or      %0, %0, t3\n\t"                                        \
> +       "2:",  THEAD_VENDOR_ID,                                         \
> +               ERRATA_THEAD_PBMT, CONFIG_ERRATA_THEAD_PBMT)            \
> +       : "+r"(_val)                                                    \
> +       : "0"(_val),                                                    \
> +         "I"(_PAGE_MTMASK_THEAD >> ALT_THEAD_PBMT_SHIFT),              \
> +         "I"(_PAGE_PMA_THEAD >> ALT_THEAD_PBMT_SHIFT),                 \
> +         "I"(ALT_THEAD_PBMT_SHIFT))
> +#else
> +#define ALT_THEAD_PMA(_val)
> +#endif
>
>  #endif /* __ASSEMBLY__ */
>
> diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
> index 2354501f0203..e4ff3e0ab887 100644
> --- a/arch/riscv/include/asm/pgtable-64.h
> +++ b/arch/riscv/include/asm/pgtable-64.h
> @@ -86,6 +86,18 @@ typedef struct {
>  #define _PAGE_IO_SVPBMT                (1UL << 62)
>  #define _PAGE_MTMASK_SVPBMT    (_PAGE_NOCACHE_SVPBMT | _PAGE_IO_SVPBMT)
>
> +/*
> + * [63:59] T-Head Memory Type definitions:
> + *
> + * 00000 - NC   Weakly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
> + * 01110 - PMA  Weakly-ordered, Cacheable, Bufferable, Shareable, Non-trustable
> + * 10000 - IO   Strongly-ordered, Non-cacheable, Non-bufferable, Non-shareable, Non-trustable
> + */
> +#define _PAGE_PMA_THEAD                ((1UL << 62) | (1UL << 61) | (1UL << 60))
> +#define _PAGE_NOCACHE_THEAD    0UL
> +#define _PAGE_IO_THEAD         (1UL << 63)
> +#define _PAGE_MTMASK_THEAD     (_PAGE_PMA_THEAD | _PAGE_IO_THEAD | (1UL << 59))
> +
>  static inline u64 riscv_page_mtmask(void)
>  {
>         u64 val;
> @@ -193,7 +205,11 @@ static inline bool mm_pud_folded(struct mm_struct *mm)
>
>  static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
>  {
> -       return __pmd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
> +       unsigned long prot_val = pgprot_val(prot);
> +
> +       ALT_THEAD_PMA(prot_val);
> +
> +       return __pmd((pfn << _PAGE_PFN_SHIFT) | prot_val);
>  }
>
>  static inline unsigned long _pmd_pfn(pmd_t pmd)
> diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
> index c55341b72de1..6f0a260d3f2c 100644
> --- a/arch/riscv/include/asm/pgtable.h
> +++ b/arch/riscv/include/asm/pgtable.h
> @@ -250,7 +250,11 @@ static inline void pmd_clear(pmd_t *pmdp)
>
>  static inline pgd_t pfn_pgd(unsigned long pfn, pgprot_t prot)
>  {
> -       return __pgd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
> +       unsigned long prot_val = pgprot_val(prot);
> +
> +       ALT_THEAD_PMA(prot_val);
> +
> +       return __pgd((pfn << _PAGE_PFN_SHIFT) | prot_val);
>  }
>
>  static inline unsigned long _pgd_pfn(pgd_t pgd)
> @@ -289,7 +293,11 @@ static inline unsigned long pte_pfn(pte_t pte)
>  /* Constructs a page table entry */
>  static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)
>  {
> -       return __pte((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
> +       unsigned long prot_val = pgprot_val(prot);
> +
> +       ALT_THEAD_PMA(prot_val);
> +
> +       return __pte((pfn << _PAGE_PFN_SHIFT) | prot_val);
>  }
>
>  #define mk_pte(page, prot)       pfn_pte(page_to_pfn(page), prot)
> @@ -398,7 +406,11 @@ static inline int pmd_protnone(pmd_t pmd)
>  /* Modify page protection bits */
>  static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
>  {
> -       return __pte((pte_val(pte) & _PAGE_CHG_MASK) | pgprot_val(newprot));
> +       unsigned long newprot_val = pgprot_val(newprot);
> +
> +       ALT_THEAD_PMA(newprot_val);
> +
> +       return __pte((pte_val(pte) & _PAGE_CHG_MASK) | newprot_val);
>  }
>
>  #define pgd_ERROR(e) \
> diff --git a/arch/riscv/include/asm/vendorid_list.h b/arch/riscv/include/asm/vendorid_list.h
> index 9d934215b3c8..cb89af3f0704 100644
> --- a/arch/riscv/include/asm/vendorid_list.h
> +++ b/arch/riscv/include/asm/vendorid_list.h
> @@ -6,5 +6,6 @@
>  #define ASM_VENDOR_LIST_H
>
>  #define SIFIVE_VENDOR_ID       0x489
> +#define THEAD_VENDOR_ID                0x5b7
>
>  #endif
> diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
> index 0f8348ac30f1..bf3876a77ed7 100644
> --- a/arch/riscv/kernel/Makefile
> +++ b/arch/riscv/kernel/Makefile
> @@ -14,6 +14,20 @@ ifdef CONFIG_KEXEC
>  AFLAGS_kexec_relocate.o := -mcmodel=medany $(call cc-option,-mno-relax)
>  endif
>
> +# cmodel=medany and notrace when patching early
> +ifdef CONFIG_RISCV_ALTERNATIVE_EARLY
> +CFLAGS_alternative.o := -mcmodel=medany
> +CFLAGS_cpufeature.o := -mcmodel=medany
> +ifdef CONFIG_FTRACE
> +CFLAGS_REMOVE_alternative.o = $(CC_FLAGS_FTRACE)
> +CFLAGS_REMOVE_cpufeature.o = $(CC_FLAGS_FTRACE)
> +endif
> +ifdef CONFIG_KASAN
> +KASAN_SANITIZE_alternative.o := n
> +KASAN_SANITIZE_cpufeature.o := n
> +endif
> +endif
> +
>  extra-y += head.o
>  extra-y += vmlinux.lds
>
> diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
> index 27f722ae452b..c9d0d3c53223 100644
> --- a/arch/riscv/kernel/alternative.c
> +++ b/arch/riscv/kernel/alternative.c
> @@ -42,6 +42,11 @@ static void __init_or_module riscv_fill_cpu_mfr_info(struct cpu_manufacturer_inf
>         case SIFIVE_VENDOR_ID:
>                 cpu_mfr_info->vendor_patch_func = sifive_errata_patch_func;
>                 break;
> +#endif
> +#ifdef CONFIG_ERRATA_THEAD
> +       case THEAD_VENDOR_ID:
> +               cpu_mfr_info->vendor_patch_func = thead_errata_patch_func;
> +               break;
>  #endif
>         default:
>                 cpu_mfr_info->vendor_patch_func = NULL;
> @@ -82,6 +87,27 @@ void __init apply_boot_alternatives(void)
>                             RISCV_ALTERNATIVES_BOOT);
>  }
>
> +/*
> + * apply_early_boot_alternatives() is called from setup_vm() with MMU-off.
> + *
> + * Following requirements should be honoured for it to work correctly:
> + * 1) It should use PC-relative addressing for accessing kernel symbols.
> + *    To achieve this we always use GCC cmodel=medany.
> + * 2) The compiler instrumentation for FTRACE will not work for setup_vm()
> + *    so disable compiler instrumentation when FTRACE is enabled.
> + *
> + * Currently, the above requirements are honoured by using custom CFLAGS
> + * for alternative.o in kernel/Makefile.
> + */
> +void __init apply_early_boot_alternatives(void)
> +{
> +#ifdef CONFIG_RISCV_ALTERNATIVE_EARLY
> +       _apply_alternatives((struct alt_entry *)__alt_start,
> +                           (struct alt_entry *)__alt_end,
> +                           RISCV_ALTERNATIVES_EARLY_BOOT);
> +#endif
> +}
> +
>  #ifdef CONFIG_MODULES
>  void apply_module_alternatives(void *start, size_t length)
>  {
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index f514b949c6a7..dea3ea19deee 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -254,7 +254,12 @@ struct cpufeature_info {
>  static bool __init_or_module cpufeature_svpbmt_check_func(unsigned int stage)
>  {
>  #ifdef CONFIG_RISCV_ISA_SVPBMT
> -       return riscv_isa_extension_available(NULL, SVPBMT);
> +       switch (stage) {
> +       case RISCV_ALTERNATIVES_EARLY_BOOT:
> +               return false;
> +       default:
> +               return riscv_isa_extension_available(NULL, SVPBMT);
> +       }
>  #endif
>
>         return false;
> diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
> index b0793dc0c291..45a4bf107bd2 100644
> --- a/arch/riscv/mm/init.c
> +++ b/arch/riscv/mm/init.c
> @@ -936,6 +936,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa)
>         BUG_ON((kernel_map.virt_addr + kernel_map.size) > ADDRESS_SPACE_END - SZ_4K);
>  #endif
>
> +       apply_early_boot_alternatives();
>         pt_ops_set_early();
>
>         /* Setup early PGD for fixmap */
> --
> 2.35.1
>


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

Re: [PATCH v10 00/12] riscv: support for Svpbmt and D1 memory types

[Heiko Stuebner]

Am Freitag, 13. Mai 2022, 05:32:49 CEST schrieb Palmer Dabbelt:
> On Wed, 11 May 2022 12:29:09 PDT (-0700), heiko@sntech.de wrote:
> > Svpbmt is an extension defining "Supervisor-mode: page-based memory types"
> > for things like non-cacheable pages or I/O memory pages.
> >
> >
> > So this is my 2nd try at implementing Svpbmt (and the diverging D1 memory
> > types) using the alternatives framework.
> >
> > This includes a number of changes to the alternatives mechanism itself.
> > The biggest one being the move to a more central location, as I expect
> > in the future, nearly every chip needing some sort of patching, be it
> > either for erratas or for optional features (svpbmt or others).
> >
> > Detection of the svpbmt functionality is done via Atish's isa extension
> > handling series [0] and thus does not need any dt-parsing of its own
> > anymore.
> >
> > The series also introduces support for the memory types of the D1
> > which are implemented differently to svpbmt. But when patching anyway
> > it's pretty clean to add the D1 variant via ALTERNATIVE_2 to the same
> > location.
> >
> > The only slightly bigger difference is that the "normal" type is not 0
> > as with svpbmt, so kernel patches for this PMA type need to be applied
> > even before the MMU is brought up, so the series introduces a separate
> > stage for that.
> >
> >
> > In theory this series is 2 parts:
> > - alternatives improvements
> > - svpbmt+d1
> >
> > I picked the recipient list from the previous versions, hopefully
> > I didn't forget anybody.
> >
> > I tested the series on:
> > - qemu-rv32 + buildroot rootfs
> > - qemu-rv64 + debian roots
> > - Allwinner D1-Nezha
> > - BeagleV - it at least reached the same point as without the series
> 
> IMO that's fine, it's also broken due to issues around non-coherence but 
> it has an entirely different way of handling things than.
> 
> > I also ran Palmers CI environment on 5.18-rc6 + this series and
> > it passed with all testcases now.
> 
> Thanks, I know that's a bit of a mess.  If I ever get some time I'll try 
> and clean it up, but it keeps finding issues so I'm sort of stuck with 
> it for now.

No worries, once I got it to run, it was easy to use, though needs
quite a bit of time to build everything.

But now that I made it to run, I'll try to use in the future as well :-) .

> As expected it now passes locally, so I've put this on 
> for-next.  I hadn't noticed your testing was on rc6, I put this on top 
> of rc1 -- that's what I usually do for merge window stuff, but if 
> there's something specific between rc1 and rc6 this depends on then LMK 
> and I'll sort it out.

In the past I also had it run on -rc1 without any issues.
Just after refreshing with with your recent changes, it was back at
5.17-something and then I went to 5.18-rc6 as base.

So there wasn't any real reason for -rc6 except was the most current
release :-)

Heiko

> > changes in v10:
> > - add received review-tags
> > - put early patching behind a kconfig symbol
> > - adapt compiler flags of sources in use by early patching
> >   similar to other riscv arch-parts.
> >   This fixes the medlow cmodel issue on rv32 and also issues
> >   with Kasan.
> >
> > changes in v9:
> > - rebase onto 5.18-rc1
> > - drop the sbi null-ptr patch
> >   While I still think this to be non-ideal as is, it isn't really
> >   necessary for svpbmt support anymore
> > - merge cpufeature + svpbmt patch, as otherwise some empty shells
> >   cause build warnings when a bisection stops between these two
> >   patches
> > - address review comments from Christoph Hellwig:
> >   - keep alternatives optional, they now get selected by its
> >     users (erratas and also the newly introduced svpbmt kconfig)
> >   - wrap long lines and keep things below 80 characters
> >   - restyle svpbmt + thead errata assembly
> >   - introduce a helper for the repeated calls to
> >     (val & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT
> >
> > changes in v8:
> > - rebase onto 5.17-final + isa extension series
> >   We're halfway through the merge-window, so this series
> >   should be merge after that
> > - adapt to fix limiting alternatives to non-xip-kernels
> > - add .norelax option for alternatives
> > - fix unused cpu_apply_errata in thead errata
> > - don't use static globals to store cpu-manufacturer data
> >   as it makes machines hang if done too early
> >
> > changes in v7:
> > - fix typo in patch1 (Atish)
> > - moved to Atish's isa-extension framework
> > - and therefore move regular boot-alternatives directly behind fill_hwcaps
> > - change T-Head errata Kconfig text (Atish)
> >
> > changes in v6:
> > - rebase onto 5.17-rc1
> > - handle sbi null-ptr differently
> > - improve commit messages
> > - use riscv,mmu as property name
> >
> > changes in v5:
> > - move to use alternatives for runtime-patching
> > - add D1 variant
> >
> >
> > [0] https://lore.kernel.org/r/20220222204811.2281949-2-atishp@rivosinc.com
> >
> > Heiko Stuebner (12):
> >   riscv: integrate alternatives better into the main architecture
> >   riscv: allow different stages with alternatives
> >   riscv: implement module alternatives
> >   riscv: implement ALTERNATIVE_2 macro
> >   riscv: extend concatenated alternatives-lines to the same length
> >   riscv: prevent compressed instructions in alternatives
> >   riscv: move boot alternatives to after fill_hwcap
> >   riscv: Fix accessing pfn bits in PTEs for non-32bit variants
> >   riscv: add RISC-V Svpbmt extension support
> >   riscv: remove FIXMAP_PAGE_IO and fall back to its default value
> >   riscv: don't use global static vars to store alternative data
> >   riscv: add memory-type errata for T-Head
> >
> >  arch/riscv/Kconfig                          |  28 +++++
> >  arch/riscv/Kconfig.erratas                  |  34 ++++--
> >  arch/riscv/Kconfig.socs                     |   1 -
> >  arch/riscv/Makefile                         |   2 +-
> >  arch/riscv/errata/Makefile                  |   2 +-
> >  arch/riscv/errata/alternative.c             |  75 ------------
> >  arch/riscv/errata/sifive/errata.c           |  20 ++-
> >  arch/riscv/errata/thead/Makefile            |  11 ++
> >  arch/riscv/errata/thead/errata.c            |  82 +++++++++++++
> >  arch/riscv/include/asm/alternative-macros.h | 129 +++++++++++++++-----
> >  arch/riscv/include/asm/alternative.h        |  25 +++-
> >  arch/riscv/include/asm/errata_list.h        |  59 +++++++++
> >  arch/riscv/include/asm/fixmap.h             |   2 -
> >  arch/riscv/include/asm/hwcap.h              |   1 +
> >  arch/riscv/include/asm/pgtable-32.h         |  17 +++
> >  arch/riscv/include/asm/pgtable-64.h         |  79 +++++++++++-
> >  arch/riscv/include/asm/pgtable-bits.h       |  10 --
> >  arch/riscv/include/asm/pgtable.h            |  55 +++++++--
> >  arch/riscv/include/asm/vendorid_list.h      |   1 +
> >  arch/riscv/kernel/Makefile                  |  15 +++
> >  arch/riscv/kernel/alternative.c             | 118 ++++++++++++++++++
> >  arch/riscv/kernel/cpu.c                     |   1 +
> >  arch/riscv/kernel/cpufeature.c              |  80 +++++++++++-
> >  arch/riscv/kernel/module.c                  |  29 +++++
> >  arch/riscv/kernel/setup.c                   |   2 +
> >  arch/riscv/kernel/smpboot.c                 |   4 -
> >  arch/riscv/kernel/traps.c                   |   2 +-
> >  arch/riscv/mm/init.c                        |   1 +
> >  28 files changed, 724 insertions(+), 161 deletions(-)
> >  delete mode 100644 arch/riscv/errata/alternative.c
> >  create mode 100644 arch/riscv/errata/thead/Makefile
> >  create mode 100644 arch/riscv/errata/thead/errata.c
> >  create mode 100644 arch/riscv/kernel/alternative.c
> 

Re: [PATCH 01/12] riscv: integrate alternatives better into the main architecture

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 02/12] riscv: allow different stages with alternatives

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 03/12] riscv: implement module alternatives

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 04/12] riscv: implement ALTERNATIVE_2 macro

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 05/12] riscv: extend concatenated alternatives-lines to the same length

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 06/12] riscv: prevent compressed instructions in alternatives

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 08/12] riscv: Fix accessing pfn bits in PTEs for non-32bit variants

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 09/12] riscv: add RISC-V Svpbmt extension support

[Christoph Hellwig]

> +config RISCV_ISA_SVPBMT
> +	bool "SVPBMT extension support"

I don't think this prompt is very useful as it doesn't describe
what it does.  But do we even want people to disable it as it is
really essentially for a fully functioning kernel and a pity that
it took RISC-V so long to get there?

> +	depends on 64BIT && MMU
> +	select RISCV_ALTERNATIVE
> +	default y
> +	help
> +	   Adds support to dynamically detect the presence of the SVPBMT extension

overly long line here.

> index 5f1046e82d9f..dbfcd9b72bd8 100644
> --- a/arch/riscv/include/asm/errata_list.h
> +++ b/arch/riscv/include/asm/errata_list.h
> @@ -14,6 +14,9 @@
>  #define	ERRATA_SIFIVE_NUMBER 2
>  #endif
>  
> +#define	CPUFEATURE_SVPBMT 0
> +#define	CPUFEATURE_NUMBER 1

is errata_list.h really the right place for architectural features?

Otherwise looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 11/12] riscv: don't use global static vars to store alternative data

[Christoph Hellwig]

On Wed, May 11, 2022 at 09:29:20PM +0200, Heiko Stuebner wrote:
> Right now the code uses a global struct to store vendor-ids
> and another global variable to store the vendor-patch-function.
> 
> There exist specific cases where we'll need to patch the kernel
> at an even earlier stage, where trying to write to a static
> variable might actually result in hangs.
> 
> Also collecting the vendor-information consists of 3 sbi-ecalls
> (or csr-reads) which is pretty negligible in the context of
> booting a kernel.
> 
> So rework the code to not rely on static variables and instead
> collect the vendor-information when a round of alternatives is
> to be applied.
> 
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Guo Ren <guoren@kernel.org>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/kernel/alternative.c | 51 ++++++++++++++++-----------------
>  1 file changed, 24 insertions(+), 27 deletions(-)
> 
> diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
> index e6c9de9f9ba6..27f722ae452b 100644
> --- a/arch/riscv/kernel/alternative.c
> +++ b/arch/riscv/kernel/alternative.c
> @@ -16,41 +16,35 @@
>  #include <asm/sbi.h>
>  #include <asm/csr.h>
>  
> -static struct cpu_manufacturer_info_t {
> +struct cpu_manufacturer_info_t {
>  	unsigned long vendor_id;
>  	unsigned long arch_id;
>  	unsigned long imp_id;
> -} cpu_mfr_info;
> +	void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
> +				  unsigned long archid, unsigned long impid,
> +				  unsigned int stage);

Please drop the confusing vendor_ prefix for the function pointer
while you're at it.  The vendor id is just one of three inputs for
the patching.

Otherwise this looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 01/12] riscv: integrate alternatives better into the main architecture

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, May 12, 2022 at 3:29 AM Heiko Stuebner <heiko@sntech.de> wrote:
>
> Right now the alternatives need to be explicitly enabled and
> erratas are limited to SiFive ones.
>
> We want to use alternatives not only for patching soc erratas,
> but in the future also for handling different behaviour depending
> on the existence of future extensions.
>
> So move the core alternatives over to the kernel subdirectory
> and move the CONFIG_RISCV_ALTERNATIVE to be a hidden symbol
> which we expect relevant erratas and extensions to just select
> if needed.
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/Kconfig                          |  9 +++++++++
>  arch/riscv/Kconfig.erratas                  | 13 ++-----------
>  arch/riscv/Kconfig.socs                     |  1 -
>  arch/riscv/Makefile                         |  2 +-
>  arch/riscv/errata/Makefile                  |  1 -
>  arch/riscv/include/asm/alternative-macros.h |  7 ++++---
>  arch/riscv/include/asm/alternative.h        |  8 ++++++++
>  arch/riscv/kernel/Makefile                  |  1 +
>  arch/riscv/{errata => kernel}/alternative.c |  0
>  arch/riscv/kernel/smpboot.c                 |  2 --
>  arch/riscv/kernel/traps.c                   |  2 +-
>  11 files changed, 26 insertions(+), 20 deletions(-)
>  rename arch/riscv/{errata => kernel}/alternative.c (100%)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index f8a55d94016d..1ec07aa582a3 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -325,6 +325,15 @@ config NODES_SHIFT
>           Specify the maximum number of NUMA Nodes available on the target
>           system.  Increases memory reserved to accommodate various tables.
>
> +config RISCV_ALTERNATIVE
> +       bool
> +       depends on !XIP_KERNEL
> +       help
> +         This Kconfig allows the kernel to automatically patch the
> +         errata required by the execution platform at run time. The
> +         code patching is performed once in the boot stages. It means
> +         that the overhead from this mechanism is just taken once.
> +
>  config RISCV_ISA_C
>         bool "Emit compressed instructions when building Linux"
>         default y
> diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas
> index 0aacd7052585..c521c2ae2de2 100644
> --- a/arch/riscv/Kconfig.erratas
> +++ b/arch/riscv/Kconfig.erratas
> @@ -1,18 +1,9 @@
>  menu "CPU errata selection"
>
> -config RISCV_ERRATA_ALTERNATIVE
> -       bool "RISC-V alternative scheme"
> -       depends on !XIP_KERNEL
> -       default y
> -       help
> -         This Kconfig allows the kernel to automatically patch the
> -         errata required by the execution platform at run time. The
> -         code patching is performed once in the boot stages. It means
> -         that the overhead from this mechanism is just taken once.
> -
>  config ERRATA_SIFIVE
>         bool "SiFive errata"
> -       depends on RISCV_ERRATA_ALTERNATIVE
> +       depends on !XIP_KERNEL
> +       select RISCV_ALTERNATIVE
>         help
>           All SiFive errata Kconfig depend on this Kconfig. Disabling
>           this Kconfig will disable all SiFive errata. Please say "Y"
> diff --git a/arch/riscv/Kconfig.socs b/arch/riscv/Kconfig.socs
> index f6ef358d8a2c..85670dc9fe95 100644
> --- a/arch/riscv/Kconfig.socs
> +++ b/arch/riscv/Kconfig.socs
> @@ -14,7 +14,6 @@ config SOC_SIFIVE
>         select CLK_SIFIVE
>         select CLK_SIFIVE_PRCI
>         select SIFIVE_PLIC
> -       select RISCV_ERRATA_ALTERNATIVE if !XIP_KERNEL
>         select ERRATA_SIFIVE if !XIP_KERNEL
>         help
>           This enables support for SiFive SoC platform hardware.
> diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
> index 7d81102cffd4..a7ed47ce9311 100644
> --- a/arch/riscv/Makefile
> +++ b/arch/riscv/Makefile
> @@ -103,7 +103,7 @@ endif
>
>  head-y := arch/riscv/kernel/head.o
>
> -core-$(CONFIG_RISCV_ERRATA_ALTERNATIVE) += arch/riscv/errata/
> +core-y += arch/riscv/errata/
>  core-$(CONFIG_KVM) += arch/riscv/kvm/
>
>  libs-y += arch/riscv/lib/
> diff --git a/arch/riscv/errata/Makefile b/arch/riscv/errata/Makefile
> index b8f8740a3e44..0ca1c5281a2d 100644
> --- a/arch/riscv/errata/Makefile
> +++ b/arch/riscv/errata/Makefile
> @@ -1,2 +1 @@
> -obj-y  += alternative.o
>  obj-$(CONFIG_ERRATA_SIFIVE) += sifive/
> diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
> index 67406c376389..5dd8d03a13da 100644
> --- a/arch/riscv/include/asm/alternative-macros.h
> +++ b/arch/riscv/include/asm/alternative-macros.h
> @@ -2,7 +2,7 @@
>  #ifndef __ASM_ALTERNATIVE_MACROS_H
>  #define __ASM_ALTERNATIVE_MACROS_H
>
> -#ifdef CONFIG_RISCV_ERRATA_ALTERNATIVE
> +#ifdef CONFIG_RISCV_ALTERNATIVE
>
>  #ifdef __ASSEMBLY__
>
> @@ -76,7 +76,7 @@
>
>  #endif /* __ASSEMBLY__ */
>
> -#else /* !CONFIG_RISCV_ERRATA_ALTERNATIVE*/
> +#else /* CONFIG_RISCV_ALTERNATIVE */
>  #ifdef __ASSEMBLY__
>
>  .macro __ALTERNATIVE_CFG old_c
> @@ -95,7 +95,8 @@
>         __ALTERNATIVE_CFG(old_c)
>
>  #endif /* __ASSEMBLY__ */
> -#endif /* CONFIG_RISCV_ERRATA_ALTERNATIVE */
> +#endif /* CONFIG_RISCV_ALTERNATIVE */
> +
>  /*
>   * Usage:
>   *   ALTERNATIVE(old_content, new_content, vendor_id, errata_id, CONFIG_k)
> diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
> index e625d3cafbed..7b42bcef0ecf 100644
> --- a/arch/riscv/include/asm/alternative.h
> +++ b/arch/riscv/include/asm/alternative.h
> @@ -12,6 +12,8 @@
>
>  #ifndef __ASSEMBLY__
>
> +#ifdef CONFIG_RISCV_ALTERNATIVE
> +
>  #include <linux/init.h>
>  #include <linux/types.h>
>  #include <linux/stddef.h>
> @@ -35,5 +37,11 @@ struct errata_checkfunc_id {
>  void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
>                               unsigned long archid, unsigned long impid);
>
> +#else /* CONFIG_RISCV_ALTERNATIVE */
> +
> +static inline void apply_boot_alternatives(void) { }
> +
> +#endif /* CONFIG_RISCV_ALTERNATIVE */
> +
>  #endif
>  #endif
> diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
> index 87adbe47bc15..0f8348ac30f1 100644
> --- a/arch/riscv/kernel/Makefile
> +++ b/arch/riscv/kernel/Makefile
> @@ -18,6 +18,7 @@ extra-y += head.o
>  extra-y += vmlinux.lds
>
>  obj-y  += soc.o
> +obj-$(CONFIG_RISCV_ALTERNATIVE) += alternative.o
>  obj-y  += cpu.o
>  obj-y  += cpufeature.o
>  obj-y  += entry.o
> diff --git a/arch/riscv/errata/alternative.c b/arch/riscv/kernel/alternative.c
> similarity index 100%
> rename from arch/riscv/errata/alternative.c
> rename to arch/riscv/kernel/alternative.c
> diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
> index 622f226454d5..a6d13dca1403 100644
> --- a/arch/riscv/kernel/smpboot.c
> +++ b/arch/riscv/kernel/smpboot.c
> @@ -41,9 +41,7 @@ static DECLARE_COMPLETION(cpu_running);
>  void __init smp_prepare_boot_cpu(void)
>  {
>         init_cpu_topology();
> -#ifdef CONFIG_RISCV_ERRATA_ALTERNATIVE
>         apply_boot_alternatives();
> -#endif
>  }
>
>  void __init smp_prepare_cpus(unsigned int max_cpus)
> diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> index fe92e119e6a3..efa693b325a1 100644
> --- a/arch/riscv/kernel/traps.c
> +++ b/arch/riscv/kernel/traps.c
> @@ -86,7 +86,7 @@ static void do_trap_error(struct pt_regs *regs, int signo, int code,
>         }
>  }
>
> -#if defined (CONFIG_XIP_KERNEL) && defined (CONFIG_RISCV_ERRATA_ALTERNATIVE)
> +#if defined (CONFIG_XIP_KERNEL) && defined (CONFIG_RISCV_ALTERNATIVE)
>  #define __trap_section         __section(".xip.traps")
>  #else
>  #define __trap_section
> --
> 2.35.1
>


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

Re: [PATCH 02/12] riscv: allow different stages with alternatives

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, May 12, 2022 at 3:29 AM Heiko Stuebner <heiko@sntech.de> wrote:
>
> Future features may need to be applied at a different
> time during boot, so allow defining stages for alternatives
> and handling them differently depending on the stage.
>
> Also make the alternatives-location more flexible so that
> future stages may provide their own location.
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/errata/sifive/errata.c    |  3 ++-
>  arch/riscv/include/asm/alternative.h |  5 ++++-
>  arch/riscv/kernel/alternative.c      | 26 +++++++++++++++++---------
>  3 files changed, 23 insertions(+), 11 deletions(-)
>
> diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
> index f5e5ae70e829..4fe03ac41fd7 100644
> --- a/arch/riscv/errata/sifive/errata.c
> +++ b/arch/riscv/errata/sifive/errata.c
> @@ -80,7 +80,8 @@ static void __init warn_miss_errata(u32 miss_errata)
>  }
>
>  void __init sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
> -                                    unsigned long archid, unsigned long impid)
> +                                    unsigned long archid, unsigned long impid,
> +                                    unsigned int stage)
>  {
>         struct alt_entry *alt;
>         u32 cpu_req_errata = sifive_errata_probe(archid, impid);
> diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
> index 7b42bcef0ecf..0ff550667e94 100644
> --- a/arch/riscv/include/asm/alternative.h
> +++ b/arch/riscv/include/asm/alternative.h
> @@ -19,6 +19,8 @@
>  #include <linux/stddef.h>
>  #include <asm/hwcap.h>
>
> +#define RISCV_ALTERNATIVES_BOOT                0 /* alternatives applied during regular boot */
> +
>  void __init apply_boot_alternatives(void);
>
>  struct alt_entry {
> @@ -35,7 +37,8 @@ struct errata_checkfunc_id {
>  };
>
>  void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
> -                             unsigned long archid, unsigned long impid);
> +                             unsigned long archid, unsigned long impid,
> +                             unsigned int stage);
>
>  #else /* CONFIG_RISCV_ALTERNATIVE */
>
> diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
> index e8b4a0fe488c..02db62f55bac 100644
> --- a/arch/riscv/kernel/alternative.c
> +++ b/arch/riscv/kernel/alternative.c
> @@ -22,8 +22,8 @@ static struct cpu_manufacturer_info_t {
>  } cpu_mfr_info;
>
>  static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
> -                                unsigned long archid,
> -                                unsigned long impid) __initdata;
> +                                unsigned long archid, unsigned long impid,
> +                                unsigned int stage) __initdata;
>
>  static inline void __init riscv_fill_cpu_mfr_info(void)
>  {
> @@ -58,6 +58,18 @@ static void __init init_alternative(void)
>   * a feature detect on the boot CPU). No need to worry about other CPUs
>   * here.
>   */
> +static void __init _apply_alternatives(struct alt_entry *begin,
> +                                      struct alt_entry *end,
> +                                      unsigned int stage)
> +{
> +       if (!vendor_patch_func)
> +               return;
> +
> +       vendor_patch_func(begin, end,
> +                         cpu_mfr_info.arch_id, cpu_mfr_info.imp_id,
> +                         stage);
> +}
> +
>  void __init apply_boot_alternatives(void)
>  {
>         /* If called on non-boot cpu things could go wrong */
> @@ -65,11 +77,7 @@ void __init apply_boot_alternatives(void)
>
>         init_alternative();
>
> -       if (!vendor_patch_func)
> -               return;
> -
> -       vendor_patch_func((struct alt_entry *)__alt_start,
> -                         (struct alt_entry *)__alt_end,
> -                         cpu_mfr_info.arch_id, cpu_mfr_info.imp_id);
> +       _apply_alternatives((struct alt_entry *)__alt_start,
> +                           (struct alt_entry *)__alt_end,
> +                           RISCV_ALTERNATIVES_BOOT);
>  }
> -
> --
> 2.35.1
>


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

Re: [PATCH 03/12] riscv: implement module alternatives

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, May 12, 2022 at 3:29 AM Heiko Stuebner <heiko@sntech.de> wrote:
>
> This allows alternatives to also be applied when loading modules
> and follows the implementation of other architectures (e.g. arm64).
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/errata/sifive/errata.c    | 14 +++++++++-----
>  arch/riscv/include/asm/alternative.h |  3 +++
>  arch/riscv/kernel/alternative.c      | 18 +++++++++++++----
>  arch/riscv/kernel/module.c           | 29 ++++++++++++++++++++++++++++
>  4 files changed, 55 insertions(+), 9 deletions(-)
>
> diff --git a/arch/riscv/errata/sifive/errata.c b/arch/riscv/errata/sifive/errata.c
> index 4fe03ac41fd7..3e39587a49dc 100644
> --- a/arch/riscv/errata/sifive/errata.c
> +++ b/arch/riscv/errata/sifive/errata.c
> @@ -4,6 +4,7 @@
>   */
>
>  #include <linux/kernel.h>
> +#include <linux/module.h>
>  #include <linux/string.h>
>  #include <linux/bug.h>
>  #include <asm/patch.h>
> @@ -54,7 +55,8 @@ static struct errata_info_t errata_list[ERRATA_SIFIVE_NUMBER] = {
>         },
>  };
>
> -static u32 __init sifive_errata_probe(unsigned long archid, unsigned long impid)
> +static u32 __init_or_module sifive_errata_probe(unsigned long archid,
> +                                               unsigned long impid)
>  {
>         int idx;
>         u32 cpu_req_errata = 0;
> @@ -66,7 +68,7 @@ static u32 __init sifive_errata_probe(unsigned long archid, unsigned long impid)
>         return cpu_req_errata;
>  }
>
> -static void __init warn_miss_errata(u32 miss_errata)
> +static void __init_or_module warn_miss_errata(u32 miss_errata)
>  {
>         int i;
>
> @@ -79,9 +81,11 @@ static void __init warn_miss_errata(u32 miss_errata)
>         pr_warn("----------------------------------------------------------------\n");
>  }
>
> -void __init sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
> -                                    unsigned long archid, unsigned long impid,
> -                                    unsigned int stage)
> +void __init_or_module sifive_errata_patch_func(struct alt_entry *begin,
> +                                              struct alt_entry *end,
> +                                              unsigned long archid,
> +                                              unsigned long impid,
> +                                              unsigned int stage)
>  {
>         struct alt_entry *alt;
>         u32 cpu_req_errata = sifive_errata_probe(archid, impid);
> diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h
> index 0ff550667e94..0859529ff08e 100644
> --- a/arch/riscv/include/asm/alternative.h
> +++ b/arch/riscv/include/asm/alternative.h
> @@ -20,8 +20,10 @@
>  #include <asm/hwcap.h>
>
>  #define RISCV_ALTERNATIVES_BOOT                0 /* alternatives applied during regular boot */
> +#define RISCV_ALTERNATIVES_MODULE      1 /* alternatives applied during module-init */
>
>  void __init apply_boot_alternatives(void);
> +void apply_module_alternatives(void *start, size_t length);
>
>  struct alt_entry {
>         void *old_ptr;           /* address of original instruciton or data  */
> @@ -43,6 +45,7 @@ void sifive_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
>  #else /* CONFIG_RISCV_ALTERNATIVE */
>
>  static inline void apply_boot_alternatives(void) { }
> +static inline void apply_module_alternatives(void *start, size_t length) { }
>
>  #endif /* CONFIG_RISCV_ALTERNATIVE */
>
> diff --git a/arch/riscv/kernel/alternative.c b/arch/riscv/kernel/alternative.c
> index 02db62f55bac..223770b3945c 100644
> --- a/arch/riscv/kernel/alternative.c
> +++ b/arch/riscv/kernel/alternative.c
> @@ -7,6 +7,7 @@
>   */
>
>  #include <linux/init.h>
> +#include <linux/module.h>
>  #include <linux/cpu.h>
>  #include <linux/uaccess.h>
>  #include <asm/alternative.h>
> @@ -23,7 +24,7 @@ static struct cpu_manufacturer_info_t {
>
>  static void (*vendor_patch_func)(struct alt_entry *begin, struct alt_entry *end,
>                                  unsigned long archid, unsigned long impid,
> -                                unsigned int stage) __initdata;
> +                                unsigned int stage) __initdata_or_module;
>
>  static inline void __init riscv_fill_cpu_mfr_info(void)
>  {
> @@ -58,9 +59,9 @@ static void __init init_alternative(void)
>   * a feature detect on the boot CPU). No need to worry about other CPUs
>   * here.
>   */
> -static void __init _apply_alternatives(struct alt_entry *begin,
> -                                      struct alt_entry *end,
> -                                      unsigned int stage)
> +static void __init_or_module _apply_alternatives(struct alt_entry *begin,
> +                                                struct alt_entry *end,
> +                                                unsigned int stage)
>  {
>         if (!vendor_patch_func)
>                 return;
> @@ -81,3 +82,12 @@ void __init apply_boot_alternatives(void)
>                             (struct alt_entry *)__alt_end,
>                             RISCV_ALTERNATIVES_BOOT);
>  }
> +
> +#ifdef CONFIG_MODULES
> +void apply_module_alternatives(void *start, size_t length)
> +{
> +       _apply_alternatives((struct alt_entry *)start,
> +                           (struct alt_entry *)(start + length),
> +                           RISCV_ALTERNATIVES_MODULE);
> +}
> +#endif
> diff --git a/arch/riscv/kernel/module.c b/arch/riscv/kernel/module.c
> index c29cef90d1dd..91fe16bfaa07 100644
> --- a/arch/riscv/kernel/module.c
> +++ b/arch/riscv/kernel/module.c
> @@ -11,6 +11,7 @@
>  #include <linux/vmalloc.h>
>  #include <linux/sizes.h>
>  #include <linux/pgtable.h>
> +#include <asm/alternative.h>
>  #include <asm/sections.h>
>
>  /*
> @@ -427,3 +428,31 @@ void *module_alloc(unsigned long size)
>                                     __builtin_return_address(0));
>  }
>  #endif
> +
> +static const Elf_Shdr *find_section(const Elf_Ehdr *hdr,
> +                                   const Elf_Shdr *sechdrs,
> +                                   const char *name)
> +{
> +       const Elf_Shdr *s, *se;
> +       const char *secstrs = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset;
> +
> +       for (s = sechdrs, se = sechdrs + hdr->e_shnum; s < se; s++) {
> +               if (strcmp(name, secstrs + s->sh_name) == 0)
> +                       return s;
> +       }
> +
> +       return NULL;
> +}
> +
> +int module_finalize(const Elf_Ehdr *hdr,
> +                   const Elf_Shdr *sechdrs,
> +                   struct module *me)
> +{
> +       const Elf_Shdr *s;
> +
> +       s = find_section(hdr, sechdrs, ".alternative");
> +       if (s)
> +               apply_module_alternatives((void *)s->sh_addr, s->sh_size);
> +
> +       return 0;
> +}
> --
> 2.35.1
>


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

Re: [PATCH 04/12] riscv: implement ALTERNATIVE_2 macro

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, May 12, 2022 at 3:29 AM Heiko Stuebner <heiko@sntech.de> wrote:
>
> When the alternatives were added the commit already provided a template
> on how to implement 2 different alternatives for one piece of code.
>
> Make this usable.
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/include/asm/alternative-macros.h | 78 +++++++++++++++------
>  1 file changed, 58 insertions(+), 20 deletions(-)
>
> diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
> index 5dd8d03a13da..9e04cd53afc8 100644
> --- a/arch/riscv/include/asm/alternative-macros.h
> +++ b/arch/riscv/include/asm/alternative-macros.h
> @@ -39,6 +39,24 @@
>  #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
>         __ALTERNATIVE_CFG old_c, new_c, vendor_id, errata_id, IS_ENABLED(CONFIG_k)
>
> +.macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
> +                                 new_c_2, vendor_id_2, errata_id_2, enable_2
> +886 :
> +       \old_c
> +887 :
> +       ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
> +       ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
> +.endm
> +
> +#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,   \
> +                                       CONFIG_k_1,                     \
> +                                 new_c_2, vendor_id_2, errata_id_2,    \
> +                                       CONFIG_k_2)                     \
> +       __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1,   \
> +                                       IS_ENABLED(CONFIG_k_1),         \
> +                                  new_c_2, vendor_id_2, errata_id_2,   \
> +                                       IS_ENABLED(CONFIG_k_2)
> +
>  #else /* !__ASSEMBLY__ */
>
>  #include <asm/asm.h>
> @@ -74,6 +92,25 @@
>  #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
>         __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, IS_ENABLED(CONFIG_k))
>
> +#define __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,  \
> +                                       enable_1,                       \
> +                                  new_c_2, vendor_id_2, errata_id_2,   \
> +                                       enable_2)                       \
> +       "886 :\n"                                                       \
> +       old_c "\n"                                                      \
> +       "887 :\n"                                                       \
> +       ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1)    \
> +       ALT_NEW_CONTENT(vendor_id_2, errata_id_2, enable_2, new_c_2)
> +
> +#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,   \
> +                                       CONFIG_k_1,                     \
> +                                 new_c_2, vendor_id_2, errata_id_2,    \
> +                                       CONFIG_k_2)                     \
> +       __ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,   \
> +                                       IS_ENABLED(CONFIG_k_1),         \
> +                                  new_c_2, vendor_id_2, errata_id_2,   \
> +                                       IS_ENABLED(CONFIG_k_2))
> +
>  #endif /* __ASSEMBLY__ */
>
>  #else /* CONFIG_RISCV_ALTERNATIVE */
> @@ -86,6 +123,12 @@
>  #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
>         __ALTERNATIVE_CFG old_c
>
> +#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,   \
> +                                       CONFIG_k_1,                     \
> +                                 new_c_2, vendor_id_2, errata_id_2,    \
> +                                       CONFIG_k_2)                     \
> +       __ALTERNATIVE_CFG old_c
> +
>  #else /* !__ASSEMBLY__ */
>
>  #define __ALTERNATIVE_CFG(old_c)  \
> @@ -94,6 +137,12 @@
>  #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
>         __ALTERNATIVE_CFG(old_c)
>
> +#define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1,   \
> +                                       CONFIG_k_1,                     \
> +                                 new_c_2, vendor_id_2, errata_id_2,    \
> +                                       CONFIG_k_2) \
> +       __ALTERNATIVE_CFG(old_c)
> +
>  #endif /* __ASSEMBLY__ */
>  #endif /* CONFIG_RISCV_ALTERNATIVE */
>
> @@ -119,25 +168,14 @@
>   * this case, this vendor can create a new macro ALTERNATIVE_2() based
>   * on the following sample code and then replace ALTERNATIVE() with
>   * ALTERNATIVE_2() to append its customized content.
> - *
> - * .macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
> - *                                   new_c_2, vendor_id_2, errata_id_2, enable_2
> - * 886 :
> - *      \old_c
> - * 887 :
> - *      ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
> - *      ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
> - * .endm
> - *
> - * #define _ALTERNATIVE_CFG_2(old_c, new_c_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
> - *                                   new_c_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
> - *        __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, IS_ENABLED(CONFIG_k_1), \
> - *                                   new_c_2, vendor_id_2, errata_id_2, IS_ENABLED(CONFIG_k_2) \
> - *
> - * #define ALTERNATIVE_2(old_content, new_content_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
> - *                                    new_content_2, vendor_id_2, errata_id_2, CONFIG_k_2) \
> - *         _ALTERNATIVE_CFG_2(old_content, new_content_1, vendor_id_1, errata_id_1, CONFIG_k_1, \
> - *                                         new_content_2, vendor_id_2, errata_id_2, CONFIG_k_2)
> - *
>   */
> +#define ALTERNATIVE_2(old_content, new_content_1, vendor_id_1,         \
> +                                       errata_id_1, CONFIG_k_1,        \
> +                                  new_content_2, vendor_id_2,          \
> +                                       errata_id_2, CONFIG_k_2)        \
> +       _ALTERNATIVE_CFG_2(old_content, new_content_1, vendor_id_1,     \
> +                                           errata_id_1, CONFIG_k_1,    \
> +                                       new_content_2, vendor_id_2,     \
> +                                           errata_id_2, CONFIG_k_2)
> +
>  #endif
> --
> 2.35.1
>


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

Re: [PATCH 05/12] riscv: extend concatenated alternatives-lines to the same length

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, May 12, 2022 at 3:29 AM Heiko Stuebner <heiko@sntech.de> wrote:
>
> ALT_NEW_CONTENT already uses same-length assembler lines, so
> extend this to the other elements as well.
>
> This makes it more readable when these elements need to be extended
> in the future.
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/include/asm/alternative-macros.h | 20 ++++++++++----------
>  1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
> index 9e04cd53afc8..8c2bbc7bbe50 100644
> --- a/arch/riscv/include/asm/alternative-macros.h
> +++ b/arch/riscv/include/asm/alternative-macros.h
> @@ -62,14 +62,14 @@
>  #include <asm/asm.h>
>  #include <linux/stringify.h>
>
> -#define ALT_ENTRY(oldptr, newptr, vendor_id, errata_id, newlen) \
> -       RISCV_PTR " " oldptr "\n" \
> -       RISCV_PTR " " newptr "\n" \
> -       REG_ASM " " vendor_id "\n" \
> -       REG_ASM " " newlen "\n" \
> +#define ALT_ENTRY(oldptr, newptr, vendor_id, errata_id, newlen)                \
> +       RISCV_PTR " " oldptr "\n"                                       \
> +       RISCV_PTR " " newptr "\n"                                       \
> +       REG_ASM " " vendor_id "\n"                                      \
> +       REG_ASM " " newlen "\n"                                         \
>         ".word " errata_id "\n"
>
> -#define ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c) \
> +#define ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)           \
>         ".if " __stringify(enable) " == 1\n"                            \
>         ".pushsection .alternative, \"a\"\n"                            \
>         ALT_ENTRY("886b", "888f", __stringify(vendor_id), __stringify(errata_id), "889f - 888f") \
> @@ -83,10 +83,10 @@
>         ".org   . - (889b - 888b) + (887b - 886b)\n"                    \
>         ".endif\n"
>
> -#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable) \
> -       "886 :\n"       \
> -       old_c "\n"      \
> -       "887 :\n"       \
> +#define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable)  \
> +       "886 :\n"                                                       \
> +       old_c "\n"                                                      \
> +       "887 :\n"                                                       \
>         ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)
>
>  #define _ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, CONFIG_k) \
> --
> 2.35.1
>


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

Re: [PATCH 06/12] riscv: prevent compressed instructions in alternatives

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, May 12, 2022 at 3:29 AM Heiko Stuebner <heiko@sntech.de> wrote:
>
> Instructions are opportunistically compressed by the RISC-V assembler
> when possible, but in alternatives-blocks both the old and new content
> need to be the same size, so having the toolchain do somewhat random
> optimizations will cause strange side-effects like
> "attempt to move .org backwards" compile-time errors.
>
> Already a simple "and" used in alternatives assembly will cause these
> mismatched code sizes.
>
> So prevent compressed instructions to be generated in alternatives-
> code and use option-push and -pop to only limit this to the relevant
> code blocks
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/include/asm/alternative-macros.h | 24 +++++++++++++++++++++
>  1 file changed, 24 insertions(+)
>
> diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h
> index 8c2bbc7bbe50..e13b1f6bb400 100644
> --- a/arch/riscv/include/asm/alternative-macros.h
> +++ b/arch/riscv/include/asm/alternative-macros.h
> @@ -21,7 +21,11 @@
>         .popsection
>         .subsection 1
>  888 :
> +       .option push
> +       .option norvc
> +       .option norelax
>         \new_c
> +       .option pop
>  889 :
>         .previous
>         .org    . - (889b - 888b) + (887b - 886b)
> @@ -31,7 +35,11 @@
>
>  .macro __ALTERNATIVE_CFG old_c, new_c, vendor_id, errata_id, enable
>  886 :
> +       .option push
> +       .option norvc
> +       .option norelax
>         \old_c
> +       .option pop
>  887 :
>         ALT_NEW_CONTENT \vendor_id, \errata_id, \enable, \new_c
>  .endm
> @@ -42,7 +50,11 @@
>  .macro __ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, errata_id_1, enable_1, \
>                                   new_c_2, vendor_id_2, errata_id_2, enable_2
>  886 :
> +       .option push
> +       .option norvc
> +       .option norelax
>         \old_c
> +       .option pop
>  887 :
>         ALT_NEW_CONTENT \vendor_id_1, \errata_id_1, \enable_1, \new_c_1
>         ALT_NEW_CONTENT \vendor_id_2, \errata_id_2, \enable_2, \new_c_2
> @@ -76,7 +88,11 @@
>         ".popsection\n"                                                 \
>         ".subsection 1\n"                                               \
>         "888 :\n"                                                       \
> +       ".option push\n"                                                \
> +       ".option norvc\n"                                               \
> +       ".option norelax\n"                                             \
>         new_c "\n"                                                      \
> +       ".option pop\n"                                                 \
>         "889 :\n"                                                       \
>         ".previous\n"                                                   \
>         ".org   . - (887b - 886b) + (889b - 888b)\n"                    \
> @@ -85,7 +101,11 @@
>
>  #define __ALTERNATIVE_CFG(old_c, new_c, vendor_id, errata_id, enable)  \
>         "886 :\n"                                                       \
> +       ".option push\n"                                                \
> +       ".option norvc\n"                                               \
> +       ".option norelax\n"                                             \
>         old_c "\n"                                                      \
> +       ".option pop\n"                                                 \
>         "887 :\n"                                                       \
>         ALT_NEW_CONTENT(vendor_id, errata_id, enable, new_c)
>
> @@ -97,7 +117,11 @@
>                                    new_c_2, vendor_id_2, errata_id_2,   \
>                                         enable_2)                       \
>         "886 :\n"                                                       \
> +       ".option push\n"                                                \
> +       ".option norvc\n"                                               \
> +       ".option norelax\n"                                             \
>         old_c "\n"                                                      \
> +       ".option pop\n"                                                 \
>         "887 :\n"                                                       \
>         ALT_NEW_CONTENT(vendor_id_1, errata_id_1, enable_1, new_c_1)    \
>         ALT_NEW_CONTENT(vendor_id_2, errata_id_2, enable_2, new_c_2)
> --
> 2.35.1
>


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

Re: [PATCH 08/12] riscv: Fix accessing pfn bits in PTEs for non-32bit variants

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, May 12, 2022 at 3:29 AM Heiko Stuebner <heiko@sntech.de> wrote:
>
> On rv32 the PFN part of PTEs is defined to use bits [xlen-1:10]
> while on rv64 it is defined to use bits [53:10], leaving [63:54]
> as reserved.
>
> With upcoming optional extensions like svpbmt these previously
> reserved bits will get used so simply right-shifting the PTE
> to get the PFN won't be enough.
>
> So introduce a _PAGE_PFN_MASK constant to mask the correct bits
> for both rv32 and rv64 before shifting.
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/include/asm/pgtable-32.h   |  8 ++++++++
>  arch/riscv/include/asm/pgtable-64.h   | 14 +++++++++++---
>  arch/riscv/include/asm/pgtable-bits.h |  6 ------
>  arch/riscv/include/asm/pgtable.h      |  8 +++++---
>  4 files changed, 24 insertions(+), 12 deletions(-)
>
> diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
> index 5b2e79e5bfa5..e266a4fe7f43 100644
> --- a/arch/riscv/include/asm/pgtable-32.h
> +++ b/arch/riscv/include/asm/pgtable-32.h
> @@ -7,6 +7,7 @@
>  #define _ASM_RISCV_PGTABLE_32_H
>
>  #include <asm-generic/pgtable-nopmd.h>
> +#include <linux/bits.h>
>  #include <linux/const.h>
>
>  /* Size of region mapped by a page global directory */
> @@ -16,4 +17,11 @@
>
>  #define MAX_POSSIBLE_PHYSMEM_BITS 34
>
> +/*
> + * rv32 PTE format:
> + * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> + *       PFN      reserved for SW   D   A   G   U   X   W   R   V
> + */
> +#define _PAGE_PFN_MASK  GENMASK(31, 10)
> +
>  #endif /* _ASM_RISCV_PGTABLE_32_H */
> diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
> index 7e246e9f8d70..15f3ad5aee4f 100644
> --- a/arch/riscv/include/asm/pgtable-64.h
> +++ b/arch/riscv/include/asm/pgtable-64.h
> @@ -6,6 +6,7 @@
>  #ifndef _ASM_RISCV_PGTABLE_64_H
>  #define _ASM_RISCV_PGTABLE_64_H
>
> +#include <linux/bits.h>
>  #include <linux/const.h>
>
>  extern bool pgtable_l4_enabled;
> @@ -65,6 +66,13 @@ typedef struct {
>
>  #define PTRS_PER_PMD    (PAGE_SIZE / sizeof(pmd_t))
>
> +/*
> + * rv64 PTE format:
> + * | 63 | 62 61 | 60 54 | 53  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> + *   N      MT     RSV    PFN      reserved for SW   D   A   G   U   X   W   R   V
> + */
> +#define _PAGE_PFN_MASK  GENMASK(53, 10)
> +
>  static inline int pud_present(pud_t pud)
>  {
>         return (pud_val(pud) & _PAGE_PRESENT);
> @@ -108,12 +116,12 @@ static inline unsigned long _pud_pfn(pud_t pud)
>
>  static inline pmd_t *pud_pgtable(pud_t pud)
>  {
> -       return (pmd_t *)pfn_to_virt(pud_val(pud) >> _PAGE_PFN_SHIFT);
> +       return (pmd_t *)pfn_to_virt(__page_val_to_pfn(pud_val(pud)));
>  }
>
>  static inline struct page *pud_page(pud_t pud)
>  {
> -       return pfn_to_page(pud_val(pud) >> _PAGE_PFN_SHIFT);
> +       return pfn_to_page(__page_val_to_pfn(pud_val(pud)));
>  }
>
>  #define mm_p4d_folded  mm_p4d_folded
> @@ -143,7 +151,7 @@ static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
>
>  static inline unsigned long _pmd_pfn(pmd_t pmd)
>  {
> -       return pmd_val(pmd) >> _PAGE_PFN_SHIFT;
> +       return __page_val_to_pfn(pmd_val(pmd));
>  }
>
>  #define mk_pmd(page, prot)    pfn_pmd(page_to_pfn(page), prot)
> diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
> index a6b0c89824c2..e571fa954afc 100644
> --- a/arch/riscv/include/asm/pgtable-bits.h
> +++ b/arch/riscv/include/asm/pgtable-bits.h
> @@ -6,12 +6,6 @@
>  #ifndef _ASM_RISCV_PGTABLE_BITS_H
>  #define _ASM_RISCV_PGTABLE_BITS_H
>
> -/*
> - * PTE format:
> - * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> - *       PFN      reserved for SW   D   A   G   U   X   W   R   V
> - */
> -
>  #define _PAGE_ACCESSED_OFFSET 6
>
>  #define _PAGE_PRESENT   (1 << 0)
> diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
> index 046b44225623..faba543e2b08 100644
> --- a/arch/riscv/include/asm/pgtable.h
> +++ b/arch/riscv/include/asm/pgtable.h
> @@ -108,6 +108,8 @@
>  #include <asm/tlbflush.h>
>  #include <linux/mm_types.h>
>
> +#define __page_val_to_pfn(_val)  (((_val) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT)
> +
>  #ifdef CONFIG_64BIT
>  #include <asm/pgtable-64.h>
>  #else
> @@ -261,12 +263,12 @@ static inline unsigned long _pgd_pfn(pgd_t pgd)
>
>  static inline struct page *pmd_page(pmd_t pmd)
>  {
> -       return pfn_to_page(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
> +       return pfn_to_page(__page_val_to_pfn(pmd_val(pmd)));
>  }
>
>  static inline unsigned long pmd_page_vaddr(pmd_t pmd)
>  {
> -       return (unsigned long)pfn_to_virt(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
> +       return (unsigned long)pfn_to_virt(__page_val_to_pfn(pmd_val(pmd)));
>  }
>
>  static inline pte_t pmd_pte(pmd_t pmd)
> @@ -282,7 +284,7 @@ static inline pte_t pud_pte(pud_t pud)
>  /* Yields the page frame number (PFN) of a page table entry */
>  static inline unsigned long pte_pfn(pte_t pte)
>  {
> -       return (pte_val(pte) >> _PAGE_PFN_SHIFT);
> +       return __page_val_to_pfn(pte_val(pte));
>  }
>
>  #define pte_page(x)     pfn_to_page(pte_pfn(x))
> --
> 2.35.1
>


-- 
Best Regards
 Guo Ren

ML: https://lore.kernel.org/linux-csky/

Re: [PATCH 09/12] riscv: add RISC-V Svpbmt extension support

[Philipp Tomsich]

On Mon, 16 May 2022 at 08:11, Christoph Hellwig <hch@lst.de> wrote:
>
> > +config RISCV_ISA_SVPBMT
> > +     bool "SVPBMT extension support"
>
> I don't think this prompt is very useful as it doesn't describe
> what it does.  But do we even want people to disable it as it is
> really essentially for a fully functioning kernel and a pity that
> it took RISC-V so long to get there?

Given that RISC-V is (in some ways) an ISA construction set, there
will be valid use cases for embedded users to disable this (e.g. if
they have their own non-standard way to configure these).  So while
kernels for binary distributions (and desktop, server, or
general-purpose embedded) will always enable these, I would fully
expect some users to want to turn these off.

@Heiko: I would request that we have a longer help text on this, which
explains what it is and ends with the usual "When in doubt, say Y."

> > +     depends on 64BIT && MMU
> > +     select RISCV_ALTERNATIVE
> > +     default y
> > +     help
> > +        Adds support to dynamically detect the presence of the SVPBMT extension
>
> overly long line here.
>
> > index 5f1046e82d9f..dbfcd9b72bd8 100644
> > --- a/arch/riscv/include/asm/errata_list.h
> > +++ b/arch/riscv/include/asm/errata_list.h
> > @@ -14,6 +14,9 @@
> >  #define      ERRATA_SIFIVE_NUMBER 2
> >  #endif
> >
> > +#define      CPUFEATURE_SVPBMT 0
> > +#define      CPUFEATURE_NUMBER 1
>
> is errata_list.h really the right place for architectural features?
>
> Otherwise looks good:
>
> Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 09/12] riscv: add RISC-V Svpbmt extension support

[Heiko Stübner]

Am Montag, 16. Mai 2022, 11:09:12 CEST schrieb Philipp Tomsich:
> On Mon, 16 May 2022 at 08:11, Christoph Hellwig <hch@lst.de> wrote:
> >
> > > +config RISCV_ISA_SVPBMT
> > > +     bool "SVPBMT extension support"
> >
> > I don't think this prompt is very useful as it doesn't describe
> > what it does.  But do we even want people to disable it as it is
> > really essentially for a fully functioning kernel and a pity that
> > it took RISC-V so long to get there?
> 
> Given that RISC-V is (in some ways) an ISA construction set, there
> will be valid use cases for embedded users to disable this (e.g. if
> they have their own non-standard way to configure these).  So while
> kernels for binary distributions (and desktop, server, or
> general-purpose embedded) will always enable these, I would fully
> expect some users to want to turn these off.

Also, enabling the SVPBMT extension will pull in the alternative-patching
as well of course, and having a way to disable that was a review-request
a version in the past.


> @Heiko: I would request that we have a longer help text on this, which
> explains what it is and ends with the usual "When in doubt, say Y."

ok, will do


> > > +     depends on 64BIT && MMU
> > > +     select RISCV_ALTERNATIVE
> > > +     default y
> > > +     help
> > > +        Adds support to dynamically detect the presence of the SVPBMT extension
> >
> > overly long line here.

will fix together with Philipp's help-text wish


> > > index 5f1046e82d9f..dbfcd9b72bd8 100644
> > > --- a/arch/riscv/include/asm/errata_list.h
> > > +++ b/arch/riscv/include/asm/errata_list.h
> > > @@ -14,6 +14,9 @@
> > >  #define      ERRATA_SIFIVE_NUMBER 2
> > >  #endif
> > >
> > > +#define      CPUFEATURE_SVPBMT 0
> > > +#define      CPUFEATURE_NUMBER 1
> >
> > is errata_list.h really the right place for architectural features?

That probably stems from the alternatives being exclusively used
for erratas in the past.

I guess making this "alternative-list.h" might be a better naming?
Or are there even better suggestions?

> > Otherwise looks good:
> >
> > Reviewed-by: Christoph Hellwig <hch@lst.de>

Thanks
Heiko

Re: [PATCH 08/12] riscv: Fix accessing pfn bits in PTEs for non-32bit variants

[Alexandre Ghiti]

Hi Heiko,

On Wed, May 11, 2022 at 9:31 PM Heiko Stuebner <heiko@sntech.de> wrote:
>
> On rv32 the PFN part of PTEs is defined to use bits [xlen-1:10]
> while on rv64 it is defined to use bits [53:10], leaving [63:54]
> as reserved.
>
> With upcoming optional extensions like svpbmt these previously
> reserved bits will get used so simply right-shifting the PTE
> to get the PFN won't be enough.
>
> So introduce a _PAGE_PFN_MASK constant to mask the correct bits
> for both rv32 and rv64 before shifting.
>
> Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> ---
>  arch/riscv/include/asm/pgtable-32.h   |  8 ++++++++
>  arch/riscv/include/asm/pgtable-64.h   | 14 +++++++++++---
>  arch/riscv/include/asm/pgtable-bits.h |  6 ------
>  arch/riscv/include/asm/pgtable.h      |  8 +++++---
>  4 files changed, 24 insertions(+), 12 deletions(-)
>
> diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
> index 5b2e79e5bfa5..e266a4fe7f43 100644
> --- a/arch/riscv/include/asm/pgtable-32.h
> +++ b/arch/riscv/include/asm/pgtable-32.h
> @@ -7,6 +7,7 @@
>  #define _ASM_RISCV_PGTABLE_32_H
>
>  #include <asm-generic/pgtable-nopmd.h>
> +#include <linux/bits.h>
>  #include <linux/const.h>
>
>  /* Size of region mapped by a page global directory */
> @@ -16,4 +17,11 @@
>
>  #define MAX_POSSIBLE_PHYSMEM_BITS 34
>
> +/*
> + * rv32 PTE format:
> + * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> + *       PFN      reserved for SW   D   A   G   U   X   W   R   V
> + */
> +#define _PAGE_PFN_MASK  GENMASK(31, 10)
> +
>  #endif /* _ASM_RISCV_PGTABLE_32_H */
> diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
> index 7e246e9f8d70..15f3ad5aee4f 100644
> --- a/arch/riscv/include/asm/pgtable-64.h
> +++ b/arch/riscv/include/asm/pgtable-64.h
> @@ -6,6 +6,7 @@
>  #ifndef _ASM_RISCV_PGTABLE_64_H
>  #define _ASM_RISCV_PGTABLE_64_H
>
> +#include <linux/bits.h>
>  #include <linux/const.h>
>
>  extern bool pgtable_l4_enabled;
> @@ -65,6 +66,13 @@ typedef struct {
>
>  #define PTRS_PER_PMD    (PAGE_SIZE / sizeof(pmd_t))
>
> +/*
> + * rv64 PTE format:
> + * | 63 | 62 61 | 60 54 | 53  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> + *   N      MT     RSV    PFN      reserved for SW   D   A   G   U   X   W   R   V
> + */
> +#define _PAGE_PFN_MASK  GENMASK(53, 10)
> +
>  static inline int pud_present(pud_t pud)
>  {
>         return (pud_val(pud) & _PAGE_PRESENT);
> @@ -108,12 +116,12 @@ static inline unsigned long _pud_pfn(pud_t pud)
>
>  static inline pmd_t *pud_pgtable(pud_t pud)
>  {
> -       return (pmd_t *)pfn_to_virt(pud_val(pud) >> _PAGE_PFN_SHIFT);
> +       return (pmd_t *)pfn_to_virt(__page_val_to_pfn(pud_val(pud)));
>  }
>
>  static inline struct page *pud_page(pud_t pud)
>  {
> -       return pfn_to_page(pud_val(pud) >> _PAGE_PFN_SHIFT);
> +       return pfn_to_page(__page_val_to_pfn(pud_val(pud)));
>  }
>
>  #define mm_p4d_folded  mm_p4d_folded
> @@ -143,7 +151,7 @@ static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
>
>  static inline unsigned long _pmd_pfn(pmd_t pmd)
>  {
> -       return pmd_val(pmd) >> _PAGE_PFN_SHIFT;
> +       return __page_val_to_pfn(pmd_val(pmd));
>  }
>
>  #define mk_pmd(page, prot)    pfn_pmd(page_to_pfn(page), prot)
> diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
> index a6b0c89824c2..e571fa954afc 100644
> --- a/arch/riscv/include/asm/pgtable-bits.h
> +++ b/arch/riscv/include/asm/pgtable-bits.h
> @@ -6,12 +6,6 @@
>  #ifndef _ASM_RISCV_PGTABLE_BITS_H
>  #define _ASM_RISCV_PGTABLE_BITS_H
>
> -/*
> - * PTE format:
> - * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> - *       PFN      reserved for SW   D   A   G   U   X   W   R   V
> - */
> -
>  #define _PAGE_ACCESSED_OFFSET 6
>
>  #define _PAGE_PRESENT   (1 << 0)
> diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
> index 046b44225623..faba543e2b08 100644
> --- a/arch/riscv/include/asm/pgtable.h
> +++ b/arch/riscv/include/asm/pgtable.h
> @@ -108,6 +108,8 @@
>  #include <asm/tlbflush.h>
>  #include <linux/mm_types.h>
>
> +#define __page_val_to_pfn(_val)  (((_val) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT)
> +
>  #ifdef CONFIG_64BIT
>  #include <asm/pgtable-64.h>
>  #else
> @@ -261,12 +263,12 @@ static inline unsigned long _pgd_pfn(pgd_t pgd)
>
>  static inline struct page *pmd_page(pmd_t pmd)
>  {
> -       return pfn_to_page(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
> +       return pfn_to_page(__page_val_to_pfn(pmd_val(pmd)));
>  }
>
>  static inline unsigned long pmd_page_vaddr(pmd_t pmd)
>  {
> -       return (unsigned long)pfn_to_virt(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
> +       return (unsigned long)pfn_to_virt(__page_val_to_pfn(pmd_val(pmd)));
>  }
>
>  static inline pte_t pmd_pte(pmd_t pmd)
> @@ -282,7 +284,7 @@ static inline pte_t pud_pte(pud_t pud)
>  /* Yields the page frame number (PFN) of a page table entry */
>  static inline unsigned long pte_pfn(pte_t pte)
>  {
> -       return (pte_val(pte) >> _PAGE_PFN_SHIFT);
> +       return __page_val_to_pfn(pte_val(pte));
>  }
>
>  #define pte_page(x)     pfn_to_page(pte_pfn(x))
> --
> 2.35.1
>
>
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv

I had this weird bug and it took some time to figure out that _pgd_pfn
is missing the "& _PAGE_PFN_MASK" here.

And then I grepped _PAGE_PFN_SHIFT and saw that _pud_pfn, p4d_pgtable,
__pmd_to_phys and stage2_pte_page_vaddr are also missing this mask. I
agree that we need something similar to your helper __page_val_to_pfn
(even if __page prefix is a bit weird to me) but I think we should go
further: forbid the use of _PAGE_PFN_SHIFT and contain this in a
single function that should be used to access the pfn minus the upper
bits, that would avoid future oversights.

I can come up with something if needs be, up to you!

Thanks,

Alex

Re: [PATCH 08/12] riscv: Fix accessing pfn bits in PTEs for non-32bit variants

[Heiko Stübner]

Hi Alexandre,

Am Montag, 23. Mai 2022, 16:03:10 CEST schrieb Alexandre Ghiti:
> Hi Heiko,
> 
> On Wed, May 11, 2022 at 9:31 PM Heiko Stuebner <heiko@sntech.de> wrote:
> >
> > On rv32 the PFN part of PTEs is defined to use bits [xlen-1:10]
> > while on rv64 it is defined to use bits [53:10], leaving [63:54]
> > as reserved.
> >
> > With upcoming optional extensions like svpbmt these previously
> > reserved bits will get used so simply right-shifting the PTE
> > to get the PFN won't be enough.
> >
> > So introduce a _PAGE_PFN_MASK constant to mask the correct bits
> > for both rv32 and rv64 before shifting.
> >
> > Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> > Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> > ---
> >  arch/riscv/include/asm/pgtable-32.h   |  8 ++++++++
> >  arch/riscv/include/asm/pgtable-64.h   | 14 +++++++++++---
> >  arch/riscv/include/asm/pgtable-bits.h |  6 ------
> >  arch/riscv/include/asm/pgtable.h      |  8 +++++---
> >  4 files changed, 24 insertions(+), 12 deletions(-)
> >
> > diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
> > index 5b2e79e5bfa5..e266a4fe7f43 100644
> > --- a/arch/riscv/include/asm/pgtable-32.h
> > +++ b/arch/riscv/include/asm/pgtable-32.h
> > @@ -7,6 +7,7 @@
> >  #define _ASM_RISCV_PGTABLE_32_H
> >
> >  #include <asm-generic/pgtable-nopmd.h>
> > +#include <linux/bits.h>
> >  #include <linux/const.h>
> >
> >  /* Size of region mapped by a page global directory */
> > @@ -16,4 +17,11 @@
> >
> >  #define MAX_POSSIBLE_PHYSMEM_BITS 34
> >
> > +/*
> > + * rv32 PTE format:
> > + * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> > + *       PFN      reserved for SW   D   A   G   U   X   W   R   V
> > + */
> > +#define _PAGE_PFN_MASK  GENMASK(31, 10)
> > +
> >  #endif /* _ASM_RISCV_PGTABLE_32_H */
> > diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
> > index 7e246e9f8d70..15f3ad5aee4f 100644
> > --- a/arch/riscv/include/asm/pgtable-64.h
> > +++ b/arch/riscv/include/asm/pgtable-64.h
> > @@ -6,6 +6,7 @@
> >  #ifndef _ASM_RISCV_PGTABLE_64_H
> >  #define _ASM_RISCV_PGTABLE_64_H
> >
> > +#include <linux/bits.h>
> >  #include <linux/const.h>
> >
> >  extern bool pgtable_l4_enabled;
> > @@ -65,6 +66,13 @@ typedef struct {
> >
> >  #define PTRS_PER_PMD    (PAGE_SIZE / sizeof(pmd_t))
> >
> > +/*
> > + * rv64 PTE format:
> > + * | 63 | 62 61 | 60 54 | 53  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> > + *   N      MT     RSV    PFN      reserved for SW   D   A   G   U   X   W   R   V
> > + */
> > +#define _PAGE_PFN_MASK  GENMASK(53, 10)
> > +
> >  static inline int pud_present(pud_t pud)
> >  {
> >         return (pud_val(pud) & _PAGE_PRESENT);
> > @@ -108,12 +116,12 @@ static inline unsigned long _pud_pfn(pud_t pud)
> >
> >  static inline pmd_t *pud_pgtable(pud_t pud)
> >  {
> > -       return (pmd_t *)pfn_to_virt(pud_val(pud) >> _PAGE_PFN_SHIFT);
> > +       return (pmd_t *)pfn_to_virt(__page_val_to_pfn(pud_val(pud)));
> >  }
> >
> >  static inline struct page *pud_page(pud_t pud)
> >  {
> > -       return pfn_to_page(pud_val(pud) >> _PAGE_PFN_SHIFT);
> > +       return pfn_to_page(__page_val_to_pfn(pud_val(pud)));
> >  }
> >
> >  #define mm_p4d_folded  mm_p4d_folded
> > @@ -143,7 +151,7 @@ static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
> >
> >  static inline unsigned long _pmd_pfn(pmd_t pmd)
> >  {
> > -       return pmd_val(pmd) >> _PAGE_PFN_SHIFT;
> > +       return __page_val_to_pfn(pmd_val(pmd));
> >  }
> >
> >  #define mk_pmd(page, prot)    pfn_pmd(page_to_pfn(page), prot)
> > diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
> > index a6b0c89824c2..e571fa954afc 100644
> > --- a/arch/riscv/include/asm/pgtable-bits.h
> > +++ b/arch/riscv/include/asm/pgtable-bits.h
> > @@ -6,12 +6,6 @@
> >  #ifndef _ASM_RISCV_PGTABLE_BITS_H
> >  #define _ASM_RISCV_PGTABLE_BITS_H
> >
> > -/*
> > - * PTE format:
> > - * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> > - *       PFN      reserved for SW   D   A   G   U   X   W   R   V
> > - */
> > -
> >  #define _PAGE_ACCESSED_OFFSET 6
> >
> >  #define _PAGE_PRESENT   (1 << 0)
> > diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
> > index 046b44225623..faba543e2b08 100644
> > --- a/arch/riscv/include/asm/pgtable.h
> > +++ b/arch/riscv/include/asm/pgtable.h
> > @@ -108,6 +108,8 @@
> >  #include <asm/tlbflush.h>
> >  #include <linux/mm_types.h>
> >
> > +#define __page_val_to_pfn(_val)  (((_val) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT)
> > +
> >  #ifdef CONFIG_64BIT
> >  #include <asm/pgtable-64.h>
> >  #else
> > @@ -261,12 +263,12 @@ static inline unsigned long _pgd_pfn(pgd_t pgd)
> >
> >  static inline struct page *pmd_page(pmd_t pmd)
> >  {
> > -       return pfn_to_page(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
> > +       return pfn_to_page(__page_val_to_pfn(pmd_val(pmd)));
> >  }
> >
> >  static inline unsigned long pmd_page_vaddr(pmd_t pmd)
> >  {
> > -       return (unsigned long)pfn_to_virt(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
> > +       return (unsigned long)pfn_to_virt(__page_val_to_pfn(pmd_val(pmd)));
> >  }
> >
> >  static inline pte_t pmd_pte(pmd_t pmd)
> > @@ -282,7 +284,7 @@ static inline pte_t pud_pte(pud_t pud)
> >  /* Yields the page frame number (PFN) of a page table entry */
> >  static inline unsigned long pte_pfn(pte_t pte)
> >  {
> > -       return (pte_val(pte) >> _PAGE_PFN_SHIFT);
> > +       return __page_val_to_pfn(pte_val(pte));
> >  }
> >
> >  #define pte_page(x)     pfn_to_page(pte_pfn(x))
> > --
> > 2.35.1
> >
> >
> > _______________________________________________
> > linux-riscv mailing list
> > linux-riscv@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-riscv
> 
> I had this weird bug and it took some time to figure out that _pgd_pfn
> is missing the "& _PAGE_PFN_MASK" here.
> 
> And then I grepped _PAGE_PFN_SHIFT and saw that _pud_pfn, p4d_pgtable,
> __pmd_to_phys and stage2_pte_page_vaddr are also missing this mask. I
> agree that we need something similar to your helper __page_val_to_pfn
> (even if __page prefix is a bit weird to me) but I think we should go
> further: forbid the use of _PAGE_PFN_SHIFT and contain this in a
> single function that should be used to access the pfn minus the upper
> bits, that would avoid future oversights.
> 
> I can come up with something if needs be, up to you!

as you seem to be stuck in the topic already, you might be faster
than me with that I guess?

Re: [PATCH 08/12] riscv: Fix accessing pfn bits in PTEs for non-32bit variants

[Alexandre Ghiti]

On Wed, May 25, 2022 at 5:22 PM Heiko Stübner <heiko@sntech.de> wrote:
>
> Hi Alexandre,
>
> Am Montag, 23. Mai 2022, 16:03:10 CEST schrieb Alexandre Ghiti:
> > Hi Heiko,
> >
> > On Wed, May 11, 2022 at 9:31 PM Heiko Stuebner <heiko@sntech.de> wrote:
> > >
> > > On rv32 the PFN part of PTEs is defined to use bits [xlen-1:10]
> > > while on rv64 it is defined to use bits [53:10], leaving [63:54]
> > > as reserved.
> > >
> > > With upcoming optional extensions like svpbmt these previously
> > > reserved bits will get used so simply right-shifting the PTE
> > > to get the PFN won't be enough.
> > >
> > > So introduce a _PAGE_PFN_MASK constant to mask the correct bits
> > > for both rv32 and rv64 before shifting.
> > >
> > > Signed-off-by: Heiko Stuebner <heiko@sntech.de>
> > > Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
> > > ---
> > >  arch/riscv/include/asm/pgtable-32.h   |  8 ++++++++
> > >  arch/riscv/include/asm/pgtable-64.h   | 14 +++++++++++---
> > >  arch/riscv/include/asm/pgtable-bits.h |  6 ------
> > >  arch/riscv/include/asm/pgtable.h      |  8 +++++---
> > >  4 files changed, 24 insertions(+), 12 deletions(-)
> > >
> > > diff --git a/arch/riscv/include/asm/pgtable-32.h b/arch/riscv/include/asm/pgtable-32.h
> > > index 5b2e79e5bfa5..e266a4fe7f43 100644
> > > --- a/arch/riscv/include/asm/pgtable-32.h
> > > +++ b/arch/riscv/include/asm/pgtable-32.h
> > > @@ -7,6 +7,7 @@
> > >  #define _ASM_RISCV_PGTABLE_32_H
> > >
> > >  #include <asm-generic/pgtable-nopmd.h>
> > > +#include <linux/bits.h>
> > >  #include <linux/const.h>
> > >
> > >  /* Size of region mapped by a page global directory */
> > > @@ -16,4 +17,11 @@
> > >
> > >  #define MAX_POSSIBLE_PHYSMEM_BITS 34
> > >
> > > +/*
> > > + * rv32 PTE format:
> > > + * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> > > + *       PFN      reserved for SW   D   A   G   U   X   W   R   V
> > > + */
> > > +#define _PAGE_PFN_MASK  GENMASK(31, 10)
> > > +
> > >  #endif /* _ASM_RISCV_PGTABLE_32_H */
> > > diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h
> > > index 7e246e9f8d70..15f3ad5aee4f 100644
> > > --- a/arch/riscv/include/asm/pgtable-64.h
> > > +++ b/arch/riscv/include/asm/pgtable-64.h
> > > @@ -6,6 +6,7 @@
> > >  #ifndef _ASM_RISCV_PGTABLE_64_H
> > >  #define _ASM_RISCV_PGTABLE_64_H
> > >
> > > +#include <linux/bits.h>
> > >  #include <linux/const.h>
> > >
> > >  extern bool pgtable_l4_enabled;
> > > @@ -65,6 +66,13 @@ typedef struct {
> > >
> > >  #define PTRS_PER_PMD    (PAGE_SIZE / sizeof(pmd_t))
> > >
> > > +/*
> > > + * rv64 PTE format:
> > > + * | 63 | 62 61 | 60 54 | 53  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> > > + *   N      MT     RSV    PFN      reserved for SW   D   A   G   U   X   W   R   V
> > > + */
> > > +#define _PAGE_PFN_MASK  GENMASK(53, 10)
> > > +
> > >  static inline int pud_present(pud_t pud)
> > >  {
> > >         return (pud_val(pud) & _PAGE_PRESENT);
> > > @@ -108,12 +116,12 @@ static inline unsigned long _pud_pfn(pud_t pud)
> > >
> > >  static inline pmd_t *pud_pgtable(pud_t pud)
> > >  {
> > > -       return (pmd_t *)pfn_to_virt(pud_val(pud) >> _PAGE_PFN_SHIFT);
> > > +       return (pmd_t *)pfn_to_virt(__page_val_to_pfn(pud_val(pud)));
> > >  }
> > >
> > >  static inline struct page *pud_page(pud_t pud)
> > >  {
> > > -       return pfn_to_page(pud_val(pud) >> _PAGE_PFN_SHIFT);
> > > +       return pfn_to_page(__page_val_to_pfn(pud_val(pud)));
> > >  }
> > >
> > >  #define mm_p4d_folded  mm_p4d_folded
> > > @@ -143,7 +151,7 @@ static inline pmd_t pfn_pmd(unsigned long pfn, pgprot_t prot)
> > >
> > >  static inline unsigned long _pmd_pfn(pmd_t pmd)
> > >  {
> > > -       return pmd_val(pmd) >> _PAGE_PFN_SHIFT;
> > > +       return __page_val_to_pfn(pmd_val(pmd));
> > >  }
> > >
> > >  #define mk_pmd(page, prot)    pfn_pmd(page_to_pfn(page), prot)
> > > diff --git a/arch/riscv/include/asm/pgtable-bits.h b/arch/riscv/include/asm/pgtable-bits.h
> > > index a6b0c89824c2..e571fa954afc 100644
> > > --- a/arch/riscv/include/asm/pgtable-bits.h
> > > +++ b/arch/riscv/include/asm/pgtable-bits.h
> > > @@ -6,12 +6,6 @@
> > >  #ifndef _ASM_RISCV_PGTABLE_BITS_H
> > >  #define _ASM_RISCV_PGTABLE_BITS_H
> > >
> > > -/*
> > > - * PTE format:
> > > - * | XLEN-1  10 | 9             8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
> > > - *       PFN      reserved for SW   D   A   G   U   X   W   R   V
> > > - */
> > > -
> > >  #define _PAGE_ACCESSED_OFFSET 6
> > >
> > >  #define _PAGE_PRESENT   (1 << 0)
> > > diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
> > > index 046b44225623..faba543e2b08 100644
> > > --- a/arch/riscv/include/asm/pgtable.h
> > > +++ b/arch/riscv/include/asm/pgtable.h
> > > @@ -108,6 +108,8 @@
> > >  #include <asm/tlbflush.h>
> > >  #include <linux/mm_types.h>
> > >
> > > +#define __page_val_to_pfn(_val)  (((_val) & _PAGE_PFN_MASK) >> _PAGE_PFN_SHIFT)
> > > +
> > >  #ifdef CONFIG_64BIT
> > >  #include <asm/pgtable-64.h>
> > >  #else
> > > @@ -261,12 +263,12 @@ static inline unsigned long _pgd_pfn(pgd_t pgd)
> > >
> > >  static inline struct page *pmd_page(pmd_t pmd)
> > >  {
> > > -       return pfn_to_page(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
> > > +       return pfn_to_page(__page_val_to_pfn(pmd_val(pmd)));
> > >  }
> > >
> > >  static inline unsigned long pmd_page_vaddr(pmd_t pmd)
> > >  {
> > > -       return (unsigned long)pfn_to_virt(pmd_val(pmd) >> _PAGE_PFN_SHIFT);
> > > +       return (unsigned long)pfn_to_virt(__page_val_to_pfn(pmd_val(pmd)));
> > >  }
> > >
> > >  static inline pte_t pmd_pte(pmd_t pmd)
> > > @@ -282,7 +284,7 @@ static inline pte_t pud_pte(pud_t pud)
> > >  /* Yields the page frame number (PFN) of a page table entry */
> > >  static inline unsigned long pte_pfn(pte_t pte)
> > >  {
> > > -       return (pte_val(pte) >> _PAGE_PFN_SHIFT);
> > > +       return __page_val_to_pfn(pte_val(pte));
> > >  }
> > >
> > >  #define pte_page(x)     pfn_to_page(pte_pfn(x))
> > > --
> > > 2.35.1
> > >
> > >
> > > _______________________________________________
> > > linux-riscv mailing list
> > > linux-riscv@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-riscv
> >
> > I had this weird bug and it took some time to figure out that _pgd_pfn
> > is missing the "& _PAGE_PFN_MASK" here.
> >
> > And then I grepped _PAGE_PFN_SHIFT and saw that _pud_pfn, p4d_pgtable,
> > __pmd_to_phys and stage2_pte_page_vaddr are also missing this mask. I
> > agree that we need something similar to your helper __page_val_to_pfn
> > (even if __page prefix is a bit weird to me) but I think we should go
> > further: forbid the use of _PAGE_PFN_SHIFT and contain this in a
> > single function that should be used to access the pfn minus the upper
> > bits, that would avoid future oversights.
> >
> > I can come up with something if needs be, up to you!
>
> as you seem to be stuck in the topic already, you might be faster
> than me with that I guess?

I'll propose the quick fix next week and will think about the proper
solution later.

Thanks

>
>