[PATCH v3 0/6] introduce mirrored memory support for arm64

[PATCH v3 0/6] introduce mirrored memory support for arm64

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
introduced mirrored memory support for x86. This support rely on UEFI to
report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:

  http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf

Memory mirroring is a technique used to separate memory into two separate
channels, usually on a memory device, like a server. In memory mirroring,
one channel is copied to another to create redundancy. This method makes
input/output (I/O) registers and memory appear with more than one address
range because the same physical byte is accessible at more than one
address. Using memory mirroring, higher memory reliability and a higher
level of memory consolidation are possible.

These EFI memory regions have various attributes, and the "mirrored"
attribute is one of them. The physical memory region whose descriptors
in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
mirrored. The address range mirroring feature of the kernel arranges such
mirrored regions into normal zones and other regions into movable zones.

Arm64 can support this too. So mirrored memory support is added to support
arm64.

The main purpose of this patch set is to introduce mirrored support for
arm64 and we have already fixed the problems we had which is shown in
patch #5 to patch #8 and try to bring total isolation in patch #9 which
will disable mirror feature if kernelcore is not specified.

In order to test this support in arm64:
- patch this patch set
- add kernelcore=mirror in kernel parameter
- start you kernel

Patch #1-#2 introduce mirrored memory support form arm64.
Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
Patch #6 disable mirror feature if kernelcore is not specified.

Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
memory if kaslr is enabled.

[1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/

Changelog since v2:
- remove efi_fake_mem support
- remove Commit ("remove some redundant code in ia64 efi_init") since
  efi_print_memmap() is not public
- add mirror flag back on initrd memory

Changelog since v1:
- update changelog in cover letter
- use PHYS_PFN in patch #7

Ma Wupeng (6):
  efi: Make efi_find_mirror() public
  arm64/mirror: arm64 enabling - find mirrored memory ranges
  mm: Ratelimited mirrored memory related warning messages
  mm: Demote warning message in vmemmap_verify() to debug level
  mm: Add mirror flag back on initrd memory
  efi: Disable mirror feature if kernelcore is not specified

 .../admin-guide/kernel-parameters.txt         |  2 +-
 arch/arm64/kernel/setup.c                     |  1 +
 arch/arm64/mm/init.c                          |  9 +++++++
 arch/x86/include/asm/efi.h                    |  4 ---
 arch/x86/platform/efi/efi.c                   | 23 ----------------
 drivers/firmware/efi/efi.c                    | 26 +++++++++++++++++++
 include/linux/efi.h                           |  3 +++
 include/linux/memblock.h                      |  1 +
 include/linux/mm.h                            |  2 ++
 mm/memblock.c                                 | 24 +++++++++++++++--
 mm/page_alloc.c                               |  2 +-
 mm/sparse-vmemmap.c                           |  2 +-
 12 files changed, 67 insertions(+), 32 deletions(-)

-- 
2.25.1

[PATCH v3 0/6] introduce mirrored memory support for arm64

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
introduced mirrored memory support for x86. This support rely on UEFI to
report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:

  http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf

Memory mirroring is a technique used to separate memory into two separate
channels, usually on a memory device, like a server. In memory mirroring,
one channel is copied to another to create redundancy. This method makes
input/output (I/O) registers and memory appear with more than one address
range because the same physical byte is accessible at more than one
address. Using memory mirroring, higher memory reliability and a higher
level of memory consolidation are possible.

These EFI memory regions have various attributes, and the "mirrored"
attribute is one of them. The physical memory region whose descriptors
in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
mirrored. The address range mirroring feature of the kernel arranges such
mirrored regions into normal zones and other regions into movable zones.

Arm64 can support this too. So mirrored memory support is added to support
arm64.

The main purpose of this patch set is to introduce mirrored support for
arm64 and we have already fixed the problems we had which is shown in
patch #5 to patch #8 and try to bring total isolation in patch #9 which
will disable mirror feature if kernelcore is not specified.

In order to test this support in arm64:
- patch this patch set
- add kernelcore=mirror in kernel parameter
- start you kernel

Patch #1-#2 introduce mirrored memory support form arm64.
Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
Patch #6 disable mirror feature if kernelcore is not specified.

Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
memory if kaslr is enabled.

[1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/

Changelog since v2:
- remove efi_fake_mem support
- remove Commit ("remove some redundant code in ia64 efi_init") since
  efi_print_memmap() is not public
- add mirror flag back on initrd memory

Changelog since v1:
- update changelog in cover letter
- use PHYS_PFN in patch #7

Ma Wupeng (6):
  efi: Make efi_find_mirror() public
  arm64/mirror: arm64 enabling - find mirrored memory ranges
  mm: Ratelimited mirrored memory related warning messages
  mm: Demote warning message in vmemmap_verify() to debug level
  mm: Add mirror flag back on initrd memory
  efi: Disable mirror feature if kernelcore is not specified

 .../admin-guide/kernel-parameters.txt         |  2 +-
 arch/arm64/kernel/setup.c                     |  1 +
 arch/arm64/mm/init.c                          |  9 +++++++
 arch/x86/include/asm/efi.h                    |  4 ---
 arch/x86/platform/efi/efi.c                   | 23 ----------------
 drivers/firmware/efi/efi.c                    | 26 +++++++++++++++++++
 include/linux/efi.h                           |  3 +++
 include/linux/memblock.h                      |  1 +
 include/linux/mm.h                            |  2 ++
 mm/memblock.c                                 | 24 +++++++++++++++--
 mm/page_alloc.c                               |  2 +-
 mm/sparse-vmemmap.c                           |  2 +-
 12 files changed, 67 insertions(+), 32 deletions(-)

-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v3 0/6] introduce mirrored memory support for arm64

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
introduced mirrored memory support for x86. This support rely on UEFI to
report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:

  http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf

Memory mirroring is a technique used to separate memory into two separate
channels, usually on a memory device, like a server. In memory mirroring,
one channel is copied to another to create redundancy. This method makes
input/output (I/O) registers and memory appear with more than one address
range because the same physical byte is accessible at more than one
address. Using memory mirroring, higher memory reliability and a higher
level of memory consolidation are possible.

These EFI memory regions have various attributes, and the "mirrored"
attribute is one of them. The physical memory region whose descriptors
in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
mirrored. The address range mirroring feature of the kernel arranges such
mirrored regions into normal zones and other regions into movable zones.

Arm64 can support this too. So mirrored memory support is added to support
arm64.

The main purpose of this patch set is to introduce mirrored support for
arm64 and we have already fixed the problems we had which is shown in
patch #5 to patch #8 and try to bring total isolation in patch #9 which
will disable mirror feature if kernelcore is not specified.

In order to test this support in arm64:
- patch this patch set
- add kernelcore=mirror in kernel parameter
- start you kernel

Patch #1-#2 introduce mirrored memory support form arm64.
Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
Patch #6 disable mirror feature if kernelcore is not specified.

Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
memory if kaslr is enabled.

[1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/

Changelog since v2:
- remove efi_fake_mem support
- remove Commit ("remove some redundant code in ia64 efi_init") since
  efi_print_memmap() is not public
- add mirror flag back on initrd memory

Changelog since v1:
- update changelog in cover letter
- use PHYS_PFN in patch #7

Ma Wupeng (6):
  efi: Make efi_find_mirror() public
  arm64/mirror: arm64 enabling - find mirrored memory ranges
  mm: Ratelimited mirrored memory related warning messages
  mm: Demote warning message in vmemmap_verify() to debug level
  mm: Add mirror flag back on initrd memory
  efi: Disable mirror feature if kernelcore is not specified

 .../admin-guide/kernel-parameters.txt         |  2 +-
 arch/arm64/kernel/setup.c                     |  1 +
 arch/arm64/mm/init.c                          |  9 +++++++
 arch/x86/include/asm/efi.h                    |  4 ---
 arch/x86/platform/efi/efi.c                   | 23 ----------------
 drivers/firmware/efi/efi.c                    | 26 +++++++++++++++++++
 include/linux/efi.h                           |  3 +++
 include/linux/memblock.h                      |  1 +
 include/linux/mm.h                            |  2 ++
 mm/memblock.c                                 | 24 +++++++++++++++--
 mm/page_alloc.c                               |  2 +-
 mm/sparse-vmemmap.c                           |  2 +-
 12 files changed, 67 insertions(+), 32 deletions(-)

-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v3 1/6] efi: Make efi_find_mirror() public

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory
ranges") introduce the efi_find_mirror function on x86. In order to reuse
the API we make it public in preparation for arm64 to support mirrord
memory.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 arch/x86/include/asm/efi.h  |  4 ----
 arch/x86/platform/efi/efi.c | 23 -----------------------
 drivers/firmware/efi/efi.c  | 23 +++++++++++++++++++++++
 include/linux/efi.h         |  3 +++
 4 files changed, 26 insertions(+), 27 deletions(-)

diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
index 71943dce691e..eb90206eae80 100644
--- a/arch/x86/include/asm/efi.h
+++ b/arch/x86/include/asm/efi.h
@@ -383,7 +383,6 @@ static inline bool efi_is_64bit(void)
 extern bool efi_reboot_required(void);
 extern bool efi_is_table_address(unsigned long phys_addr);
 
-extern void efi_find_mirror(void);
 extern void efi_reserve_boot_services(void);
 #else
 static inline void parse_efi_setup(u64 phys_addr, u32 data_len) {}
@@ -395,9 +394,6 @@ static inline  bool efi_is_table_address(unsigned long phys_addr)
 {
 	return false;
 }
-static inline void efi_find_mirror(void)
-{
-}
 static inline void efi_reserve_boot_services(void)
 {
 }
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index 1591d67e0bcd..6e598bd78eef 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -108,29 +108,6 @@ static int __init setup_add_efi_memmap(char *arg)
 }
 early_param("add_efi_memmap", setup_add_efi_memmap);
 
-void __init efi_find_mirror(void)
-{
-	efi_memory_desc_t *md;
-	u64 mirror_size = 0, total_size = 0;
-
-	if (!efi_enabled(EFI_MEMMAP))
-		return;
-
-	for_each_efi_memory_desc(md) {
-		unsigned long long start = md->phys_addr;
-		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
-
-		total_size += size;
-		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
-			memblock_mark_mirror(start, size);
-			mirror_size += size;
-		}
-	}
-	if (mirror_size)
-		pr_info("Memory: %lldM/%lldM mirrored memory\n",
-			mirror_size>>20, total_size>>20);
-}
-
 /*
  * Tell the kernel about the EFI memory map.  This might include
  * more than the max 128 entries that can fit in the passed in e820
diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 860534bcfdac..79c232e07de7 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -446,6 +446,29 @@ static int __init efisubsys_init(void)
 
 subsys_initcall(efisubsys_init);
 
+void __init efi_find_mirror(void)
+{
+	efi_memory_desc_t *md;
+	u64 mirror_size = 0, total_size = 0;
+
+	if (!efi_enabled(EFI_MEMMAP))
+		return;
+
+	for_each_efi_memory_desc(md) {
+		unsigned long long start = md->phys_addr;
+		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
+
+		total_size += size;
+		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
+			memblock_mark_mirror(start, size);
+			mirror_size += size;
+		}
+	}
+	if (mirror_size)
+		pr_info("Memory: %lldM/%lldM mirrored memory\n",
+			mirror_size>>20, total_size>>20);
+}
+
 /*
  * Find the efi memory descriptor for a given physical address.  Given a
  * physical address, determine if it exists within an EFI Memory Map entry,
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 7d9b0bb47eb3..53f64c14a525 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -872,6 +872,7 @@ static inline bool efi_rt_services_supported(unsigned int mask)
 {
 	return (efi.runtime_supported_mask & mask) == mask;
 }
+extern void efi_find_mirror(void);
 #else
 static inline bool efi_enabled(int feature)
 {
@@ -889,6 +890,8 @@ static inline bool efi_rt_services_supported(unsigned int mask)
 {
 	return false;
 }
+
+static inline void efi_find_mirror(void) {}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);
-- 
2.25.1

[PATCH v3 1/6] efi: Make efi_find_mirror() public

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory
ranges") introduce the efi_find_mirror function on x86. In order to reuse
the API we make it public in preparation for arm64 to support mirrord
memory.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 arch/x86/include/asm/efi.h  |  4 ----
 arch/x86/platform/efi/efi.c | 23 -----------------------
 drivers/firmware/efi/efi.c  | 23 +++++++++++++++++++++++
 include/linux/efi.h         |  3 +++
 4 files changed, 26 insertions(+), 27 deletions(-)

diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
index 71943dce691e..eb90206eae80 100644
--- a/arch/x86/include/asm/efi.h
+++ b/arch/x86/include/asm/efi.h
@@ -383,7 +383,6 @@ static inline bool efi_is_64bit(void)
 extern bool efi_reboot_required(void);
 extern bool efi_is_table_address(unsigned long phys_addr);
 
-extern void efi_find_mirror(void);
 extern void efi_reserve_boot_services(void);
 #else
 static inline void parse_efi_setup(u64 phys_addr, u32 data_len) {}
@@ -395,9 +394,6 @@ static inline  bool efi_is_table_address(unsigned long phys_addr)
 {
 	return false;
 }
-static inline void efi_find_mirror(void)
-{
-}
 static inline void efi_reserve_boot_services(void)
 {
 }
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index 1591d67e0bcd..6e598bd78eef 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -108,29 +108,6 @@ static int __init setup_add_efi_memmap(char *arg)
 }
 early_param("add_efi_memmap", setup_add_efi_memmap);
 
-void __init efi_find_mirror(void)
-{
-	efi_memory_desc_t *md;
-	u64 mirror_size = 0, total_size = 0;
-
-	if (!efi_enabled(EFI_MEMMAP))
-		return;
-
-	for_each_efi_memory_desc(md) {
-		unsigned long long start = md->phys_addr;
-		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
-
-		total_size += size;
-		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
-			memblock_mark_mirror(start, size);
-			mirror_size += size;
-		}
-	}
-	if (mirror_size)
-		pr_info("Memory: %lldM/%lldM mirrored memory\n",
-			mirror_size>>20, total_size>>20);
-}
-
 /*
  * Tell the kernel about the EFI memory map.  This might include
  * more than the max 128 entries that can fit in the passed in e820
diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 860534bcfdac..79c232e07de7 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -446,6 +446,29 @@ static int __init efisubsys_init(void)
 
 subsys_initcall(efisubsys_init);
 
+void __init efi_find_mirror(void)
+{
+	efi_memory_desc_t *md;
+	u64 mirror_size = 0, total_size = 0;
+
+	if (!efi_enabled(EFI_MEMMAP))
+		return;
+
+	for_each_efi_memory_desc(md) {
+		unsigned long long start = md->phys_addr;
+		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
+
+		total_size += size;
+		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
+			memblock_mark_mirror(start, size);
+			mirror_size += size;
+		}
+	}
+	if (mirror_size)
+		pr_info("Memory: %lldM/%lldM mirrored memory\n",
+			mirror_size>>20, total_size>>20);
+}
+
 /*
  * Find the efi memory descriptor for a given physical address.  Given a
  * physical address, determine if it exists within an EFI Memory Map entry,
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 7d9b0bb47eb3..53f64c14a525 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -872,6 +872,7 @@ static inline bool efi_rt_services_supported(unsigned int mask)
 {
 	return (efi.runtime_supported_mask & mask) == mask;
 }
+extern void efi_find_mirror(void);
 #else
 static inline bool efi_enabled(int feature)
 {
@@ -889,6 +890,8 @@ static inline bool efi_rt_services_supported(unsigned int mask)
 {
 	return false;
 }
+
+static inline void efi_find_mirror(void) {}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);
-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v3 1/6] efi: Make efi_find_mirror() public

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory
ranges") introduce the efi_find_mirror function on x86. In order to reuse
the API we make it public in preparation for arm64 to support mirrord
memory.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 arch/x86/include/asm/efi.h  |  4 ----
 arch/x86/platform/efi/efi.c | 23 -----------------------
 drivers/firmware/efi/efi.c  | 23 +++++++++++++++++++++++
 include/linux/efi.h         |  3 +++
 4 files changed, 26 insertions(+), 27 deletions(-)

diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
index 71943dce691e..eb90206eae80 100644
--- a/arch/x86/include/asm/efi.h
+++ b/arch/x86/include/asm/efi.h
@@ -383,7 +383,6 @@ static inline bool efi_is_64bit(void)
 extern bool efi_reboot_required(void);
 extern bool efi_is_table_address(unsigned long phys_addr);
 
-extern void efi_find_mirror(void);
 extern void efi_reserve_boot_services(void);
 #else
 static inline void parse_efi_setup(u64 phys_addr, u32 data_len) {}
@@ -395,9 +394,6 @@ static inline  bool efi_is_table_address(unsigned long phys_addr)
 {
 	return false;
 }
-static inline void efi_find_mirror(void)
-{
-}
 static inline void efi_reserve_boot_services(void)
 {
 }
diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
index 1591d67e0bcd..6e598bd78eef 100644
--- a/arch/x86/platform/efi/efi.c
+++ b/arch/x86/platform/efi/efi.c
@@ -108,29 +108,6 @@ static int __init setup_add_efi_memmap(char *arg)
 }
 early_param("add_efi_memmap", setup_add_efi_memmap);
 
-void __init efi_find_mirror(void)
-{
-	efi_memory_desc_t *md;
-	u64 mirror_size = 0, total_size = 0;
-
-	if (!efi_enabled(EFI_MEMMAP))
-		return;
-
-	for_each_efi_memory_desc(md) {
-		unsigned long long start = md->phys_addr;
-		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
-
-		total_size += size;
-		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
-			memblock_mark_mirror(start, size);
-			mirror_size += size;
-		}
-	}
-	if (mirror_size)
-		pr_info("Memory: %lldM/%lldM mirrored memory\n",
-			mirror_size>>20, total_size>>20);
-}
-
 /*
  * Tell the kernel about the EFI memory map.  This might include
  * more than the max 128 entries that can fit in the passed in e820
diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 860534bcfdac..79c232e07de7 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -446,6 +446,29 @@ static int __init efisubsys_init(void)
 
 subsys_initcall(efisubsys_init);
 
+void __init efi_find_mirror(void)
+{
+	efi_memory_desc_t *md;
+	u64 mirror_size = 0, total_size = 0;
+
+	if (!efi_enabled(EFI_MEMMAP))
+		return;
+
+	for_each_efi_memory_desc(md) {
+		unsigned long long start = md->phys_addr;
+		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
+
+		total_size += size;
+		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
+			memblock_mark_mirror(start, size);
+			mirror_size += size;
+		}
+	}
+	if (mirror_size)
+		pr_info("Memory: %lldM/%lldM mirrored memory\n",
+			mirror_size>>20, total_size>>20);
+}
+
 /*
  * Find the efi memory descriptor for a given physical address.  Given a
  * physical address, determine if it exists within an EFI Memory Map entry,
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 7d9b0bb47eb3..53f64c14a525 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -872,6 +872,7 @@ static inline bool efi_rt_services_supported(unsigned int mask)
 {
 	return (efi.runtime_supported_mask & mask) == mask;
 }
+extern void efi_find_mirror(void);
 #else
 static inline bool efi_enabled(int feature)
 {
@@ -889,6 +890,8 @@ static inline bool efi_rt_services_supported(unsigned int mask)
 {
 	return false;
 }
+
+static inline void efi_find_mirror(void) {}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
introduced mirrored memory support for x86 and this could be used on arm64.

Since we only support this feature on arm64, efi_find_mirror() won't be placed
into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
to scan the memory map and mark mirrored memory in memblock.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 2 +-
 arch/arm64/kernel/setup.c                       | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 8090130b544b..e3537646b6f7 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2301,7 +2301,7 @@
 
 	keepinitrd	[HW,ARM]
 
-	kernelcore=	[KNL,X86,IA-64,PPC]
+	kernelcore=	[KNL,X86,IA-64,PPC,ARM64]
 			Format: nn[KMGTPE] | nn% | "mirror"
 			This parameter specifies the amount of memory usable by
 			the kernel for non-movable allocations.  The requested
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index cf3a759f10d4..6e9acd7ecf0f 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
 
 	xen_early_init();
 	efi_init();
+	efi_find_mirror();
 
 	if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
 	     pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");
-- 
2.25.1

[PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
introduced mirrored memory support for x86 and this could be used on arm64.

Since we only support this feature on arm64, efi_find_mirror() won't be placed
into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
to scan the memory map and mark mirrored memory in memblock.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 2 +-
 arch/arm64/kernel/setup.c                       | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 8090130b544b..e3537646b6f7 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2301,7 +2301,7 @@
 
 	keepinitrd	[HW,ARM]
 
-	kernelcore=	[KNL,X86,IA-64,PPC]
+	kernelcore=	[KNL,X86,IA-64,PPC,ARM64]
 			Format: nn[KMGTPE] | nn% | "mirror"
 			This parameter specifies the amount of memory usable by
 			the kernel for non-movable allocations.  The requested
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index cf3a759f10d4..6e9acd7ecf0f 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
 
 	xen_early_init();
 	efi_init();
+	efi_find_mirror();
 
 	if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
 	     pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");
-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
introduced mirrored memory support for x86 and this could be used on arm64.

Since we only support this feature on arm64, efi_find_mirror() won't be placed
into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
to scan the memory map and mark mirrored memory in memblock.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 2 +-
 arch/arm64/kernel/setup.c                       | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 8090130b544b..e3537646b6f7 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2301,7 +2301,7 @@
 
 	keepinitrd	[HW,ARM]
 
-	kernelcore=	[KNL,X86,IA-64,PPC]
+	kernelcore=	[KNL,X86,IA-64,PPC,ARM64]
 			Format: nn[KMGTPE] | nn% | "mirror"
 			This parameter specifies the amount of memory usable by
 			the kernel for non-movable allocations.  The requested
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index cf3a759f10d4..6e9acd7ecf0f 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
 
 	xen_early_init();
 	efi_init();
+	efi_find_mirror();
 
 	if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
 	     pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

If system has mirrored memory, memblock will try to allocate mirrored
memory firstly and fallback to non-mirrored memory when fails, but if with
limited mirrored memory or some numa node without mirrored memory, lots of
warning message about memblock allocation will occur.

This patch ratelimit the warning message to avoid a very long print during
bootup.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 mm/memblock.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/memblock.c b/mm/memblock.c
index e4f03a6e8e56..b1d2a0009733 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
 					    NUMA_NO_NODE, flags);
 
 	if (!ret && (flags & MEMBLOCK_MIRROR)) {
-		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
+		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
 			&size);
 		flags &= ~MEMBLOCK_MIRROR;
 		goto again;
@@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
 
 	if (flags & MEMBLOCK_MIRROR) {
 		flags &= ~MEMBLOCK_MIRROR;
-		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
+		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
 			&size);
 		goto again;
 	}
-- 
2.25.1

[PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

If system has mirrored memory, memblock will try to allocate mirrored
memory firstly and fallback to non-mirrored memory when fails, but if with
limited mirrored memory or some numa node without mirrored memory, lots of
warning message about memblock allocation will occur.

This patch ratelimit the warning message to avoid a very long print during
bootup.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 mm/memblock.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/memblock.c b/mm/memblock.c
index e4f03a6e8e56..b1d2a0009733 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
 					    NUMA_NO_NODE, flags);
 
 	if (!ret && (flags & MEMBLOCK_MIRROR)) {
-		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
+		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
 			&size);
 		flags &= ~MEMBLOCK_MIRROR;
 		goto again;
@@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
 
 	if (flags & MEMBLOCK_MIRROR) {
 		flags &= ~MEMBLOCK_MIRROR;
-		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
+		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
 			&size);
 		goto again;
 	}
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

If system has mirrored memory, memblock will try to allocate mirrored
memory firstly and fallback to non-mirrored memory when fails, but if with
limited mirrored memory or some numa node without mirrored memory, lots of
warning message about memblock allocation will occur.

This patch ratelimit the warning message to avoid a very long print during
bootup.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 mm/memblock.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/memblock.c b/mm/memblock.c
index e4f03a6e8e56..b1d2a0009733 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
 					    NUMA_NO_NODE, flags);
 
 	if (!ret && (flags & MEMBLOCK_MIRROR)) {
-		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
+		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
 			&size);
 		flags &= ~MEMBLOCK_MIRROR;
 		goto again;
@@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
 
 	if (flags & MEMBLOCK_MIRROR) {
 		flags &= ~MEMBLOCK_MIRROR;
-		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
+		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
 			&size);
 		goto again;
 	}
-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

For a system only have limited mirrored memory or some numa node without
mirrored memory, the per node vmemmap page_structs prefer to allocate
memory from mirrored region, which will lead to vmemmap_verify() in
vmemmap_populate_basepages() report lots of warning message.

This patch demote the "potential offnode page_structs" warning messages
to debug level to avoid a very long print during bootup.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 mm/sparse-vmemmap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
index f4fa61dbbee3..78debdb89eb1 100644
--- a/mm/sparse-vmemmap.c
+++ b/mm/sparse-vmemmap.c
@@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
 	int actual_node = early_pfn_to_nid(pfn);
 
 	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
-		pr_warn("[%lx-%lx] potential offnode page_structs\n",
+		pr_debug("[%lx-%lx] potential offnode page_structs\n",
 			start, end - 1);
 }
 
-- 
2.25.1

[PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

For a system only have limited mirrored memory or some numa node without
mirrored memory, the per node vmemmap page_structs prefer to allocate
memory from mirrored region, which will lead to vmemmap_verify() in
vmemmap_populate_basepages() report lots of warning message.

This patch demote the "potential offnode page_structs" warning messages
to debug level to avoid a very long print during bootup.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 mm/sparse-vmemmap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
index f4fa61dbbee3..78debdb89eb1 100644
--- a/mm/sparse-vmemmap.c
+++ b/mm/sparse-vmemmap.c
@@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
 	int actual_node = early_pfn_to_nid(pfn);
 
 	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
-		pr_warn("[%lx-%lx] potential offnode page_structs\n",
+		pr_debug("[%lx-%lx] potential offnode page_structs\n",
 			start, end - 1);
 }
 
-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

For a system only have limited mirrored memory or some numa node without
mirrored memory, the per node vmemmap page_structs prefer to allocate
memory from mirrored region, which will lead to vmemmap_verify() in
vmemmap_populate_basepages() report lots of warning message.

This patch demote the "potential offnode page_structs" warning messages
to debug level to avoid a very long print during bootup.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 mm/sparse-vmemmap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
index f4fa61dbbee3..78debdb89eb1 100644
--- a/mm/sparse-vmemmap.c
+++ b/mm/sparse-vmemmap.c
@@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
 	int actual_node = early_pfn_to_nid(pfn);
 
 	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
-		pr_warn("[%lx-%lx] potential offnode page_structs\n",
+		pr_debug("[%lx-%lx] potential offnode page_structs\n",
 			start, end - 1);
 }
 
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Initrd memory will be removed and then added in arm64_memblock_init() and this
will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
the lower 4G range has some non-mirrored memory.

In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
reinstalled if the origin memblock has this flag.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 arch/arm64/mm/init.c     |  9 +++++++++
 include/linux/memblock.h |  1 +
 mm/memblock.c            | 20 ++++++++++++++++++++
 3 files changed, 30 insertions(+)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 339ee84e5a61..11641f924d08 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
 			"initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
 			phys_initrd_size = 0;
 		} else {
+			int flags, ret;
+
+			ret = memblock_get_flags(base, &flags);
+			if (ret)
+				flags = 0;
+
 			memblock_remove(base, size); /* clear MEMBLOCK_ flags */
 			memblock_add(base, size);
 			memblock_reserve(base, size);
+
+			if (flags & MEMBLOCK_MIRROR)
+				memblock_mark_mirror(base, size);
 		}
 	}
 
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index 50ad19662a32..3d6a382ac9c8 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -487,6 +487,7 @@ bool memblock_is_map_memory(phys_addr_t addr);
 bool memblock_is_region_memory(phys_addr_t base, phys_addr_t size);
 bool memblock_is_reserved(phys_addr_t addr);
 bool memblock_is_region_reserved(phys_addr_t base, phys_addr_t size);
+int memblock_get_flags(phys_addr_t base, int *flags);
 
 void memblock_dump_all(void);
 
diff --git a/mm/memblock.c b/mm/memblock.c
index b1d2a0009733..0c5b5699af6e 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -1796,6 +1796,26 @@ int __init_memblock memblock_search_pfn_nid(unsigned long pfn,
 	return memblock_get_region_node(&type->regions[mid]);
 }
 
+/**
+ * memblock_get_flags - get a single memblock flags
+ * @base: base of memeblock to get
+ *
+ * Get the flags of memeblock with base: @base
+ *
+ * Return:
+ * 0 if ok, non-zero if fail
+ */
+int __init_memblock memblock_get_flags(phys_addr_t base, int *flags)
+{
+	int idx = memblock_search(&memblock.memory, base);
+
+	if (idx == -1)
+		return -EINVAL;
+
+	*flags = memblock.memory.regions[idx].flags;
+	return 0;
+}
+
 /**
  * memblock_is_region_memory - check if a region is a subset of memory
  * @base: base of region to check
-- 
2.25.1

[PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Initrd memory will be removed and then added in arm64_memblock_init() and this
will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
the lower 4G range has some non-mirrored memory.

In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
reinstalled if the origin memblock has this flag.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 arch/arm64/mm/init.c     |  9 +++++++++
 include/linux/memblock.h |  1 +
 mm/memblock.c            | 20 ++++++++++++++++++++
 3 files changed, 30 insertions(+)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 339ee84e5a61..11641f924d08 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
 			"initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
 			phys_initrd_size = 0;
 		} else {
+			int flags, ret;
+
+			ret = memblock_get_flags(base, &flags);
+			if (ret)
+				flags = 0;
+
 			memblock_remove(base, size); /* clear MEMBLOCK_ flags */
 			memblock_add(base, size);
 			memblock_reserve(base, size);
+
+			if (flags & MEMBLOCK_MIRROR)
+				memblock_mark_mirror(base, size);
 		}
 	}
 
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index 50ad19662a32..3d6a382ac9c8 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -487,6 +487,7 @@ bool memblock_is_map_memory(phys_addr_t addr);
 bool memblock_is_region_memory(phys_addr_t base, phys_addr_t size);
 bool memblock_is_reserved(phys_addr_t addr);
 bool memblock_is_region_reserved(phys_addr_t base, phys_addr_t size);
+int memblock_get_flags(phys_addr_t base, int *flags);
 
 void memblock_dump_all(void);
 
diff --git a/mm/memblock.c b/mm/memblock.c
index b1d2a0009733..0c5b5699af6e 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -1796,6 +1796,26 @@ int __init_memblock memblock_search_pfn_nid(unsigned long pfn,
 	return memblock_get_region_node(&type->regions[mid]);
 }
 
+/**
+ * memblock_get_flags - get a single memblock flags
+ * @base: base of memeblock to get
+ *
+ * Get the flags of memeblock with base: @base
+ *
+ * Return:
+ * 0 if ok, non-zero if fail
+ */
+int __init_memblock memblock_get_flags(phys_addr_t base, int *flags)
+{
+	int idx = memblock_search(&memblock.memory, base);
+
+	if (idx == -1)
+		return -EINVAL;
+
+	*flags = memblock.memory.regions[idx].flags;
+	return 0;
+}
+
 /**
  * memblock_is_region_memory - check if a region is a subset of memory
  * @base: base of region to check
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

Initrd memory will be removed and then added in arm64_memblock_init() and this
will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
the lower 4G range has some non-mirrored memory.

In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
reinstalled if the origin memblock has this flag.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 arch/arm64/mm/init.c     |  9 +++++++++
 include/linux/memblock.h |  1 +
 mm/memblock.c            | 20 ++++++++++++++++++++
 3 files changed, 30 insertions(+)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 339ee84e5a61..11641f924d08 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
 			"initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
 			phys_initrd_size = 0;
 		} else {
+			int flags, ret;
+
+			ret = memblock_get_flags(base, &flags);
+			if (ret)
+				flags = 0;
+
 			memblock_remove(base, size); /* clear MEMBLOCK_ flags */
 			memblock_add(base, size);
 			memblock_reserve(base, size);
+
+			if (flags & MEMBLOCK_MIRROR)
+				memblock_mark_mirror(base, size);
 		}
 	}
 
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index 50ad19662a32..3d6a382ac9c8 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -487,6 +487,7 @@ bool memblock_is_map_memory(phys_addr_t addr);
 bool memblock_is_region_memory(phys_addr_t base, phys_addr_t size);
 bool memblock_is_reserved(phys_addr_t addr);
 bool memblock_is_region_reserved(phys_addr_t base, phys_addr_t size);
+int memblock_get_flags(phys_addr_t base, int *flags);
 
 void memblock_dump_all(void);
 
diff --git a/mm/memblock.c b/mm/memblock.c
index b1d2a0009733..0c5b5699af6e 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -1796,6 +1796,26 @@ int __init_memblock memblock_search_pfn_nid(unsigned long pfn,
 	return memblock_get_region_node(&type->regions[mid]);
 }
 
+/**
+ * memblock_get_flags - get a single memblock flags
+ * @base: base of memeblock to get
+ *
+ * Get the flags of memeblock with base: @base
+ *
+ * Return:
+ * 0 if ok, non-zero if fail
+ */
+int __init_memblock memblock_get_flags(phys_addr_t base, int *flags)
+{
+	int idx = memblock_search(&memblock.memory, base);
+
+	if (idx == -1)
+		return -EINVAL;
+
+	*flags = memblock.memory.regions[idx].flags;
+	return 0;
+}
+
 /**
  * memblock_is_region_memory - check if a region is a subset of memory
  * @base: base of region to check
-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

If system have some mirrored memory and mirrored feature is not specified
in boot parameter, the basic mirrored feature will be enabled and this will
lead to the following situations:

- memblock memory allocation prefers mirrored region. This may have some
  unexpected influence on numa affinity.

- contiguous memory will be split into several parts if parts of them
  is mirrored memory via memblock_mark_mirror().

To fix this, variable mirrored_kernelcore will be checked before calling
efi_find_mirror() which will enable basic mirrored feature and this
variable is true if kernelcore=mirror is added in the kernel parameters.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 drivers/firmware/efi/efi.c | 3 +++
 include/linux/mm.h         | 2 ++
 mm/page_alloc.c            | 2 +-
 3 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 79c232e07de7..8a5edcb0dd82 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
 	if (!efi_enabled(EFI_MEMMAP))
 		return;
 
+	if (!mirrored_kernelcore)
+		return;
+
 	for_each_efi_memory_desc(md) {
 		unsigned long long start = md->phys_addr;
 		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
diff --git a/include/linux/mm.h b/include/linux/mm.h
index bc8f326be0ce..741ac7d022c3 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
 			unsigned long *start_pfn, unsigned long *end_pfn);
 extern unsigned long find_min_pfn_with_active_regions(void);
 
+extern bool mirrored_kernelcore;
+
 #ifndef CONFIG_NUMA
 static inline int early_pfn_to_nid(unsigned long pfn)
 {
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index e008a3df0485..cf6f70aba787 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
 static unsigned long required_movablecore __initdata;
 static unsigned long required_movablecore_percent __initdata;
 static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
-static bool mirrored_kernelcore __meminitdata;
+bool mirrored_kernelcore __meminitdata;
 
 /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
 int movable_zone;
-- 
2.25.1

[PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

If system have some mirrored memory and mirrored feature is not specified
in boot parameter, the basic mirrored feature will be enabled and this will
lead to the following situations:

- memblock memory allocation prefers mirrored region. This may have some
  unexpected influence on numa affinity.

- contiguous memory will be split into several parts if parts of them
  is mirrored memory via memblock_mark_mirror().

To fix this, variable mirrored_kernelcore will be checked before calling
efi_find_mirror() which will enable basic mirrored feature and this
variable is true if kernelcore=mirror is added in the kernel parameters.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 drivers/firmware/efi/efi.c | 3 +++
 include/linux/mm.h         | 2 ++
 mm/page_alloc.c            | 2 +-
 3 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 79c232e07de7..8a5edcb0dd82 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
 	if (!efi_enabled(EFI_MEMMAP))
 		return;
 
+	if (!mirrored_kernelcore)
+		return;
+
 	for_each_efi_memory_desc(md) {
 		unsigned long long start = md->phys_addr;
 		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
diff --git a/include/linux/mm.h b/include/linux/mm.h
index bc8f326be0ce..741ac7d022c3 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
 			unsigned long *start_pfn, unsigned long *end_pfn);
 extern unsigned long find_min_pfn_with_active_regions(void);
 
+extern bool mirrored_kernelcore;
+
 #ifndef CONFIG_NUMA
 static inline int early_pfn_to_nid(unsigned long pfn)
 {
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index e008a3df0485..cf6f70aba787 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
 static unsigned long required_movablecore __initdata;
 static unsigned long required_movablecore_percent __initdata;
 static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
-static bool mirrored_kernelcore __meminitdata;
+bool mirrored_kernelcore __meminitdata;
 
 /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
 int movable_zone;
-- 
2.25.1


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Wupeng Ma]

From: Ma Wupeng <mawupeng1@huawei.com>

If system have some mirrored memory and mirrored feature is not specified
in boot parameter, the basic mirrored feature will be enabled and this will
lead to the following situations:

- memblock memory allocation prefers mirrored region. This may have some
  unexpected influence on numa affinity.

- contiguous memory will be split into several parts if parts of them
  is mirrored memory via memblock_mark_mirror().

To fix this, variable mirrored_kernelcore will be checked before calling
efi_find_mirror() which will enable basic mirrored feature and this
variable is true if kernelcore=mirror is added in the kernel parameters.

Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
---
 drivers/firmware/efi/efi.c | 3 +++
 include/linux/mm.h         | 2 ++
 mm/page_alloc.c            | 2 +-
 3 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index 79c232e07de7..8a5edcb0dd82 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
 	if (!efi_enabled(EFI_MEMMAP))
 		return;
 
+	if (!mirrored_kernelcore)
+		return;
+
 	for_each_efi_memory_desc(md) {
 		unsigned long long start = md->phys_addr;
 		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
diff --git a/include/linux/mm.h b/include/linux/mm.h
index bc8f326be0ce..741ac7d022c3 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
 			unsigned long *start_pfn, unsigned long *end_pfn);
 extern unsigned long find_min_pfn_with_active_regions(void);
 
+extern bool mirrored_kernelcore;
+
 #ifndef CONFIG_NUMA
 static inline int early_pfn_to_nid(unsigned long pfn)
 {
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index e008a3df0485..cf6f70aba787 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
 static unsigned long required_movablecore __initdata;
 static unsigned long required_movablecore_percent __initdata;
 static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
-static bool mirrored_kernelcore __meminitdata;
+bool mirrored_kernelcore __meminitdata;
 
 /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
 int movable_zone;
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> Initrd memory will be removed and then added in arm64_memblock_init() and this
> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> the lower 4G range has some non-mirrored memory.
> 
> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> reinstalled if the origin memblock has this flag.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  arch/arm64/mm/init.c     |  9 +++++++++
>  include/linux/memblock.h |  1 +
>  mm/memblock.c            | 20 ++++++++++++++++++++
>  3 files changed, 30 insertions(+)
> 
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 339ee84e5a61..11641f924d08 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>  			"initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>  			phys_initrd_size = 0;
>  		} else {
> +			int flags, ret;
> +
> +			ret = memblock_get_flags(base, &flags);
> +			if (ret)
> +				flags = 0;
> +
>  			memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>  			memblock_add(base, size);
>  			memblock_reserve(base, size);

Can you explain why we're removing+re-adding here exactly? Is it just to
clear flags as the comment indicates?


If it's really just about clearing flags, I wonder if we rather want to
have an interface that does exactly that, and hides the way this is
actually implemented (obtain flags, remove, re-add ...), internally.

But most probably there is more magic in the code and clearing flags
isn't all it ends up doing.

-- 
Thanks,

David / dhildenb


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> Initrd memory will be removed and then added in arm64_memblock_init() and this
> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> the lower 4G range has some non-mirrored memory.
> 
> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> reinstalled if the origin memblock has this flag.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  arch/arm64/mm/init.c     |  9 +++++++++
>  include/linux/memblock.h |  1 +
>  mm/memblock.c            | 20 ++++++++++++++++++++
>  3 files changed, 30 insertions(+)
> 
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 339ee84e5a61..11641f924d08 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>  			"initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>  			phys_initrd_size = 0;
>  		} else {
> +			int flags, ret;
> +
> +			ret = memblock_get_flags(base, &flags);
> +			if (ret)
> +				flags = 0;
> +
>  			memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>  			memblock_add(base, size);
>  			memblock_reserve(base, size);

Can you explain why we're removing+re-adding here exactly? Is it just to
clear flags as the comment indicates?


If it's really just about clearing flags, I wonder if we rather want to
have an interface that does exactly that, and hides the way this is
actually implemented (obtain flags, remove, re-add ...), internally.

But most probably there is more magic in the code and clearing flags
isn't all it ends up doing.

-- 
Thanks,

David / dhildenb

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> Initrd memory will be removed and then added in arm64_memblock_init() and this
> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> the lower 4G range has some non-mirrored memory.
> 
> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> reinstalled if the origin memblock has this flag.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  arch/arm64/mm/init.c     |  9 +++++++++
>  include/linux/memblock.h |  1 +
>  mm/memblock.c            | 20 ++++++++++++++++++++
>  3 files changed, 30 insertions(+)
> 
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 339ee84e5a61..11641f924d08 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>  			"initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>  			phys_initrd_size = 0;
>  		} else {
> +			int flags, ret;
> +
> +			ret = memblock_get_flags(base, &flags);
> +			if (ret)
> +				flags = 0;
> +
>  			memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>  			memblock_add(base, size);
>  			memblock_reserve(base, size);

Can you explain why we're removing+re-adding here exactly? Is it just to
clear flags as the comment indicates?


If it's really just about clearing flags, I wonder if we rather want to
have an interface that does exactly that, and hides the way this is
actually implemented (obtain flags, remove, re-add ...), internally.

But most probably there is more magic in the code and clearing flags
isn't all it ends up doing.

-- 
Thanks,

David / dhildenb


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}

Reviewed-by: David Hildenbrand <david@redhat.com>

-- 
Thanks,

David / dhildenb

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}

Reviewed-by: David Hildenbrand <david@redhat.com>

-- 
Thanks,

David / dhildenb


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}

Reviewed-by: David Hildenbrand <david@redhat.com>

-- 
Thanks,

David / dhildenb


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> For a system only have limited mirrored memory or some numa node without
> mirrored memory, the per node vmemmap page_structs prefer to allocate
> memory from mirrored region, which will lead to vmemmap_verify() in
> vmemmap_populate_basepages() report lots of warning message.
> 
> This patch demote the "potential offnode page_structs" warning messages
> to debug level to avoid a very long print during bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/sparse-vmemmap.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
> index f4fa61dbbee3..78debdb89eb1 100644
> --- a/mm/sparse-vmemmap.c
> +++ b/mm/sparse-vmemmap.c
> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>  	int actual_node = early_pfn_to_nid(pfn);
>  
>  	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
> -		pr_warn("[%lx-%lx] potential offnode page_structs\n",
> +		pr_debug("[%lx-%lx] potential offnode page_structs\n",
>  			start, end - 1);
>  }
>  

This will possibly hide it in environments where this might indeed
indicate performance issues.

What about a pr_warn_once()?

-- 
Thanks,

David / dhildenb

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> For a system only have limited mirrored memory or some numa node without
> mirrored memory, the per node vmemmap page_structs prefer to allocate
> memory from mirrored region, which will lead to vmemmap_verify() in
> vmemmap_populate_basepages() report lots of warning message.
> 
> This patch demote the "potential offnode page_structs" warning messages
> to debug level to avoid a very long print during bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/sparse-vmemmap.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
> index f4fa61dbbee3..78debdb89eb1 100644
> --- a/mm/sparse-vmemmap.c
> +++ b/mm/sparse-vmemmap.c
> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>  	int actual_node = early_pfn_to_nid(pfn);
>  
>  	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
> -		pr_warn("[%lx-%lx] potential offnode page_structs\n",
> +		pr_debug("[%lx-%lx] potential offnode page_structs\n",
>  			start, end - 1);
>  }
>  

This will possibly hide it in environments where this might indeed
indicate performance issues.

What about a pr_warn_once()?

-- 
Thanks,

David / dhildenb


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[David Hildenbrand]

On 07.06.22 11:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> For a system only have limited mirrored memory or some numa node without
> mirrored memory, the per node vmemmap page_structs prefer to allocate
> memory from mirrored region, which will lead to vmemmap_verify() in
> vmemmap_populate_basepages() report lots of warning message.
> 
> This patch demote the "potential offnode page_structs" warning messages
> to debug level to avoid a very long print during bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/sparse-vmemmap.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
> index f4fa61dbbee3..78debdb89eb1 100644
> --- a/mm/sparse-vmemmap.c
> +++ b/mm/sparse-vmemmap.c
> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>  	int actual_node = early_pfn_to_nid(pfn);
>  
>  	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
> -		pr_warn("[%lx-%lx] potential offnode page_structs\n",
> +		pr_debug("[%lx-%lx] potential offnode page_structs\n",
>  			start, end - 1);
>  }
>  

This will possibly hide it in environments where this might indeed
indicate performance issues.

What about a pr_warn_once()?

-- 
Thanks,

David / dhildenb


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>
> On 07.06.22 11:38, Wupeng Ma wrote:
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Initrd memory will be removed and then added in arm64_memblock_init() and this
> > will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> > flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> > the lower 4G range has some non-mirrored memory.
> >
> > In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> > reinstalled if the origin memblock has this flag.
> >
> > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > ---
> >  arch/arm64/mm/init.c     |  9 +++++++++
> >  include/linux/memblock.h |  1 +
> >  mm/memblock.c            | 20 ++++++++++++++++++++
> >  3 files changed, 30 insertions(+)
> >
> > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> > index 339ee84e5a61..11641f924d08 100644
> > --- a/arch/arm64/mm/init.c
> > +++ b/arch/arm64/mm/init.c
> > @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >                       "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >                       phys_initrd_size = 0;
> >               } else {
> > +                     int flags, ret;
> > +
> > +                     ret = memblock_get_flags(base, &flags);
> > +                     if (ret)
> > +                             flags = 0;
> > +
> >                       memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >                       memblock_add(base, size);
> >                       memblock_reserve(base, size);
>
> Can you explain why we're removing+re-adding here exactly? Is it just to
> clear flags as the comment indicates?
>

This should only happen if the placement of the initrd conflicts with
a mem= command line parameter or it is not covered by memblock for
some other reason.

IOW, this should never happen, and if re-memblock_add'ing this memory
unconditionally is causing problems, we should fix that instead of
working around it.

> If it's really just about clearing flags, I wonder if we rather want to
> have an interface that does exactly that, and hides the way this is
> actually implemented (obtain flags, remove, re-add ...), internally.
>
> But most probably there is more magic in the code and clearing flags
> isn't all it ends up doing.
>

I don't remember exactly why we needed to clear the flags, but I think
it had to do with some corner case we hit when the initrd was
partially covered.

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>
> On 07.06.22 11:38, Wupeng Ma wrote:
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Initrd memory will be removed and then added in arm64_memblock_init() and this
> > will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> > flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> > the lower 4G range has some non-mirrored memory.
> >
> > In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> > reinstalled if the origin memblock has this flag.
> >
> > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > ---
> >  arch/arm64/mm/init.c     |  9 +++++++++
> >  include/linux/memblock.h |  1 +
> >  mm/memblock.c            | 20 ++++++++++++++++++++
> >  3 files changed, 30 insertions(+)
> >
> > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> > index 339ee84e5a61..11641f924d08 100644
> > --- a/arch/arm64/mm/init.c
> > +++ b/arch/arm64/mm/init.c
> > @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >                       "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >                       phys_initrd_size = 0;
> >               } else {
> > +                     int flags, ret;
> > +
> > +                     ret = memblock_get_flags(base, &flags);
> > +                     if (ret)
> > +                             flags = 0;
> > +
> >                       memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >                       memblock_add(base, size);
> >                       memblock_reserve(base, size);
>
> Can you explain why we're removing+re-adding here exactly? Is it just to
> clear flags as the comment indicates?
>

This should only happen if the placement of the initrd conflicts with
a mem= command line parameter or it is not covered by memblock for
some other reason.

IOW, this should never happen, and if re-memblock_add'ing this memory
unconditionally is causing problems, we should fix that instead of
working around it.

> If it's really just about clearing flags, I wonder if we rather want to
> have an interface that does exactly that, and hides the way this is
> actually implemented (obtain flags, remove, re-add ...), internally.
>
> But most probably there is more magic in the code and clearing flags
> isn't all it ends up doing.
>

I don't remember exactly why we needed to clear the flags, but I think
it had to do with some corner case we hit when the initrd was
partially covered.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>
> On 07.06.22 11:38, Wupeng Ma wrote:
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Initrd memory will be removed and then added in arm64_memblock_init() and this
> > will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> > flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> > the lower 4G range has some non-mirrored memory.
> >
> > In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> > reinstalled if the origin memblock has this flag.
> >
> > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > ---
> >  arch/arm64/mm/init.c     |  9 +++++++++
> >  include/linux/memblock.h |  1 +
> >  mm/memblock.c            | 20 ++++++++++++++++++++
> >  3 files changed, 30 insertions(+)
> >
> > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> > index 339ee84e5a61..11641f924d08 100644
> > --- a/arch/arm64/mm/init.c
> > +++ b/arch/arm64/mm/init.c
> > @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >                       "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >                       phys_initrd_size = 0;
> >               } else {
> > +                     int flags, ret;
> > +
> > +                     ret = memblock_get_flags(base, &flags);
> > +                     if (ret)
> > +                             flags = 0;
> > +
> >                       memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >                       memblock_add(base, size);
> >                       memblock_reserve(base, size);
>
> Can you explain why we're removing+re-adding here exactly? Is it just to
> clear flags as the comment indicates?
>

This should only happen if the placement of the initrd conflicts with
a mem= command line parameter or it is not covered by memblock for
some other reason.

IOW, this should never happen, and if re-memblock_add'ing this memory
unconditionally is causing problems, we should fix that instead of
working around it.

> If it's really just about clearing flags, I wonder if we rather want to
> have an interface that does exactly that, and hides the way this is
> actually implemented (obtain flags, remove, re-add ...), internally.
>
> But most probably there is more magic in the code and clearing flags
> isn't all it ends up doing.
>

I don't remember exactly why we needed to clear the flags, but I think
it had to do with some corner case we hit when the initrd was
partially covered.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[mawupeng]

在 2022/6/7 20:25, David Hildenbrand 写道:
> On 07.06.22 11:38, Wupeng Ma wrote:
>> From: Ma Wupeng <mawupeng1@huawei.com>
>>
>> For a system only have limited mirrored memory or some numa node without
>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>> memory from mirrored region, which will lead to vmemmap_verify() in
>> vmemmap_populate_basepages() report lots of warning message.
>>
>> This patch demote the "potential offnode page_structs" warning messages
>> to debug level to avoid a very long print during bootup.
>>
>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>> ---
>>   mm/sparse-vmemmap.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>> index f4fa61dbbee3..78debdb89eb1 100644
>> --- a/mm/sparse-vmemmap.c
>> +++ b/mm/sparse-vmemmap.c
>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>   	int actual_node = early_pfn_to_nid(pfn);
>>   
>>   	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>> -		pr_warn("[%lx-%lx] potential offnode page_structs\n",
>> +		pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>   			start, end - 1);
>>   }
>>   
> 
> This will possibly hide it in environments where this might indeed
> indicate performance issues.
> 
> What about a pr_warn_once()?
> 

Sure.

This will works. We can certainly use a pr_warn_once().

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[mawupeng]

在 2022/6/7 20:25, David Hildenbrand 写道:
> On 07.06.22 11:38, Wupeng Ma wrote:
>> From: Ma Wupeng <mawupeng1@huawei.com>
>>
>> For a system only have limited mirrored memory or some numa node without
>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>> memory from mirrored region, which will lead to vmemmap_verify() in
>> vmemmap_populate_basepages() report lots of warning message.
>>
>> This patch demote the "potential offnode page_structs" warning messages
>> to debug level to avoid a very long print during bootup.
>>
>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>> ---
>>   mm/sparse-vmemmap.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>> index f4fa61dbbee3..78debdb89eb1 100644
>> --- a/mm/sparse-vmemmap.c
>> +++ b/mm/sparse-vmemmap.c
>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>   	int actual_node = early_pfn_to_nid(pfn);
>>   
>>   	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>> -		pr_warn("[%lx-%lx] potential offnode page_structs\n",
>> +		pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>   			start, end - 1);
>>   }
>>   
> 
> This will possibly hide it in environments where this might indeed
> indicate performance issues.
> 
> What about a pr_warn_once()?
> 

Sure.

This will works. We can certainly use a pr_warn_once().

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[mawupeng]

在 2022/6/7 20:25, David Hildenbrand 写道:
> On 07.06.22 11:38, Wupeng Ma wrote:
>> From: Ma Wupeng <mawupeng1@huawei.com>
>>
>> For a system only have limited mirrored memory or some numa node without
>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>> memory from mirrored region, which will lead to vmemmap_verify() in
>> vmemmap_populate_basepages() report lots of warning message.
>>
>> This patch demote the "potential offnode page_structs" warning messages
>> to debug level to avoid a very long print during bootup.
>>
>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>> ---
>>   mm/sparse-vmemmap.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>> index f4fa61dbbee3..78debdb89eb1 100644
>> --- a/mm/sparse-vmemmap.c
>> +++ b/mm/sparse-vmemmap.c
>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>   	int actual_node = early_pfn_to_nid(pfn);
>>   
>>   	if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>> -		pr_warn("[%lx-%lx] potential offnode page_structs\n",
>> +		pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>   			start, end - 1);
>>   }
>>   
> 
> This will possibly hide it in environments where this might indeed
> indicate performance issues.
> 
> What about a pr_warn_once()?
> 

Sure.

This will works. We can certainly use a pr_warn_once().

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[mawupeng]

在 2022/6/7 22:49, Ard Biesheuvel 写道:
> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>
>> On 07.06.22 11:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>> the lower 4G range has some non-mirrored memory.
>>>
>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>> reinstalled if the origin memblock has this flag.
>>>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>   arch/arm64/mm/init.c     |  9 +++++++++
>>>   include/linux/memblock.h |  1 +
>>>   mm/memblock.c            | 20 ++++++++++++++++++++
>>>   3 files changed, 30 insertions(+)
>>>
>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>> index 339ee84e5a61..11641f924d08 100644
>>> --- a/arch/arm64/mm/init.c
>>> +++ b/arch/arm64/mm/init.c
>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>                        phys_initrd_size = 0;
>>>                } else {
>>> +                     int flags, ret;
>>> +
>>> +                     ret = memblock_get_flags(base, &flags);
>>> +                     if (ret)
>>> +                             flags = 0;
>>> +
>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>                        memblock_add(base, size);
>>>                        memblock_reserve(base, size);
>>
>> Can you explain why we're removing+re-adding here exactly? Is it just to
>> clear flags as the comment indicates?
>>
> 
> This should only happen if the placement of the initrd conflicts with
> a mem= command line parameter or it is not covered by memblock for
> some other reason.
> 
> IOW, this should never happen, and if re-memblock_add'ing this memory
> unconditionally is causing problems, we should fix that instead of
> working around it.

This will happen if we use initrdmem=3G,100M to reserve initrd memory below
the 4G limit to test this scenario(just for testing, I have trouble to boot
qemu with initrd enabled and memory below 4G are all mirror memory).

Re-memblock_add'ing this memory unconditionally seems fine but clear all
flags(especially MEMBLOCK_MIRROR) may lead to some error log.

> 
>> If it's really just about clearing flags, I wonder if we rather want to
>> have an interface that does exactly that, and hides the way this is
>> actually implemented (obtain flags, remove, re-add ...), internally.
>>
>> But most probably there is more magic in the code and clearing flags
>> isn't all it ends up doing.
>>
> 
> I don't remember exactly why we needed to clear the flags, but I think
> it had to do with some corner case we hit when the initrd was
> partially covered.
If "mem=" is set in command line, memblock_mem_limit_remove_map() will
remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
memory back if this initrd mem has the MEMBLOCK_NOMAP flag?

The rfc version [1] introduce and use memblock_clear_nomap() to clear the
MEMBLOCK_NOMAP of this initrd memblock.
So maybe the usage of memblock_remove() is just to avoid introducing new
function(memblock_clear_nomap)?

Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
to solve this problem rather than bring flag MEMBLOCK_MIRROR back?

[1] https://lore.kernel.org/linux-arm-kernel/20160202180622.GP10166@arm.com/T/#t
> .

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[mawupeng]

在 2022/6/7 22:49, Ard Biesheuvel 写道:
> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>
>> On 07.06.22 11:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>> the lower 4G range has some non-mirrored memory.
>>>
>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>> reinstalled if the origin memblock has this flag.
>>>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>   arch/arm64/mm/init.c     |  9 +++++++++
>>>   include/linux/memblock.h |  1 +
>>>   mm/memblock.c            | 20 ++++++++++++++++++++
>>>   3 files changed, 30 insertions(+)
>>>
>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>> index 339ee84e5a61..11641f924d08 100644
>>> --- a/arch/arm64/mm/init.c
>>> +++ b/arch/arm64/mm/init.c
>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>                        phys_initrd_size = 0;
>>>                } else {
>>> +                     int flags, ret;
>>> +
>>> +                     ret = memblock_get_flags(base, &flags);
>>> +                     if (ret)
>>> +                             flags = 0;
>>> +
>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>                        memblock_add(base, size);
>>>                        memblock_reserve(base, size);
>>
>> Can you explain why we're removing+re-adding here exactly? Is it just to
>> clear flags as the comment indicates?
>>
> 
> This should only happen if the placement of the initrd conflicts with
> a mem= command line parameter or it is not covered by memblock for
> some other reason.
> 
> IOW, this should never happen, and if re-memblock_add'ing this memory
> unconditionally is causing problems, we should fix that instead of
> working around it.

This will happen if we use initrdmem=3G,100M to reserve initrd memory below
the 4G limit to test this scenario(just for testing, I have trouble to boot
qemu with initrd enabled and memory below 4G are all mirror memory).

Re-memblock_add'ing this memory unconditionally seems fine but clear all
flags(especially MEMBLOCK_MIRROR) may lead to some error log.

> 
>> If it's really just about clearing flags, I wonder if we rather want to
>> have an interface that does exactly that, and hides the way this is
>> actually implemented (obtain flags, remove, re-add ...), internally.
>>
>> But most probably there is more magic in the code and clearing flags
>> isn't all it ends up doing.
>>
> 
> I don't remember exactly why we needed to clear the flags, but I think
> it had to do with some corner case we hit when the initrd was
> partially covered.
If "mem=" is set in command line, memblock_mem_limit_remove_map() will
remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
memory back if this initrd mem has the MEMBLOCK_NOMAP flag?

The rfc version [1] introduce and use memblock_clear_nomap() to clear the
MEMBLOCK_NOMAP of this initrd memblock.
So maybe the usage of memblock_remove() is just to avoid introducing new
function(memblock_clear_nomap)?

Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
to solve this problem rather than bring flag MEMBLOCK_MIRROR back?

[1] https://lore.kernel.org/linux-arm-kernel/20160202180622.GP10166@arm.com/T/#t
> .

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[mawupeng]

在 2022/6/7 22:49, Ard Biesheuvel 写道:
> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>
>> On 07.06.22 11:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>> the lower 4G range has some non-mirrored memory.
>>>
>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>> reinstalled if the origin memblock has this flag.
>>>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>   arch/arm64/mm/init.c     |  9 +++++++++
>>>   include/linux/memblock.h |  1 +
>>>   mm/memblock.c            | 20 ++++++++++++++++++++
>>>   3 files changed, 30 insertions(+)
>>>
>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>> index 339ee84e5a61..11641f924d08 100644
>>> --- a/arch/arm64/mm/init.c
>>> +++ b/arch/arm64/mm/init.c
>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>                        phys_initrd_size = 0;
>>>                } else {
>>> +                     int flags, ret;
>>> +
>>> +                     ret = memblock_get_flags(base, &flags);
>>> +                     if (ret)
>>> +                             flags = 0;
>>> +
>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>                        memblock_add(base, size);
>>>                        memblock_reserve(base, size);
>>
>> Can you explain why we're removing+re-adding here exactly? Is it just to
>> clear flags as the comment indicates?
>>
> 
> This should only happen if the placement of the initrd conflicts with
> a mem= command line parameter or it is not covered by memblock for
> some other reason.
> 
> IOW, this should never happen, and if re-memblock_add'ing this memory
> unconditionally is causing problems, we should fix that instead of
> working around it.

This will happen if we use initrdmem=3G,100M to reserve initrd memory below
the 4G limit to test this scenario(just for testing, I have trouble to boot
qemu with initrd enabled and memory below 4G are all mirror memory).

Re-memblock_add'ing this memory unconditionally seems fine but clear all
flags(especially MEMBLOCK_MIRROR) may lead to some error log.

> 
>> If it's really just about clearing flags, I wonder if we rather want to
>> have an interface that does exactly that, and hides the way this is
>> actually implemented (obtain flags, remove, re-add ...), internally.
>>
>> But most probably there is more magic in the code and clearing flags
>> isn't all it ends up doing.
>>
> 
> I don't remember exactly why we needed to clear the flags, but I think
> it had to do with some corner case we hit when the initrd was
> partially covered.
If "mem=" is set in command line, memblock_mem_limit_remove_map() will
remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
memory back if this initrd mem has the MEMBLOCK_NOMAP flag?

The rfc version [1] introduce and use memblock_clear_nomap() to clear the
MEMBLOCK_NOMAP of this initrd memblock.
So maybe the usage of memblock_remove() is just to avoid introducing new
function(memblock_clear_nomap)?

Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
to solve this problem rather than bring flag MEMBLOCK_MIRROR back?

[1] https://lore.kernel.org/linux-arm-kernel/20160202180622.GP10166@arm.com/T/#t
> .

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Mike Rapoport]

On Tue, Jun 07, 2022 at 05:38:02PM +0800, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>

Acked-by: Mike Rapoport <rppt@linux.ibm.com>

> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}
> -- 
> 2.25.1
> 

-- 
Sincerely yours,
Mike.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Mike Rapoport]

On Tue, Jun 07, 2022 at 05:38:02PM +0800, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>

Acked-by: Mike Rapoport <rppt@linux.ibm.com>

> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}
> -- 
> 2.25.1
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Mike Rapoport]

On Tue, Jun 07, 2022 at 05:38:02PM +0800, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>

Acked-by: Mike Rapoport <rppt@linux.ibm.com>

> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}
> -- 
> 2.25.1
> 

-- 
Sincerely yours,
Mike.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Anshuman Khandual]

On 6/8/22 06:56, mawupeng wrote:
> 
> 
> 在 2022/6/7 20:25, David Hildenbrand 写道:
>> On 07.06.22 11:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> For a system only have limited mirrored memory or some numa node without
>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>> vmemmap_populate_basepages() report lots of warning message.
>>>
>>> This patch demote the "potential offnode page_structs" warning messages
>>> to debug level to avoid a very long print during bootup.
>>>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>   mm/sparse-vmemmap.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>> index f4fa61dbbee3..78debdb89eb1 100644
>>> --- a/mm/sparse-vmemmap.c
>>> +++ b/mm/sparse-vmemmap.c
>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>>       int actual_node = early_pfn_to_nid(pfn);
>>>         if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>               start, end - 1);
>>>   }
>>>   
>>
>> This will possibly hide it in environments where this might indeed
>> indicate performance issues.
>>
>> What about a pr_warn_once()?
>>
> 
> Sure.
> 
> This will works. We can certainly use a pr_warn_once().

Why not pr_warn_ratelimited() like in the previous patch ?

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Anshuman Khandual]

On 6/8/22 06:56, mawupeng wrote:
> 
> 
> 在 2022/6/7 20:25, David Hildenbrand 写道:
>> On 07.06.22 11:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> For a system only have limited mirrored memory or some numa node without
>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>> vmemmap_populate_basepages() report lots of warning message.
>>>
>>> This patch demote the "potential offnode page_structs" warning messages
>>> to debug level to avoid a very long print during bootup.
>>>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>   mm/sparse-vmemmap.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>> index f4fa61dbbee3..78debdb89eb1 100644
>>> --- a/mm/sparse-vmemmap.c
>>> +++ b/mm/sparse-vmemmap.c
>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>>       int actual_node = early_pfn_to_nid(pfn);
>>>         if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>               start, end - 1);
>>>   }
>>>   
>>
>> This will possibly hide it in environments where this might indeed
>> indicate performance issues.
>>
>> What about a pr_warn_once()?
>>
> 
> Sure.
> 
> This will works. We can certainly use a pr_warn_once().

Why not pr_warn_ratelimited() like in the previous patch ?

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Anshuman Khandual]

On 6/8/22 06:56, mawupeng wrote:
> 
> 
> 在 2022/6/7 20:25, David Hildenbrand 写道:
>> On 07.06.22 11:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> For a system only have limited mirrored memory or some numa node without
>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>> vmemmap_populate_basepages() report lots of warning message.
>>>
>>> This patch demote the "potential offnode page_structs" warning messages
>>> to debug level to avoid a very long print during bootup.
>>>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>   mm/sparse-vmemmap.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>> index f4fa61dbbee3..78debdb89eb1 100644
>>> --- a/mm/sparse-vmemmap.c
>>> +++ b/mm/sparse-vmemmap.c
>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>>       int actual_node = early_pfn_to_nid(pfn);
>>>         if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>               start, end - 1);
>>>   }
>>>   
>>
>> This will possibly hide it in environments where this might indeed
>> indicate performance issues.
>>
>> What about a pr_warn_once()?
>>
> 
> Sure.
> 
> This will works. We can certainly use a pr_warn_once().

Why not pr_warn_ratelimited() like in the previous patch ?

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Anshuman Khandual]

On 6/7/22 15:08, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}

Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Anshuman Khandual]

On 6/7/22 15:08, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}

Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Anshuman Khandual]

On 6/7/22 15:08, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
> 
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
> 
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
> 
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>  mm/memblock.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>  					    NUMA_NO_NODE, flags);
>  
>  	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		flags &= ~MEMBLOCK_MIRROR;
>  		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  
>  	if (flags & MEMBLOCK_MIRROR) {
>  		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>  			&size);
>  		goto again;
>  	}

Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Mike Rapoport]

On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> 
> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> > On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> > > 
> > > On 07.06.22 11:38, Wupeng Ma wrote:
> > > > From: Ma Wupeng <mawupeng1@huawei.com>
> > > > 
> > > > Initrd memory will be removed and then added in arm64_memblock_init() and this
> > > > will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> > > > flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> > > > the lower 4G range has some non-mirrored memory.
> > > > 
> > > > In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> > > > reinstalled if the origin memblock has this flag.
> > > > 
> > > > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > > > ---
> > > >   arch/arm64/mm/init.c     |  9 +++++++++
> > > >   include/linux/memblock.h |  1 +
> > > >   mm/memblock.c            | 20 ++++++++++++++++++++
> > > >   3 files changed, 30 insertions(+)
> > > > 
> > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> > > > index 339ee84e5a61..11641f924d08 100644
> > > > --- a/arch/arm64/mm/init.c
> > > > +++ b/arch/arm64/mm/init.c
> > > > @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> > > >                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> > > >                        phys_initrd_size = 0;
> > > >                } else {
> > > > +                     int flags, ret;
> > > > +
> > > > +                     ret = memblock_get_flags(base, &flags);
> > > > +                     if (ret)
> > > > +                             flags = 0;
> > > > +
> > > >                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> > > >                        memblock_add(base, size);
> > > >                        memblock_reserve(base, size);
> > > 
> > > Can you explain why we're removing+re-adding here exactly? Is it just to
> > > clear flags as the comment indicates?
> > > 
> > 
> > This should only happen if the placement of the initrd conflicts with
> > a mem= command line parameter or it is not covered by memblock for
> > some other reason.
> > 
> > IOW, this should never happen, and if re-memblock_add'ing this memory
> > unconditionally is causing problems, we should fix that instead of
> > working around it.
> 
> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> the 4G limit to test this scenario(just for testing, I have trouble to boot
> qemu with initrd enabled and memory below 4G are all mirror memory).
> 
> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> 
> > 
> > > If it's really just about clearing flags, I wonder if we rather want to
> > > have an interface that does exactly that, and hides the way this is
> > > actually implemented (obtain flags, remove, re-add ...), internally.
> > > 
> > > But most probably there is more magic in the code and clearing flags
> > > isn't all it ends up doing.
> > > 
> > 
> > I don't remember exactly why we needed to clear the flags, but I think
> > it had to do with some corner case we hit when the initrd was
> > partially covered.
> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> 
> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> MEMBLOCK_NOMAP of this initrd memblock.
> So maybe the usage of memblock_remove() is just to avoid introducing new
> function(memblock_clear_nomap)?
> 
> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?

AFAICT, there are two corner cases that re-adding initrd memory covers:
* initrd memory is not a part of the memory reported to memblock, either
because of firmware weirdness or because it was cut out with mem=
* initrd memory overlaps a NOMAP region

So to make sure initrd memory is mapped properly and retains
MEMBLOCK_MIRROR I think the best we can do is

	memblock_add();
	memblock_clear_nomap();
	memblock_reserve();

 
> [1] https://lore.kernel.org/linux-arm-kernel/20160202180622.GP10166@arm.com/T/#t
> > .

-- 
Sincerely yours,
Mike.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Mike Rapoport]

On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> 
> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> > On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> > > 
> > > On 07.06.22 11:38, Wupeng Ma wrote:
> > > > From: Ma Wupeng <mawupeng1@huawei.com>
> > > > 
> > > > Initrd memory will be removed and then added in arm64_memblock_init() and this
> > > > will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> > > > flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> > > > the lower 4G range has some non-mirrored memory.
> > > > 
> > > > In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> > > > reinstalled if the origin memblock has this flag.
> > > > 
> > > > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > > > ---
> > > >   arch/arm64/mm/init.c     |  9 +++++++++
> > > >   include/linux/memblock.h |  1 +
> > > >   mm/memblock.c            | 20 ++++++++++++++++++++
> > > >   3 files changed, 30 insertions(+)
> > > > 
> > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> > > > index 339ee84e5a61..11641f924d08 100644
> > > > --- a/arch/arm64/mm/init.c
> > > > +++ b/arch/arm64/mm/init.c
> > > > @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> > > >                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> > > >                        phys_initrd_size = 0;
> > > >                } else {
> > > > +                     int flags, ret;
> > > > +
> > > > +                     ret = memblock_get_flags(base, &flags);
> > > > +                     if (ret)
> > > > +                             flags = 0;
> > > > +
> > > >                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> > > >                        memblock_add(base, size);
> > > >                        memblock_reserve(base, size);
> > > 
> > > Can you explain why we're removing+re-adding here exactly? Is it just to
> > > clear flags as the comment indicates?
> > > 
> > 
> > This should only happen if the placement of the initrd conflicts with
> > a mem= command line parameter or it is not covered by memblock for
> > some other reason.
> > 
> > IOW, this should never happen, and if re-memblock_add'ing this memory
> > unconditionally is causing problems, we should fix that instead of
> > working around it.
> 
> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> the 4G limit to test this scenario(just for testing, I have trouble to boot
> qemu with initrd enabled and memory below 4G are all mirror memory).
> 
> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> 
> > 
> > > If it's really just about clearing flags, I wonder if we rather want to
> > > have an interface that does exactly that, and hides the way this is
> > > actually implemented (obtain flags, remove, re-add ...), internally.
> > > 
> > > But most probably there is more magic in the code and clearing flags
> > > isn't all it ends up doing.
> > > 
> > 
> > I don't remember exactly why we needed to clear the flags, but I think
> > it had to do with some corner case we hit when the initrd was
> > partially covered.
> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> 
> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> MEMBLOCK_NOMAP of this initrd memblock.
> So maybe the usage of memblock_remove() is just to avoid introducing new
> function(memblock_clear_nomap)?
> 
> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?

AFAICT, there are two corner cases that re-adding initrd memory covers:
* initrd memory is not a part of the memory reported to memblock, either
because of firmware weirdness or because it was cut out with mem=
* initrd memory overlaps a NOMAP region

So to make sure initrd memory is mapped properly and retains
MEMBLOCK_MIRROR I think the best we can do is

	memblock_add();
	memblock_clear_nomap();
	memblock_reserve();

 
> [1] https://lore.kernel.org/linux-arm-kernel/20160202180622.GP10166@arm.com/T/#t
> > .

-- 
Sincerely yours,
Mike.

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Mike Rapoport]

On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> 
> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> > On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> > > 
> > > On 07.06.22 11:38, Wupeng Ma wrote:
> > > > From: Ma Wupeng <mawupeng1@huawei.com>
> > > > 
> > > > Initrd memory will be removed and then added in arm64_memblock_init() and this
> > > > will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> > > > flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> > > > the lower 4G range has some non-mirrored memory.
> > > > 
> > > > In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> > > > reinstalled if the origin memblock has this flag.
> > > > 
> > > > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > > > ---
> > > >   arch/arm64/mm/init.c     |  9 +++++++++
> > > >   include/linux/memblock.h |  1 +
> > > >   mm/memblock.c            | 20 ++++++++++++++++++++
> > > >   3 files changed, 30 insertions(+)
> > > > 
> > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> > > > index 339ee84e5a61..11641f924d08 100644
> > > > --- a/arch/arm64/mm/init.c
> > > > +++ b/arch/arm64/mm/init.c
> > > > @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> > > >                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> > > >                        phys_initrd_size = 0;
> > > >                } else {
> > > > +                     int flags, ret;
> > > > +
> > > > +                     ret = memblock_get_flags(base, &flags);
> > > > +                     if (ret)
> > > > +                             flags = 0;
> > > > +
> > > >                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> > > >                        memblock_add(base, size);
> > > >                        memblock_reserve(base, size);
> > > 
> > > Can you explain why we're removing+re-adding here exactly? Is it just to
> > > clear flags as the comment indicates?
> > > 
> > 
> > This should only happen if the placement of the initrd conflicts with
> > a mem= command line parameter or it is not covered by memblock for
> > some other reason.
> > 
> > IOW, this should never happen, and if re-memblock_add'ing this memory
> > unconditionally is causing problems, we should fix that instead of
> > working around it.
> 
> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> the 4G limit to test this scenario(just for testing, I have trouble to boot
> qemu with initrd enabled and memory below 4G are all mirror memory).
> 
> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> 
> > 
> > > If it's really just about clearing flags, I wonder if we rather want to
> > > have an interface that does exactly that, and hides the way this is
> > > actually implemented (obtain flags, remove, re-add ...), internally.
> > > 
> > > But most probably there is more magic in the code and clearing flags
> > > isn't all it ends up doing.
> > > 
> > 
> > I don't remember exactly why we needed to clear the flags, but I think
> > it had to do with some corner case we hit when the initrd was
> > partially covered.
> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> 
> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> MEMBLOCK_NOMAP of this initrd memblock.
> So maybe the usage of memblock_remove() is just to avoid introducing new
> function(memblock_clear_nomap)?
> 
> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?

AFAICT, there are two corner cases that re-adding initrd memory covers:
* initrd memory is not a part of the memory reported to memblock, either
because of firmware weirdness or because it was cut out with mem=
* initrd memory overlaps a NOMAP region

So to make sure initrd memory is mapped properly and retains
MEMBLOCK_MIRROR I think the best we can do is

	memblock_add();
	memblock_clear_nomap();
	memblock_reserve();

 
> [1] https://lore.kernel.org/linux-arm-kernel/20160202180622.GP10166@arm.com/T/#t
> > .

-- 
Sincerely yours,
Mike.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[David Hildenbrand]

On 08.06.22 12:02, Mike Rapoport wrote:
> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
>>
>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>>>
>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>
>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>>>> the lower 4G range has some non-mirrored memory.
>>>>>
>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>>>> reinstalled if the origin memblock has this flag.
>>>>>
>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>> ---
>>>>>   arch/arm64/mm/init.c     |  9 +++++++++
>>>>>   include/linux/memblock.h |  1 +
>>>>>   mm/memblock.c            | 20 ++++++++++++++++++++
>>>>>   3 files changed, 30 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>>>> index 339ee84e5a61..11641f924d08 100644
>>>>> --- a/arch/arm64/mm/init.c
>>>>> +++ b/arch/arm64/mm/init.c
>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>>>                        phys_initrd_size = 0;
>>>>>                } else {
>>>>> +                     int flags, ret;
>>>>> +
>>>>> +                     ret = memblock_get_flags(base, &flags);
>>>>> +                     if (ret)
>>>>> +                             flags = 0;
>>>>> +
>>>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>>>                        memblock_add(base, size);
>>>>>                        memblock_reserve(base, size);
>>>>
>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
>>>> clear flags as the comment indicates?
>>>>
>>>
>>> This should only happen if the placement of the initrd conflicts with
>>> a mem= command line parameter or it is not covered by memblock for
>>> some other reason.
>>>
>>> IOW, this should never happen, and if re-memblock_add'ing this memory
>>> unconditionally is causing problems, we should fix that instead of
>>> working around it.
>>
>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
>> the 4G limit to test this scenario(just for testing, I have trouble to boot
>> qemu with initrd enabled and memory below 4G are all mirror memory).
>>
>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
>>
>>>
>>>> If it's really just about clearing flags, I wonder if we rather want to
>>>> have an interface that does exactly that, and hides the way this is
>>>> actually implemented (obtain flags, remove, re-add ...), internally.
>>>>
>>>> But most probably there is more magic in the code and clearing flags
>>>> isn't all it ends up doing.
>>>>
>>>
>>> I don't remember exactly why we needed to clear the flags, but I think
>>> it had to do with some corner case we hit when the initrd was
>>> partially covered.
>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
>>
>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
>> MEMBLOCK_NOMAP of this initrd memblock.
>> So maybe the usage of memblock_remove() is just to avoid introducing new
>> function(memblock_clear_nomap)?
>>
>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> 
> AFAICT, there are two corner cases that re-adding initrd memory covers:
> * initrd memory is not a part of the memory reported to memblock, either
> because of firmware weirdness or because it was cut out with mem=
> * initrd memory overlaps a NOMAP region
> 
> So to make sure initrd memory is mapped properly and retains
> MEMBLOCK_MIRROR I think the best we can do is
> 
> 	memblock_add();
> 	memblock_clear_nomap();
> 	memblock_reserve();

Would simply detect+rejecting to boot on such setups be an option? The
replies so far indicate to me that this is rather a corner case than a
reasonable use case.

-- 
Thanks,

David / dhildenb

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[David Hildenbrand]

On 08.06.22 12:02, Mike Rapoport wrote:
> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
>>
>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>>>
>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>
>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>>>> the lower 4G range has some non-mirrored memory.
>>>>>
>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>>>> reinstalled if the origin memblock has this flag.
>>>>>
>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>> ---
>>>>>   arch/arm64/mm/init.c     |  9 +++++++++
>>>>>   include/linux/memblock.h |  1 +
>>>>>   mm/memblock.c            | 20 ++++++++++++++++++++
>>>>>   3 files changed, 30 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>>>> index 339ee84e5a61..11641f924d08 100644
>>>>> --- a/arch/arm64/mm/init.c
>>>>> +++ b/arch/arm64/mm/init.c
>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>>>                        phys_initrd_size = 0;
>>>>>                } else {
>>>>> +                     int flags, ret;
>>>>> +
>>>>> +                     ret = memblock_get_flags(base, &flags);
>>>>> +                     if (ret)
>>>>> +                             flags = 0;
>>>>> +
>>>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>>>                        memblock_add(base, size);
>>>>>                        memblock_reserve(base, size);
>>>>
>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
>>>> clear flags as the comment indicates?
>>>>
>>>
>>> This should only happen if the placement of the initrd conflicts with
>>> a mem= command line parameter or it is not covered by memblock for
>>> some other reason.
>>>
>>> IOW, this should never happen, and if re-memblock_add'ing this memory
>>> unconditionally is causing problems, we should fix that instead of
>>> working around it.
>>
>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
>> the 4G limit to test this scenario(just for testing, I have trouble to boot
>> qemu with initrd enabled and memory below 4G are all mirror memory).
>>
>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
>>
>>>
>>>> If it's really just about clearing flags, I wonder if we rather want to
>>>> have an interface that does exactly that, and hides the way this is
>>>> actually implemented (obtain flags, remove, re-add ...), internally.
>>>>
>>>> But most probably there is more magic in the code and clearing flags
>>>> isn't all it ends up doing.
>>>>
>>>
>>> I don't remember exactly why we needed to clear the flags, but I think
>>> it had to do with some corner case we hit when the initrd was
>>> partially covered.
>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
>>
>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
>> MEMBLOCK_NOMAP of this initrd memblock.
>> So maybe the usage of memblock_remove() is just to avoid introducing new
>> function(memblock_clear_nomap)?
>>
>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> 
> AFAICT, there are two corner cases that re-adding initrd memory covers:
> * initrd memory is not a part of the memory reported to memblock, either
> because of firmware weirdness or because it was cut out with mem=
> * initrd memory overlaps a NOMAP region
> 
> So to make sure initrd memory is mapped properly and retains
> MEMBLOCK_MIRROR I think the best we can do is
> 
> 	memblock_add();
> 	memblock_clear_nomap();
> 	memblock_reserve();

Would simply detect+rejecting to boot on such setups be an option? The
replies so far indicate to me that this is rather a corner case than a
reasonable use case.

-- 
Thanks,

David / dhildenb


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[David Hildenbrand]

On 08.06.22 12:02, Mike Rapoport wrote:
> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
>>
>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>>>
>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>
>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>>>> the lower 4G range has some non-mirrored memory.
>>>>>
>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>>>> reinstalled if the origin memblock has this flag.
>>>>>
>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>> ---
>>>>>   arch/arm64/mm/init.c     |  9 +++++++++
>>>>>   include/linux/memblock.h |  1 +
>>>>>   mm/memblock.c            | 20 ++++++++++++++++++++
>>>>>   3 files changed, 30 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>>>> index 339ee84e5a61..11641f924d08 100644
>>>>> --- a/arch/arm64/mm/init.c
>>>>> +++ b/arch/arm64/mm/init.c
>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>>>                        phys_initrd_size = 0;
>>>>>                } else {
>>>>> +                     int flags, ret;
>>>>> +
>>>>> +                     ret = memblock_get_flags(base, &flags);
>>>>> +                     if (ret)
>>>>> +                             flags = 0;
>>>>> +
>>>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>>>                        memblock_add(base, size);
>>>>>                        memblock_reserve(base, size);
>>>>
>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
>>>> clear flags as the comment indicates?
>>>>
>>>
>>> This should only happen if the placement of the initrd conflicts with
>>> a mem= command line parameter or it is not covered by memblock for
>>> some other reason.
>>>
>>> IOW, this should never happen, and if re-memblock_add'ing this memory
>>> unconditionally is causing problems, we should fix that instead of
>>> working around it.
>>
>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
>> the 4G limit to test this scenario(just for testing, I have trouble to boot
>> qemu with initrd enabled and memory below 4G are all mirror memory).
>>
>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
>>
>>>
>>>> If it's really just about clearing flags, I wonder if we rather want to
>>>> have an interface that does exactly that, and hides the way this is
>>>> actually implemented (obtain flags, remove, re-add ...), internally.
>>>>
>>>> But most probably there is more magic in the code and clearing flags
>>>> isn't all it ends up doing.
>>>>
>>>
>>> I don't remember exactly why we needed to clear the flags, but I think
>>> it had to do with some corner case we hit when the initrd was
>>> partially covered.
>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
>>
>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
>> MEMBLOCK_NOMAP of this initrd memblock.
>> So maybe the usage of memblock_remove() is just to avoid introducing new
>> function(memblock_clear_nomap)?
>>
>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> 
> AFAICT, there are two corner cases that re-adding initrd memory covers:
> * initrd memory is not a part of the memory reported to memblock, either
> because of firmware weirdness or because it was cut out with mem=
> * initrd memory overlaps a NOMAP region
> 
> So to make sure initrd memory is mapped properly and retains
> MEMBLOCK_MIRROR I think the best we can do is
> 
> 	memblock_add();
> 	memblock_clear_nomap();
> 	memblock_reserve();

Would simply detect+rejecting to boot on such setups be an option? The
replies so far indicate to me that this is rather a corner case than a
reasonable use case.

-- 
Thanks,

David / dhildenb


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
>
> On 08.06.22 12:02, Mike Rapoport wrote:
> > On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> >>
> >> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> >>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> >>>>
> >>>> On 07.06.22 11:38, Wupeng Ma wrote:
> >>>>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>
> >>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
> >>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> >>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> >>>>> the lower 4G range has some non-mirrored memory.
> >>>>>
> >>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> >>>>> reinstalled if the origin memblock has this flag.
> >>>>>
> >>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> >>>>> ---
> >>>>>   arch/arm64/mm/init.c     |  9 +++++++++
> >>>>>   include/linux/memblock.h |  1 +
> >>>>>   mm/memblock.c            | 20 ++++++++++++++++++++
> >>>>>   3 files changed, 30 insertions(+)
> >>>>>
> >>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> >>>>> index 339ee84e5a61..11641f924d08 100644
> >>>>> --- a/arch/arm64/mm/init.c
> >>>>> +++ b/arch/arm64/mm/init.c
> >>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >>>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >>>>>                        phys_initrd_size = 0;
> >>>>>                } else {
> >>>>> +                     int flags, ret;
> >>>>> +
> >>>>> +                     ret = memblock_get_flags(base, &flags);
> >>>>> +                     if (ret)
> >>>>> +                             flags = 0;
> >>>>> +
> >>>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >>>>>                        memblock_add(base, size);
> >>>>>                        memblock_reserve(base, size);
> >>>>
> >>>> Can you explain why we're removing+re-adding here exactly? Is it just to
> >>>> clear flags as the comment indicates?
> >>>>
> >>>
> >>> This should only happen if the placement of the initrd conflicts with
> >>> a mem= command line parameter or it is not covered by memblock for
> >>> some other reason.
> >>>
> >>> IOW, this should never happen, and if re-memblock_add'ing this memory
> >>> unconditionally is causing problems, we should fix that instead of
> >>> working around it.
> >>
> >> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> >> the 4G limit to test this scenario(just for testing, I have trouble to boot
> >> qemu with initrd enabled and memory below 4G are all mirror memory).
> >>
> >> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> >> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> >>
> >>>
> >>>> If it's really just about clearing flags, I wonder if we rather want to
> >>>> have an interface that does exactly that, and hides the way this is
> >>>> actually implemented (obtain flags, remove, re-add ...), internally.
> >>>>
> >>>> But most probably there is more magic in the code and clearing flags
> >>>> isn't all it ends up doing.
> >>>>
> >>>
> >>> I don't remember exactly why we needed to clear the flags, but I think
> >>> it had to do with some corner case we hit when the initrd was
> >>> partially covered.
> >> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> >> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> >> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> >>
> >> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> >> MEMBLOCK_NOMAP of this initrd memblock.
> >> So maybe the usage of memblock_remove() is just to avoid introducing new
> >> function(memblock_clear_nomap)?
> >>
> >> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> >> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> >> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> >
> > AFAICT, there are two corner cases that re-adding initrd memory covers:
> > * initrd memory is not a part of the memory reported to memblock, either
> > because of firmware weirdness or because it was cut out with mem=
> > * initrd memory overlaps a NOMAP region
> >
> > So to make sure initrd memory is mapped properly and retains
> > MEMBLOCK_MIRROR I think the best we can do is
> >
> >       memblock_add();
> >       memblock_clear_nomap();
> >       memblock_reserve();
>
> Would simply detect+rejecting to boot on such setups be an option? The
> replies so far indicate to me that this is rather a corner case than a
> reasonable use case.
>

The sad reality is that mem= is known to be used in production for
limiting the amount of memory that the kernel takes control of, in
order to allow the remainder to be used in platform specific ways.

Of course, there are much better ways to achieve that, but given that
we currently support it, I don't think we can easily back that out.

I do think that there is no need to go out of our way to make this
case work seamlessly with mirrored memory, though. So I'd prefer to
make the remove+re-add conditional on there actually being a need to
do so. That way, we don't break the old use case or mirrored memory,
and whatever happens when the two are combined is DONTCARE.

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
>
> On 08.06.22 12:02, Mike Rapoport wrote:
> > On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> >>
> >> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> >>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> >>>>
> >>>> On 07.06.22 11:38, Wupeng Ma wrote:
> >>>>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>
> >>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
> >>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> >>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> >>>>> the lower 4G range has some non-mirrored memory.
> >>>>>
> >>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> >>>>> reinstalled if the origin memblock has this flag.
> >>>>>
> >>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> >>>>> ---
> >>>>>   arch/arm64/mm/init.c     |  9 +++++++++
> >>>>>   include/linux/memblock.h |  1 +
> >>>>>   mm/memblock.c            | 20 ++++++++++++++++++++
> >>>>>   3 files changed, 30 insertions(+)
> >>>>>
> >>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> >>>>> index 339ee84e5a61..11641f924d08 100644
> >>>>> --- a/arch/arm64/mm/init.c
> >>>>> +++ b/arch/arm64/mm/init.c
> >>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >>>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >>>>>                        phys_initrd_size = 0;
> >>>>>                } else {
> >>>>> +                     int flags, ret;
> >>>>> +
> >>>>> +                     ret = memblock_get_flags(base, &flags);
> >>>>> +                     if (ret)
> >>>>> +                             flags = 0;
> >>>>> +
> >>>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >>>>>                        memblock_add(base, size);
> >>>>>                        memblock_reserve(base, size);
> >>>>
> >>>> Can you explain why we're removing+re-adding here exactly? Is it just to
> >>>> clear flags as the comment indicates?
> >>>>
> >>>
> >>> This should only happen if the placement of the initrd conflicts with
> >>> a mem= command line parameter or it is not covered by memblock for
> >>> some other reason.
> >>>
> >>> IOW, this should never happen, and if re-memblock_add'ing this memory
> >>> unconditionally is causing problems, we should fix that instead of
> >>> working around it.
> >>
> >> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> >> the 4G limit to test this scenario(just for testing, I have trouble to boot
> >> qemu with initrd enabled and memory below 4G are all mirror memory).
> >>
> >> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> >> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> >>
> >>>
> >>>> If it's really just about clearing flags, I wonder if we rather want to
> >>>> have an interface that does exactly that, and hides the way this is
> >>>> actually implemented (obtain flags, remove, re-add ...), internally.
> >>>>
> >>>> But most probably there is more magic in the code and clearing flags
> >>>> isn't all it ends up doing.
> >>>>
> >>>
> >>> I don't remember exactly why we needed to clear the flags, but I think
> >>> it had to do with some corner case we hit when the initrd was
> >>> partially covered.
> >> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> >> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> >> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> >>
> >> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> >> MEMBLOCK_NOMAP of this initrd memblock.
> >> So maybe the usage of memblock_remove() is just to avoid introducing new
> >> function(memblock_clear_nomap)?
> >>
> >> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> >> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> >> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> >
> > AFAICT, there are two corner cases that re-adding initrd memory covers:
> > * initrd memory is not a part of the memory reported to memblock, either
> > because of firmware weirdness or because it was cut out with mem=
> > * initrd memory overlaps a NOMAP region
> >
> > So to make sure initrd memory is mapped properly and retains
> > MEMBLOCK_MIRROR I think the best we can do is
> >
> >       memblock_add();
> >       memblock_clear_nomap();
> >       memblock_reserve();
>
> Would simply detect+rejecting to boot on such setups be an option? The
> replies so far indicate to me that this is rather a corner case than a
> reasonable use case.
>

The sad reality is that mem= is known to be used in production for
limiting the amount of memory that the kernel takes control of, in
order to allow the remainder to be used in platform specific ways.

Of course, there are much better ways to achieve that, but given that
we currently support it, I don't think we can easily back that out.

I do think that there is no need to go out of our way to make this
case work seamlessly with mirrored memory, though. So I'd prefer to
make the remove+re-add conditional on there actually being a need to
do so. That way, we don't break the old use case or mirrored memory,
and whatever happens when the two are combined is DONTCARE.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
>
> On 08.06.22 12:02, Mike Rapoport wrote:
> > On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> >>
> >> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> >>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> >>>>
> >>>> On 07.06.22 11:38, Wupeng Ma wrote:
> >>>>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>
> >>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
> >>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> >>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> >>>>> the lower 4G range has some non-mirrored memory.
> >>>>>
> >>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> >>>>> reinstalled if the origin memblock has this flag.
> >>>>>
> >>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> >>>>> ---
> >>>>>   arch/arm64/mm/init.c     |  9 +++++++++
> >>>>>   include/linux/memblock.h |  1 +
> >>>>>   mm/memblock.c            | 20 ++++++++++++++++++++
> >>>>>   3 files changed, 30 insertions(+)
> >>>>>
> >>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> >>>>> index 339ee84e5a61..11641f924d08 100644
> >>>>> --- a/arch/arm64/mm/init.c
> >>>>> +++ b/arch/arm64/mm/init.c
> >>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >>>>>                        "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >>>>>                        phys_initrd_size = 0;
> >>>>>                } else {
> >>>>> +                     int flags, ret;
> >>>>> +
> >>>>> +                     ret = memblock_get_flags(base, &flags);
> >>>>> +                     if (ret)
> >>>>> +                             flags = 0;
> >>>>> +
> >>>>>                        memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >>>>>                        memblock_add(base, size);
> >>>>>                        memblock_reserve(base, size);
> >>>>
> >>>> Can you explain why we're removing+re-adding here exactly? Is it just to
> >>>> clear flags as the comment indicates?
> >>>>
> >>>
> >>> This should only happen if the placement of the initrd conflicts with
> >>> a mem= command line parameter or it is not covered by memblock for
> >>> some other reason.
> >>>
> >>> IOW, this should never happen, and if re-memblock_add'ing this memory
> >>> unconditionally is causing problems, we should fix that instead of
> >>> working around it.
> >>
> >> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> >> the 4G limit to test this scenario(just for testing, I have trouble to boot
> >> qemu with initrd enabled and memory below 4G are all mirror memory).
> >>
> >> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> >> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> >>
> >>>
> >>>> If it's really just about clearing flags, I wonder if we rather want to
> >>>> have an interface that does exactly that, and hides the way this is
> >>>> actually implemented (obtain flags, remove, re-add ...), internally.
> >>>>
> >>>> But most probably there is more magic in the code and clearing flags
> >>>> isn't all it ends up doing.
> >>>>
> >>>
> >>> I don't remember exactly why we needed to clear the flags, but I think
> >>> it had to do with some corner case we hit when the initrd was
> >>> partially covered.
> >> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> >> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> >> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> >>
> >> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> >> MEMBLOCK_NOMAP of this initrd memblock.
> >> So maybe the usage of memblock_remove() is just to avoid introducing new
> >> function(memblock_clear_nomap)?
> >>
> >> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> >> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> >> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> >
> > AFAICT, there are two corner cases that re-adding initrd memory covers:
> > * initrd memory is not a part of the memory reported to memblock, either
> > because of firmware weirdness or because it was cut out with mem=
> > * initrd memory overlaps a NOMAP region
> >
> > So to make sure initrd memory is mapped properly and retains
> > MEMBLOCK_MIRROR I think the best we can do is
> >
> >       memblock_add();
> >       memblock_clear_nomap();
> >       memblock_reserve();
>
> Would simply detect+rejecting to boot on such setups be an option? The
> replies so far indicate to me that this is rather a corner case than a
> reasonable use case.
>

The sad reality is that mem= is known to be used in production for
limiting the amount of memory that the kernel takes control of, in
order to allow the remainder to be used in platform specific ways.

Of course, there are much better ways to achieve that, but given that
we currently support it, I don't think we can easily back that out.

I do think that there is no need to go out of our way to make this
case work seamlessly with mirrored memory, though. So I'd prefer to
make the remove+re-add conditional on there actually being a need to
do so. That way, we don't break the old use case or mirrored memory,
and whatever happens when the two are combined is DONTCARE.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[mawupeng]

在 2022/6/8 18:00, Anshuman Khandual 写道:
> 
> 
> On 6/8/22 06:56, mawupeng wrote:
>>
>>
>> 在 2022/6/7 20:25, David Hildenbrand 写道:
>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>
>>>> For a system only have limited mirrored memory or some numa node without
>>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>>> vmemmap_populate_basepages() report lots of warning message.
>>>>
>>>> This patch demote the "potential offnode page_structs" warning messages
>>>> to debug level to avoid a very long print during bootup.
>>>>
>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>> ---
>>>>    mm/sparse-vmemmap.c | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>>> index f4fa61dbbee3..78debdb89eb1 100644
>>>> --- a/mm/sparse-vmemmap.c
>>>> +++ b/mm/sparse-vmemmap.c
>>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>>>        int actual_node = early_pfn_to_nid(pfn);
>>>>          if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>>                start, end - 1);
>>>>    }
>>>>    
>>>
>>> This will possibly hide it in environments where this might indeed
>>> indicate performance issues.
>>>
>>> What about a pr_warn_once()?
>>>
>>
>> Sure.
>>
>> This will works. We can certainly use a pr_warn_once().
> 
> Why not pr_warn_ratelimited() like in the previous patch ?

Function vmemmap_populate_basepages() is used to populate base pages.
System with huge memory will produce lots lots of warning message
during this populate process even with ratelimited. This may lead to slow
startup.

Thanks for reviewing.

> .

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[mawupeng]

在 2022/6/8 18:00, Anshuman Khandual 写道:
> 
> 
> On 6/8/22 06:56, mawupeng wrote:
>>
>>
>> 在 2022/6/7 20:25, David Hildenbrand 写道:
>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>
>>>> For a system only have limited mirrored memory or some numa node without
>>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>>> vmemmap_populate_basepages() report lots of warning message.
>>>>
>>>> This patch demote the "potential offnode page_structs" warning messages
>>>> to debug level to avoid a very long print during bootup.
>>>>
>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>> ---
>>>>    mm/sparse-vmemmap.c | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>>> index f4fa61dbbee3..78debdb89eb1 100644
>>>> --- a/mm/sparse-vmemmap.c
>>>> +++ b/mm/sparse-vmemmap.c
>>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>>>        int actual_node = early_pfn_to_nid(pfn);
>>>>          if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>>                start, end - 1);
>>>>    }
>>>>    
>>>
>>> This will possibly hide it in environments where this might indeed
>>> indicate performance issues.
>>>
>>> What about a pr_warn_once()?
>>>
>>
>> Sure.
>>
>> This will works. We can certainly use a pr_warn_once().
> 
> Why not pr_warn_ratelimited() like in the previous patch ?

Function vmemmap_populate_basepages() is used to populate base pages.
System with huge memory will produce lots lots of warning message
during this populate process even with ratelimited. This may lead to slow
startup.

Thanks for reviewing.

> .

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[mawupeng]

在 2022/6/8 18:00, Anshuman Khandual 写道:
> 
> 
> On 6/8/22 06:56, mawupeng wrote:
>>
>>
>> 在 2022/6/7 20:25, David Hildenbrand 写道:
>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>
>>>> For a system only have limited mirrored memory or some numa node without
>>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>>> vmemmap_populate_basepages() report lots of warning message.
>>>>
>>>> This patch demote the "potential offnode page_structs" warning messages
>>>> to debug level to avoid a very long print during bootup.
>>>>
>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>> ---
>>>>    mm/sparse-vmemmap.c | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>>> index f4fa61dbbee3..78debdb89eb1 100644
>>>> --- a/mm/sparse-vmemmap.c
>>>> +++ b/mm/sparse-vmemmap.c
>>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>>>>        int actual_node = early_pfn_to_nid(pfn);
>>>>          if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>>                start, end - 1);
>>>>    }
>>>>    
>>>
>>> This will possibly hide it in environments where this might indeed
>>> indicate performance issues.
>>>
>>> What about a pr_warn_once()?
>>>
>>
>> Sure.
>>
>> This will works. We can certainly use a pr_warn_once().
> 
> Why not pr_warn_ratelimited() like in the previous patch ?

Function vmemmap_populate_basepages() is used to populate base pages.
System with huge memory will produce lots lots of warning message
during this populate process even with ratelimited. This may lead to slow
startup.

Thanks for reviewing.

> .

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[mawupeng]

在 2022/6/8 18:12, Ard Biesheuvel 写道:
> On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
>>
>> On 08.06.22 12:02, Mike Rapoport wrote:
>>> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
>>>>
>>>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
>>>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>>>>>
>>>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>>>
>>>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>>>>>> the lower 4G range has some non-mirrored memory.
>>>>>>>
>>>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>>>>>> reinstalled if the origin memblock has this flag.
>>>>>>>
>>>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>>>> ---
>>>>>>>    arch/arm64/mm/init.c     |  9 +++++++++
>>>>>>>    include/linux/memblock.h |  1 +
>>>>>>>    mm/memblock.c            | 20 ++++++++++++++++++++
>>>>>>>    3 files changed, 30 insertions(+)
>>>>>>>
>>>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>>>>>> index 339ee84e5a61..11641f924d08 100644
>>>>>>> --- a/arch/arm64/mm/init.c
>>>>>>> +++ b/arch/arm64/mm/init.c
>>>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>>>>>                         "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>>>>>                         phys_initrd_size = 0;
>>>>>>>                 } else {
>>>>>>> +                     int flags, ret;
>>>>>>> +
>>>>>>> +                     ret = memblock_get_flags(base, &flags);
>>>>>>> +                     if (ret)
>>>>>>> +                             flags = 0;
>>>>>>> +
>>>>>>>                         memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>>>>>                         memblock_add(base, size);
>>>>>>>                         memblock_reserve(base, size);
>>>>>>
>>>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
>>>>>> clear flags as the comment indicates?
>>>>>>
>>>>>
>>>>> This should only happen if the placement of the initrd conflicts with
>>>>> a mem= command line parameter or it is not covered by memblock for
>>>>> some other reason.
>>>>>
>>>>> IOW, this should never happen, and if re-memblock_add'ing this memory
>>>>> unconditionally is causing problems, we should fix that instead of
>>>>> working around it.
>>>>
>>>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
>>>> the 4G limit to test this scenario(just for testing, I have trouble to boot
>>>> qemu with initrd enabled and memory below 4G are all mirror memory).
>>>>
>>>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
>>>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
>>>>
>>>>>
>>>>>> If it's really just about clearing flags, I wonder if we rather want to
>>>>>> have an interface that does exactly that, and hides the way this is
>>>>>> actually implemented (obtain flags, remove, re-add ...), internally.
>>>>>>
>>>>>> But most probably there is more magic in the code and clearing flags
>>>>>> isn't all it ends up doing.
>>>>>>
>>>>>
>>>>> I don't remember exactly why we needed to clear the flags, but I think
>>>>> it had to do with some corner case we hit when the initrd was
>>>>> partially covered.
>>>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
>>>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
>>>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
>>>>
>>>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
>>>> MEMBLOCK_NOMAP of this initrd memblock.
>>>> So maybe the usage of memblock_remove() is just to avoid introducing new
>>>> function(memblock_clear_nomap)?
>>>>
>>>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
>>>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
>>>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
>>>
>>> AFAICT, there are two corner cases that re-adding initrd memory covers:
>>> * initrd memory is not a part of the memory reported to memblock, either
>>> because of firmware weirdness or because it was cut out with mem=
>>> * initrd memory overlaps a NOMAP region
>>>
>>> So to make sure initrd memory is mapped properly and retains
>>> MEMBLOCK_MIRROR I think the best we can do is
>>>
>>>        memblock_add();
>>>        memblock_clear_nomap();
>>>        memblock_reserve();
>>
>> Would simply detect+rejecting to boot on such setups be an option? The
>> replies so far indicate to me that this is rather a corner case than a
>> reasonable use case.
>>
> 
> The sad reality is that mem= is known to be used in production for
> limiting the amount of memory that the kernel takes control of, in
> order to allow the remainder to be used in platform specific ways.
> 
> Of course, there are much better ways to achieve that, but given that
> we currently support it, I don't think we can easily back that out.
> 
> I do think that there is no need to go out of our way to make this
> case work seamlessly with mirrored memory, though. So I'd prefer to
> make the remove+re-add conditional on there actually being a need to
> do so. That way, we don't break the old use case or mirrored memory,
> and whatever happens when the two are combined is DONTCARE.

Does that mean that we don't need to care about this scenario with
mirror memory?

Thanks for reviewing.

> .

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[mawupeng]

在 2022/6/8 18:12, Ard Biesheuvel 写道:
> On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
>>
>> On 08.06.22 12:02, Mike Rapoport wrote:
>>> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
>>>>
>>>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
>>>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>>>>>
>>>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>>>
>>>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>>>>>> the lower 4G range has some non-mirrored memory.
>>>>>>>
>>>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>>>>>> reinstalled if the origin memblock has this flag.
>>>>>>>
>>>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>>>> ---
>>>>>>>    arch/arm64/mm/init.c     |  9 +++++++++
>>>>>>>    include/linux/memblock.h |  1 +
>>>>>>>    mm/memblock.c            | 20 ++++++++++++++++++++
>>>>>>>    3 files changed, 30 insertions(+)
>>>>>>>
>>>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>>>>>> index 339ee84e5a61..11641f924d08 100644
>>>>>>> --- a/arch/arm64/mm/init.c
>>>>>>> +++ b/arch/arm64/mm/init.c
>>>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>>>>>                         "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>>>>>                         phys_initrd_size = 0;
>>>>>>>                 } else {
>>>>>>> +                     int flags, ret;
>>>>>>> +
>>>>>>> +                     ret = memblock_get_flags(base, &flags);
>>>>>>> +                     if (ret)
>>>>>>> +                             flags = 0;
>>>>>>> +
>>>>>>>                         memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>>>>>                         memblock_add(base, size);
>>>>>>>                         memblock_reserve(base, size);
>>>>>>
>>>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
>>>>>> clear flags as the comment indicates?
>>>>>>
>>>>>
>>>>> This should only happen if the placement of the initrd conflicts with
>>>>> a mem= command line parameter or it is not covered by memblock for
>>>>> some other reason.
>>>>>
>>>>> IOW, this should never happen, and if re-memblock_add'ing this memory
>>>>> unconditionally is causing problems, we should fix that instead of
>>>>> working around it.
>>>>
>>>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
>>>> the 4G limit to test this scenario(just for testing, I have trouble to boot
>>>> qemu with initrd enabled and memory below 4G are all mirror memory).
>>>>
>>>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
>>>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
>>>>
>>>>>
>>>>>> If it's really just about clearing flags, I wonder if we rather want to
>>>>>> have an interface that does exactly that, and hides the way this is
>>>>>> actually implemented (obtain flags, remove, re-add ...), internally.
>>>>>>
>>>>>> But most probably there is more magic in the code and clearing flags
>>>>>> isn't all it ends up doing.
>>>>>>
>>>>>
>>>>> I don't remember exactly why we needed to clear the flags, but I think
>>>>> it had to do with some corner case we hit when the initrd was
>>>>> partially covered.
>>>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
>>>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
>>>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
>>>>
>>>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
>>>> MEMBLOCK_NOMAP of this initrd memblock.
>>>> So maybe the usage of memblock_remove() is just to avoid introducing new
>>>> function(memblock_clear_nomap)?
>>>>
>>>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
>>>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
>>>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
>>>
>>> AFAICT, there are two corner cases that re-adding initrd memory covers:
>>> * initrd memory is not a part of the memory reported to memblock, either
>>> because of firmware weirdness or because it was cut out with mem=
>>> * initrd memory overlaps a NOMAP region
>>>
>>> So to make sure initrd memory is mapped properly and retains
>>> MEMBLOCK_MIRROR I think the best we can do is
>>>
>>>        memblock_add();
>>>        memblock_clear_nomap();
>>>        memblock_reserve();
>>
>> Would simply detect+rejecting to boot on such setups be an option? The
>> replies so far indicate to me that this is rather a corner case than a
>> reasonable use case.
>>
> 
> The sad reality is that mem= is known to be used in production for
> limiting the amount of memory that the kernel takes control of, in
> order to allow the remainder to be used in platform specific ways.
> 
> Of course, there are much better ways to achieve that, but given that
> we currently support it, I don't think we can easily back that out.
> 
> I do think that there is no need to go out of our way to make this
> case work seamlessly with mirrored memory, though. So I'd prefer to
> make the remove+re-add conditional on there actually being a need to
> do so. That way, we don't break the old use case or mirrored memory,
> and whatever happens when the two are combined is DONTCARE.

Does that mean that we don't need to care about this scenario with
mirror memory?

Thanks for reviewing.

> .

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[mawupeng]

在 2022/6/8 18:12, Ard Biesheuvel 写道:
> On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
>>
>> On 08.06.22 12:02, Mike Rapoport wrote:
>>> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
>>>>
>>>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
>>>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
>>>>>>
>>>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>>>
>>>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
>>>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
>>>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
>>>>>>> the lower 4G range has some non-mirrored memory.
>>>>>>>
>>>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
>>>>>>> reinstalled if the origin memblock has this flag.
>>>>>>>
>>>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>>>> ---
>>>>>>>    arch/arm64/mm/init.c     |  9 +++++++++
>>>>>>>    include/linux/memblock.h |  1 +
>>>>>>>    mm/memblock.c            | 20 ++++++++++++++++++++
>>>>>>>    3 files changed, 30 insertions(+)
>>>>>>>
>>>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
>>>>>>> index 339ee84e5a61..11641f924d08 100644
>>>>>>> --- a/arch/arm64/mm/init.c
>>>>>>> +++ b/arch/arm64/mm/init.c
>>>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
>>>>>>>                         "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
>>>>>>>                         phys_initrd_size = 0;
>>>>>>>                 } else {
>>>>>>> +                     int flags, ret;
>>>>>>> +
>>>>>>> +                     ret = memblock_get_flags(base, &flags);
>>>>>>> +                     if (ret)
>>>>>>> +                             flags = 0;
>>>>>>> +
>>>>>>>                         memblock_remove(base, size); /* clear MEMBLOCK_ flags */
>>>>>>>                         memblock_add(base, size);
>>>>>>>                         memblock_reserve(base, size);
>>>>>>
>>>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
>>>>>> clear flags as the comment indicates?
>>>>>>
>>>>>
>>>>> This should only happen if the placement of the initrd conflicts with
>>>>> a mem= command line parameter or it is not covered by memblock for
>>>>> some other reason.
>>>>>
>>>>> IOW, this should never happen, and if re-memblock_add'ing this memory
>>>>> unconditionally is causing problems, we should fix that instead of
>>>>> working around it.
>>>>
>>>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
>>>> the 4G limit to test this scenario(just for testing, I have trouble to boot
>>>> qemu with initrd enabled and memory below 4G are all mirror memory).
>>>>
>>>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
>>>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
>>>>
>>>>>
>>>>>> If it's really just about clearing flags, I wonder if we rather want to
>>>>>> have an interface that does exactly that, and hides the way this is
>>>>>> actually implemented (obtain flags, remove, re-add ...), internally.
>>>>>>
>>>>>> But most probably there is more magic in the code and clearing flags
>>>>>> isn't all it ends up doing.
>>>>>>
>>>>>
>>>>> I don't remember exactly why we needed to clear the flags, but I think
>>>>> it had to do with some corner case we hit when the initrd was
>>>>> partially covered.
>>>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
>>>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
>>>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
>>>>
>>>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
>>>> MEMBLOCK_NOMAP of this initrd memblock.
>>>> So maybe the usage of memblock_remove() is just to avoid introducing new
>>>> function(memblock_clear_nomap)?
>>>>
>>>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
>>>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
>>>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
>>>
>>> AFAICT, there are two corner cases that re-adding initrd memory covers:
>>> * initrd memory is not a part of the memory reported to memblock, either
>>> because of firmware weirdness or because it was cut out with mem=
>>> * initrd memory overlaps a NOMAP region
>>>
>>> So to make sure initrd memory is mapped properly and retains
>>> MEMBLOCK_MIRROR I think the best we can do is
>>>
>>>        memblock_add();
>>>        memblock_clear_nomap();
>>>        memblock_reserve();
>>
>> Would simply detect+rejecting to boot on such setups be an option? The
>> replies so far indicate to me that this is rather a corner case than a
>> reasonable use case.
>>
> 
> The sad reality is that mem= is known to be used in production for
> limiting the amount of memory that the kernel takes control of, in
> order to allow the remainder to be used in platform specific ways.
> 
> Of course, there are much better ways to achieve that, but given that
> we currently support it, I don't think we can easily back that out.
> 
> I do think that there is no need to go out of our way to make this
> case work seamlessly with mirrored memory, though. So I'd prefer to
> make the remove+re-add conditional on there actually being a need to
> do so. That way, we don't break the old use case or mirrored memory,
> and whatever happens when the two are combined is DONTCARE.

Does that mean that we don't need to care about this scenario with
mirror memory?

Thanks for reviewing.

> .

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 1/6] efi: Make efi_find_mirror() public

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory
> ranges") introduce the efi_find_mirror function on x86. In order to reuse
> the API we make it public in preparation for arm64 to support mirrord
> memory.
Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   arch/x86/include/asm/efi.h  |  4 ----
>   arch/x86/platform/efi/efi.c | 23 -----------------------
>   drivers/firmware/efi/efi.c  | 23 +++++++++++++++++++++++
>   include/linux/efi.h         |  3 +++
>   4 files changed, 26 insertions(+), 27 deletions(-)
>
> diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
> index 71943dce691e..eb90206eae80 100644
> --- a/arch/x86/include/asm/efi.h
> +++ b/arch/x86/include/asm/efi.h
> @@ -383,7 +383,6 @@ static inline bool efi_is_64bit(void)
>   extern bool efi_reboot_required(void);
>   extern bool efi_is_table_address(unsigned long phys_addr);
>   
> -extern void efi_find_mirror(void);
>   extern void efi_reserve_boot_services(void);
>   #else
>   static inline void parse_efi_setup(u64 phys_addr, u32 data_len) {}
> @@ -395,9 +394,6 @@ static inline  bool efi_is_table_address(unsigned long phys_addr)
>   {
>   	return false;
>   }
> -static inline void efi_find_mirror(void)
> -{
> -}
>   static inline void efi_reserve_boot_services(void)
>   {
>   }
> diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
> index 1591d67e0bcd..6e598bd78eef 100644
> --- a/arch/x86/platform/efi/efi.c
> +++ b/arch/x86/platform/efi/efi.c
> @@ -108,29 +108,6 @@ static int __init setup_add_efi_memmap(char *arg)
>   }
>   early_param("add_efi_memmap", setup_add_efi_memmap);
>   
> -void __init efi_find_mirror(void)
> -{
> -	efi_memory_desc_t *md;
> -	u64 mirror_size = 0, total_size = 0;
> -
> -	if (!efi_enabled(EFI_MEMMAP))
> -		return;
> -
> -	for_each_efi_memory_desc(md) {
> -		unsigned long long start = md->phys_addr;
> -		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> -
> -		total_size += size;
> -		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
> -			memblock_mark_mirror(start, size);
> -			mirror_size += size;
> -		}
> -	}
> -	if (mirror_size)
> -		pr_info("Memory: %lldM/%lldM mirrored memory\n",
> -			mirror_size>>20, total_size>>20);
> -}
> -
>   /*
>    * Tell the kernel about the EFI memory map.  This might include
>    * more than the max 128 entries that can fit in the passed in e820
> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
> index 860534bcfdac..79c232e07de7 100644
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -446,6 +446,29 @@ static int __init efisubsys_init(void)
>   
>   subsys_initcall(efisubsys_init);
>   
> +void __init efi_find_mirror(void)
> +{
> +	efi_memory_desc_t *md;
> +	u64 mirror_size = 0, total_size = 0;
> +
> +	if (!efi_enabled(EFI_MEMMAP))
> +		return;
> +
> +	for_each_efi_memory_desc(md) {
> +		unsigned long long start = md->phys_addr;
> +		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> +
> +		total_size += size;
> +		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
> +			memblock_mark_mirror(start, size);
> +			mirror_size += size;
> +		}
> +	}
> +	if (mirror_size)
> +		pr_info("Memory: %lldM/%lldM mirrored memory\n",
> +			mirror_size>>20, total_size>>20);
> +}
> +
>   /*
>    * Find the efi memory descriptor for a given physical address.  Given a
>    * physical address, determine if it exists within an EFI Memory Map entry,
> diff --git a/include/linux/efi.h b/include/linux/efi.h
> index 7d9b0bb47eb3..53f64c14a525 100644
> --- a/include/linux/efi.h
> +++ b/include/linux/efi.h
> @@ -872,6 +872,7 @@ static inline bool efi_rt_services_supported(unsigned int mask)
>   {
>   	return (efi.runtime_supported_mask & mask) == mask;
>   }
> +extern void efi_find_mirror(void);
>   #else
>   static inline bool efi_enabled(int feature)
>   {
> @@ -889,6 +890,8 @@ static inline bool efi_rt_services_supported(unsigned int mask)
>   {
>   	return false;
>   }
> +
> +static inline void efi_find_mirror(void) {}
>   #endif
>   
>   extern int efi_status_to_err(efi_status_t status);

Re: [PATCH v3 1/6] efi: Make efi_find_mirror() public

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory
> ranges") introduce the efi_find_mirror function on x86. In order to reuse
> the API we make it public in preparation for arm64 to support mirrord
> memory.
Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   arch/x86/include/asm/efi.h  |  4 ----
>   arch/x86/platform/efi/efi.c | 23 -----------------------
>   drivers/firmware/efi/efi.c  | 23 +++++++++++++++++++++++
>   include/linux/efi.h         |  3 +++
>   4 files changed, 26 insertions(+), 27 deletions(-)
>
> diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
> index 71943dce691e..eb90206eae80 100644
> --- a/arch/x86/include/asm/efi.h
> +++ b/arch/x86/include/asm/efi.h
> @@ -383,7 +383,6 @@ static inline bool efi_is_64bit(void)
>   extern bool efi_reboot_required(void);
>   extern bool efi_is_table_address(unsigned long phys_addr);
>   
> -extern void efi_find_mirror(void);
>   extern void efi_reserve_boot_services(void);
>   #else
>   static inline void parse_efi_setup(u64 phys_addr, u32 data_len) {}
> @@ -395,9 +394,6 @@ static inline  bool efi_is_table_address(unsigned long phys_addr)
>   {
>   	return false;
>   }
> -static inline void efi_find_mirror(void)
> -{
> -}
>   static inline void efi_reserve_boot_services(void)
>   {
>   }
> diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
> index 1591d67e0bcd..6e598bd78eef 100644
> --- a/arch/x86/platform/efi/efi.c
> +++ b/arch/x86/platform/efi/efi.c
> @@ -108,29 +108,6 @@ static int __init setup_add_efi_memmap(char *arg)
>   }
>   early_param("add_efi_memmap", setup_add_efi_memmap);
>   
> -void __init efi_find_mirror(void)
> -{
> -	efi_memory_desc_t *md;
> -	u64 mirror_size = 0, total_size = 0;
> -
> -	if (!efi_enabled(EFI_MEMMAP))
> -		return;
> -
> -	for_each_efi_memory_desc(md) {
> -		unsigned long long start = md->phys_addr;
> -		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> -
> -		total_size += size;
> -		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
> -			memblock_mark_mirror(start, size);
> -			mirror_size += size;
> -		}
> -	}
> -	if (mirror_size)
> -		pr_info("Memory: %lldM/%lldM mirrored memory\n",
> -			mirror_size>>20, total_size>>20);
> -}
> -
>   /*
>    * Tell the kernel about the EFI memory map.  This might include
>    * more than the max 128 entries that can fit in the passed in e820
> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
> index 860534bcfdac..79c232e07de7 100644
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -446,6 +446,29 @@ static int __init efisubsys_init(void)
>   
>   subsys_initcall(efisubsys_init);
>   
> +void __init efi_find_mirror(void)
> +{
> +	efi_memory_desc_t *md;
> +	u64 mirror_size = 0, total_size = 0;
> +
> +	if (!efi_enabled(EFI_MEMMAP))
> +		return;
> +
> +	for_each_efi_memory_desc(md) {
> +		unsigned long long start = md->phys_addr;
> +		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> +
> +		total_size += size;
> +		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
> +			memblock_mark_mirror(start, size);
> +			mirror_size += size;
> +		}
> +	}
> +	if (mirror_size)
> +		pr_info("Memory: %lldM/%lldM mirrored memory\n",
> +			mirror_size>>20, total_size>>20);
> +}
> +
>   /*
>    * Find the efi memory descriptor for a given physical address.  Given a
>    * physical address, determine if it exists within an EFI Memory Map entry,
> diff --git a/include/linux/efi.h b/include/linux/efi.h
> index 7d9b0bb47eb3..53f64c14a525 100644
> --- a/include/linux/efi.h
> +++ b/include/linux/efi.h
> @@ -872,6 +872,7 @@ static inline bool efi_rt_services_supported(unsigned int mask)
>   {
>   	return (efi.runtime_supported_mask & mask) == mask;
>   }
> +extern void efi_find_mirror(void);
>   #else
>   static inline bool efi_enabled(int feature)
>   {
> @@ -889,6 +890,8 @@ static inline bool efi_rt_services_supported(unsigned int mask)
>   {
>   	return false;
>   }
> +
> +static inline void efi_find_mirror(void) {}
>   #endif
>   
>   extern int efi_status_to_err(efi_status_t status);

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 1/6] efi: Make efi_find_mirror() public

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory
> ranges") introduce the efi_find_mirror function on x86. In order to reuse
> the API we make it public in preparation for arm64 to support mirrord
> memory.
Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   arch/x86/include/asm/efi.h  |  4 ----
>   arch/x86/platform/efi/efi.c | 23 -----------------------
>   drivers/firmware/efi/efi.c  | 23 +++++++++++++++++++++++
>   include/linux/efi.h         |  3 +++
>   4 files changed, 26 insertions(+), 27 deletions(-)
>
> diff --git a/arch/x86/include/asm/efi.h b/arch/x86/include/asm/efi.h
> index 71943dce691e..eb90206eae80 100644
> --- a/arch/x86/include/asm/efi.h
> +++ b/arch/x86/include/asm/efi.h
> @@ -383,7 +383,6 @@ static inline bool efi_is_64bit(void)
>   extern bool efi_reboot_required(void);
>   extern bool efi_is_table_address(unsigned long phys_addr);
>   
> -extern void efi_find_mirror(void);
>   extern void efi_reserve_boot_services(void);
>   #else
>   static inline void parse_efi_setup(u64 phys_addr, u32 data_len) {}
> @@ -395,9 +394,6 @@ static inline  bool efi_is_table_address(unsigned long phys_addr)
>   {
>   	return false;
>   }
> -static inline void efi_find_mirror(void)
> -{
> -}
>   static inline void efi_reserve_boot_services(void)
>   {
>   }
> diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c
> index 1591d67e0bcd..6e598bd78eef 100644
> --- a/arch/x86/platform/efi/efi.c
> +++ b/arch/x86/platform/efi/efi.c
> @@ -108,29 +108,6 @@ static int __init setup_add_efi_memmap(char *arg)
>   }
>   early_param("add_efi_memmap", setup_add_efi_memmap);
>   
> -void __init efi_find_mirror(void)
> -{
> -	efi_memory_desc_t *md;
> -	u64 mirror_size = 0, total_size = 0;
> -
> -	if (!efi_enabled(EFI_MEMMAP))
> -		return;
> -
> -	for_each_efi_memory_desc(md) {
> -		unsigned long long start = md->phys_addr;
> -		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> -
> -		total_size += size;
> -		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
> -			memblock_mark_mirror(start, size);
> -			mirror_size += size;
> -		}
> -	}
> -	if (mirror_size)
> -		pr_info("Memory: %lldM/%lldM mirrored memory\n",
> -			mirror_size>>20, total_size>>20);
> -}
> -
>   /*
>    * Tell the kernel about the EFI memory map.  This might include
>    * more than the max 128 entries that can fit in the passed in e820
> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
> index 860534bcfdac..79c232e07de7 100644
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -446,6 +446,29 @@ static int __init efisubsys_init(void)
>   
>   subsys_initcall(efisubsys_init);
>   
> +void __init efi_find_mirror(void)
> +{
> +	efi_memory_desc_t *md;
> +	u64 mirror_size = 0, total_size = 0;
> +
> +	if (!efi_enabled(EFI_MEMMAP))
> +		return;
> +
> +	for_each_efi_memory_desc(md) {
> +		unsigned long long start = md->phys_addr;
> +		unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> +
> +		total_size += size;
> +		if (md->attribute & EFI_MEMORY_MORE_RELIABLE) {
> +			memblock_mark_mirror(start, size);
> +			mirror_size += size;
> +		}
> +	}
> +	if (mirror_size)
> +		pr_info("Memory: %lldM/%lldM mirrored memory\n",
> +			mirror_size>>20, total_size>>20);
> +}
> +
>   /*
>    * Find the efi memory descriptor for a given physical address.  Given a
>    * physical address, determine if it exists within an EFI Memory Map entry,
> diff --git a/include/linux/efi.h b/include/linux/efi.h
> index 7d9b0bb47eb3..53f64c14a525 100644
> --- a/include/linux/efi.h
> +++ b/include/linux/efi.h
> @@ -872,6 +872,7 @@ static inline bool efi_rt_services_supported(unsigned int mask)
>   {
>   	return (efi.runtime_supported_mask & mask) == mask;
>   }
> +extern void efi_find_mirror(void);
>   #else
>   static inline bool efi_enabled(int feature)
>   {
> @@ -889,6 +890,8 @@ static inline bool efi_rt_services_supported(unsigned int mask)
>   {
>   	return false;
>   }
> +
> +static inline void efi_find_mirror(void) {}
>   #endif
>   
>   extern int efi_status_to_err(efi_status_t status);

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> introduced mirrored memory support for x86 and this could be used on arm64.
>
> Since we only support this feature on arm64, efi_find_mirror() won't be placed
> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> to scan the memory map and mark mirrored memory in memblock.

Move into efi_init() looks better, it won't bring negative effects on 
arm/riscv.

but let's maintainer to make a decision.

Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>

>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   Documentation/admin-guide/kernel-parameters.txt | 2 +-
>   arch/arm64/kernel/setup.c                       | 1 +
>   2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 8090130b544b..e3537646b6f7 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -2301,7 +2301,7 @@
>   
>   	keepinitrd	[HW,ARM]
>   
> -	kernelcore=	[KNL,X86,IA-64,PPC]
> +	kernelcore=	[KNL,X86,IA-64,PPC,ARM64]
>   			Format: nn[KMGTPE] | nn% | "mirror"
>   			This parameter specifies the amount of memory usable by
>   			the kernel for non-movable allocations.  The requested
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index cf3a759f10d4..6e9acd7ecf0f 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>   
>   	xen_early_init();
>   	efi_init();
> +	efi_find_mirror();
>   
>   	if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
>   	     pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> introduced mirrored memory support for x86 and this could be used on arm64.
>
> Since we only support this feature on arm64, efi_find_mirror() won't be placed
> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> to scan the memory map and mark mirrored memory in memblock.

Move into efi_init() looks better, it won't bring negative effects on 
arm/riscv.

but let's maintainer to make a decision.

Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>

>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   Documentation/admin-guide/kernel-parameters.txt | 2 +-
>   arch/arm64/kernel/setup.c                       | 1 +
>   2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 8090130b544b..e3537646b6f7 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -2301,7 +2301,7 @@
>   
>   	keepinitrd	[HW,ARM]
>   
> -	kernelcore=	[KNL,X86,IA-64,PPC]
> +	kernelcore=	[KNL,X86,IA-64,PPC,ARM64]
>   			Format: nn[KMGTPE] | nn% | "mirror"
>   			This parameter specifies the amount of memory usable by
>   			the kernel for non-movable allocations.  The requested
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index cf3a759f10d4..6e9acd7ecf0f 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>   
>   	xen_early_init();
>   	efi_init();
> +	efi_find_mirror();
>   
>   	if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
>   	     pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> introduced mirrored memory support for x86 and this could be used on arm64.
>
> Since we only support this feature on arm64, efi_find_mirror() won't be placed
> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> to scan the memory map and mark mirrored memory in memblock.

Move into efi_init() looks better, it won't bring negative effects on 
arm/riscv.

but let's maintainer to make a decision.

Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>

>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   Documentation/admin-guide/kernel-parameters.txt | 2 +-
>   arch/arm64/kernel/setup.c                       | 1 +
>   2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 8090130b544b..e3537646b6f7 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -2301,7 +2301,7 @@
>   
>   	keepinitrd	[HW,ARM]
>   
> -	kernelcore=	[KNL,X86,IA-64,PPC]
> +	kernelcore=	[KNL,X86,IA-64,PPC,ARM64]
>   			Format: nn[KMGTPE] | nn% | "mirror"
>   			This parameter specifies the amount of memory usable by
>   			the kernel for non-movable allocations.  The requested
> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> index cf3a759f10d4..6e9acd7ecf0f 100644
> --- a/arch/arm64/kernel/setup.c
> +++ b/arch/arm64/kernel/setup.c
> @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>   
>   	xen_early_init();
>   	efi_init();
> +	efi_find_mirror();
>   
>   	if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
>   	     pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
>
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   mm/memblock.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>   					    NUMA_NO_NODE, flags);
>   
>   	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>   			&size);
>   		flags &= ~MEMBLOCK_MIRROR;
>   		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>   
>   	if (flags & MEMBLOCK_MIRROR) {
>   		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>   			&size);
>   		goto again;
>   	}
Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
>
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   mm/memblock.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>   					    NUMA_NO_NODE, flags);
>   
>   	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>   			&size);
>   		flags &= ~MEMBLOCK_MIRROR;
>   		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>   
>   	if (flags & MEMBLOCK_MIRROR) {
>   		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>   			&size);
>   		goto again;
>   	}
Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 3/6] mm: Ratelimited mirrored memory related warning messages

[Kefeng Wang]

On 2022/6/7 17:38, Wupeng Ma wrote:
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> If system has mirrored memory, memblock will try to allocate mirrored
> memory firstly and fallback to non-mirrored memory when fails, but if with
> limited mirrored memory or some numa node without mirrored memory, lots of
> warning message about memblock allocation will occur.
>
> This patch ratelimit the warning message to avoid a very long print during
> bootup.
>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> ---
>   mm/memblock.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/memblock.c b/mm/memblock.c
> index e4f03a6e8e56..b1d2a0009733 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -327,7 +327,7 @@ static phys_addr_t __init_memblock memblock_find_in_range(phys_addr_t start,
>   					    NUMA_NO_NODE, flags);
>   
>   	if (!ret && (flags & MEMBLOCK_MIRROR)) {
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>   			&size);
>   		flags &= ~MEMBLOCK_MIRROR;
>   		goto again;
> @@ -1384,7 +1384,7 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>   
>   	if (flags & MEMBLOCK_MIRROR) {
>   		flags &= ~MEMBLOCK_MIRROR;
> -		pr_warn("Could not allocate %pap bytes of mirrored memory\n",
> +		pr_warn_ratelimited("Could not allocate %pap bytes of mirrored memory\n",
>   			&size);
>   		goto again;
>   	}
Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
>
> On 2022/6/7 17:38, Wupeng Ma wrote:
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86 and this could be used on arm64.
> >
> > Since we only support this feature on arm64, efi_find_mirror() won't be placed
> > into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> > to scan the memory map and mark mirrored memory in memblock.
>
> Move into efi_init() looks better, it won't bring negative effects on
> arm/riscv.
>
> but let's maintainer to make a decision.
>
> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>

I don't disagree with that in principle, but x86 calls the efi fake
memmap stuff between efi_init() and efi_find_mirror(), and I already
indicated that I don't want to enable fake memmap on !X86.

But I do think there is some room for improvement here: we could move
things like efi_mokvar_table_init() and efi_esrt_init() into
efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
we can move it into efi_init() too.

> >
> > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > ---
> >   Documentation/admin-guide/kernel-parameters.txt | 2 +-
> >   arch/arm64/kernel/setup.c                       | 1 +
> >   2 files changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 8090130b544b..e3537646b6f7 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -2301,7 +2301,7 @@
> >
> >       keepinitrd      [HW,ARM]
> >
> > -     kernelcore=     [KNL,X86,IA-64,PPC]
> > +     kernelcore=     [KNL,X86,IA-64,PPC,ARM64]
> >                       Format: nn[KMGTPE] | nn% | "mirror"
> >                       This parameter specifies the amount of memory usable by
> >                       the kernel for non-movable allocations.  The requested
> > diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> > index cf3a759f10d4..6e9acd7ecf0f 100644
> > --- a/arch/arm64/kernel/setup.c
> > +++ b/arch/arm64/kernel/setup.c
> > @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
> >
> >       xen_early_init();
> >       efi_init();
> > +     efi_find_mirror();
> >
> >       if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
> >            pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
>
> On 2022/6/7 17:38, Wupeng Ma wrote:
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86 and this could be used on arm64.
> >
> > Since we only support this feature on arm64, efi_find_mirror() won't be placed
> > into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> > to scan the memory map and mark mirrored memory in memblock.
>
> Move into efi_init() looks better, it won't bring negative effects on
> arm/riscv.
>
> but let's maintainer to make a decision.
>
> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>

I don't disagree with that in principle, but x86 calls the efi fake
memmap stuff between efi_init() and efi_find_mirror(), and I already
indicated that I don't want to enable fake memmap on !X86.

But I do think there is some room for improvement here: we could move
things like efi_mokvar_table_init() and efi_esrt_init() into
efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
we can move it into efi_init() too.

> >
> > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > ---
> >   Documentation/admin-guide/kernel-parameters.txt | 2 +-
> >   arch/arm64/kernel/setup.c                       | 1 +
> >   2 files changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 8090130b544b..e3537646b6f7 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -2301,7 +2301,7 @@
> >
> >       keepinitrd      [HW,ARM]
> >
> > -     kernelcore=     [KNL,X86,IA-64,PPC]
> > +     kernelcore=     [KNL,X86,IA-64,PPC,ARM64]
> >                       Format: nn[KMGTPE] | nn% | "mirror"
> >                       This parameter specifies the amount of memory usable by
> >                       the kernel for non-movable allocations.  The requested
> > diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> > index cf3a759f10d4..6e9acd7ecf0f 100644
> > --- a/arch/arm64/kernel/setup.c
> > +++ b/arch/arm64/kernel/setup.c
> > @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
> >
> >       xen_early_init();
> >       efi_init();
> > +     efi_find_mirror();
> >
> >       if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
> >            pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
>
> On 2022/6/7 17:38, Wupeng Ma wrote:
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86 and this could be used on arm64.
> >
> > Since we only support this feature on arm64, efi_find_mirror() won't be placed
> > into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> > to scan the memory map and mark mirrored memory in memblock.
>
> Move into efi_init() looks better, it won't bring negative effects on
> arm/riscv.
>
> but let's maintainer to make a decision.
>
> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>

I don't disagree with that in principle, but x86 calls the efi fake
memmap stuff between efi_init() and efi_find_mirror(), and I already
indicated that I don't want to enable fake memmap on !X86.

But I do think there is some room for improvement here: we could move
things like efi_mokvar_table_init() and efi_esrt_init() into
efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
we can move it into efi_init() too.

> >
> > Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> > ---
> >   Documentation/admin-guide/kernel-parameters.txt | 2 +-
> >   arch/arm64/kernel/setup.c                       | 1 +
> >   2 files changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 8090130b544b..e3537646b6f7 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -2301,7 +2301,7 @@
> >
> >       keepinitrd      [HW,ARM]
> >
> > -     kernelcore=     [KNL,X86,IA-64,PPC]
> > +     kernelcore=     [KNL,X86,IA-64,PPC,ARM64]
> >                       Format: nn[KMGTPE] | nn% | "mirror"
> >                       This parameter specifies the amount of memory usable by
> >                       the kernel for non-movable allocations.  The requested
> > diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
> > index cf3a759f10d4..6e9acd7ecf0f 100644
> > --- a/arch/arm64/kernel/setup.c
> > +++ b/arch/arm64/kernel/setup.c
> > @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
> >
> >       xen_early_init();
> >       efi_init();
> > +     efi_find_mirror();
> >
> >       if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
> >            pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Kefeng Wang]

On 2022/6/9 16:13, mawupeng wrote:
>
>
> 在 2022/6/8 18:00, Anshuman Khandual 写道:
>>
>>
>> On 6/8/22 06:56, mawupeng wrote:
>>>
>>>
>>> 在 2022/6/7 20:25, David Hildenbrand 写道:
>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>
>>>>> For a system only have limited mirrored memory or some numa node 
>>>>> without
>>>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>>>> vmemmap_populate_basepages() report lots of warning message.
>>>>>
>>>>> This patch demote the "potential offnode page_structs" warning 
>>>>> messages
>>>>> to debug level to avoid a very long print during bootup.
>>>>>
>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>> ---
>>>>>    mm/sparse-vmemmap.c | 2 +-
>>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>>>> index f4fa61dbbee3..78debdb89eb1 100644
>>>>> --- a/mm/sparse-vmemmap.c
>>>>> +++ b/mm/sparse-vmemmap.c
>>>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int 
>>>>> node,
>>>>>        int actual_node = early_pfn_to_nid(pfn);
>>>>>          if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>>>                start, end - 1);
>>>>>    }
>>>>
>>>> This will possibly hide it in environments where this might indeed
>>>> indicate performance issues.
>>>>
>>>> What about a pr_warn_once()?
>>>>
>>>
>>> Sure.
>>>
>>> This will works. We can certainly use a pr_warn_once().
>>
>> Why not pr_warn_ratelimited() like in the previous patch ?
>
> Function vmemmap_populate_basepages() is used to populate base pages.
> System with huge memory will produce lots lots of warning message
> during this populate process even with ratelimited. This may lead to slow
> startup.

I think pr_warn_once is better, the memblock_alloc fallback is not frequent,

but vmemmap_verify will verify each memory and print a lot.

>
> Thanks for reviewing.
>
>> .
> .

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Kefeng Wang]

On 2022/6/9 16:13, mawupeng wrote:
>
>
> 在 2022/6/8 18:00, Anshuman Khandual 写道:
>>
>>
>> On 6/8/22 06:56, mawupeng wrote:
>>>
>>>
>>> 在 2022/6/7 20:25, David Hildenbrand 写道:
>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>
>>>>> For a system only have limited mirrored memory or some numa node 
>>>>> without
>>>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>>>> vmemmap_populate_basepages() report lots of warning message.
>>>>>
>>>>> This patch demote the "potential offnode page_structs" warning 
>>>>> messages
>>>>> to debug level to avoid a very long print during bootup.
>>>>>
>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>> ---
>>>>>    mm/sparse-vmemmap.c | 2 +-
>>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>>>> index f4fa61dbbee3..78debdb89eb1 100644
>>>>> --- a/mm/sparse-vmemmap.c
>>>>> +++ b/mm/sparse-vmemmap.c
>>>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int 
>>>>> node,
>>>>>        int actual_node = early_pfn_to_nid(pfn);
>>>>>          if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>>>                start, end - 1);
>>>>>    }
>>>>
>>>> This will possibly hide it in environments where this might indeed
>>>> indicate performance issues.
>>>>
>>>> What about a pr_warn_once()?
>>>>
>>>
>>> Sure.
>>>
>>> This will works. We can certainly use a pr_warn_once().
>>
>> Why not pr_warn_ratelimited() like in the previous patch ?
>
> Function vmemmap_populate_basepages() is used to populate base pages.
> System with huge memory will produce lots lots of warning message
> during this populate process even with ratelimited. This may lead to slow
> startup.

I think pr_warn_once is better, the memblock_alloc fallback is not frequent,

but vmemmap_verify will verify each memory and print a lot.

>
> Thanks for reviewing.
>
>> .
> .

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 4/6] mm: Demote warning message in vmemmap_verify() to debug level

[Kefeng Wang]

On 2022/6/9 16:13, mawupeng wrote:
>
>
> 在 2022/6/8 18:00, Anshuman Khandual 写道:
>>
>>
>> On 6/8/22 06:56, mawupeng wrote:
>>>
>>>
>>> 在 2022/6/7 20:25, David Hildenbrand 写道:
>>>> On 07.06.22 11:38, Wupeng Ma wrote:
>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>>>
>>>>> For a system only have limited mirrored memory or some numa node 
>>>>> without
>>>>> mirrored memory, the per node vmemmap page_structs prefer to allocate
>>>>> memory from mirrored region, which will lead to vmemmap_verify() in
>>>>> vmemmap_populate_basepages() report lots of warning message.
>>>>>
>>>>> This patch demote the "potential offnode page_structs" warning 
>>>>> messages
>>>>> to debug level to avoid a very long print during bootup.
>>>>>
>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>>>> ---
>>>>>    mm/sparse-vmemmap.c | 2 +-
>>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>>>>> index f4fa61dbbee3..78debdb89eb1 100644
>>>>> --- a/mm/sparse-vmemmap.c
>>>>> +++ b/mm/sparse-vmemmap.c
>>>>> @@ -528,7 +528,7 @@ void __meminit vmemmap_verify(pte_t *pte, int 
>>>>> node,
>>>>>        int actual_node = early_pfn_to_nid(pfn);
>>>>>          if (node_distance(actual_node, node) > LOCAL_DISTANCE)
>>>>> -        pr_warn("[%lx-%lx] potential offnode page_structs\n",
>>>>> +        pr_debug("[%lx-%lx] potential offnode page_structs\n",
>>>>>                start, end - 1);
>>>>>    }
>>>>
>>>> This will possibly hide it in environments where this might indeed
>>>> indicate performance issues.
>>>>
>>>> What about a pr_warn_once()?
>>>>
>>>
>>> Sure.
>>>
>>> This will works. We can certainly use a pr_warn_once().
>>
>> Why not pr_warn_ratelimited() like in the previous patch ?
>
> Function vmemmap_populate_basepages() is used to populate base pages.
> System with huge memory will produce lots lots of warning message
> during this populate process even with ratelimited. This may lead to slow
> startup.

I think pr_warn_once is better, the memblock_alloc fallback is not frequent,

but vmemmap_verify will verify each memory and print a lot.

>
> Thanks for reviewing.
>
>> .
> .

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Kefeng Wang]

On 2022/6/10 17:34, Ard Biesheuvel wrote:
> On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>>
>> On 2022/6/7 17:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
>>> introduced mirrored memory support for x86 and this could be used on arm64.
>>>
>>> Since we only support this feature on arm64, efi_find_mirror() won't be placed
>>> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
>>> to scan the memory map and mark mirrored memory in memblock.
>> Move into efi_init() looks better, it won't bring negative effects on
>> arm/riscv.
>>
>> but let's maintainer to make a decision.
>>
>> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>>
> I don't disagree with that in principle, but x86 calls the efi fake
> memmap stuff between efi_init() and efi_find_mirror(), and I already
> indicated that I don't want to enable fake memmap on !X86.

I mean move into efi_init() in drivers/firmware/efi/efi-init.c which 
only used

in arm32/arm64/riscv.

>
> But I do think there is some room for improvement here: we could move
> things like efi_mokvar_table_init() and efi_esrt_init() into
> efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
> we can move it into efi_init() too.

x86 has own efi_init() in arch/x86/platform/efi/efi.c, it seams that all 
the above

function could move into x86's own efi_init(), not sure, but we could do 
it later

if it possible.

>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>    Documentation/admin-guide/kernel-parameters.txt | 2 +-
>>>    arch/arm64/kernel/setup.c                       | 1 +
>>>    2 files changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>>> index 8090130b544b..e3537646b6f7 100644
>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>> @@ -2301,7 +2301,7 @@
>>>
>>>        keepinitrd      [HW,ARM]
>>>
>>> -     kernelcore=     [KNL,X86,IA-64,PPC]
>>> +     kernelcore=     [KNL,X86,IA-64,PPC,ARM64]
>>>                        Format: nn[KMGTPE] | nn% | "mirror"
>>>                        This parameter specifies the amount of memory usable by
>>>                        the kernel for non-movable allocations.  The requested
>>> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
>>> index cf3a759f10d4..6e9acd7ecf0f 100644
>>> --- a/arch/arm64/kernel/setup.c
>>> +++ b/arch/arm64/kernel/setup.c
>>> @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>>>
>>>        xen_early_init();
>>>        efi_init();
>>> +     efi_find_mirror();
>>>
>>>        if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
>>>             pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");
> .

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Kefeng Wang]

On 2022/6/10 17:34, Ard Biesheuvel wrote:
> On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>>
>> On 2022/6/7 17:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
>>> introduced mirrored memory support for x86 and this could be used on arm64.
>>>
>>> Since we only support this feature on arm64, efi_find_mirror() won't be placed
>>> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
>>> to scan the memory map and mark mirrored memory in memblock.
>> Move into efi_init() looks better, it won't bring negative effects on
>> arm/riscv.
>>
>> but let's maintainer to make a decision.
>>
>> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>>
> I don't disagree with that in principle, but x86 calls the efi fake
> memmap stuff between efi_init() and efi_find_mirror(), and I already
> indicated that I don't want to enable fake memmap on !X86.

I mean move into efi_init() in drivers/firmware/efi/efi-init.c which 
only used

in arm32/arm64/riscv.

>
> But I do think there is some room for improvement here: we could move
> things like efi_mokvar_table_init() and efi_esrt_init() into
> efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
> we can move it into efi_init() too.

x86 has own efi_init() in arch/x86/platform/efi/efi.c, it seams that all 
the above

function could move into x86's own efi_init(), not sure, but we could do 
it later

if it possible.

>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>    Documentation/admin-guide/kernel-parameters.txt | 2 +-
>>>    arch/arm64/kernel/setup.c                       | 1 +
>>>    2 files changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>>> index 8090130b544b..e3537646b6f7 100644
>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>> @@ -2301,7 +2301,7 @@
>>>
>>>        keepinitrd      [HW,ARM]
>>>
>>> -     kernelcore=     [KNL,X86,IA-64,PPC]
>>> +     kernelcore=     [KNL,X86,IA-64,PPC,ARM64]
>>>                        Format: nn[KMGTPE] | nn% | "mirror"
>>>                        This parameter specifies the amount of memory usable by
>>>                        the kernel for non-movable allocations.  The requested
>>> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
>>> index cf3a759f10d4..6e9acd7ecf0f 100644
>>> --- a/arch/arm64/kernel/setup.c
>>> +++ b/arch/arm64/kernel/setup.c
>>> @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>>>
>>>        xen_early_init();
>>>        efi_init();
>>> +     efi_find_mirror();
>>>
>>>        if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
>>>             pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");
> .

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Kefeng Wang]

On 2022/6/10 17:34, Ard Biesheuvel wrote:
> On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>>
>> On 2022/6/7 17:38, Wupeng Ma wrote:
>>> From: Ma Wupeng <mawupeng1@huawei.com>
>>>
>>> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
>>> introduced mirrored memory support for x86 and this could be used on arm64.
>>>
>>> Since we only support this feature on arm64, efi_find_mirror() won't be placed
>>> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
>>> to scan the memory map and mark mirrored memory in memblock.
>> Move into efi_init() looks better, it won't bring negative effects on
>> arm/riscv.
>>
>> but let's maintainer to make a decision.
>>
>> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>>
> I don't disagree with that in principle, but x86 calls the efi fake
> memmap stuff between efi_init() and efi_find_mirror(), and I already
> indicated that I don't want to enable fake memmap on !X86.

I mean move into efi_init() in drivers/firmware/efi/efi-init.c which 
only used

in arm32/arm64/riscv.

>
> But I do think there is some room for improvement here: we could move
> things like efi_mokvar_table_init() and efi_esrt_init() into
> efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
> we can move it into efi_init() too.

x86 has own efi_init() in arch/x86/platform/efi/efi.c, it seams that all 
the above

function could move into x86's own efi_init(), not sure, but we could do 
it later

if it possible.

>
>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>>    Documentation/admin-guide/kernel-parameters.txt | 2 +-
>>>    arch/arm64/kernel/setup.c                       | 1 +
>>>    2 files changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>>> index 8090130b544b..e3537646b6f7 100644
>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>> @@ -2301,7 +2301,7 @@
>>>
>>>        keepinitrd      [HW,ARM]
>>>
>>> -     kernelcore=     [KNL,X86,IA-64,PPC]
>>> +     kernelcore=     [KNL,X86,IA-64,PPC,ARM64]
>>>                        Format: nn[KMGTPE] | nn% | "mirror"
>>>                        This parameter specifies the amount of memory usable by
>>>                        the kernel for non-movable allocations.  The requested
>>> diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
>>> index cf3a759f10d4..6e9acd7ecf0f 100644
>>> --- a/arch/arm64/kernel/setup.c
>>> +++ b/arch/arm64/kernel/setup.c
>>> @@ -328,6 +328,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>>>
>>>        xen_early_init();
>>>        efi_init();
>>> +     efi_find_mirror();
>>>
>>>        if (!efi_enabled(EFI_BOOT) && ((u64)_text % MIN_KIMG_ALIGN) != 0)
>>>             pr_warn(FW_BUG "Kernel image misaligned at boot, please fix your bootloader!");
> .

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Thu, 9 Jun 2022 at 10:16, mawupeng <mawupeng1@huawei.com> wrote:
>
>
>
> 在 2022/6/8 18:12, Ard Biesheuvel 写道:
> > On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
> >>
> >> On 08.06.22 12:02, Mike Rapoport wrote:
> >>> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> >>>>
> >>>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> >>>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> >>>>>>
> >>>>>> On 07.06.22 11:38, Wupeng Ma wrote:
> >>>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>>>
> >>>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
> >>>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> >>>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> >>>>>>> the lower 4G range has some non-mirrored memory.
> >>>>>>>
> >>>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> >>>>>>> reinstalled if the origin memblock has this flag.
> >>>>>>>
> >>>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>>> ---
> >>>>>>>    arch/arm64/mm/init.c     |  9 +++++++++
> >>>>>>>    include/linux/memblock.h |  1 +
> >>>>>>>    mm/memblock.c            | 20 ++++++++++++++++++++
> >>>>>>>    3 files changed, 30 insertions(+)
> >>>>>>>
> >>>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> >>>>>>> index 339ee84e5a61..11641f924d08 100644
> >>>>>>> --- a/arch/arm64/mm/init.c
> >>>>>>> +++ b/arch/arm64/mm/init.c
> >>>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >>>>>>>                         "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >>>>>>>                         phys_initrd_size = 0;
> >>>>>>>                 } else {
> >>>>>>> +                     int flags, ret;
> >>>>>>> +
> >>>>>>> +                     ret = memblock_get_flags(base, &flags);
> >>>>>>> +                     if (ret)
> >>>>>>> +                             flags = 0;
> >>>>>>> +
> >>>>>>>                         memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >>>>>>>                         memblock_add(base, size);
> >>>>>>>                         memblock_reserve(base, size);
> >>>>>>
> >>>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
> >>>>>> clear flags as the comment indicates?
> >>>>>>
> >>>>>
> >>>>> This should only happen if the placement of the initrd conflicts with
> >>>>> a mem= command line parameter or it is not covered by memblock for
> >>>>> some other reason.
> >>>>>
> >>>>> IOW, this should never happen, and if re-memblock_add'ing this memory
> >>>>> unconditionally is causing problems, we should fix that instead of
> >>>>> working around it.
> >>>>
> >>>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> >>>> the 4G limit to test this scenario(just for testing, I have trouble to boot
> >>>> qemu with initrd enabled and memory below 4G are all mirror memory).
> >>>>
> >>>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> >>>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> >>>>
> >>>>>
> >>>>>> If it's really just about clearing flags, I wonder if we rather want to
> >>>>>> have an interface that does exactly that, and hides the way this is
> >>>>>> actually implemented (obtain flags, remove, re-add ...), internally.
> >>>>>>
> >>>>>> But most probably there is more magic in the code and clearing flags
> >>>>>> isn't all it ends up doing.
> >>>>>>
> >>>>>
> >>>>> I don't remember exactly why we needed to clear the flags, but I think
> >>>>> it had to do with some corner case we hit when the initrd was
> >>>>> partially covered.
> >>>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> >>>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> >>>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> >>>>
> >>>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> >>>> MEMBLOCK_NOMAP of this initrd memblock.
> >>>> So maybe the usage of memblock_remove() is just to avoid introducing new
> >>>> function(memblock_clear_nomap)?
> >>>>
> >>>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> >>>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> >>>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> >>>
> >>> AFAICT, there are two corner cases that re-adding initrd memory covers:
> >>> * initrd memory is not a part of the memory reported to memblock, either
> >>> because of firmware weirdness or because it was cut out with mem=
> >>> * initrd memory overlaps a NOMAP region
> >>>
> >>> So to make sure initrd memory is mapped properly and retains
> >>> MEMBLOCK_MIRROR I think the best we can do is
> >>>
> >>>        memblock_add();
> >>>        memblock_clear_nomap();
> >>>        memblock_reserve();
> >>
> >> Would simply detect+rejecting to boot on such setups be an option? The
> >> replies so far indicate to me that this is rather a corner case than a
> >> reasonable use case.
> >>
> >
> > The sad reality is that mem= is known to be used in production for
> > limiting the amount of memory that the kernel takes control of, in
> > order to allow the remainder to be used in platform specific ways.
> >
> > Of course, there are much better ways to achieve that, but given that
> > we currently support it, I don't think we can easily back that out.
> >
> > I do think that there is no need to go out of our way to make this
> > case work seamlessly with mirrored memory, though. So I'd prefer to
> > make the remove+re-add conditional on there actually being a need to
> > do so. That way, we don't break the old use case or mirrored memory,
> > and whatever happens when the two are combined is DONTCARE.
>
> Does that mean that we don't need to care about this scenario with
> mirror memory?
>

We shouldn't, but we do, unfortunately.

So we should probably adopt the sequence suggested by Mike.

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Thu, 9 Jun 2022 at 10:16, mawupeng <mawupeng1@huawei.com> wrote:
>
>
>
> 在 2022/6/8 18:12, Ard Biesheuvel 写道:
> > On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
> >>
> >> On 08.06.22 12:02, Mike Rapoport wrote:
> >>> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> >>>>
> >>>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> >>>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> >>>>>>
> >>>>>> On 07.06.22 11:38, Wupeng Ma wrote:
> >>>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>>>
> >>>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
> >>>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> >>>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> >>>>>>> the lower 4G range has some non-mirrored memory.
> >>>>>>>
> >>>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> >>>>>>> reinstalled if the origin memblock has this flag.
> >>>>>>>
> >>>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>>> ---
> >>>>>>>    arch/arm64/mm/init.c     |  9 +++++++++
> >>>>>>>    include/linux/memblock.h |  1 +
> >>>>>>>    mm/memblock.c            | 20 ++++++++++++++++++++
> >>>>>>>    3 files changed, 30 insertions(+)
> >>>>>>>
> >>>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> >>>>>>> index 339ee84e5a61..11641f924d08 100644
> >>>>>>> --- a/arch/arm64/mm/init.c
> >>>>>>> +++ b/arch/arm64/mm/init.c
> >>>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >>>>>>>                         "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >>>>>>>                         phys_initrd_size = 0;
> >>>>>>>                 } else {
> >>>>>>> +                     int flags, ret;
> >>>>>>> +
> >>>>>>> +                     ret = memblock_get_flags(base, &flags);
> >>>>>>> +                     if (ret)
> >>>>>>> +                             flags = 0;
> >>>>>>> +
> >>>>>>>                         memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >>>>>>>                         memblock_add(base, size);
> >>>>>>>                         memblock_reserve(base, size);
> >>>>>>
> >>>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
> >>>>>> clear flags as the comment indicates?
> >>>>>>
> >>>>>
> >>>>> This should only happen if the placement of the initrd conflicts with
> >>>>> a mem= command line parameter or it is not covered by memblock for
> >>>>> some other reason.
> >>>>>
> >>>>> IOW, this should never happen, and if re-memblock_add'ing this memory
> >>>>> unconditionally is causing problems, we should fix that instead of
> >>>>> working around it.
> >>>>
> >>>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> >>>> the 4G limit to test this scenario(just for testing, I have trouble to boot
> >>>> qemu with initrd enabled and memory below 4G are all mirror memory).
> >>>>
> >>>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> >>>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> >>>>
> >>>>>
> >>>>>> If it's really just about clearing flags, I wonder if we rather want to
> >>>>>> have an interface that does exactly that, and hides the way this is
> >>>>>> actually implemented (obtain flags, remove, re-add ...), internally.
> >>>>>>
> >>>>>> But most probably there is more magic in the code and clearing flags
> >>>>>> isn't all it ends up doing.
> >>>>>>
> >>>>>
> >>>>> I don't remember exactly why we needed to clear the flags, but I think
> >>>>> it had to do with some corner case we hit when the initrd was
> >>>>> partially covered.
> >>>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> >>>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> >>>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> >>>>
> >>>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> >>>> MEMBLOCK_NOMAP of this initrd memblock.
> >>>> So maybe the usage of memblock_remove() is just to avoid introducing new
> >>>> function(memblock_clear_nomap)?
> >>>>
> >>>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> >>>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> >>>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> >>>
> >>> AFAICT, there are two corner cases that re-adding initrd memory covers:
> >>> * initrd memory is not a part of the memory reported to memblock, either
> >>> because of firmware weirdness or because it was cut out with mem=
> >>> * initrd memory overlaps a NOMAP region
> >>>
> >>> So to make sure initrd memory is mapped properly and retains
> >>> MEMBLOCK_MIRROR I think the best we can do is
> >>>
> >>>        memblock_add();
> >>>        memblock_clear_nomap();
> >>>        memblock_reserve();
> >>
> >> Would simply detect+rejecting to boot on such setups be an option? The
> >> replies so far indicate to me that this is rather a corner case than a
> >> reasonable use case.
> >>
> >
> > The sad reality is that mem= is known to be used in production for
> > limiting the amount of memory that the kernel takes control of, in
> > order to allow the remainder to be used in platform specific ways.
> >
> > Of course, there are much better ways to achieve that, but given that
> > we currently support it, I don't think we can easily back that out.
> >
> > I do think that there is no need to go out of our way to make this
> > case work seamlessly with mirrored memory, though. So I'd prefer to
> > make the remove+re-add conditional on there actually being a need to
> > do so. That way, we don't break the old use case or mirrored memory,
> > and whatever happens when the two are combined is DONTCARE.
>
> Does that mean that we don't need to care about this scenario with
> mirror memory?
>

We shouldn't, but we do, unfortunately.

So we should probably adopt the sequence suggested by Mike.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory

[Ard Biesheuvel]

On Thu, 9 Jun 2022 at 10:16, mawupeng <mawupeng1@huawei.com> wrote:
>
>
>
> 在 2022/6/8 18:12, Ard Biesheuvel 写道:
> > On Wed, 8 Jun 2022 at 12:08, David Hildenbrand <david@redhat.com> wrote:
> >>
> >> On 08.06.22 12:02, Mike Rapoport wrote:
> >>> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote:
> >>>>
> >>>> 在 2022/6/7 22:49, Ard Biesheuvel 写道:
> >>>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand <david@redhat.com> wrote:
> >>>>>>
> >>>>>> On 07.06.22 11:38, Wupeng Ma wrote:
> >>>>>>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>>>
> >>>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this
> >>>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR
> >>>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if
> >>>>>>> the lower 4G range has some non-mirrored memory.
> >>>>>>>
> >>>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be
> >>>>>>> reinstalled if the origin memblock has this flag.
> >>>>>>>
> >>>>>>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> >>>>>>> ---
> >>>>>>>    arch/arm64/mm/init.c     |  9 +++++++++
> >>>>>>>    include/linux/memblock.h |  1 +
> >>>>>>>    mm/memblock.c            | 20 ++++++++++++++++++++
> >>>>>>>    3 files changed, 30 insertions(+)
> >>>>>>>
> >>>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> >>>>>>> index 339ee84e5a61..11641f924d08 100644
> >>>>>>> --- a/arch/arm64/mm/init.c
> >>>>>>> +++ b/arch/arm64/mm/init.c
> >>>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void)
> >>>>>>>                         "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) {
> >>>>>>>                         phys_initrd_size = 0;
> >>>>>>>                 } else {
> >>>>>>> +                     int flags, ret;
> >>>>>>> +
> >>>>>>> +                     ret = memblock_get_flags(base, &flags);
> >>>>>>> +                     if (ret)
> >>>>>>> +                             flags = 0;
> >>>>>>> +
> >>>>>>>                         memblock_remove(base, size); /* clear MEMBLOCK_ flags */
> >>>>>>>                         memblock_add(base, size);
> >>>>>>>                         memblock_reserve(base, size);
> >>>>>>
> >>>>>> Can you explain why we're removing+re-adding here exactly? Is it just to
> >>>>>> clear flags as the comment indicates?
> >>>>>>
> >>>>>
> >>>>> This should only happen if the placement of the initrd conflicts with
> >>>>> a mem= command line parameter or it is not covered by memblock for
> >>>>> some other reason.
> >>>>>
> >>>>> IOW, this should never happen, and if re-memblock_add'ing this memory
> >>>>> unconditionally is causing problems, we should fix that instead of
> >>>>> working around it.
> >>>>
> >>>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below
> >>>> the 4G limit to test this scenario(just for testing, I have trouble to boot
> >>>> qemu with initrd enabled and memory below 4G are all mirror memory).
> >>>>
> >>>> Re-memblock_add'ing this memory unconditionally seems fine but clear all
> >>>> flags(especially MEMBLOCK_MIRROR) may lead to some error log.
> >>>>
> >>>>>
> >>>>>> If it's really just about clearing flags, I wonder if we rather want to
> >>>>>> have an interface that does exactly that, and hides the way this is
> >>>>>> actually implemented (obtain flags, remove, re-add ...), internally.
> >>>>>>
> >>>>>> But most probably there is more magic in the code and clearing flags
> >>>>>> isn't all it ends up doing.
> >>>>>>
> >>>>>
> >>>>> I don't remember exactly why we needed to clear the flags, but I think
> >>>>> it had to do with some corner case we hit when the initrd was
> >>>>> partially covered.
> >>>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will
> >>>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the
> >>>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag?
> >>>>
> >>>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the
> >>>> MEMBLOCK_NOMAP of this initrd memblock.
> >>>> So maybe the usage of memblock_remove() is just to avoid introducing new
> >>>> function(memblock_clear_nomap)?
> >>>>
> >>>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already
> >>>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP
> >>>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back?
> >>>
> >>> AFAICT, there are two corner cases that re-adding initrd memory covers:
> >>> * initrd memory is not a part of the memory reported to memblock, either
> >>> because of firmware weirdness or because it was cut out with mem=
> >>> * initrd memory overlaps a NOMAP region
> >>>
> >>> So to make sure initrd memory is mapped properly and retains
> >>> MEMBLOCK_MIRROR I think the best we can do is
> >>>
> >>>        memblock_add();
> >>>        memblock_clear_nomap();
> >>>        memblock_reserve();
> >>
> >> Would simply detect+rejecting to boot on such setups be an option? The
> >> replies so far indicate to me that this is rather a corner case than a
> >> reasonable use case.
> >>
> >
> > The sad reality is that mem= is known to be used in production for
> > limiting the amount of memory that the kernel takes control of, in
> > order to allow the remainder to be used in platform specific ways.
> >
> > Of course, there are much better ways to achieve that, but given that
> > we currently support it, I don't think we can easily back that out.
> >
> > I do think that there is no need to go out of our way to make this
> > case work seamlessly with mirrored memory, though. So I'd prefer to
> > make the remove+re-add conditional on there actually being a need to
> > do so. That way, we don't break the old use case or mirrored memory,
> > and whatever happens when the two are combined is DONTCARE.
>
> Does that mean that we don't need to care about this scenario with
> mirror memory?
>

We shouldn't, but we do, unfortunately.

So we should probably adopt the sequence suggested by Mike.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 12:24, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
>
> On 2022/6/10 17:34, Ard Biesheuvel wrote:
> > On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
> >>
> >> On 2022/6/7 17:38, Wupeng Ma wrote:
> >>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>
> >>> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> >>> introduced mirrored memory support for x86 and this could be used on arm64.
> >>>
> >>> Since we only support this feature on arm64, efi_find_mirror() won't be placed
> >>> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> >>> to scan the memory map and mark mirrored memory in memblock.
> >> Move into efi_init() looks better, it won't bring negative effects on
> >> arm/riscv.
> >>
> >> but let's maintainer to make a decision.
> >>
> >> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> >>
> > I don't disagree with that in principle, but x86 calls the efi fake
> > memmap stuff between efi_init() and efi_find_mirror(), and I already
> > indicated that I don't want to enable fake memmap on !X86.
>
> I mean move into efi_init() in drivers/firmware/efi/efi-init.c which
> only used
>
> in arm32/arm64/riscv.
>
> >
> > But I do think there is some room for improvement here: we could move
> > things like efi_mokvar_table_init() and efi_esrt_init() into
> > efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
> > we can move it into efi_init() too.
>
> x86 has own efi_init() in arch/x86/platform/efi/efi.c, it seams that all
> the above
>
> function could move into x86's own efi_init(), not sure, but we could do
> it later
>
> if it possible.
>

Yes, I see what you mean now. I agree that would be better.

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 12:24, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
>
> On 2022/6/10 17:34, Ard Biesheuvel wrote:
> > On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
> >>
> >> On 2022/6/7 17:38, Wupeng Ma wrote:
> >>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>
> >>> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> >>> introduced mirrored memory support for x86 and this could be used on arm64.
> >>>
> >>> Since we only support this feature on arm64, efi_find_mirror() won't be placed
> >>> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> >>> to scan the memory map and mark mirrored memory in memblock.
> >> Move into efi_init() looks better, it won't bring negative effects on
> >> arm/riscv.
> >>
> >> but let's maintainer to make a decision.
> >>
> >> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> >>
> > I don't disagree with that in principle, but x86 calls the efi fake
> > memmap stuff between efi_init() and efi_find_mirror(), and I already
> > indicated that I don't want to enable fake memmap on !X86.
>
> I mean move into efi_init() in drivers/firmware/efi/efi-init.c which
> only used
>
> in arm32/arm64/riscv.
>
> >
> > But I do think there is some room for improvement here: we could move
> > things like efi_mokvar_table_init() and efi_esrt_init() into
> > efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
> > we can move it into efi_init() too.
>
> x86 has own efi_init() in arch/x86/platform/efi/efi.c, it seams that all
> the above
>
> function could move into x86's own efi_init(), not sure, but we could do
> it later
>
> if it possible.
>

Yes, I see what you mean now. I agree that would be better.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 2/6] arm64/mirror: arm64 enabling - find mirrored memory ranges

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 12:24, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
>
> On 2022/6/10 17:34, Ard Biesheuvel wrote:
> > On Fri, 10 Jun 2022 at 11:27, Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
> >>
> >> On 2022/6/7 17:38, Wupeng Ma wrote:
> >>> From: Ma Wupeng <mawupeng1@huawei.com>
> >>>
> >>> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> >>> introduced mirrored memory support for x86 and this could be used on arm64.
> >>>
> >>> Since we only support this feature on arm64, efi_find_mirror() won't be placed
> >>> into efi_init(), which is used by riscv/arm/arm64, it is added in setup_arch()
> >>> to scan the memory map and mark mirrored memory in memblock.
> >> Move into efi_init() looks better, it won't bring negative effects on
> >> arm/riscv.
> >>
> >> but let's maintainer to make a decision.
> >>
> >> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> >>
> > I don't disagree with that in principle, but x86 calls the efi fake
> > memmap stuff between efi_init() and efi_find_mirror(), and I already
> > indicated that I don't want to enable fake memmap on !X86.
>
> I mean move into efi_init() in drivers/firmware/efi/efi-init.c which
> only used
>
> in arm32/arm64/riscv.
>
> >
> > But I do think there is some room for improvement here: we could move
> > things like efi_mokvar_table_init() and efi_esrt_init() into
> > efi_init() as well, and make efi_fake_memmap() do nothing on !X86 so
> > we can move it into efi_init() too.
>
> x86 has own efi_init() in arch/x86/platform/efi/efi.c, it seams that all
> the above
>
> function could move into x86's own efi_init(), not sure, but we could do
> it later
>
> if it possible.
>

Yes, I see what you mean now. I agree that would be better.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> If system have some mirrored memory and mirrored feature is not specified
> in boot parameter, the basic mirrored feature will be enabled and this will
> lead to the following situations:
>
> - memblock memory allocation prefers mirrored region. This may have some
>   unexpected influence on numa affinity.
>
> - contiguous memory will be split into several parts if parts of them
>   is mirrored memory via memblock_mark_mirror().
>
> To fix this, variable mirrored_kernelcore will be checked before calling
> efi_find_mirror() which will enable basic mirrored feature and this
> variable is true if kernelcore=mirror is added in the kernel parameters.
>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>

This seems like the wrong place to do this. If mirrored memory is
irrelevant to memblock, it should ignore the attribute. So I think
this check belongs in mm/memblock.c instead.


> ---
>  drivers/firmware/efi/efi.c | 3 +++
>  include/linux/mm.h         | 2 ++
>  mm/page_alloc.c            | 2 +-
>  3 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
> index 79c232e07de7..8a5edcb0dd82 100644
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
>         if (!efi_enabled(EFI_MEMMAP))
>                 return;
>
> +       if (!mirrored_kernelcore)
> +               return;
> +
>         for_each_efi_memory_desc(md) {
>                 unsigned long long start = md->phys_addr;
>                 unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index bc8f326be0ce..741ac7d022c3 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
>                         unsigned long *start_pfn, unsigned long *end_pfn);
>  extern unsigned long find_min_pfn_with_active_regions(void);
>
> +extern bool mirrored_kernelcore;
> +
>  #ifndef CONFIG_NUMA
>  static inline int early_pfn_to_nid(unsigned long pfn)
>  {
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index e008a3df0485..cf6f70aba787 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
>  static unsigned long required_movablecore __initdata;
>  static unsigned long required_movablecore_percent __initdata;
>  static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
> -static bool mirrored_kernelcore __meminitdata;
> +bool mirrored_kernelcore __meminitdata;
>
>  /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
>  int movable_zone;
> --
> 2.25.1
>

Re: [PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> If system have some mirrored memory and mirrored feature is not specified
> in boot parameter, the basic mirrored feature will be enabled and this will
> lead to the following situations:
>
> - memblock memory allocation prefers mirrored region. This may have some
>   unexpected influence on numa affinity.
>
> - contiguous memory will be split into several parts if parts of them
>   is mirrored memory via memblock_mark_mirror().
>
> To fix this, variable mirrored_kernelcore will be checked before calling
> efi_find_mirror() which will enable basic mirrored feature and this
> variable is true if kernelcore=mirror is added in the kernel parameters.
>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>

This seems like the wrong place to do this. If mirrored memory is
irrelevant to memblock, it should ignore the attribute. So I think
this check belongs in mm/memblock.c instead.


> ---
>  drivers/firmware/efi/efi.c | 3 +++
>  include/linux/mm.h         | 2 ++
>  mm/page_alloc.c            | 2 +-
>  3 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
> index 79c232e07de7..8a5edcb0dd82 100644
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
>         if (!efi_enabled(EFI_MEMMAP))
>                 return;
>
> +       if (!mirrored_kernelcore)
> +               return;
> +
>         for_each_efi_memory_desc(md) {
>                 unsigned long long start = md->phys_addr;
>                 unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index bc8f326be0ce..741ac7d022c3 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
>                         unsigned long *start_pfn, unsigned long *end_pfn);
>  extern unsigned long find_min_pfn_with_active_regions(void);
>
> +extern bool mirrored_kernelcore;
> +
>  #ifndef CONFIG_NUMA
>  static inline int early_pfn_to_nid(unsigned long pfn)
>  {
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index e008a3df0485..cf6f70aba787 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
>  static unsigned long required_movablecore __initdata;
>  static unsigned long required_movablecore_percent __initdata;
>  static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
> -static bool mirrored_kernelcore __meminitdata;
> +bool mirrored_kernelcore __meminitdata;
>
>  /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
>  int movable_zone;
> --
> 2.25.1
>

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> If system have some mirrored memory and mirrored feature is not specified
> in boot parameter, the basic mirrored feature will be enabled and this will
> lead to the following situations:
>
> - memblock memory allocation prefers mirrored region. This may have some
>   unexpected influence on numa affinity.
>
> - contiguous memory will be split into several parts if parts of them
>   is mirrored memory via memblock_mark_mirror().
>
> To fix this, variable mirrored_kernelcore will be checked before calling
> efi_find_mirror() which will enable basic mirrored feature and this
> variable is true if kernelcore=mirror is added in the kernel parameters.
>
> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>

This seems like the wrong place to do this. If mirrored memory is
irrelevant to memblock, it should ignore the attribute. So I think
this check belongs in mm/memblock.c instead.


> ---
>  drivers/firmware/efi/efi.c | 3 +++
>  include/linux/mm.h         | 2 ++
>  mm/page_alloc.c            | 2 +-
>  3 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
> index 79c232e07de7..8a5edcb0dd82 100644
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
>         if (!efi_enabled(EFI_MEMMAP))
>                 return;
>
> +       if (!mirrored_kernelcore)
> +               return;
> +
>         for_each_efi_memory_desc(md) {
>                 unsigned long long start = md->phys_addr;
>                 unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index bc8f326be0ce..741ac7d022c3 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
>                         unsigned long *start_pfn, unsigned long *end_pfn);
>  extern unsigned long find_min_pfn_with_active_regions(void);
>
> +extern bool mirrored_kernelcore;
> +
>  #ifndef CONFIG_NUMA
>  static inline int early_pfn_to_nid(unsigned long pfn)
>  {
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index e008a3df0485..cf6f70aba787 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
>  static unsigned long required_movablecore __initdata;
>  static unsigned long required_movablecore_percent __initdata;
>  static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
> -static bool mirrored_kernelcore __meminitdata;
> +bool mirrored_kernelcore __meminitdata;
>
>  /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
>  int movable_zone;
> --
> 2.25.1
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> introduced mirrored memory support for x86. This support rely on UEFI to
> report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
>
>   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
>
> Memory mirroring is a technique used to separate memory into two separate
> channels, usually on a memory device, like a server. In memory mirroring,
> one channel is copied to another to create redundancy. This method makes
> input/output (I/O) registers and memory appear with more than one address
> range because the same physical byte is accessible at more than one
> address. Using memory mirroring, higher memory reliability and a higher
> level of memory consolidation are possible.
>
> These EFI memory regions have various attributes, and the "mirrored"
> attribute is one of them. The physical memory region whose descriptors
> in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> mirrored. The address range mirroring feature of the kernel arranges such
> mirrored regions into normal zones and other regions into movable zones.
>
> Arm64 can support this too. So mirrored memory support is added to support
> arm64.
>
> The main purpose of this patch set is to introduce mirrored support for
> arm64 and we have already fixed the problems we had which is shown in
> patch #5 to patch #8 and try to bring total isolation in patch #9 which
> will disable mirror feature if kernelcore is not specified.
>
> In order to test this support in arm64:
> - patch this patch set
> - add kernelcore=mirror in kernel parameter
> - start you kernel
>
> Patch #1-#2 introduce mirrored memory support form arm64.
> Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> Patch #6 disable mirror feature if kernelcore is not specified.
>
> Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> memory if kaslr is enabled.
>
> [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
>
> Changelog since v2:
> - remove efi_fake_mem support
> - remove Commit ("remove some redundant code in ia64 efi_init") since
>   efi_print_memmap() is not public
> - add mirror flag back on initrd memory
>
> Changelog since v1:
> - update changelog in cover letter
> - use PHYS_PFN in patch #7
>
> Ma Wupeng (6):
>   efi: Make efi_find_mirror() public
>   arm64/mirror: arm64 enabling - find mirrored memory ranges
>   mm: Ratelimited mirrored memory related warning messages
>   mm: Demote warning message in vmemmap_verify() to debug level
>   mm: Add mirror flag back on initrd memory
>   efi: Disable mirror feature if kernelcore is not specified
>

I have tested these changes on QEMU/arm64 with the patch below, and
things seem to work as expected. We have some minor issues to work out
but the general shape of this code is good.

As for the mm/ changes: does anyone mind if I take those through the
EFI tree as well? I don't think the EFI and -mm changes depend on each
other, so they can go into -mm separately as well.

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> introduced mirrored memory support for x86. This support rely on UEFI to
> report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
>
>   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
>
> Memory mirroring is a technique used to separate memory into two separate
> channels, usually on a memory device, like a server. In memory mirroring,
> one channel is copied to another to create redundancy. This method makes
> input/output (I/O) registers and memory appear with more than one address
> range because the same physical byte is accessible at more than one
> address. Using memory mirroring, higher memory reliability and a higher
> level of memory consolidation are possible.
>
> These EFI memory regions have various attributes, and the "mirrored"
> attribute is one of them. The physical memory region whose descriptors
> in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> mirrored. The address range mirroring feature of the kernel arranges such
> mirrored regions into normal zones and other regions into movable zones.
>
> Arm64 can support this too. So mirrored memory support is added to support
> arm64.
>
> The main purpose of this patch set is to introduce mirrored support for
> arm64 and we have already fixed the problems we had which is shown in
> patch #5 to patch #8 and try to bring total isolation in patch #9 which
> will disable mirror feature if kernelcore is not specified.
>
> In order to test this support in arm64:
> - patch this patch set
> - add kernelcore=mirror in kernel parameter
> - start you kernel
>
> Patch #1-#2 introduce mirrored memory support form arm64.
> Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> Patch #6 disable mirror feature if kernelcore is not specified.
>
> Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> memory if kaslr is enabled.
>
> [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
>
> Changelog since v2:
> - remove efi_fake_mem support
> - remove Commit ("remove some redundant code in ia64 efi_init") since
>   efi_print_memmap() is not public
> - add mirror flag back on initrd memory
>
> Changelog since v1:
> - update changelog in cover letter
> - use PHYS_PFN in patch #7
>
> Ma Wupeng (6):
>   efi: Make efi_find_mirror() public
>   arm64/mirror: arm64 enabling - find mirrored memory ranges
>   mm: Ratelimited mirrored memory related warning messages
>   mm: Demote warning message in vmemmap_verify() to debug level
>   mm: Add mirror flag back on initrd memory
>   efi: Disable mirror feature if kernelcore is not specified
>

I have tested these changes on QEMU/arm64 with the patch below, and
things seem to work as expected. We have some minor issues to work out
but the general shape of this code is good.

As for the mm/ changes: does anyone mind if I take those through the
EFI tree as well? I don't think the EFI and -mm changes depend on each
other, so they can go into -mm separately as well.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Ard Biesheuvel]

On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>
> From: Ma Wupeng <mawupeng1@huawei.com>
>
> Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> introduced mirrored memory support for x86. This support rely on UEFI to
> report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
>
>   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
>
> Memory mirroring is a technique used to separate memory into two separate
> channels, usually on a memory device, like a server. In memory mirroring,
> one channel is copied to another to create redundancy. This method makes
> input/output (I/O) registers and memory appear with more than one address
> range because the same physical byte is accessible at more than one
> address. Using memory mirroring, higher memory reliability and a higher
> level of memory consolidation are possible.
>
> These EFI memory regions have various attributes, and the "mirrored"
> attribute is one of them. The physical memory region whose descriptors
> in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> mirrored. The address range mirroring feature of the kernel arranges such
> mirrored regions into normal zones and other regions into movable zones.
>
> Arm64 can support this too. So mirrored memory support is added to support
> arm64.
>
> The main purpose of this patch set is to introduce mirrored support for
> arm64 and we have already fixed the problems we had which is shown in
> patch #5 to patch #8 and try to bring total isolation in patch #9 which
> will disable mirror feature if kernelcore is not specified.
>
> In order to test this support in arm64:
> - patch this patch set
> - add kernelcore=mirror in kernel parameter
> - start you kernel
>
> Patch #1-#2 introduce mirrored memory support form arm64.
> Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> Patch #6 disable mirror feature if kernelcore is not specified.
>
> Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> memory if kaslr is enabled.
>
> [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
>
> Changelog since v2:
> - remove efi_fake_mem support
> - remove Commit ("remove some redundant code in ia64 efi_init") since
>   efi_print_memmap() is not public
> - add mirror flag back on initrd memory
>
> Changelog since v1:
> - update changelog in cover letter
> - use PHYS_PFN in patch #7
>
> Ma Wupeng (6):
>   efi: Make efi_find_mirror() public
>   arm64/mirror: arm64 enabling - find mirrored memory ranges
>   mm: Ratelimited mirrored memory related warning messages
>   mm: Demote warning message in vmemmap_verify() to debug level
>   mm: Add mirror flag back on initrd memory
>   efi: Disable mirror feature if kernelcore is not specified
>

I have tested these changes on QEMU/arm64 with the patch below, and
things seem to work as expected. We have some minor issues to work out
but the general shape of this code is good.

As for the mm/ changes: does anyone mind if I take those through the
EFI tree as well? I don't think the EFI and -mm changes depend on each
other, so they can go into -mm separately as well.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 13:23, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
> >
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86. This support rely on UEFI to
> > report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
> >
> >   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
> >
> > Memory mirroring is a technique used to separate memory into two separate
> > channels, usually on a memory device, like a server. In memory mirroring,
> > one channel is copied to another to create redundancy. This method makes
> > input/output (I/O) registers and memory appear with more than one address
> > range because the same physical byte is accessible at more than one
> > address. Using memory mirroring, higher memory reliability and a higher
> > level of memory consolidation are possible.
> >
> > These EFI memory regions have various attributes, and the "mirrored"
> > attribute is one of them. The physical memory region whose descriptors
> > in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> > mirrored. The address range mirroring feature of the kernel arranges such
> > mirrored regions into normal zones and other regions into movable zones.
> >
> > Arm64 can support this too. So mirrored memory support is added to support
> > arm64.
> >
> > The main purpose of this patch set is to introduce mirrored support for
> > arm64 and we have already fixed the problems we had which is shown in
> > patch #5 to patch #8 and try to bring total isolation in patch #9 which
> > will disable mirror feature if kernelcore is not specified.
> >
> > In order to test this support in arm64:
> > - patch this patch set
> > - add kernelcore=mirror in kernel parameter
> > - start you kernel
> >
> > Patch #1-#2 introduce mirrored memory support form arm64.
> > Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> > Patch #6 disable mirror feature if kernelcore is not specified.
> >
> > Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> > memory if kaslr is enabled.
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
> >
> > Changelog since v2:
> > - remove efi_fake_mem support
> > - remove Commit ("remove some redundant code in ia64 efi_init") since
> >   efi_print_memmap() is not public
> > - add mirror flag back on initrd memory
> >
> > Changelog since v1:
> > - update changelog in cover letter
> > - use PHYS_PFN in patch #7
> >
> > Ma Wupeng (6):
> >   efi: Make efi_find_mirror() public
> >   arm64/mirror: arm64 enabling - find mirrored memory ranges
> >   mm: Ratelimited mirrored memory related warning messages
> >   mm: Demote warning message in vmemmap_verify() to debug level
> >   mm: Add mirror flag back on initrd memory
> >   efi: Disable mirror feature if kernelcore is not specified
> >
>
> I have tested these changes on QEMU/arm64 with the patch below, and
> things seem to work as expected. We have some minor issues to work out
> but the general shape of this code is good.
>
> As for the mm/ changes: does anyone mind if I take those through the
> EFI tree as well? I don't think the EFI and -mm changes depend on each
> other, so they can go into -mm separately as well.

diff --git a/drivers/firmware/efi/libstub/efi-stub.c
b/drivers/firmware/efi/libstub/efi-stub.c
index f515394cce6e..1d4dd8aca3e6 100644
--- a/drivers/firmware/efi/libstub/efi-stub.c
+++ b/drivers/firmware/efi/libstub/efi-stub.c
@@ -136,6 +136,7 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
        enum efi_secureboot_mode secure_boot;
        struct screen_info *si;
        efi_properties_table_t *prop_tbl;
+       const efi_dxe_services_table_t *efi_dxe_table;

        efi_system_table = sys_table_arg;

@@ -161,6 +162,20 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
                goto fail;
        }

+       efi_dxe_table = get_efi_config_table(EFI_DXE_SERVICES_TABLE_GUID);
+       if (efi_dxe_table) {
+               efi_physical_addr_t pa = 0x60000000;
+               efi_gcd_memory_space_desc_t desc;
+
+               status = efi_dxe_call(get_memory_space_descriptor, pa, &desc);
+               if (status != EFI_SUCCESS)
+                       efi_err("Failed to get memory space
descriptor: %lx\n", status);
+               status = efi_dxe_call(set_memory_space_capabilities, pa, SZ_1G,
+                                     desc.capabilities |
EFI_MEMORY_MORE_RELIABLE);
+               if (status != EFI_SUCCESS)
+                       efi_err("Failed to set memory space
capabilities: %lx\n", status);
+       }
+
        /*
         * Get the command line from EFI, using the LOADED_IMAGE
         * protocol. We are going to copy the command line into the
diff --git a/drivers/firmware/efi/libstub/efistub.h
b/drivers/firmware/efi/libstub/efistub.h
index b0ae0a454404..bf11d85bf9b4 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -378,7 +378,8 @@ union efi_dxe_services_table {
                void *schedule;
                void *trust;
                void *process_firmware_volume;
-               void *set_memory_space_capabilities;
+               efi_status_t (__efiapi
*set_memory_space_capabilities)(efi_physical_addr_t,
+
u64, u64);
        };
        struct {
                efi_table_hdr_t hdr;

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 13:23, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
> >
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86. This support rely on UEFI to
> > report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
> >
> >   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
> >
> > Memory mirroring is a technique used to separate memory into two separate
> > channels, usually on a memory device, like a server. In memory mirroring,
> > one channel is copied to another to create redundancy. This method makes
> > input/output (I/O) registers and memory appear with more than one address
> > range because the same physical byte is accessible at more than one
> > address. Using memory mirroring, higher memory reliability and a higher
> > level of memory consolidation are possible.
> >
> > These EFI memory regions have various attributes, and the "mirrored"
> > attribute is one of them. The physical memory region whose descriptors
> > in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> > mirrored. The address range mirroring feature of the kernel arranges such
> > mirrored regions into normal zones and other regions into movable zones.
> >
> > Arm64 can support this too. So mirrored memory support is added to support
> > arm64.
> >
> > The main purpose of this patch set is to introduce mirrored support for
> > arm64 and we have already fixed the problems we had which is shown in
> > patch #5 to patch #8 and try to bring total isolation in patch #9 which
> > will disable mirror feature if kernelcore is not specified.
> >
> > In order to test this support in arm64:
> > - patch this patch set
> > - add kernelcore=mirror in kernel parameter
> > - start you kernel
> >
> > Patch #1-#2 introduce mirrored memory support form arm64.
> > Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> > Patch #6 disable mirror feature if kernelcore is not specified.
> >
> > Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> > memory if kaslr is enabled.
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
> >
> > Changelog since v2:
> > - remove efi_fake_mem support
> > - remove Commit ("remove some redundant code in ia64 efi_init") since
> >   efi_print_memmap() is not public
> > - add mirror flag back on initrd memory
> >
> > Changelog since v1:
> > - update changelog in cover letter
> > - use PHYS_PFN in patch #7
> >
> > Ma Wupeng (6):
> >   efi: Make efi_find_mirror() public
> >   arm64/mirror: arm64 enabling - find mirrored memory ranges
> >   mm: Ratelimited mirrored memory related warning messages
> >   mm: Demote warning message in vmemmap_verify() to debug level
> >   mm: Add mirror flag back on initrd memory
> >   efi: Disable mirror feature if kernelcore is not specified
> >
>
> I have tested these changes on QEMU/arm64 with the patch below, and
> things seem to work as expected. We have some minor issues to work out
> but the general shape of this code is good.
>
> As for the mm/ changes: does anyone mind if I take those through the
> EFI tree as well? I don't think the EFI and -mm changes depend on each
> other, so they can go into -mm separately as well.

diff --git a/drivers/firmware/efi/libstub/efi-stub.c
b/drivers/firmware/efi/libstub/efi-stub.c
index f515394cce6e..1d4dd8aca3e6 100644
--- a/drivers/firmware/efi/libstub/efi-stub.c
+++ b/drivers/firmware/efi/libstub/efi-stub.c
@@ -136,6 +136,7 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
        enum efi_secureboot_mode secure_boot;
        struct screen_info *si;
        efi_properties_table_t *prop_tbl;
+       const efi_dxe_services_table_t *efi_dxe_table;

        efi_system_table = sys_table_arg;

@@ -161,6 +162,20 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
                goto fail;
        }

+       efi_dxe_table = get_efi_config_table(EFI_DXE_SERVICES_TABLE_GUID);
+       if (efi_dxe_table) {
+               efi_physical_addr_t pa = 0x60000000;
+               efi_gcd_memory_space_desc_t desc;
+
+               status = efi_dxe_call(get_memory_space_descriptor, pa, &desc);
+               if (status != EFI_SUCCESS)
+                       efi_err("Failed to get memory space
descriptor: %lx\n", status);
+               status = efi_dxe_call(set_memory_space_capabilities, pa, SZ_1G,
+                                     desc.capabilities |
EFI_MEMORY_MORE_RELIABLE);
+               if (status != EFI_SUCCESS)
+                       efi_err("Failed to set memory space
capabilities: %lx\n", status);
+       }
+
        /*
         * Get the command line from EFI, using the LOADED_IMAGE
         * protocol. We are going to copy the command line into the
diff --git a/drivers/firmware/efi/libstub/efistub.h
b/drivers/firmware/efi/libstub/efistub.h
index b0ae0a454404..bf11d85bf9b4 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -378,7 +378,8 @@ union efi_dxe_services_table {
                void *schedule;
                void *trust;
                void *process_firmware_volume;
-               void *set_memory_space_capabilities;
+               efi_status_t (__efiapi
*set_memory_space_capabilities)(efi_physical_addr_t,
+
u64, u64);
        };
        struct {
                efi_table_hdr_t hdr;

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Ard Biesheuvel]

On Fri, 10 Jun 2022 at 13:23, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
> >
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86. This support rely on UEFI to
> > report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
> >
> >   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
> >
> > Memory mirroring is a technique used to separate memory into two separate
> > channels, usually on a memory device, like a server. In memory mirroring,
> > one channel is copied to another to create redundancy. This method makes
> > input/output (I/O) registers and memory appear with more than one address
> > range because the same physical byte is accessible at more than one
> > address. Using memory mirroring, higher memory reliability and a higher
> > level of memory consolidation are possible.
> >
> > These EFI memory regions have various attributes, and the "mirrored"
> > attribute is one of them. The physical memory region whose descriptors
> > in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> > mirrored. The address range mirroring feature of the kernel arranges such
> > mirrored regions into normal zones and other regions into movable zones.
> >
> > Arm64 can support this too. So mirrored memory support is added to support
> > arm64.
> >
> > The main purpose of this patch set is to introduce mirrored support for
> > arm64 and we have already fixed the problems we had which is shown in
> > patch #5 to patch #8 and try to bring total isolation in patch #9 which
> > will disable mirror feature if kernelcore is not specified.
> >
> > In order to test this support in arm64:
> > - patch this patch set
> > - add kernelcore=mirror in kernel parameter
> > - start you kernel
> >
> > Patch #1-#2 introduce mirrored memory support form arm64.
> > Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> > Patch #6 disable mirror feature if kernelcore is not specified.
> >
> > Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> > memory if kaslr is enabled.
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
> >
> > Changelog since v2:
> > - remove efi_fake_mem support
> > - remove Commit ("remove some redundant code in ia64 efi_init") since
> >   efi_print_memmap() is not public
> > - add mirror flag back on initrd memory
> >
> > Changelog since v1:
> > - update changelog in cover letter
> > - use PHYS_PFN in patch #7
> >
> > Ma Wupeng (6):
> >   efi: Make efi_find_mirror() public
> >   arm64/mirror: arm64 enabling - find mirrored memory ranges
> >   mm: Ratelimited mirrored memory related warning messages
> >   mm: Demote warning message in vmemmap_verify() to debug level
> >   mm: Add mirror flag back on initrd memory
> >   efi: Disable mirror feature if kernelcore is not specified
> >
>
> I have tested these changes on QEMU/arm64 with the patch below, and
> things seem to work as expected. We have some minor issues to work out
> but the general shape of this code is good.
>
> As for the mm/ changes: does anyone mind if I take those through the
> EFI tree as well? I don't think the EFI and -mm changes depend on each
> other, so they can go into -mm separately as well.

diff --git a/drivers/firmware/efi/libstub/efi-stub.c
b/drivers/firmware/efi/libstub/efi-stub.c
index f515394cce6e..1d4dd8aca3e6 100644
--- a/drivers/firmware/efi/libstub/efi-stub.c
+++ b/drivers/firmware/efi/libstub/efi-stub.c
@@ -136,6 +136,7 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
        enum efi_secureboot_mode secure_boot;
        struct screen_info *si;
        efi_properties_table_t *prop_tbl;
+       const efi_dxe_services_table_t *efi_dxe_table;

        efi_system_table = sys_table_arg;

@@ -161,6 +162,20 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
                goto fail;
        }

+       efi_dxe_table = get_efi_config_table(EFI_DXE_SERVICES_TABLE_GUID);
+       if (efi_dxe_table) {
+               efi_physical_addr_t pa = 0x60000000;
+               efi_gcd_memory_space_desc_t desc;
+
+               status = efi_dxe_call(get_memory_space_descriptor, pa, &desc);
+               if (status != EFI_SUCCESS)
+                       efi_err("Failed to get memory space
descriptor: %lx\n", status);
+               status = efi_dxe_call(set_memory_space_capabilities, pa, SZ_1G,
+                                     desc.capabilities |
EFI_MEMORY_MORE_RELIABLE);
+               if (status != EFI_SUCCESS)
+                       efi_err("Failed to set memory space
capabilities: %lx\n", status);
+       }
+
        /*
         * Get the command line from EFI, using the LOADED_IMAGE
         * protocol. We are going to copy the command line into the
diff --git a/drivers/firmware/efi/libstub/efistub.h
b/drivers/firmware/efi/libstub/efistub.h
index b0ae0a454404..bf11d85bf9b4 100644
--- a/drivers/firmware/efi/libstub/efistub.h
+++ b/drivers/firmware/efi/libstub/efistub.h
@@ -378,7 +378,8 @@ union efi_dxe_services_table {
                void *schedule;
                void *trust;
                void *process_firmware_volume;
-               void *set_memory_space_capabilities;
+               efi_status_t (__efiapi
*set_memory_space_capabilities)(efi_physical_addr_t,
+
u64, u64);
        };
        struct {
                efi_table_hdr_t hdr;

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Kefeng Wang]

On 2022/6/10 19:20, Ard Biesheuvel wrote:
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>> From: Ma Wupeng <mawupeng1@huawei.com>
>>
>> If system have some mirrored memory and mirrored feature is not specified
>> in boot parameter, the basic mirrored feature will be enabled and this will
>> lead to the following situations:
>>
>> - memblock memory allocation prefers mirrored region. This may have some
>>    unexpected influence on numa affinity.
>>
>> - contiguous memory will be split into several parts if parts of them
>>    is mirrored memory via memblock_mark_mirror().
>>
>> To fix this, variable mirrored_kernelcore will be checked before calling
>> efi_find_mirror() which will enable basic mirrored feature and this
>> variable is true if kernelcore=mirror is added in the kernel parameters.
>>
>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> This seems like the wrong place to do this. If mirrored memory is
> irrelevant to memblock, it should ignore the attribute. So I think
> this check belongs in mm/memblock.c instead.

Sound good,  let's add check into memblock_mark_mirror() and retain

the efi memory mirror information printing.

>
>
>> ---
>>   drivers/firmware/efi/efi.c | 3 +++
>>   include/linux/mm.h         | 2 ++
>>   mm/page_alloc.c            | 2 +-
>>   3 files changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
>> index 79c232e07de7..8a5edcb0dd82 100644
>> --- a/drivers/firmware/efi/efi.c
>> +++ b/drivers/firmware/efi/efi.c
>> @@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
>>          if (!efi_enabled(EFI_MEMMAP))
>>                  return;
>>
>> +       if (!mirrored_kernelcore)
>> +               return;
>> +
>>          for_each_efi_memory_desc(md) {
>>                  unsigned long long start = md->phys_addr;
>>                  unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index bc8f326be0ce..741ac7d022c3 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
>>                          unsigned long *start_pfn, unsigned long *end_pfn);
>>   extern unsigned long find_min_pfn_with_active_regions(void);
>>
>> +extern bool mirrored_kernelcore;
>> +
>>   #ifndef CONFIG_NUMA
>>   static inline int early_pfn_to_nid(unsigned long pfn)
>>   {
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index e008a3df0485..cf6f70aba787 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
>>   static unsigned long required_movablecore __initdata;
>>   static unsigned long required_movablecore_percent __initdata;
>>   static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
>> -static bool mirrored_kernelcore __meminitdata;
>> +bool mirrored_kernelcore __meminitdata;
>>
>>   /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
>>   int movable_zone;
>> --
>> 2.25.1
>>
> .

Re: [PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Kefeng Wang]

On 2022/6/10 19:20, Ard Biesheuvel wrote:
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>> From: Ma Wupeng <mawupeng1@huawei.com>
>>
>> If system have some mirrored memory and mirrored feature is not specified
>> in boot parameter, the basic mirrored feature will be enabled and this will
>> lead to the following situations:
>>
>> - memblock memory allocation prefers mirrored region. This may have some
>>    unexpected influence on numa affinity.
>>
>> - contiguous memory will be split into several parts if parts of them
>>    is mirrored memory via memblock_mark_mirror().
>>
>> To fix this, variable mirrored_kernelcore will be checked before calling
>> efi_find_mirror() which will enable basic mirrored feature and this
>> variable is true if kernelcore=mirror is added in the kernel parameters.
>>
>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> This seems like the wrong place to do this. If mirrored memory is
> irrelevant to memblock, it should ignore the attribute. So I think
> this check belongs in mm/memblock.c instead.

Sound good,  let's add check into memblock_mark_mirror() and retain

the efi memory mirror information printing.

>
>
>> ---
>>   drivers/firmware/efi/efi.c | 3 +++
>>   include/linux/mm.h         | 2 ++
>>   mm/page_alloc.c            | 2 +-
>>   3 files changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
>> index 79c232e07de7..8a5edcb0dd82 100644
>> --- a/drivers/firmware/efi/efi.c
>> +++ b/drivers/firmware/efi/efi.c
>> @@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
>>          if (!efi_enabled(EFI_MEMMAP))
>>                  return;
>>
>> +       if (!mirrored_kernelcore)
>> +               return;
>> +
>>          for_each_efi_memory_desc(md) {
>>                  unsigned long long start = md->phys_addr;
>>                  unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index bc8f326be0ce..741ac7d022c3 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
>>                          unsigned long *start_pfn, unsigned long *end_pfn);
>>   extern unsigned long find_min_pfn_with_active_regions(void);
>>
>> +extern bool mirrored_kernelcore;
>> +
>>   #ifndef CONFIG_NUMA
>>   static inline int early_pfn_to_nid(unsigned long pfn)
>>   {
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index e008a3df0485..cf6f70aba787 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
>>   static unsigned long required_movablecore __initdata;
>>   static unsigned long required_movablecore_percent __initdata;
>>   static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
>> -static bool mirrored_kernelcore __meminitdata;
>> +bool mirrored_kernelcore __meminitdata;
>>
>>   /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
>>   int movable_zone;
>> --
>> 2.25.1
>>
> .

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 6/6] efi: Disable mirror feature if kernelcore is not specified

[Kefeng Wang]

On 2022/6/10 19:20, Ard Biesheuvel wrote:
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
>> From: Ma Wupeng <mawupeng1@huawei.com>
>>
>> If system have some mirrored memory and mirrored feature is not specified
>> in boot parameter, the basic mirrored feature will be enabled and this will
>> lead to the following situations:
>>
>> - memblock memory allocation prefers mirrored region. This may have some
>>    unexpected influence on numa affinity.
>>
>> - contiguous memory will be split into several parts if parts of them
>>    is mirrored memory via memblock_mark_mirror().
>>
>> To fix this, variable mirrored_kernelcore will be checked before calling
>> efi_find_mirror() which will enable basic mirrored feature and this
>> variable is true if kernelcore=mirror is added in the kernel parameters.
>>
>> Signed-off-by: Ma Wupeng <mawupeng1@huawei.com>
> This seems like the wrong place to do this. If mirrored memory is
> irrelevant to memblock, it should ignore the attribute. So I think
> this check belongs in mm/memblock.c instead.

Sound good,  let's add check into memblock_mark_mirror() and retain

the efi memory mirror information printing.

>
>
>> ---
>>   drivers/firmware/efi/efi.c | 3 +++
>>   include/linux/mm.h         | 2 ++
>>   mm/page_alloc.c            | 2 +-
>>   3 files changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
>> index 79c232e07de7..8a5edcb0dd82 100644
>> --- a/drivers/firmware/efi/efi.c
>> +++ b/drivers/firmware/efi/efi.c
>> @@ -454,6 +454,9 @@ void __init efi_find_mirror(void)
>>          if (!efi_enabled(EFI_MEMMAP))
>>                  return;
>>
>> +       if (!mirrored_kernelcore)
>> +               return;
>> +
>>          for_each_efi_memory_desc(md) {
>>                  unsigned long long start = md->phys_addr;
>>                  unsigned long long size = md->num_pages << EFI_PAGE_SHIFT;
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index bc8f326be0ce..741ac7d022c3 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -2540,6 +2540,8 @@ extern void get_pfn_range_for_nid(unsigned int nid,
>>                          unsigned long *start_pfn, unsigned long *end_pfn);
>>   extern unsigned long find_min_pfn_with_active_regions(void);
>>
>> +extern bool mirrored_kernelcore;
>> +
>>   #ifndef CONFIG_NUMA
>>   static inline int early_pfn_to_nid(unsigned long pfn)
>>   {
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index e008a3df0485..cf6f70aba787 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -356,7 +356,7 @@ static unsigned long required_kernelcore_percent __initdata;
>>   static unsigned long required_movablecore __initdata;
>>   static unsigned long required_movablecore_percent __initdata;
>>   static unsigned long zone_movable_pfn[MAX_NUMNODES] __initdata;
>> -static bool mirrored_kernelcore __meminitdata;
>> +bool mirrored_kernelcore __meminitdata;
>>
>>   /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
>>   int movable_zone;
>> --
>> 2.25.1
>>
> .

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Mike Rapoport]

On Fri, Jun 10, 2022 at 01:23:34PM +0200, Ard Biesheuvel wrote:
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
> >
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86. This support rely on UEFI to
> > report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
> >
> >   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
> >
> > Memory mirroring is a technique used to separate memory into two separate
> > channels, usually on a memory device, like a server. In memory mirroring,
> > one channel is copied to another to create redundancy. This method makes
> > input/output (I/O) registers and memory appear with more than one address
> > range because the same physical byte is accessible at more than one
> > address. Using memory mirroring, higher memory reliability and a higher
> > level of memory consolidation are possible.
> >
> > These EFI memory regions have various attributes, and the "mirrored"
> > attribute is one of them. The physical memory region whose descriptors
> > in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> > mirrored. The address range mirroring feature of the kernel arranges such
> > mirrored regions into normal zones and other regions into movable zones.
> >
> > Arm64 can support this too. So mirrored memory support is added to support
> > arm64.
> >
> > The main purpose of this patch set is to introduce mirrored support for
> > arm64 and we have already fixed the problems we had which is shown in
> > patch #5 to patch #8 and try to bring total isolation in patch #9 which
> > will disable mirror feature if kernelcore is not specified.
> >
> > In order to test this support in arm64:
> > - patch this patch set
> > - add kernelcore=mirror in kernel parameter
> > - start you kernel
> >
> > Patch #1-#2 introduce mirrored memory support form arm64.
> > Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> > Patch #6 disable mirror feature if kernelcore is not specified.
> >
> > Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> > memory if kaslr is enabled.
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
> >
> > Changelog since v2:
> > - remove efi_fake_mem support
> > - remove Commit ("remove some redundant code in ia64 efi_init") since
> >   efi_print_memmap() is not public
> > - add mirror flag back on initrd memory
> >
> > Changelog since v1:
> > - update changelog in cover letter
> > - use PHYS_PFN in patch #7
> >
> > Ma Wupeng (6):
> >   efi: Make efi_find_mirror() public
> >   arm64/mirror: arm64 enabling - find mirrored memory ranges
> >   mm: Ratelimited mirrored memory related warning messages
> >   mm: Demote warning message in vmemmap_verify() to debug level
> >   mm: Add mirror flag back on initrd memory
> >   efi: Disable mirror feature if kernelcore is not specified
> >
> 
> I have tested these changes on QEMU/arm64 with the patch below, and
> things seem to work as expected. We have some minor issues to work out
> but the general shape of this code is good.
> 
> As for the mm/ changes: does anyone mind if I take those through the
> EFI tree as well?

No objections from me.

> I don't think the EFI and -mm changes depend on each other, so they
> can go into -mm separately as well.

--
Sincerely yours,
Mike.

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Mike Rapoport]

On Fri, Jun 10, 2022 at 01:23:34PM +0200, Ard Biesheuvel wrote:
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
> >
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86. This support rely on UEFI to
> > report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
> >
> >   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
> >
> > Memory mirroring is a technique used to separate memory into two separate
> > channels, usually on a memory device, like a server. In memory mirroring,
> > one channel is copied to another to create redundancy. This method makes
> > input/output (I/O) registers and memory appear with more than one address
> > range because the same physical byte is accessible at more than one
> > address. Using memory mirroring, higher memory reliability and a higher
> > level of memory consolidation are possible.
> >
> > These EFI memory regions have various attributes, and the "mirrored"
> > attribute is one of them. The physical memory region whose descriptors
> > in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> > mirrored. The address range mirroring feature of the kernel arranges such
> > mirrored regions into normal zones and other regions into movable zones.
> >
> > Arm64 can support this too. So mirrored memory support is added to support
> > arm64.
> >
> > The main purpose of this patch set is to introduce mirrored support for
> > arm64 and we have already fixed the problems we had which is shown in
> > patch #5 to patch #8 and try to bring total isolation in patch #9 which
> > will disable mirror feature if kernelcore is not specified.
> >
> > In order to test this support in arm64:
> > - patch this patch set
> > - add kernelcore=mirror in kernel parameter
> > - start you kernel
> >
> > Patch #1-#2 introduce mirrored memory support form arm64.
> > Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> > Patch #6 disable mirror feature if kernelcore is not specified.
> >
> > Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> > memory if kaslr is enabled.
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
> >
> > Changelog since v2:
> > - remove efi_fake_mem support
> > - remove Commit ("remove some redundant code in ia64 efi_init") since
> >   efi_print_memmap() is not public
> > - add mirror flag back on initrd memory
> >
> > Changelog since v1:
> > - update changelog in cover letter
> > - use PHYS_PFN in patch #7
> >
> > Ma Wupeng (6):
> >   efi: Make efi_find_mirror() public
> >   arm64/mirror: arm64 enabling - find mirrored memory ranges
> >   mm: Ratelimited mirrored memory related warning messages
> >   mm: Demote warning message in vmemmap_verify() to debug level
> >   mm: Add mirror flag back on initrd memory
> >   efi: Disable mirror feature if kernelcore is not specified
> >
> 
> I have tested these changes on QEMU/arm64 with the patch below, and
> things seem to work as expected. We have some minor issues to work out
> but the general shape of this code is good.
> 
> As for the mm/ changes: does anyone mind if I take those through the
> EFI tree as well?

No objections from me.

> I don't think the EFI and -mm changes depend on each other, so they
> can go into -mm separately as well.

--
Sincerely yours,
Mike.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3 0/6] introduce mirrored memory support for arm64

[Mike Rapoport]

On Fri, Jun 10, 2022 at 01:23:34PM +0200, Ard Biesheuvel wrote:
> On Tue, 7 Jun 2022 at 11:16, Wupeng Ma <mawupeng1@huawei.com> wrote:
> >
> > From: Ma Wupeng <mawupeng1@huawei.com>
> >
> > Commit b05b9f5f9dcf ("x86, mirror: x86 enabling - find mirrored memory ranges")
> > introduced mirrored memory support for x86. This support rely on UEFI to
> > report mirrored memory address ranges.  See UEFI 2.5 spec pages 157-158:
> >
> >   http://www.uefi.org/sites/default/files/resources/UEFI%202_5.pdf
> >
> > Memory mirroring is a technique used to separate memory into two separate
> > channels, usually on a memory device, like a server. In memory mirroring,
> > one channel is copied to another to create redundancy. This method makes
> > input/output (I/O) registers and memory appear with more than one address
> > range because the same physical byte is accessible at more than one
> > address. Using memory mirroring, higher memory reliability and a higher
> > level of memory consolidation are possible.
> >
> > These EFI memory regions have various attributes, and the "mirrored"
> > attribute is one of them. The physical memory region whose descriptors
> > in EFI memory map has EFI_MEMORY_MORE_RELIABLE attribute (bit: 16) are
> > mirrored. The address range mirroring feature of the kernel arranges such
> > mirrored regions into normal zones and other regions into movable zones.
> >
> > Arm64 can support this too. So mirrored memory support is added to support
> > arm64.
> >
> > The main purpose of this patch set is to introduce mirrored support for
> > arm64 and we have already fixed the problems we had which is shown in
> > patch #5 to patch #8 and try to bring total isolation in patch #9 which
> > will disable mirror feature if kernelcore is not specified.
> >
> > In order to test this support in arm64:
> > - patch this patch set
> > - add kernelcore=mirror in kernel parameter
> > - start you kernel
> >
> > Patch #1-#2 introduce mirrored memory support form arm64.
> > Patch #3-#5 fix some bugs for arm64 if memory reliable is enabled.
> > Patch #6 disable mirror feature if kernelcore is not specified.
> >
> > Thanks to Ard Biesheuvel's hard work [1], now kernel will perfer mirrored
> > memory if kaslr is enabled.
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/CAMj1kXEPVEzMgOM4+Yj6PxHA-jFuDOAUdDJSiSxy_XaP4P7LSw@mail.gmail.com/T/
> >
> > Changelog since v2:
> > - remove efi_fake_mem support
> > - remove Commit ("remove some redundant code in ia64 efi_init") since
> >   efi_print_memmap() is not public
> > - add mirror flag back on initrd memory
> >
> > Changelog since v1:
> > - update changelog in cover letter
> > - use PHYS_PFN in patch #7
> >
> > Ma Wupeng (6):
> >   efi: Make efi_find_mirror() public
> >   arm64/mirror: arm64 enabling - find mirrored memory ranges
> >   mm: Ratelimited mirrored memory related warning messages
> >   mm: Demote warning message in vmemmap_verify() to debug level
> >   mm: Add mirror flag back on initrd memory
> >   efi: Disable mirror feature if kernelcore is not specified
> >
> 
> I have tested these changes on QEMU/arm64 with the patch below, and
> things seem to work as expected. We have some minor issues to work out
> but the general shape of this code is good.
> 
> As for the mm/ changes: does anyone mind if I take those through the
> EFI tree as well?

No objections from me.

> I don't think the EFI and -mm changes depend on each other, so they
> can go into -mm separately as well.

--
Sincerely yours,
Mike.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel