* [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM
@ 2020-07-18 16:26 Mike Rapoport
2020-07-18 16:26 ` [PATCH v3 1/3] m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM Mike Rapoport
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: Mike Rapoport @ 2020-07-18 16:26 UTC (permalink / raw)
To: linux-m68k
Cc: Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain,
John Paul Adrian Glaubitz, Michael Schmitz, Mike Rapoport,
Mike Rapoport
From: Mike Rapoport <rppt@linux.ibm.com>
Hi,
It took me a while to get back to this, but better late than never :)
These patches replace DISCONTIGMEM with SPARSEMEM on m68k for systems with
!SINGLE_MEMORY_CHUNK set.
With SPARSEMEM there is a single node for the entire physical memory and to
cope with holes in the physical address space it is divided to sections of
several megabytes.
Each section has it's own memory map which size depends on actual populated
memory.
The section size of 16M was chosen pretty much arbitrarily as I couldn't
find specs for systems with e.g. Zorro memory extensions.
Yet, we cannot use smaller sections and still be able to address the entire
4G of the physical memory because the section number is encoded in the page
flags and there are only 8 bits available.
For systems with several small memory chunks and with small (several megs)
holes between them, 16M section size would cause wasted parts in the memory
map and it is desirable to allow smaller section size at the expense of
limiting the addressable memory.
I've hesitated between adding Kconfig option as Finn suggested [1] and
other options like SPARSE_VMEMMAP or ARCH_HAS_HOLES_MEMORYMODEL. In the
end, I think Kconfig is the simplest one from the implementation point of
view and would work fine for people that run mainline kernels on vintage
hardware.
For the systems with ST-RAM and FastRAM, the switch to SPARSEMEM does not
change the limitation that if the kernel is loaded into the FastRam the
ST-RAM remains unmapped. It only ensures that if the kernel is loaded in
ST-RAM, the memory map is allocated from high physical addresses and then
atari/stram.c is able to reserve the frame buffer memory. If the kernel is
loaded to FastRAM, the ST-RAM remains unmapped as with DISCONTIGMEM and the
atari/stram.c maps it as IOMEM.
[1] https://marc.info/?l=linux-m68k&m=156309170500921&w=2
v3 changes:
* rebase on the current upstream
* alias ARCH_PFN_BASE to m68k_memory[0].addr
* add configuration option to allow two variants of section size.
v2 changes:
* rebase on the current upstream
* make ColdFire MMU select SINGLE_MEMORY_CHUNK in Kconfig.cpu
Mike Rapoport (3):
m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM
m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM
m68k/mm: switch from DISCONTIGMEM to SPARSEMEM
arch/m68k/Kconfig.cpu | 14 ++++++--
arch/m68k/include/asm/page.h | 2 ++
arch/m68k/include/asm/page_mm.h | 6 +++-
arch/m68k/include/asm/sparsemem.h | 8 +++++
arch/m68k/include/asm/virtconvert.h | 2 +-
arch/m68k/mm/init.c | 8 ++---
arch/m68k/mm/motorola.c | 64 ++++++++++++++++++++++++++++++-------
7 files changed, 84 insertions(+), 20 deletions(-)
create mode 100644 arch/m68k/include/asm/sparsemem.h
--
2.7.4
*** BLURB HERE ***
Mike Rapoport (3):
m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM
m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM
m68k/mm: switch from DISCONTIGMEM to SPARSEMEM
arch/m68k/Kconfig.cpu | 23 ++++++++++++++---
arch/m68k/include/asm/page.h | 2 ++
arch/m68k/include/asm/page_mm.h | 7 +++++-
arch/m68k/include/asm/virtconvert.h | 2 +-
arch/m68k/mm/init.c | 8 +++---
arch/m68k/mm/motorola.c | 39 ++++++++++++++++++++++++++---
6 files changed, 69 insertions(+), 12 deletions(-)
--
2.26.2
^ permalink raw reply [flat|nested] 14+ messages in thread* [PATCH v3 1/3] m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM 2020-07-18 16:26 [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM Mike Rapoport @ 2020-07-18 16:26 ` Mike Rapoport 2020-07-18 16:26 ` [PATCH v3 2/3] m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM Mike Rapoport ` (2 subsequent siblings) 3 siblings, 0 replies; 14+ messages in thread From: Mike Rapoport @ 2020-07-18 16:26 UTC (permalink / raw) To: linux-m68k Cc: Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Michael Schmitz, Mike Rapoport, Mike Rapoport From: Mike Rapoport <rppt@linux.ibm.com> The pg_data_t node structures and their initialization currently depends on !CONFIG_SINGLE_MEMORY_CHUNK. Since their are required only for DISCONTIGMEM make this dependency explicit and replace usage of CONFIG_SINGLE_MEMORY_CHUNK with CONFIG_DISCONTIGMEM where appropriate. The CONFIG_SINGLE_MEMORY_CHUNK was implicitly disabled on the ColdFire MMU variant, although it always presumed a single memory bank. As there is no actual need for DISCONTIGMEM in this case, make sure that ColdFire MMU systems set CONFIG_SINGLE_MEMORY_CHUNK to 'y'. Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> --- arch/m68k/Kconfig.cpu | 6 +++--- arch/m68k/include/asm/page_mm.h | 2 +- arch/m68k/mm/init.c | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/m68k/Kconfig.cpu b/arch/m68k/Kconfig.cpu index 694c4fca9f5d..3af0fca03803 100644 --- a/arch/m68k/Kconfig.cpu +++ b/arch/m68k/Kconfig.cpu @@ -20,6 +20,7 @@ choice config M68KCLASSIC bool "Classic M68K CPU family support" + select NEED_MULTIPLE_NODES if DISCONTIGMEM config COLDFIRE bool "Coldfire CPU family support" @@ -373,8 +374,7 @@ config RMW_INSNS config SINGLE_MEMORY_CHUNK bool "Use one physical chunk of memory only" if ADVANCED && !SUN3 depends on MMU - default y if SUN3 - select NEED_MULTIPLE_NODES + default y if SUN3 || MMU_COLDFIRE help Ignore all but the first contiguous chunk of physical memory for VM purposes. This will save a few bytes kernel size and may speed up @@ -406,7 +406,7 @@ config M68K_L2_CACHE config NODES_SHIFT int default "3" - depends on !SINGLE_MEMORY_CHUNK + depends on DISCONTIGMEM config CPU_HAS_NO_BITFIELDS bool diff --git a/arch/m68k/include/asm/page_mm.h b/arch/m68k/include/asm/page_mm.h index e6b75992192b..0e794051d3bb 100644 --- a/arch/m68k/include/asm/page_mm.h +++ b/arch/m68k/include/asm/page_mm.h @@ -126,7 +126,7 @@ static inline void *__va(unsigned long x) extern int m68k_virt_to_node_shift; -#ifdef CONFIG_SINGLE_MEMORY_CHUNK +#ifndef CONFIG_DISCONTIGMEM #define __virt_to_node(addr) (&pg_data_map[0]) #else extern struct pglist_data *pg_data_table[]; diff --git a/arch/m68k/mm/init.c b/arch/m68k/mm/init.c index 53040857a9ed..4b46ceace3d3 100644 --- a/arch/m68k/mm/init.c +++ b/arch/m68k/mm/init.c @@ -47,14 +47,14 @@ EXPORT_SYMBOL(pg_data_map); int m68k_virt_to_node_shift; -#ifndef CONFIG_SINGLE_MEMORY_CHUNK +#ifdef CONFIG_DISCONTIGMEM pg_data_t *pg_data_table[65]; EXPORT_SYMBOL(pg_data_table); #endif void __init m68k_setup_node(int node) { -#ifndef CONFIG_SINGLE_MEMORY_CHUNK +#ifdef CONFIG_DISCONTIGMEM struct m68k_mem_info *info = m68k_memory + node; int i, end; -- 2.26.2 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v3 2/3] m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM 2020-07-18 16:26 [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM Mike Rapoport 2020-07-18 16:26 ` [PATCH v3 1/3] m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM Mike Rapoport @ 2020-07-18 16:26 ` Mike Rapoport 2020-07-18 16:26 ` [PATCH v3 3/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM Mike Rapoport 2020-08-20 16:03 ` [PATCH v3 0/3] " Mike Rapoport 3 siblings, 0 replies; 14+ messages in thread From: Mike Rapoport @ 2020-07-18 16:26 UTC (permalink / raw) To: linux-m68k Cc: Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Michael Schmitz, Mike Rapoport, Mike Rapoport From: Mike Rapoport <rppt@linux.ibm.com> The pg_data_map and pg_data_table arrays as well as page_to_pfn() and pfn_to_page() are required only for DISCONTIGMEM. Other memory models can use the generic definitions in asm-generic/memory_model.h. Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> --- arch/m68k/include/asm/page.h | 2 ++ arch/m68k/include/asm/page_mm.h | 5 +++++ arch/m68k/include/asm/virtconvert.h | 2 +- arch/m68k/mm/init.c | 6 +++--- 4 files changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/m68k/include/asm/page.h b/arch/m68k/include/asm/page.h index 2614a1206f2f..6116d7094292 100644 --- a/arch/m68k/include/asm/page.h +++ b/arch/m68k/include/asm/page.h @@ -62,8 +62,10 @@ extern unsigned long _ramend; #include <asm/page_no.h> #endif +#ifdef CONFIG_DISCONTIGMEM #define __phys_to_pfn(paddr) ((unsigned long)((paddr) >> PAGE_SHIFT)) #define __pfn_to_phys(pfn) PFN_PHYS(pfn) +#endif #include <asm-generic/getorder.h> diff --git a/arch/m68k/include/asm/page_mm.h b/arch/m68k/include/asm/page_mm.h index 0e794051d3bb..7f5912af2a52 100644 --- a/arch/m68k/include/asm/page_mm.h +++ b/arch/m68k/include/asm/page_mm.h @@ -153,6 +153,7 @@ static inline __attribute_const__ int __virt_to_node_shift(void) pfn_to_virt(page_to_pfn(page)); \ }) +#ifdef CONFIG_DISCONTIGMEM #define pfn_to_page(pfn) ({ \ unsigned long __pfn = (pfn); \ struct pglist_data *pgdat; \ @@ -165,6 +166,10 @@ static inline __attribute_const__ int __virt_to_node_shift(void) pgdat = &pg_data_map[page_to_nid(__p)]; \ ((__p) - pgdat->node_mem_map) + pgdat->node_start_pfn; \ }) +#else +#define ARCH_PFN_OFFSET (m68k_memory[0].addr) +#include <asm-generic/memory_model.h> +#endif #define virt_addr_valid(kaddr) ((void *)(kaddr) >= (void *)PAGE_OFFSET && (void *)(kaddr) < high_memory) #define pfn_valid(pfn) virt_addr_valid(pfn_to_virt(pfn)) diff --git a/arch/m68k/include/asm/virtconvert.h b/arch/m68k/include/asm/virtconvert.h index dfe43083b579..751bb6f4aaf6 100644 --- a/arch/m68k/include/asm/virtconvert.h +++ b/arch/m68k/include/asm/virtconvert.h @@ -31,7 +31,7 @@ static inline void *phys_to_virt(unsigned long address) /* Permanent address of a page. */ #if defined(CONFIG_MMU) && defined(CONFIG_SINGLE_MEMORY_CHUNK) #define page_to_phys(page) \ - __pa(PAGE_OFFSET + (((page) - pg_data_map[0].node_mem_map) << PAGE_SHIFT)) + __pa(PAGE_OFFSET + (((page) - mem_map) << PAGE_SHIFT)) #else #define page_to_phys(page) (page_to_pfn(page) << PAGE_SHIFT) #endif diff --git a/arch/m68k/mm/init.c b/arch/m68k/mm/init.c index 4b46ceace3d3..14c1e541451c 100644 --- a/arch/m68k/mm/init.c +++ b/arch/m68k/mm/init.c @@ -42,12 +42,12 @@ EXPORT_SYMBOL(empty_zero_page); #ifdef CONFIG_MMU -pg_data_t pg_data_map[MAX_NUMNODES]; -EXPORT_SYMBOL(pg_data_map); - int m68k_virt_to_node_shift; #ifdef CONFIG_DISCONTIGMEM +pg_data_t pg_data_map[MAX_NUMNODES]; +EXPORT_SYMBOL(pg_data_map); + pg_data_t *pg_data_table[65]; EXPORT_SYMBOL(pg_data_table); #endif -- 2.26.2 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH v3 3/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-07-18 16:26 [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM Mike Rapoport 2020-07-18 16:26 ` [PATCH v3 1/3] m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM Mike Rapoport 2020-07-18 16:26 ` [PATCH v3 2/3] m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM Mike Rapoport @ 2020-07-18 16:26 ` Mike Rapoport 2020-08-20 16:03 ` [PATCH v3 0/3] " Mike Rapoport 3 siblings, 0 replies; 14+ messages in thread From: Mike Rapoport @ 2020-07-18 16:26 UTC (permalink / raw) To: linux-m68k Cc: Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Michael Schmitz, Mike Rapoport, Mike Rapoport From: Mike Rapoport <rppt@linux.ibm.com> Enable SPARSEMEM support for systems with !SINGLE_MEMORY_CHUNK. With SPARSEMEM there is a single node for the entire physical memory and to cope with holes in the physical address space it is divided to sections of up to 4M or 16M, depending on CONFIG_MEMORY_LIMIT_1G. Each section has it's own memory map which size depends on actual populated memory. The DISCONTIGMEM is marked BROKEN and will be removed in a couple of releases. Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> --- arch/m68k/Kconfig.cpu | 17 +++++++++++++++++ arch/m68k/mm/motorola.c | 39 ++++++++++++++++++++++++++++++++++++--- 2 files changed, 53 insertions(+), 3 deletions(-) diff --git a/arch/m68k/Kconfig.cpu b/arch/m68k/Kconfig.cpu index 3af0fca03803..4e8a124741d6 100644 --- a/arch/m68k/Kconfig.cpu +++ b/arch/m68k/Kconfig.cpu @@ -21,6 +21,7 @@ choice config M68KCLASSIC bool "Classic M68K CPU family support" select NEED_MULTIPLE_NODES if DISCONTIGMEM + select SPARSEMEM_STATIC if SPARSEMEM config COLDFIRE bool "Coldfire CPU family support" @@ -380,9 +381,25 @@ config SINGLE_MEMORY_CHUNK purposes. This will save a few bytes kernel size and may speed up some operations. Say N if not sure. +config MEMORY_LIMIT_1G + bool "Limit maximal physical memory to 1G" + depends on MMU && !SINGLE_MEMORY_CHUNK + help + Limit the physical memory addressable by the kernel to 1G. + On systems that have several chunks of physical memory and the + hole between these chunks is less than 16M this will prevent + wasting memory on empty memory map. Say N if not sure. + config ARCH_DISCONTIGMEM_ENABLE + def_bool no + depends on BROKEN && MMU && !SINGLE_MEMORY_CHUNK + +config ARCH_SPARSEMEM_ENABLE def_bool MMU && !SINGLE_MEMORY_CHUNK +config HAVE_ARCH_PFN_VALID + def_bool SPARSEMEM + config 060_WRITETHROUGH bool "Use write-through caching for 68060 supervisor accesses" depends on ADVANCED && M68060 diff --git a/arch/m68k/mm/motorola.c b/arch/m68k/mm/motorola.c index 2bb006bdc31c..6a513b9c919d 100644 --- a/arch/m68k/mm/motorola.c +++ b/arch/m68k/mm/motorola.c @@ -384,6 +384,39 @@ static void __init map_node(int node) #endif } +#ifdef CONFIG_SPARSEMEM +static void mem_model_init(void) +{ + m68k_setup_node(0); + node_set_state(0, N_NORMAL_MEMORY); + + /* + * Make sure the memory map is allocated from top of the + * memory. + * Otherwise for systems with both ST-RAM and FastRam, ST-RAM + * gets filled with the memory map leaving no space for + * framebuffer + */ + memblock_set_bottom_up(false); + memblocks_present(); + sparse_init(); + memblock_set_bottom_up(true); +} +#else +static void mem_model_init(void) +{ + unsigned long zones_size[MAX_NR_ZONES] = { 0, }; + int i; + + for (i = 0; i < m68k_num_memory; i++) { + m68k_setup_node(i); + if (node_present_pages(i)) + node_set_state(i, N_NORMAL_MEMORY); + } +} +#endif + + /* * paging_init() continues the virtual memory environment setup which * was begun by the code in arch/head.S. @@ -449,10 +482,8 @@ void __init paging_init(void) */ memblock_set_bottom_up(true); - for (i = 0; i < m68k_num_memory; i++) { - m68k_setup_node(i); + for (i = 0; i < m68k_num_memory; i++) map_node(i); - } flush_tlb_all(); @@ -477,6 +508,8 @@ void __init paging_init(void) if (node_present_pages(i)) node_set_state(i, N_NORMAL_MEMORY); + mem_model_init(); + max_zone_pfn[ZONE_DMA] = memblock_end_of_DRAM(); free_area_init(max_zone_pfn); } -- 2.26.2 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-07-18 16:26 [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM Mike Rapoport ` (2 preceding siblings ...) 2020-07-18 16:26 ` [PATCH v3 3/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM Mike Rapoport @ 2020-08-20 16:03 ` Mike Rapoport 2020-08-20 22:29 ` Michael Schmitz 3 siblings, 1 reply; 14+ messages in thread From: Mike Rapoport @ 2020-08-20 16:03 UTC (permalink / raw) To: linux-m68k Cc: Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Michael Schmitz, Mike Rapoport Gentle ping :) On Sat, Jul 18, 2020 at 07:26:48PM +0300, Mike Rapoport wrote: > From: Mike Rapoport <rppt@linux.ibm.com> > > Hi, > > It took me a while to get back to this, but better late than never :) > > These patches replace DISCONTIGMEM with SPARSEMEM on m68k for systems with > !SINGLE_MEMORY_CHUNK set. > > With SPARSEMEM there is a single node for the entire physical memory and to > cope with holes in the physical address space it is divided to sections of > several megabytes. > > Each section has it's own memory map which size depends on actual populated > memory. > > The section size of 16M was chosen pretty much arbitrarily as I couldn't > find specs for systems with e.g. Zorro memory extensions. > Yet, we cannot use smaller sections and still be able to address the entire > 4G of the physical memory because the section number is encoded in the page > flags and there are only 8 bits available. > > For systems with several small memory chunks and with small (several megs) > holes between them, 16M section size would cause wasted parts in the memory > map and it is desirable to allow smaller section size at the expense of > limiting the addressable memory. > > I've hesitated between adding Kconfig option as Finn suggested [1] and > other options like SPARSE_VMEMMAP or ARCH_HAS_HOLES_MEMORYMODEL. In the > end, I think Kconfig is the simplest one from the implementation point of > view and would work fine for people that run mainline kernels on vintage > hardware. > > For the systems with ST-RAM and FastRAM, the switch to SPARSEMEM does not > change the limitation that if the kernel is loaded into the FastRam the > ST-RAM remains unmapped. It only ensures that if the kernel is loaded in > ST-RAM, the memory map is allocated from high physical addresses and then > atari/stram.c is able to reserve the frame buffer memory. If the kernel is > loaded to FastRAM, the ST-RAM remains unmapped as with DISCONTIGMEM and the > atari/stram.c maps it as IOMEM. > > [1] https://marc.info/?l=linux-m68k&m=156309170500921&w=2 > > v3 changes: > * rebase on the current upstream > * alias ARCH_PFN_BASE to m68k_memory[0].addr > * add configuration option to allow two variants of section size. > > v2 changes: > * rebase on the current upstream > * make ColdFire MMU select SINGLE_MEMORY_CHUNK in Kconfig.cpu > > Mike Rapoport (3): > m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM > m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM > m68k/mm: switch from DISCONTIGMEM to SPARSEMEM > > arch/m68k/Kconfig.cpu | 14 ++++++-- > arch/m68k/include/asm/page.h | 2 ++ > arch/m68k/include/asm/page_mm.h | 6 +++- > arch/m68k/include/asm/sparsemem.h | 8 +++++ > arch/m68k/include/asm/virtconvert.h | 2 +- > arch/m68k/mm/init.c | 8 ++--- > arch/m68k/mm/motorola.c | 64 ++++++++++++++++++++++++++++++------- > 7 files changed, 84 insertions(+), 20 deletions(-) > create mode 100644 arch/m68k/include/asm/sparsemem.h > > -- > 2.7.4 > > > *** BLURB HERE *** > > Mike Rapoport (3): > m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM > m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM > m68k/mm: switch from DISCONTIGMEM to SPARSEMEM > > arch/m68k/Kconfig.cpu | 23 ++++++++++++++--- > arch/m68k/include/asm/page.h | 2 ++ > arch/m68k/include/asm/page_mm.h | 7 +++++- > arch/m68k/include/asm/virtconvert.h | 2 +- > arch/m68k/mm/init.c | 8 +++--- > arch/m68k/mm/motorola.c | 39 ++++++++++++++++++++++++++--- > 6 files changed, 69 insertions(+), 12 deletions(-) > > -- > 2.26.2 > -- Sincerely yours, Mike. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-20 16:03 ` [PATCH v3 0/3] " Mike Rapoport @ 2020-08-20 22:29 ` Michael Schmitz 2020-08-21 7:56 ` Mike Rapoport 0 siblings, 1 reply; 14+ messages in thread From: Michael Schmitz @ 2020-08-20 22:29 UTC (permalink / raw) To: Mike Rapoport, linux-m68k Cc: Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport Hi Mike, Sorry, I had seen those but didn't get around to test in time. Applying your patch series to Geert's v5.9-rc1, I get this: > CC kernel/bounds.s > CC arch/m68k/kernel/asm-offsets.s > In file included from ./include/linux/mmzone.h:19, > from ./include/linux/topology.h:33, > from ./include/linux/irq.h:19, > from ./include/asm-generic/hardirq.h:13, > from ./arch/m68k/include/generated/asm/hardirq.h:1, > from ./include/linux/hardirq.h:10, > from ./include/linux/interrupt.h:11, > from ./include/linux/kernel_stat.h:9, > from arch/m68k/kernel/asm-offsets.c:16: > ./include/linux/page-flags-layout.h:28:10: fatal error: > asm/sparsemem.h: No such file or directory > #include <asm/sparsemem.h> > ^~~~~~~~~~~~~~~~~ > compilation terminated. > make[2]: *** [arch/m68k/kernel/asm-offsets.s] Error 1 > make[1]: *** [prepare0] Error 2 > make: *** [__sub-make] Error 2 Your patch announcement blurb hat sparsemem.h listed among the changed files, but none of the three patches stats reference that file. Am I missing a prerequisite here? Cheers, Michael On 21/08/20 4:03 AM, Mike Rapoport wrote: > Gentle ping :) > > On Sat, Jul 18, 2020 at 07:26:48PM +0300, Mike Rapoport wrote: >> From: Mike Rapoport <rppt@linux.ibm.com> >> >> Hi, >> >> It took me a while to get back to this, but better late than never :) >> >> These patches replace DISCONTIGMEM with SPARSEMEM on m68k for systems with >> !SINGLE_MEMORY_CHUNK set. >> >> With SPARSEMEM there is a single node for the entire physical memory and to >> cope with holes in the physical address space it is divided to sections of >> several megabytes. >> >> Each section has it's own memory map which size depends on actual populated >> memory. >> >> The section size of 16M was chosen pretty much arbitrarily as I couldn't >> find specs for systems with e.g. Zorro memory extensions. >> Yet, we cannot use smaller sections and still be able to address the entire >> 4G of the physical memory because the section number is encoded in the page >> flags and there are only 8 bits available. >> >> For systems with several small memory chunks and with small (several megs) >> holes between them, 16M section size would cause wasted parts in the memory >> map and it is desirable to allow smaller section size at the expense of >> limiting the addressable memory. >> >> I've hesitated between adding Kconfig option as Finn suggested [1] and >> other options like SPARSE_VMEMMAP or ARCH_HAS_HOLES_MEMORYMODEL. In the >> end, I think Kconfig is the simplest one from the implementation point of >> view and would work fine for people that run mainline kernels on vintage >> hardware. >> >> For the systems with ST-RAM and FastRAM, the switch to SPARSEMEM does not >> change the limitation that if the kernel is loaded into the FastRam the >> ST-RAM remains unmapped. It only ensures that if the kernel is loaded in >> ST-RAM, the memory map is allocated from high physical addresses and then >> atari/stram.c is able to reserve the frame buffer memory. If the kernel is >> loaded to FastRAM, the ST-RAM remains unmapped as with DISCONTIGMEM and the >> atari/stram.c maps it as IOMEM. >> >> [1] https://marc.info/?l=linux-m68k&m=156309170500921&w=2 >> >> v3 changes: >> * rebase on the current upstream >> * alias ARCH_PFN_BASE to m68k_memory[0].addr >> * add configuration option to allow two variants of section size. >> >> v2 changes: >> * rebase on the current upstream >> * make ColdFire MMU select SINGLE_MEMORY_CHUNK in Kconfig.cpu >> >> Mike Rapoport (3): >> m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM >> m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM >> m68k/mm: switch from DISCONTIGMEM to SPARSEMEM >> >> arch/m68k/Kconfig.cpu | 14 ++++++-- >> arch/m68k/include/asm/page.h | 2 ++ >> arch/m68k/include/asm/page_mm.h | 6 +++- >> arch/m68k/include/asm/q.h | 8 +++++ >> arch/m68k/include/asm/virtconvert.h | 2 +- >> arch/m68k/mm/init.c | 8 ++--- >> arch/m68k/mm/motorola.c | 64 ++++++++++++++++++++++++++++++------- >> 7 files changed, 84 insertions(+), 20 deletions(-) >> create mode 100644 arch/m68k/include/asm/sparsemem.h >> >> -- >> 2.7.4 >> >> >> *** BLURB HERE *** >> >> Mike Rapoport (3): >> m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM >> m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM >> m68k/mm: switch from DISCONTIGMEM to SPARSEMEM >> >> arch/m68k/Kconfig.cpu | 23 ++++++++++++++--- >> arch/m68k/include/asm/page.h | 2 ++ >> arch/m68k/include/asm/page_mm.h | 7 +++++- >> arch/m68k/include/asm/virtconvert.h | 2 +- >> arch/m68k/mm/init.c | 8 +++--- >> arch/m68k/mm/motorola.c | 39 ++++++++++++++++++++++++++--- >> 6 files changed, 69 insertions(+), 12 deletions(-) >> >> -- >> 2.26.2 >> ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-20 22:29 ` Michael Schmitz @ 2020-08-21 7:56 ` Mike Rapoport 2020-08-21 20:58 ` Michael Schmitz 2020-08-21 23:27 ` Michael Schmitz 0 siblings, 2 replies; 14+ messages in thread From: Mike Rapoport @ 2020-08-21 7:56 UTC (permalink / raw) To: Michael Schmitz Cc: linux-m68k, Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport On Fri, Aug 21, 2020 at 10:29:13AM +1200, Michael Schmitz wrote: > Hi Mike, > > Sorry, I had seen those but didn't get around to test in time. > > Applying your patch series to Geert's v5.9-rc1, I get this: > > > CC kernel/bounds.s > > CC arch/m68k/kernel/asm-offsets.s > > In file included from ./include/linux/mmzone.h:19, > > from ./include/linux/topology.h:33, > > from ./include/linux/irq.h:19, > > from ./include/asm-generic/hardirq.h:13, > > from ./arch/m68k/include/generated/asm/hardirq.h:1, > > from ./include/linux/hardirq.h:10, > > from ./include/linux/interrupt.h:11, > > from ./include/linux/kernel_stat.h:9, > > from arch/m68k/kernel/asm-offsets.c:16: > > ./include/linux/page-flags-layout.h:28:10: fatal error: asm/sparsemem.h: > > No such file or directory > > #include <asm/sparsemem.h> > > ^~~~~~~~~~~~~~~~~ > > compilation terminated. > > make[2]: *** [arch/m68k/kernel/asm-offsets.s] Error 1 > > make[1]: *** [prepare0] Error 2 > > make: *** [__sub-make] Error 2 > > Your patch announcement blurb hat sparsemem.h listed among the changed > files, but none of the three patches stats reference that file. Am I missing > a prerequisite here? Argh, I forgot to 'git add asm/sparsemem.h'. I'll resend a refreshed version in a couple of days. > Cheers, > > Michael > > > On 21/08/20 4:03 AM, Mike Rapoport wrote: > > Gentle ping :) > > > > On Sat, Jul 18, 2020 at 07:26:48PM +0300, Mike Rapoport wrote: > > > From: Mike Rapoport <rppt@linux.ibm.com> > > > > > > Hi, > > > > > > It took me a while to get back to this, but better late than never :) > > > > > > These patches replace DISCONTIGMEM with SPARSEMEM on m68k for systems with > > > !SINGLE_MEMORY_CHUNK set. > > > > > > With SPARSEMEM there is a single node for the entire physical memory and to > > > cope with holes in the physical address space it is divided to sections of > > > several megabytes. > > > > > > Each section has it's own memory map which size depends on actual populated > > > memory. > > > > > > The section size of 16M was chosen pretty much arbitrarily as I couldn't > > > find specs for systems with e.g. Zorro memory extensions. > > > Yet, we cannot use smaller sections and still be able to address the entire > > > 4G of the physical memory because the section number is encoded in the page > > > flags and there are only 8 bits available. > > > > > > For systems with several small memory chunks and with small (several megs) > > > holes between them, 16M section size would cause wasted parts in the memory > > > map and it is desirable to allow smaller section size at the expense of > > > limiting the addressable memory. > > > > > > I've hesitated between adding Kconfig option as Finn suggested [1] and > > > other options like SPARSE_VMEMMAP or ARCH_HAS_HOLES_MEMORYMODEL. In the > > > end, I think Kconfig is the simplest one from the implementation point of > > > view and would work fine for people that run mainline kernels on vintage > > > hardware. > > > > > > For the systems with ST-RAM and FastRAM, the switch to SPARSEMEM does not > > > change the limitation that if the kernel is loaded into the FastRam the > > > ST-RAM remains unmapped. It only ensures that if the kernel is loaded in > > > ST-RAM, the memory map is allocated from high physical addresses and then > > > atari/stram.c is able to reserve the frame buffer memory. If the kernel is > > > loaded to FastRAM, the ST-RAM remains unmapped as with DISCONTIGMEM and the > > > atari/stram.c maps it as IOMEM. > > > > > > [1] https://marc.info/?l=linux-m68k&m=156309170500921&w=2 > > > > > > v3 changes: > > > * rebase on the current upstream > > > * alias ARCH_PFN_BASE to m68k_memory[0].addr > > > * add configuration option to allow two variants of section size. > > > > > > v2 changes: > > > * rebase on the current upstream > > > * make ColdFire MMU select SINGLE_MEMORY_CHUNK in Kconfig.cpu > > > > > > Mike Rapoport (3): > > > m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM > > > m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM > > > m68k/mm: switch from DISCONTIGMEM to SPARSEMEM > > > > > > arch/m68k/Kconfig.cpu | 14 ++++++-- > > > arch/m68k/include/asm/page.h | 2 ++ > > > arch/m68k/include/asm/page_mm.h | 6 +++- > > > arch/m68k/include/asm/q.h | 8 +++++ > > > arch/m68k/include/asm/virtconvert.h | 2 +- > > > arch/m68k/mm/init.c | 8 ++--- > > > arch/m68k/mm/motorola.c | 64 ++++++++++++++++++++++++++++++------- > > > 7 files changed, 84 insertions(+), 20 deletions(-) > > > create mode 100644 arch/m68k/include/asm/sparsemem.h > > > > > > -- > > > 2.7.4 > > > > > > > > > *** BLURB HERE *** > > > > > > Mike Rapoport (3): > > > m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM > > > m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM > > > m68k/mm: switch from DISCONTIGMEM to SPARSEMEM > > > > > > arch/m68k/Kconfig.cpu | 23 ++++++++++++++--- > > > arch/m68k/include/asm/page.h | 2 ++ > > > arch/m68k/include/asm/page_mm.h | 7 +++++- > > > arch/m68k/include/asm/virtconvert.h | 2 +- > > > arch/m68k/mm/init.c | 8 +++--- > > > arch/m68k/mm/motorola.c | 39 ++++++++++++++++++++++++++--- > > > 6 files changed, 69 insertions(+), 12 deletions(-) > > > > > > -- > > > 2.26.2 > > > -- Sincerely yours, Mike. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-21 7:56 ` Mike Rapoport @ 2020-08-21 20:58 ` Michael Schmitz 2020-08-21 23:27 ` Michael Schmitz 1 sibling, 0 replies; 14+ messages in thread From: Michael Schmitz @ 2020-08-21 20:58 UTC (permalink / raw) To: Mike Rapoport Cc: linux-m68k, Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport Hi Mike, Am 21.08.2020 um 19:56 schrieb Mike Rapoport: > On Fri, Aug 21, 2020 at 10:29:13AM +1200, Michael Schmitz wrote: >> Hi Mike, >> >> Sorry, I had seen those but didn't get around to test in time. >> >> Applying your patch series to Geert's v5.9-rc1, I get this: >> >>> CC kernel/bounds.s >>> CC arch/m68k/kernel/asm-offsets.s >>> In file included from ./include/linux/mmzone.h:19, >>> from ./include/linux/topology.h:33, >>> from ./include/linux/irq.h:19, >>> from ./include/asm-generic/hardirq.h:13, >>> from ./arch/m68k/include/generated/asm/hardirq.h:1, >>> from ./include/linux/hardirq.h:10, >>> from ./include/linux/interrupt.h:11, >>> from ./include/linux/kernel_stat.h:9, >>> from arch/m68k/kernel/asm-offsets.c:16: >>> ./include/linux/page-flags-layout.h:28:10: fatal error: asm/sparsemem.h: >>> No such file or directory >>> #include <asm/sparsemem.h> >>> ^~~~~~~~~~~~~~~~~ >>> compilation terminated. >>> make[2]: *** [arch/m68k/kernel/asm-offsets.s] Error 1 >>> make[1]: *** [prepare0] Error 2 >>> make: *** [__sub-make] Error 2 >> >> Your patch announcement blurb hat sparsemem.h listed among the changed >> files, but none of the three patches stats reference that file. Am I missing >> a prerequisite here? > > Argh, I forgot to 'git add asm/sparsemem.h'. OK, that should be easy enough to overcome. I'll try setting the section size bits according to the 16 MB section size, and test with that for now. Cheers, Michael > I'll resend a refreshed version in a couple of days. > >> Cheers, >> >> Michael >> >> >> On 21/08/20 4:03 AM, Mike Rapoport wrote: >>> Gentle ping :) >>> >>> On Sat, Jul 18, 2020 at 07:26:48PM +0300, Mike Rapoport wrote: >>>> From: Mike Rapoport <rppt@linux.ibm.com> >>>> >>>> Hi, >>>> >>>> It took me a while to get back to this, but better late than never :) >>>> >>>> These patches replace DISCONTIGMEM with SPARSEMEM on m68k for systems with >>>> !SINGLE_MEMORY_CHUNK set. >>>> >>>> With SPARSEMEM there is a single node for the entire physical memory and to >>>> cope with holes in the physical address space it is divided to sections of >>>> several megabytes. >>>> >>>> Each section has it's own memory map which size depends on actual populated >>>> memory. >>>> >>>> The section size of 16M was chosen pretty much arbitrarily as I couldn't >>>> find specs for systems with e.g. Zorro memory extensions. >>>> Yet, we cannot use smaller sections and still be able to address the entire >>>> 4G of the physical memory because the section number is encoded in the page >>>> flags and there are only 8 bits available. >>>> >>>> For systems with several small memory chunks and with small (several megs) >>>> holes between them, 16M section size would cause wasted parts in the memory >>>> map and it is desirable to allow smaller section size at the expense of >>>> limiting the addressable memory. >>>> >>>> I've hesitated between adding Kconfig option as Finn suggested [1] and >>>> other options like SPARSE_VMEMMAP or ARCH_HAS_HOLES_MEMORYMODEL. In the >>>> end, I think Kconfig is the simplest one from the implementation point of >>>> view and would work fine for people that run mainline kernels on vintage >>>> hardware. >>>> >>>> For the systems with ST-RAM and FastRAM, the switch to SPARSEMEM does not >>>> change the limitation that if the kernel is loaded into the FastRam the >>>> ST-RAM remains unmapped. It only ensures that if the kernel is loaded in >>>> ST-RAM, the memory map is allocated from high physical addresses and then >>>> atari/stram.c is able to reserve the frame buffer memory. If the kernel is >>>> loaded to FastRAM, the ST-RAM remains unmapped as with DISCONTIGMEM and the >>>> atari/stram.c maps it as IOMEM. >>>> >>>> [1] https://marc.info/?l=linux-m68k&m=156309170500921&w=2 >>>> >>>> v3 changes: >>>> * rebase on the current upstream >>>> * alias ARCH_PFN_BASE to m68k_memory[0].addr >>>> * add configuration option to allow two variants of section size. >>>> >>>> v2 changes: >>>> * rebase on the current upstream >>>> * make ColdFire MMU select SINGLE_MEMORY_CHUNK in Kconfig.cpu >>>> >>>> Mike Rapoport (3): >>>> m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM >>>> m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM >>>> m68k/mm: switch from DISCONTIGMEM to SPARSEMEM >>>> >>>> arch/m68k/Kconfig.cpu | 14 ++++++-- >>>> arch/m68k/include/asm/page.h | 2 ++ >>>> arch/m68k/include/asm/page_mm.h | 6 +++- >>>> arch/m68k/include/asm/q.h | 8 +++++ >>>> arch/m68k/include/asm/virtconvert.h | 2 +- >>>> arch/m68k/mm/init.c | 8 ++--- >>>> arch/m68k/mm/motorola.c | 64 ++++++++++++++++++++++++++++++------- >>>> 7 files changed, 84 insertions(+), 20 deletions(-) >>>> create mode 100644 arch/m68k/include/asm/sparsemem.h >>>> >>>> -- >>>> 2.7.4 >>>> >>>> >>>> *** BLURB HERE *** >>>> >>>> Mike Rapoport (3): >>>> m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM >>>> m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM >>>> m68k/mm: switch from DISCONTIGMEM to SPARSEMEM >>>> >>>> arch/m68k/Kconfig.cpu | 23 ++++++++++++++--- >>>> arch/m68k/include/asm/page.h | 2 ++ >>>> arch/m68k/include/asm/page_mm.h | 7 +++++- >>>> arch/m68k/include/asm/virtconvert.h | 2 +- >>>> arch/m68k/mm/init.c | 8 +++--- >>>> arch/m68k/mm/motorola.c | 39 ++++++++++++++++++++++++++--- >>>> 6 files changed, 69 insertions(+), 12 deletions(-) >>>> >>>> -- >>>> 2.26.2 >>>> > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-21 7:56 ` Mike Rapoport 2020-08-21 20:58 ` Michael Schmitz @ 2020-08-21 23:27 ` Michael Schmitz 2020-08-22 9:51 ` Mike Rapoport 1 sibling, 1 reply; 14+ messages in thread From: Michael Schmitz @ 2020-08-21 23:27 UTC (permalink / raw) To: Mike Rapoport Cc: linux-m68k, Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport Hi Mike, only tested on ARAnyM, but appears to work (after adding sparsemem.h and changing memblocks_present() to non-static). 20 kB more total memory with sparsemem. Less memory used when booting to FastRAM, more used when booting to ST-RAM (checked right after boot on an idle system). The latter probably isn't significant. Cheers, Michael Am 21.08.2020 um 19:56 schrieb Mike Rapoport: > On Fri, Aug 21, 2020 at 10:29:13AM +1200, Michael Schmitz wrote: >> Hi Mike, >> >> Sorry, I had seen those but didn't get around to test in time. >> >> Applying your patch series to Geert's v5.9-rc1, I get this: >> >>> CC kernel/bounds.s >>> CC arch/m68k/kernel/asm-offsets.s >>> In file included from ./include/linux/mmzone.h:19, >>> from ./include/linux/topology.h:33, >>> from ./include/linux/irq.h:19, >>> from ./include/asm-generic/hardirq.h:13, >>> from ./arch/m68k/include/generated/asm/hardirq.h:1, >>> from ./include/linux/hardirq.h:10, >>> from ./include/linux/interrupt.h:11, >>> from ./include/linux/kernel_stat.h:9, >>> from arch/m68k/kernel/asm-offsets.c:16: >>> ./include/linux/page-flags-layout.h:28:10: fatal error: asm/sparsemem.h: >>> No such file or directory >>> #include <asm/sparsemem.h> >>> ^~~~~~~~~~~~~~~~~ >>> compilation terminated. >>> make[2]: *** [arch/m68k/kernel/asm-offsets.s] Error 1 >>> make[1]: *** [prepare0] Error 2 >>> make: *** [__sub-make] Error 2 >> >> Your patch announcement blurb hat sparsemem.h listed among the changed >> files, but none of the three patches stats reference that file. Am I missing >> a prerequisite here? > > Argh, I forgot to 'git add asm/sparsemem.h'. > I'll resend a refreshed version in a couple of days. > >> Cheers, >> >> Michael >> >> >> On 21/08/20 4:03 AM, Mike Rapoport wrote: >>> Gentle ping :) >>> >>> On Sat, Jul 18, 2020 at 07:26:48PM +0300, Mike Rapoport wrote: >>>> From: Mike Rapoport <rppt@linux.ibm.com> >>>> >>>> Hi, >>>> >>>> It took me a while to get back to this, but better late than never :) >>>> >>>> These patches replace DISCONTIGMEM with SPARSEMEM on m68k for systems with >>>> !SINGLE_MEMORY_CHUNK set. >>>> >>>> With SPARSEMEM there is a single node for the entire physical memory and to >>>> cope with holes in the physical address space it is divided to sections of >>>> several megabytes. >>>> >>>> Each section has it's own memory map which size depends on actual populated >>>> memory. >>>> >>>> The section size of 16M was chosen pretty much arbitrarily as I couldn't >>>> find specs for systems with e.g. Zorro memory extensions. >>>> Yet, we cannot use smaller sections and still be able to address the entire >>>> 4G of the physical memory because the section number is encoded in the page >>>> flags and there are only 8 bits available. >>>> >>>> For systems with several small memory chunks and with small (several megs) >>>> holes between them, 16M section size would cause wasted parts in the memory >>>> map and it is desirable to allow smaller section size at the expense of >>>> limiting the addressable memory. >>>> >>>> I've hesitated between adding Kconfig option as Finn suggested [1] and >>>> other options like SPARSE_VMEMMAP or ARCH_HAS_HOLES_MEMORYMODEL. In the >>>> end, I think Kconfig is the simplest one from the implementation point of >>>> view and would work fine for people that run mainline kernels on vintage >>>> hardware. >>>> >>>> For the systems with ST-RAM and FastRAM, the switch to SPARSEMEM does not >>>> change the limitation that if the kernel is loaded into the FastRam the >>>> ST-RAM remains unmapped. It only ensures that if the kernel is loaded in >>>> ST-RAM, the memory map is allocated from high physical addresses and then >>>> atari/stram.c is able to reserve the frame buffer memory. If the kernel is >>>> loaded to FastRAM, the ST-RAM remains unmapped as with DISCONTIGMEM and the >>>> atari/stram.c maps it as IOMEM. >>>> >>>> [1] https://marc.info/?l=linux-m68k&m=156309170500921&w=2 >>>> >>>> v3 changes: >>>> * rebase on the current upstream >>>> * alias ARCH_PFN_BASE to m68k_memory[0].addr >>>> * add configuration option to allow two variants of section size. >>>> >>>> v2 changes: >>>> * rebase on the current upstream >>>> * make ColdFire MMU select SINGLE_MEMORY_CHUNK in Kconfig.cpu >>>> >>>> Mike Rapoport (3): >>>> m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM >>>> m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM >>>> m68k/mm: switch from DISCONTIGMEM to SPARSEMEM >>>> >>>> arch/m68k/Kconfig.cpu | 14 ++++++-- >>>> arch/m68k/include/asm/page.h | 2 ++ >>>> arch/m68k/include/asm/page_mm.h | 6 +++- >>>> arch/m68k/include/asm/q.h | 8 +++++ >>>> arch/m68k/include/asm/virtconvert.h | 2 +- >>>> arch/m68k/mm/init.c | 8 ++--- >>>> arch/m68k/mm/motorola.c | 64 ++++++++++++++++++++++++++++++------- >>>> 7 files changed, 84 insertions(+), 20 deletions(-) >>>> create mode 100644 arch/m68k/include/asm/sparsemem.h >>>> >>>> -- >>>> 2.7.4 >>>> >>>> >>>> *** BLURB HERE *** >>>> >>>> Mike Rapoport (3): >>>> m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM >>>> m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM >>>> m68k/mm: switch from DISCONTIGMEM to SPARSEMEM >>>> >>>> arch/m68k/Kconfig.cpu | 23 ++++++++++++++--- >>>> arch/m68k/include/asm/page.h | 2 ++ >>>> arch/m68k/include/asm/page_mm.h | 7 +++++- >>>> arch/m68k/include/asm/virtconvert.h | 2 +- >>>> arch/m68k/mm/init.c | 8 +++--- >>>> arch/m68k/mm/motorola.c | 39 ++++++++++++++++++++++++++--- >>>> 6 files changed, 69 insertions(+), 12 deletions(-) >>>> >>>> -- >>>> 2.26.2 >>>> > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-21 23:27 ` Michael Schmitz @ 2020-08-22 9:51 ` Mike Rapoport 2020-08-22 19:16 ` Michael Schmitz 0 siblings, 1 reply; 14+ messages in thread From: Mike Rapoport @ 2020-08-22 9:51 UTC (permalink / raw) To: Michael Schmitz Cc: linux-m68k, Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport Hi Michael, On Sat, Aug 22, 2020 at 11:27:18AM +1200, Michael Schmitz wrote: > Hi Mike, > > only tested on ARAnyM, but appears to work (after adding sparsemem.h and > changing memblocks_present() to non-static). Cool, thanks! memblocks_present() is now called from sparse_init() and can be simply dropped. > 20 kB more total memory with sparsemem. Less memory used when booting > to FastRAM, more used when booting to ST-RAM (checked right after boot > on an idle system). The latter probably isn't significant. When booting to FastRAM we still discard ST-RAM and only use it as device memory for the framebuffer. So the total memory map size will be smaller. How many FastRAM do you have in your configuration? > Cheers, > > Michael > > > Am 21.08.2020 um 19:56 schrieb Mike Rapoport: > > On Fri, Aug 21, 2020 at 10:29:13AM +1200, Michael Schmitz wrote: > > > Hi Mike, > > > > > > Sorry, I had seen those but didn't get around to test in time. > > > > > > Applying your patch series to Geert's v5.9-rc1, I get this: > > > > > > > CC kernel/bounds.s > > > > CC arch/m68k/kernel/asm-offsets.s > > > > In file included from ./include/linux/mmzone.h:19, > > > > from ./include/linux/topology.h:33, > > > > from ./include/linux/irq.h:19, > > > > from ./include/asm-generic/hardirq.h:13, > > > > from ./arch/m68k/include/generated/asm/hardirq.h:1, > > > > from ./include/linux/hardirq.h:10, > > > > from ./include/linux/interrupt.h:11, > > > > from ./include/linux/kernel_stat.h:9, > > > > from arch/m68k/kernel/asm-offsets.c:16: > > > > ./include/linux/page-flags-layout.h:28:10: fatal error: asm/sparsemem.h: > > > > No such file or directory > > > > #include <asm/sparsemem.h> > > > > ^~~~~~~~~~~~~~~~~ > > > > compilation terminated. > > > > make[2]: *** [arch/m68k/kernel/asm-offsets.s] Error 1 > > > > make[1]: *** [prepare0] Error 2 > > > > make: *** [__sub-make] Error 2 > > > > > > Your patch announcement blurb hat sparsemem.h listed among the changed > > > files, but none of the three patches stats reference that file. Am I missing > > > a prerequisite here? > > > > Argh, I forgot to 'git add asm/sparsemem.h'. > > I'll resend a refreshed version in a couple of days. > > > > > Cheers, > > > > > > Michael > > > > > > > > > On 21/08/20 4:03 AM, Mike Rapoport wrote: > > > > Gentle ping :) > > > > > > > > On Sat, Jul 18, 2020 at 07:26:48PM +0300, Mike Rapoport wrote: > > > > > From: Mike Rapoport <rppt@linux.ibm.com> > > > > > > > > > > Hi, > > > > > > > > > > It took me a while to get back to this, but better late than never :) > > > > > > > > > > These patches replace DISCONTIGMEM with SPARSEMEM on m68k for systems with > > > > > !SINGLE_MEMORY_CHUNK set. > > > > > > > > > > With SPARSEMEM there is a single node for the entire physical memory and to > > > > > cope with holes in the physical address space it is divided to sections of > > > > > several megabytes. > > > > > > > > > > Each section has it's own memory map which size depends on actual populated > > > > > memory. > > > > > > > > > > The section size of 16M was chosen pretty much arbitrarily as I couldn't > > > > > find specs for systems with e.g. Zorro memory extensions. > > > > > Yet, we cannot use smaller sections and still be able to address the entire > > > > > 4G of the physical memory because the section number is encoded in the page > > > > > flags and there are only 8 bits available. > > > > > > > > > > For systems with several small memory chunks and with small (several megs) > > > > > holes between them, 16M section size would cause wasted parts in the memory > > > > > map and it is desirable to allow smaller section size at the expense of > > > > > limiting the addressable memory. > > > > > > > > > > I've hesitated between adding Kconfig option as Finn suggested [1] and > > > > > other options like SPARSE_VMEMMAP or ARCH_HAS_HOLES_MEMORYMODEL. In the > > > > > end, I think Kconfig is the simplest one from the implementation point of > > > > > view and would work fine for people that run mainline kernels on vintage > > > > > hardware. > > > > > > > > > > For the systems with ST-RAM and FastRAM, the switch to SPARSEMEM does not > > > > > change the limitation that if the kernel is loaded into the FastRam the > > > > > ST-RAM remains unmapped. It only ensures that if the kernel is loaded in > > > > > ST-RAM, the memory map is allocated from high physical addresses and then > > > > > atari/stram.c is able to reserve the frame buffer memory. If the kernel is > > > > > loaded to FastRAM, the ST-RAM remains unmapped as with DISCONTIGMEM and the > > > > > atari/stram.c maps it as IOMEM. > > > > > > > > > > [1] https://marc.info/?l=linux-m68k&m=156309170500921&w=2 > > > > > > > > > > v3 changes: > > > > > * rebase on the current upstream > > > > > * alias ARCH_PFN_BASE to m68k_memory[0].addr > > > > > * add configuration option to allow two variants of section size. > > > > > > > > > > v2 changes: > > > > > * rebase on the current upstream > > > > > * make ColdFire MMU select SINGLE_MEMORY_CHUNK in Kconfig.cpu > > > > > > > > > > Mike Rapoport (3): > > > > > m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM > > > > > m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM > > > > > m68k/mm: switch from DISCONTIGMEM to SPARSEMEM > > > > > > > > > > arch/m68k/Kconfig.cpu | 14 ++++++-- > > > > > arch/m68k/include/asm/page.h | 2 ++ > > > > > arch/m68k/include/asm/page_mm.h | 6 +++- > > > > > arch/m68k/include/asm/q.h | 8 +++++ > > > > > arch/m68k/include/asm/virtconvert.h | 2 +- > > > > > arch/m68k/mm/init.c | 8 ++--- > > > > > arch/m68k/mm/motorola.c | 64 ++++++++++++++++++++++++++++++------- > > > > > 7 files changed, 84 insertions(+), 20 deletions(-) > > > > > create mode 100644 arch/m68k/include/asm/sparsemem.h > > > > > > > > > > -- > > > > > 2.7.4 > > > > > > > > > > > > > > > *** BLURB HERE *** > > > > > > > > > > Mike Rapoport (3): > > > > > m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM > > > > > m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM > > > > > m68k/mm: switch from DISCONTIGMEM to SPARSEMEM > > > > > > > > > > arch/m68k/Kconfig.cpu | 23 ++++++++++++++--- > > > > > arch/m68k/include/asm/page.h | 2 ++ > > > > > arch/m68k/include/asm/page_mm.h | 7 +++++- > > > > > arch/m68k/include/asm/virtconvert.h | 2 +- > > > > > arch/m68k/mm/init.c | 8 +++--- > > > > > arch/m68k/mm/motorola.c | 39 ++++++++++++++++++++++++++--- > > > > > 6 files changed, 69 insertions(+), 12 deletions(-) > > > > > > > > > > -- > > > > > 2.26.2 > > > > > > > -- Sincerely yours, Mike. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-22 9:51 ` Mike Rapoport @ 2020-08-22 19:16 ` Michael Schmitz 2020-08-23 8:06 ` Mike Rapoport 0 siblings, 1 reply; 14+ messages in thread From: Michael Schmitz @ 2020-08-22 19:16 UTC (permalink / raw) To: Mike Rapoport Cc: linux-m68k, Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport Hi Mike, Am 22.08.2020 um 21:51 schrieb Mike Rapoport: > Hi Michael, > > On Sat, Aug 22, 2020 at 11:27:18AM +1200, Michael Schmitz wrote: >> Hi Mike, >> >> only tested on ARAnyM, but appears to work (after adding sparsemem.h and >> changing memblocks_present() to non-static). > > Cool, thanks! > memblocks_present() is now called from sparse_init() and can be simply > dropped. Right you are - I missed that in the hurry. > >> 20 kB more total memory with sparsemem. Less memory used when booting >> to FastRAM, more used when booting to ST-RAM (checked right after boot >> on an idle system). The latter probably isn't significant. > > When booting to FastRAM we still discard ST-RAM and only use it as > device memory for the framebuffer. So the total memory map size will be > smaller. Makes sense, but I was still surprised that replacing discontigmem by sparsemem saves 20 kB regardless of whether or not ST-RAM is used. > How many FastRAM do you have in your configuration? Only 256 MB. Can't use much more on my old Powerbook. Cheers, Michael ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-22 19:16 ` Michael Schmitz @ 2020-08-23 8:06 ` Mike Rapoport 2020-08-24 20:47 ` Michael Schmitz 0 siblings, 1 reply; 14+ messages in thread From: Mike Rapoport @ 2020-08-23 8:06 UTC (permalink / raw) To: Michael Schmitz Cc: linux-m68k, Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport On Sun, Aug 23, 2020 at 07:16:45AM +1200, Michael Schmitz wrote: > Hi Mike, > > Am 22.08.2020 um 21:51 schrieb Mike Rapoport: > > Hi Michael, > > > > On Sat, Aug 22, 2020 at 11:27:18AM +1200, Michael Schmitz wrote: > > > Hi Mike, > > > > > > only tested on ARAnyM, but appears to work (after adding sparsemem.h and > > > changing memblocks_present() to non-static). > > > > Cool, thanks! > > memblocks_present() is now called from sparse_init() and can be simply > > dropped. > > Right you are - I missed that in the hurry. > > > > > > 20 kB more total memory with sparsemem. Less memory used when booting > > > to FastRAM, more used when booting to ST-RAM (checked right after boot > > > on an idle system). The latter probably isn't significant. > > > > When booting to FastRAM we still discard ST-RAM and only use it as > > device memory for the framebuffer. So the total memory map size will be > > smaller. > > Makes sense, but I was still surprised that replacing discontigmem by > sparsemem saves 20 kB regardless of whether or not ST-RAM is used. The problem with sparse, however, is the memory wasted for empty memmap. For example, if the section size is 16M and there is, say, 17M of FastRAM, the memory map will be created for 32M. This means that there will be 3840 unused 'struct page' objects. :( > > How many FastRAM do you have in your configuration? > > Only 256 MB. Can't use much more on my old Powerbook. > > Cheers, > > Michael > -- Sincerely yours, Mike. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-23 8:06 ` Mike Rapoport @ 2020-08-24 20:47 ` Michael Schmitz 2020-08-25 5:42 ` Mike Rapoport 0 siblings, 1 reply; 14+ messages in thread From: Michael Schmitz @ 2020-08-24 20:47 UTC (permalink / raw) To: Mike Rapoport Cc: linux-m68k, Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport Hi Mike, On 23/08/20 8:06 PM, Mike Rapoport wrote: >>>> 20 kB more total memory with sparsemem. Less memory used when booting >>>> to FastRAM, more used when booting to ST-RAM (checked right after boot >>>> on an idle system). The latter probably isn't significant. >>> When booting to FastRAM we still discard ST-RAM and only use it as >>> device memory for the framebuffer. So the total memory map size will be >>> smaller. >> Makes sense, but I was still surprised that replacing discontigmem by >> sparsemem saves 20 kB regardless of whether or not ST-RAM is used. > The problem with sparse, however, is the memory wasted for empty memmap. > For example, if the section size is 16M and there is, say, 17M of > FastRAM, the memory map will be created for 32M. This means that there > will be 3840 unused 'struct page' objects. :( No such thing as a free lunch - that would be a case for the 1G VM limit (which I admit, I did not test!)? In my case, ST-RAM is 14 MB, so only half of the last 4 MB wasted. Note that the top of the 16 MB physical area is mapped by early kernel startup as noncacheable (hardware registers). I hope this mapping is left alone by sparsemem. Now what would be required to allow use of the ST-RAM chunk (or any other memory area mapped out of order) by the kernel memory allocators? Cheers, Michael > >>> How many FastRAM do you have in your configuration? >> Only 256 MB. Can't use much more on my old Powerbook. >> >> Cheers, >> >> Michael >> ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM 2020-08-24 20:47 ` Michael Schmitz @ 2020-08-25 5:42 ` Mike Rapoport 0 siblings, 0 replies; 14+ messages in thread From: Mike Rapoport @ 2020-08-25 5:42 UTC (permalink / raw) To: Michael Schmitz Cc: linux-m68k, Geert Uytterhoeven, Greg Ungerer, Andreas Schwab, Finn Thain, John Paul Adrian Glaubitz, Mike Rapoport On Tue, Aug 25, 2020 at 08:47:41AM +1200, Michael Schmitz wrote: > Hi Mike, > > On 23/08/20 8:06 PM, Mike Rapoport wrote: > > > > > 20 kB more total memory with sparsemem. Less memory used when booting > > > > > to FastRAM, more used when booting to ST-RAM (checked right after boot > > > > > on an idle system). The latter probably isn't significant. > > > > When booting to FastRAM we still discard ST-RAM and only use it as > > > > device memory for the framebuffer. So the total memory map size will be > > > > smaller. > > > Makes sense, but I was still surprised that replacing discontigmem by > > > sparsemem saves 20 kB regardless of whether or not ST-RAM is used. > > The problem with sparse, however, is the memory wasted for empty memmap. > > For example, if the section size is 16M and there is, say, 17M of > > FastRAM, the memory map will be created for 32M. This means that there > > will be 3840 unused 'struct page' objects. :( > > No such thing as a free lunch - that would be a case for the 1G VM limit > (which I admit, I did not test!)? For 1G physical memory limit the section size is 4M, so for a system with 1M of RAM there will be 768 unused 'struct page' objects. There is an option to free unused memmap, like ARM does [1], or even make sparsemem to avoid allocating it from the beginning. This should work, at least for the -mm case, because m68k has pfn_valid() that does not depend on the memory model. > In my case, ST-RAM is 14 MB, so only half of the last 4 MB wasted. Note that > the top of the 16 MB physical area is mapped by early kernel startup as > noncacheable (hardware registers). I hope this mapping is left alone by > sparsemem. Sparsmem will allocate an empty struct page for it, but it should not be used and even initialized. And I didn't change the page table creation code, at least not intentionally :) > Now what would be required to allow use of the ST-RAM chunk (or any other > memory area mapped out of order) by the kernel memory allocators? Oh, that's a different story. This will require changes to the way we create the virtual mapping of the physical memory and to __pa()/__va() functions. [1] https://elixir.bootlin.com/linux/latest/source/arch/arm/mm/init.c#L305 -- Sincerely yours, Mike. ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2020-08-25 5:42 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-07-18 16:26 [PATCH v3 0/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM Mike Rapoport 2020-07-18 16:26 ` [PATCH v3 1/3] m68k/mm: make node data and node setup depend on CONFIG_DISCONTIGMEM Mike Rapoport 2020-07-18 16:26 ` [PATCH v3 2/3] m68k/mm: enable use of generic memory_model.h for !DISCONTIGMEM Mike Rapoport 2020-07-18 16:26 ` [PATCH v3 3/3] m68k/mm: switch from DISCONTIGMEM to SPARSEMEM Mike Rapoport 2020-08-20 16:03 ` [PATCH v3 0/3] " Mike Rapoport 2020-08-20 22:29 ` Michael Schmitz 2020-08-21 7:56 ` Mike Rapoport 2020-08-21 20:58 ` Michael Schmitz 2020-08-21 23:27 ` Michael Schmitz 2020-08-22 9:51 ` Mike Rapoport 2020-08-22 19:16 ` Michael Schmitz 2020-08-23 8:06 ` Mike Rapoport 2020-08-24 20:47 ` Michael Schmitz 2020-08-25 5:42 ` Mike Rapoport
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).