[PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>

commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4 upstream.

Size of the mem_section[] array depends on the size of the physical address space.

In preparation for boot-time switching between paging modes on x86-64
we need to make the allocation of mem_section[] dynamic, because otherwise
we waste a lot of RAM: with CONFIG_NODE_SHIFT=10, mem_section[] size is 32kB
for 4-level paging and 2MB for 5-level paging mode.

The patch allocates the array on the first call to sparse_memory_present_with_active_regions().

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@suse.de>
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20170929140821.37654-2-kirill.shutemov@linux.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 include/linux/mmzone.h |    6 +++++-
 mm/page_alloc.c        |   10 ++++++++++
 mm/sparse.c            |   17 +++++++++++------
 3 files changed, 26 insertions(+), 7 deletions(-)

--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -1152,13 +1152,17 @@ struct mem_section {
 #define SECTION_ROOT_MASK	(SECTIONS_PER_ROOT - 1)
 
 #ifdef CONFIG_SPARSEMEM_EXTREME
-extern struct mem_section *mem_section[NR_SECTION_ROOTS];
+extern struct mem_section **mem_section;
 #else
 extern struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT];
 #endif
 
 static inline struct mem_section *__nr_to_section(unsigned long nr)
 {
+#ifdef CONFIG_SPARSEMEM_EXTREME
+	if (!mem_section)
+		return NULL;
+#endif
 	if (!mem_section[SECTION_NR_TO_ROOT(nr)])
 		return NULL;
 	return &mem_section[SECTION_NR_TO_ROOT(nr)][nr & SECTION_ROOT_MASK];
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5651,6 +5651,16 @@ void __init sparse_memory_present_with_a
 	unsigned long start_pfn, end_pfn;
 	int i, this_nid;
 
+#ifdef CONFIG_SPARSEMEM_EXTREME
+	if (!mem_section) {
+		unsigned long size, align;
+
+		size = sizeof(struct mem_section) * NR_SECTION_ROOTS;
+		align = 1 << (INTERNODE_CACHE_SHIFT);
+		mem_section = memblock_virt_alloc(size, align);
+	}
+#endif
+
 	for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, &this_nid)
 		memory_present(this_nid, start_pfn, end_pfn);
 }
--- a/mm/sparse.c
+++ b/mm/sparse.c
@@ -23,8 +23,7 @@
  * 1) mem_section	- memory sections, mem_map's for valid memory
  */
 #ifdef CONFIG_SPARSEMEM_EXTREME
-struct mem_section *mem_section[NR_SECTION_ROOTS]
-	____cacheline_internodealigned_in_smp;
+struct mem_section **mem_section;
 #else
 struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT]
 	____cacheline_internodealigned_in_smp;
@@ -101,7 +100,7 @@ static inline int sparse_index_init(unsi
 int __section_nr(struct mem_section* ms)
 {
 	unsigned long root_nr;
-	struct mem_section* root;
+	struct mem_section *root = NULL;
 
 	for (root_nr = 0; root_nr < NR_SECTION_ROOTS; root_nr++) {
 		root = __nr_to_section(root_nr * SECTIONS_PER_ROOT);
@@ -112,7 +111,7 @@ int __section_nr(struct mem_section* ms)
 		     break;
 	}
 
-	VM_BUG_ON(root_nr == NR_SECTION_ROOTS);
+	VM_BUG_ON(!root);
 
 	return (root_nr * SECTIONS_PER_ROOT) + (ms - root);
 }
@@ -330,11 +329,17 @@ again:
 static void __init check_usemap_section_nr(int nid, unsigned long *usemap)
 {
 	unsigned long usemap_snr, pgdat_snr;
-	static unsigned long old_usemap_snr = NR_MEM_SECTIONS;
-	static unsigned long old_pgdat_snr = NR_MEM_SECTIONS;
+	static unsigned long old_usemap_snr;
+	static unsigned long old_pgdat_snr;
 	struct pglist_data *pgdat = NODE_DATA(nid);
 	int usemap_nid;
 
+	/* First call */
+	if (!old_usemap_snr) {
+		old_usemap_snr = NR_MEM_SECTIONS;
+		old_pgdat_snr = NR_MEM_SECTIONS;
+	}
+
 	usemap_snr = pfn_to_section_nr(__pa(usemap) >> PAGE_SHIFT);
 	pgdat_snr = pfn_to_section_nr(__pa(pgdat) >> PAGE_SHIFT);
 	if (usemap_snr == pgdat_snr)


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dan Rue]

On Fri, Dec 22, 2017 at 09:45:08AM +0100, Greg Kroah-Hartman wrote:
> 4.14-stable review patch.  If anyone has any objections, please let me know.
> 
> ------------------
> 
> From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> 
> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4 upstream.
> 
> Size of the mem_section[] array depends on the size of the physical address space.
> 
> In preparation for boot-time switching between paging modes on x86-64
> we need to make the allocation of mem_section[] dynamic, because otherwise
> we waste a lot of RAM: with CONFIG_NODE_SHIFT=10, mem_section[] size is 32kB
> for 4-level paging and 2MB for 5-level paging mode.
> 
> The patch allocates the array on the first call to sparse_memory_present_with_active_regions().
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Andy Lutomirski <luto@amacapital.net>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Cyrill Gorcunov <gorcunov@openvz.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-mm@kvack.org
> Link: http://lkml.kernel.org/r/20170929140821.37654-2-kirill.shutemov@linux.intel.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

This patch causes a boot failure on arm64.

Please drop this patch, or pick up the fix in:

    commit 629a359bdb0e0652a8227b4ff3125431995fec6e
    Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Date:   Tue Nov 7 11:33:37 2017 +0300

        mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y

See https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1527427.html

> 
> ---
>  include/linux/mmzone.h |    6 +++++-
>  mm/page_alloc.c        |   10 ++++++++++
>  mm/sparse.c            |   17 +++++++++++------
>  3 files changed, 26 insertions(+), 7 deletions(-)
> 
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -1152,13 +1152,17 @@ struct mem_section {
>  #define SECTION_ROOT_MASK	(SECTIONS_PER_ROOT - 1)
>  
>  #ifdef CONFIG_SPARSEMEM_EXTREME
> -extern struct mem_section *mem_section[NR_SECTION_ROOTS];
> +extern struct mem_section **mem_section;
>  #else
>  extern struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT];
>  #endif
>  
>  static inline struct mem_section *__nr_to_section(unsigned long nr)
>  {
> +#ifdef CONFIG_SPARSEMEM_EXTREME
> +	if (!mem_section)
> +		return NULL;
> +#endif
>  	if (!mem_section[SECTION_NR_TO_ROOT(nr)])
>  		return NULL;
>  	return &mem_section[SECTION_NR_TO_ROOT(nr)][nr & SECTION_ROOT_MASK];
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5651,6 +5651,16 @@ void __init sparse_memory_present_with_a
>  	unsigned long start_pfn, end_pfn;
>  	int i, this_nid;
>  
> +#ifdef CONFIG_SPARSEMEM_EXTREME
> +	if (!mem_section) {
> +		unsigned long size, align;
> +
> +		size = sizeof(struct mem_section) * NR_SECTION_ROOTS;
> +		align = 1 << (INTERNODE_CACHE_SHIFT);
> +		mem_section = memblock_virt_alloc(size, align);
> +	}
> +#endif
> +
>  	for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, &this_nid)
>  		memory_present(this_nid, start_pfn, end_pfn);
>  }
> --- a/mm/sparse.c
> +++ b/mm/sparse.c
> @@ -23,8 +23,7 @@
>   * 1) mem_section	- memory sections, mem_map's for valid memory
>   */
>  #ifdef CONFIG_SPARSEMEM_EXTREME
> -struct mem_section *mem_section[NR_SECTION_ROOTS]
> -	____cacheline_internodealigned_in_smp;
> +struct mem_section **mem_section;
>  #else
>  struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT]
>  	____cacheline_internodealigned_in_smp;
> @@ -101,7 +100,7 @@ static inline int sparse_index_init(unsi
>  int __section_nr(struct mem_section* ms)
>  {
>  	unsigned long root_nr;
> -	struct mem_section* root;
> +	struct mem_section *root = NULL;
>  
>  	for (root_nr = 0; root_nr < NR_SECTION_ROOTS; root_nr++) {
>  		root = __nr_to_section(root_nr * SECTIONS_PER_ROOT);
> @@ -112,7 +111,7 @@ int __section_nr(struct mem_section* ms)
>  		     break;
>  	}
>  
> -	VM_BUG_ON(root_nr == NR_SECTION_ROOTS);
> +	VM_BUG_ON(!root);
>  
>  	return (root_nr * SECTIONS_PER_ROOT) + (ms - root);
>  }
> @@ -330,11 +329,17 @@ again:
>  static void __init check_usemap_section_nr(int nid, unsigned long *usemap)
>  {
>  	unsigned long usemap_snr, pgdat_snr;
> -	static unsigned long old_usemap_snr = NR_MEM_SECTIONS;
> -	static unsigned long old_pgdat_snr = NR_MEM_SECTIONS;
> +	static unsigned long old_usemap_snr;
> +	static unsigned long old_pgdat_snr;
>  	struct pglist_data *pgdat = NODE_DATA(nid);
>  	int usemap_nid;
>  
> +	/* First call */
> +	if (!old_usemap_snr) {
> +		old_usemap_snr = NR_MEM_SECTIONS;
> +		old_pgdat_snr = NR_MEM_SECTIONS;
> +	}
> +
>  	usemap_snr = pfn_to_section_nr(__pa(usemap) >> PAGE_SHIFT);
>  	pgdat_snr = pfn_to_section_nr(__pa(pgdat) >> PAGE_SHIFT);
>  	if (usemap_snr == pgdat_snr)
> 
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Naresh Kamboju]

On 22 December 2017 at 19:48, Dan Rue <dan.rue@linaro.org> wrote:
> On Fri, Dec 22, 2017 at 09:45:08AM +0100, Greg Kroah-Hartman wrote:
>> 4.14-stable review patch.  If anyone has any objections, please let me know.
>>
>> ------------------
>>
>> From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>>
>> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4 upstream.
>>
>> Size of the mem_section[] array depends on the size of the physical address space.
>>
>> In preparation for boot-time switching between paging modes on x86-64
>> we need to make the allocation of mem_section[] dynamic, because otherwise
>> we waste a lot of RAM: with CONFIG_NODE_SHIFT=10, mem_section[] size is 32kB
>> for 4-level paging and 2MB for 5-level paging mode.
>>
>> The patch allocates the array on the first call to sparse_memory_present_with_active_regions().
>>
>> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: Andy Lutomirski <luto@amacapital.net>
>> Cc: Borislav Petkov <bp@suse.de>
>> Cc: Cyrill Gorcunov <gorcunov@openvz.org>
>> Cc: Linus Torvalds <torvalds@linux-foundation.org>
>> Cc: Peter Zijlstra <peterz@infradead.org>
>> Cc: Thomas Gleixner <tglx@linutronix.de>
>> Cc: linux-mm@kvack.org
>> Link: http://lkml.kernel.org/r/20170929140821.37654-2-kirill.shutemov@linux.intel.com
>> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>
> This patch causes a boot failure on arm64.
>
> Please drop this patch, or pick up the fix in:
>
>     commit 629a359bdb0e0652a8227b4ff3125431995fec6e
>     Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>     Date:   Tue Nov 7 11:33:37 2017 +0300
>
>         mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
>
> See https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1527427.html

+1.
Boot failed on arm64 without 629a359b
mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y

Boot Error log:
--------------------
[    0.000000] Unable to handle kernel NULL pointer dereference at
virtual address 00000000
[    0.000000] Mem abort info:
[    0.000000]   Exception class = DABT (current EL), IL = 32 bits
[    0.000000]   SET = 0, FnV = 0
[    0.000000]   EA = 0, S1PTW = 0
[    0.000000] Data abort info:
[    0.000000]   ISV = 0, ISS = 0x00000004
[    0.000000]   CM = 0, WnR = 0
[    0.000000] [0000000000000000] user address but active_mm is swapper
[    0.000000] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.9-rc1 #1
[    0.000000] Hardware name: ARM Juno development board (r2) (DT)
[    0.000000] task: ffff0000091d9380 task.stack: ffff0000091c0000
[    0.000000] PC is at memory_present+0x64/0xf4
[    0.000000] LR is at memory_present+0x38/0xf4
[    0.000000] pc : [<ffff0000090a1f54>] lr : [<ffff0000090a1f28>]
pstate: 800000c5
[    0.000000] sp : ffff0000091c3e80

More information,
https://pastebin.com/KambxUwb

- Naresh
>
>>
>> ---
>>  include/linux/mmzone.h |    6 +++++-
>>  mm/page_alloc.c        |   10 ++++++++++
>>  mm/sparse.c            |   17 +++++++++++------
>>  3 files changed, 26 insertions(+), 7 deletions(-)
>>
>> --- a/include/linux/mmzone.h
>> +++ b/include/linux/mmzone.h
>> @@ -1152,13 +1152,17 @@ struct mem_section {
>>  #define SECTION_ROOT_MASK    (SECTIONS_PER_ROOT - 1)
>>
>>  #ifdef CONFIG_SPARSEMEM_EXTREME
>> -extern struct mem_section *mem_section[NR_SECTION_ROOTS];
>> +extern struct mem_section **mem_section;
>>  #else
>>  extern struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT];
>>  #endif
>>
>>  static inline struct mem_section *__nr_to_section(unsigned long nr)
>>  {
>> +#ifdef CONFIG_SPARSEMEM_EXTREME
>> +     if (!mem_section)
>> +             return NULL;
>> +#endif
>>       if (!mem_section[SECTION_NR_TO_ROOT(nr)])
>>               return NULL;
>>       return &mem_section[SECTION_NR_TO_ROOT(nr)][nr & SECTION_ROOT_MASK];
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -5651,6 +5651,16 @@ void __init sparse_memory_present_with_a
>>       unsigned long start_pfn, end_pfn;
>>       int i, this_nid;
>>
>> +#ifdef CONFIG_SPARSEMEM_EXTREME
>> +     if (!mem_section) {
>> +             unsigned long size, align;
>> +
>> +             size = sizeof(struct mem_section) * NR_SECTION_ROOTS;
>> +             align = 1 << (INTERNODE_CACHE_SHIFT);
>> +             mem_section = memblock_virt_alloc(size, align);
>> +     }
>> +#endif
>> +
>>       for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, &this_nid)
>>               memory_present(this_nid, start_pfn, end_pfn);
>>  }
>> --- a/mm/sparse.c
>> +++ b/mm/sparse.c
>> @@ -23,8 +23,7 @@
>>   * 1) mem_section    - memory sections, mem_map's for valid memory
>>   */
>>  #ifdef CONFIG_SPARSEMEM_EXTREME
>> -struct mem_section *mem_section[NR_SECTION_ROOTS]
>> -     ____cacheline_internodealigned_in_smp;
>> +struct mem_section **mem_section;
>>  #else
>>  struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT]
>>       ____cacheline_internodealigned_in_smp;
>> @@ -101,7 +100,7 @@ static inline int sparse_index_init(unsi
>>  int __section_nr(struct mem_section* ms)
>>  {
>>       unsigned long root_nr;
>> -     struct mem_section* root;
>> +     struct mem_section *root = NULL;
>>
>>       for (root_nr = 0; root_nr < NR_SECTION_ROOTS; root_nr++) {
>>               root = __nr_to_section(root_nr * SECTIONS_PER_ROOT);
>> @@ -112,7 +111,7 @@ int __section_nr(struct mem_section* ms)
>>                    break;
>>       }
>>
>> -     VM_BUG_ON(root_nr == NR_SECTION_ROOTS);
>> +     VM_BUG_ON(!root);
>>
>>       return (root_nr * SECTIONS_PER_ROOT) + (ms - root);
>>  }
>> @@ -330,11 +329,17 @@ again:
>>  static void __init check_usemap_section_nr(int nid, unsigned long *usemap)
>>  {
>>       unsigned long usemap_snr, pgdat_snr;
>> -     static unsigned long old_usemap_snr = NR_MEM_SECTIONS;
>> -     static unsigned long old_pgdat_snr = NR_MEM_SECTIONS;
>> +     static unsigned long old_usemap_snr;
>> +     static unsigned long old_pgdat_snr;
>>       struct pglist_data *pgdat = NODE_DATA(nid);
>>       int usemap_nid;
>>
>> +     /* First call */
>> +     if (!old_usemap_snr) {
>> +             old_usemap_snr = NR_MEM_SECTIONS;
>> +             old_pgdat_snr = NR_MEM_SECTIONS;
>> +     }
>> +
>>       usemap_snr = pfn_to_section_nr(__pa(usemap) >> PAGE_SHIFT);
>>       pgdat_snr = pfn_to_section_nr(__pa(pgdat) >> PAGE_SHIFT);
>>       if (usemap_snr == pgdat_snr)
>>
>>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

On Fri, Dec 22, 2017 at 08:22:09PM +0530, Naresh Kamboju wrote:
> On 22 December 2017 at 19:48, Dan Rue <dan.rue@linaro.org> wrote:
> > On Fri, Dec 22, 2017 at 09:45:08AM +0100, Greg Kroah-Hartman wrote:
> >> 4.14-stable review patch.  If anyone has any objections, please let me know.
> >>
> >> ------------------
> >>
> >> From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> >>
> >> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4 upstream.
> >>
> >> Size of the mem_section[] array depends on the size of the physical address space.
> >>
> >> In preparation for boot-time switching between paging modes on x86-64
> >> we need to make the allocation of mem_section[] dynamic, because otherwise
> >> we waste a lot of RAM: with CONFIG_NODE_SHIFT=10, mem_section[] size is 32kB
> >> for 4-level paging and 2MB for 5-level paging mode.
> >>
> >> The patch allocates the array on the first call to sparse_memory_present_with_active_regions().
> >>
> >> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> >> Cc: Andrew Morton <akpm@linux-foundation.org>
> >> Cc: Andy Lutomirski <luto@amacapital.net>
> >> Cc: Borislav Petkov <bp@suse.de>
> >> Cc: Cyrill Gorcunov <gorcunov@openvz.org>
> >> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> >> Cc: Peter Zijlstra <peterz@infradead.org>
> >> Cc: Thomas Gleixner <tglx@linutronix.de>
> >> Cc: linux-mm@kvack.org
> >> Link: http://lkml.kernel.org/r/20170929140821.37654-2-kirill.shutemov@linux.intel.com
> >> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> >> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> >
> > This patch causes a boot failure on arm64.
> >
> > Please drop this patch, or pick up the fix in:
> >
> >     commit 629a359bdb0e0652a8227b4ff3125431995fec6e
> >     Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> >     Date:   Tue Nov 7 11:33:37 2017 +0300
> >
> >         mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
> >
> > See https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1527427.html
> 
> +1.
> Boot failed on arm64 without 629a359b
> mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
> 
> Boot Error log:
> --------------------
> [    0.000000] Unable to handle kernel NULL pointer dereference at
> virtual address 00000000
> [    0.000000] Mem abort info:
> [    0.000000]   Exception class = DABT (current EL), IL = 32 bits
> [    0.000000]   SET = 0, FnV = 0
> [    0.000000]   EA = 0, S1PTW = 0
> [    0.000000] Data abort info:
> [    0.000000]   ISV = 0, ISS = 0x00000004
> [    0.000000]   CM = 0, WnR = 0
> [    0.000000] [0000000000000000] user address but active_mm is swapper
> [    0.000000] Internal error: Oops: 96000004 [#1] PREEMPT SMP
> [    0.000000] Modules linked in:
> [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.9-rc1 #1
> [    0.000000] Hardware name: ARM Juno development board (r2) (DT)
> [    0.000000] task: ffff0000091d9380 task.stack: ffff0000091c0000
> [    0.000000] PC is at memory_present+0x64/0xf4
> [    0.000000] LR is at memory_present+0x38/0xf4
> [    0.000000] pc : [<ffff0000090a1f54>] lr : [<ffff0000090a1f28>]
> pstate: 800000c5
> [    0.000000] sp : ffff0000091c3e80
> 
> More information,
> https://pastebin.com/KambxUwb

-rc2 is out with the fix, hopefully that survives longer :)

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

On Fri, Dec 22, 2017 at 08:18:10AM -0600, Dan Rue wrote:
> On Fri, Dec 22, 2017 at 09:45:08AM +0100, Greg Kroah-Hartman wrote:
> > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > 
> > ------------------
> > 
> > From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > 
> > commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4 upstream.
> > 
> > Size of the mem_section[] array depends on the size of the physical address space.
> > 
> > In preparation for boot-time switching between paging modes on x86-64
> > we need to make the allocation of mem_section[] dynamic, because otherwise
> > we waste a lot of RAM: with CONFIG_NODE_SHIFT=10, mem_section[] size is 32kB
> > for 4-level paging and 2MB for 5-level paging mode.
> > 
> > The patch allocates the array on the first call to sparse_memory_present_with_active_regions().
> > 
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Andy Lutomirski <luto@amacapital.net>
> > Cc: Borislav Petkov <bp@suse.de>
> > Cc: Cyrill Gorcunov <gorcunov@openvz.org>
> > Cc: Linus Torvalds <torvalds@linux-foundation.org>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: linux-mm@kvack.org
> > Link: http://lkml.kernel.org/r/20170929140821.37654-2-kirill.shutemov@linux.intel.com
> > Signed-off-by: Ingo Molnar <mingo@kernel.org>
> > Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> This patch causes a boot failure on arm64.
> 
> Please drop this patch, or pick up the fix in:
> 
>     commit 629a359bdb0e0652a8227b4ff3125431995fec6e
>     Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>     Date:   Tue Nov 7 11:33:37 2017 +0300
> 
>         mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
> 
> See https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1527427.html

Now added, thanks.

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Mike Galbraith]

On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> 4.14-stable review patch.  If anyone has any objections, please let me know.

FYI, this broke kdump, or rather the makedumpfile part thereof.
 Forward looking wreckage is par for the kdump course, but...

> ------------------
> 
> From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> 
> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4 upstream.
> 
> Size of the mem_section[] array depends on the size of the physical address space.
> 
> In preparation for boot-time switching between paging modes on x86-64
> we need to make the allocation of mem_section[] dynamic, because otherwise
> we waste a lot of RAM: with CONFIG_NODE_SHIFT=10, mem_section[] size is 32kB
> for 4-level paging and 2MB for 5-level paging mode.
> 
> The patch allocates the array on the first call to sparse_memory_present_with_active_regions().
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Andy Lutomirski <luto@amacapital.net>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Cyrill Gorcunov <gorcunov@openvz.org>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-mm@kvack.org
> Link: http://lkml.kernel.org/r/20170929140821.37654-2-kirill.shutemov@linux.intel.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> ---
>  include/linux/mmzone.h |    6 +++++-
>  mm/page_alloc.c        |   10 ++++++++++
>  mm/sparse.c            |   17 +++++++++++------
>  3 files changed, 26 insertions(+), 7 deletions(-)
> 
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -1152,13 +1152,17 @@ struct mem_section {
>  #define SECTION_ROOT_MASK	(SECTIONS_PER_ROOT - 1)
>  
>  #ifdef CONFIG_SPARSEMEM_EXTREME
> -extern struct mem_section *mem_section[NR_SECTION_ROOTS];
> +extern struct mem_section **mem_section;
>  #else
>  extern struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT];
>  #endif
>  
>  static inline struct mem_section *__nr_to_section(unsigned long nr)
>  {
> +#ifdef CONFIG_SPARSEMEM_EXTREME
> +	if (!mem_section)
> +		return NULL;
> +#endif
>  	if (!mem_section[SECTION_NR_TO_ROOT(nr)])
>  		return NULL;
>  	return &mem_section[SECTION_NR_TO_ROOT(nr)][nr & SECTION_ROOT_MASK];
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5651,6 +5651,16 @@ void __init sparse_memory_present_with_a
>  	unsigned long start_pfn, end_pfn;
>  	int i, this_nid;
>  
> +#ifdef CONFIG_SPARSEMEM_EXTREME
> +	if (!mem_section) {
> +		unsigned long size, align;
> +
> +		size = sizeof(struct mem_section) * NR_SECTION_ROOTS;
> +		align = 1 << (INTERNODE_CACHE_SHIFT);
> +		mem_section = memblock_virt_alloc(size, align);
> +	}
> +#endif
> +
>  	for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, &this_nid)
>  		memory_present(this_nid, start_pfn, end_pfn);
>  }
> --- a/mm/sparse.c
> +++ b/mm/sparse.c
> @@ -23,8 +23,7 @@
>   * 1) mem_section	- memory sections, mem_map's for valid memory
>   */
>  #ifdef CONFIG_SPARSEMEM_EXTREME
> -struct mem_section *mem_section[NR_SECTION_ROOTS]
> -	____cacheline_internodealigned_in_smp;
> +struct mem_section **mem_section;
>  #else
>  struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT]
>  	____cacheline_internodealigned_in_smp;
> @@ -101,7 +100,7 @@ static inline int sparse_index_init(unsi
>  int __section_nr(struct mem_section* ms)
>  {
>  	unsigned long root_nr;
> -	struct mem_section* root;
> +	struct mem_section *root = NULL;
>  
>  	for (root_nr = 0; root_nr < NR_SECTION_ROOTS; root_nr++) {
>  		root = __nr_to_section(root_nr * SECTIONS_PER_ROOT);
> @@ -112,7 +111,7 @@ int __section_nr(struct mem_section* ms)
>  		     break;
>  	}
>  
> -	VM_BUG_ON(root_nr == NR_SECTION_ROOTS);
> +	VM_BUG_ON(!root);
>  
>  	return (root_nr * SECTIONS_PER_ROOT) + (ms - root);
>  }
> @@ -330,11 +329,17 @@ again:
>  static void __init check_usemap_section_nr(int nid, unsigned long *usemap)
>  {
>  	unsigned long usemap_snr, pgdat_snr;
> -	static unsigned long old_usemap_snr = NR_MEM_SECTIONS;
> -	static unsigned long old_pgdat_snr = NR_MEM_SECTIONS;
> +	static unsigned long old_usemap_snr;
> +	static unsigned long old_pgdat_snr;
>  	struct pglist_data *pgdat = NODE_DATA(nid);
>  	int usemap_nid;
>  
> +	/* First call */
> +	if (!old_usemap_snr) {
> +		old_usemap_snr = NR_MEM_SECTIONS;
> +		old_pgdat_snr = NR_MEM_SECTIONS;
> +	}
> +
>  	usemap_snr = pfn_to_section_nr(__pa(usemap) >> PAGE_SHIFT);
>  	pgdat_snr = pfn_to_section_nr(__pa(pgdat) >> PAGE_SHIFT);
>  	if (usemap_snr == pgdat_snr)
> 
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > 4.14-stable review patch.  If anyone has any objections, please let me know.
> 
> FYI, this broke kdump, or rather the makedumpfile part thereof.
>  Forward looking wreckage is par for the kdump course, but...

Is it also broken in Linus's tree with this patch?  Or is there an
add-on patch that I should apply to 4.14 to resolve this issue there?

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Mike Galbraith]

On Sun, 2018-01-07 at 10:11 +0100, Greg Kroah-Hartman wrote:
> On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > 
> > FYI, this broke kdump, or rather the makedumpfile part thereof.
> >  Forward looking wreckage is par for the kdump course, but...
> 
> Is it also broken in Linus's tree with this patch?  Or is there an
> add-on patch that I should apply to 4.14 to resolve this issue there?

Yeah, it's belly up.  By its very nature, it's gonna get dinged up
regularly.  I only mentioned it because it's not expected that stuff
gets dinged up retroactively.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Michal Hocko]

On Sun 07-01-18 10:11:15, Greg KH wrote:
> On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > 
> > FYI, this broke kdump, or rather the makedumpfile part thereof.
> >  Forward looking wreckage is par for the kdump course, but...
> 
> Is it also broken in Linus's tree with this patch?  Or is there an
> add-on patch that I should apply to 4.14 to resolve this issue there?

This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
I guess.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

On Sun, Jan 07, 2018 at 11:18:47AM +0100, Michal Hocko wrote:
> On Sun 07-01-18 10:11:15, Greg KH wrote:
> > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > 
> > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > >  Forward looking wreckage is par for the kdump course, but...
> > 
> > Is it also broken in Linus's tree with this patch?  Or is there an
> > add-on patch that I should apply to 4.14 to resolve this issue there?
> 
> This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> I guess.

Good, that patch is queued up for the next 4.14-stable release in a few
days.

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Mike Galbraith]

On Sun, 2018-01-07 at 11:18 +0100, Michal Hocko wrote:
> On Sun 07-01-18 10:11:15, Greg KH wrote:
> > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > 
> > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > >  Forward looking wreckage is par for the kdump course, but...
> > 
> > Is it also broken in Linus's tree with this patch?  Or is there an
> > add-on patch that I should apply to 4.14 to resolve this issue there?
> 
> This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> I guess.

That won't unbreak kdump, else master wouldn't be broken.  I don't care
deeply, or know if anyone else does, I'm just reporting it because I
met it and chased it down.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Michal Hocko]

On Sun 07-01-18 13:44:02, Mike Galbraith wrote:
> On Sun, 2018-01-07 at 11:18 +0100, Michal Hocko wrote:
> > On Sun 07-01-18 10:11:15, Greg KH wrote:
> > > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > > 
> > > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > > >  Forward looking wreckage is par for the kdump course, but...
> > > 
> > > Is it also broken in Linus's tree with this patch?  Or is there an
> > > add-on patch that I should apply to 4.14 to resolve this issue there?
> > 
> > This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> > I guess.
> 
> That won't unbreak kdump, else master wouldn't be broken.  I don't care
> deeply, or know if anyone else does, I'm just reporting it because I
> met it and chased it down.

OK, I didn't notice that d8cfbbfa0f7 ("mm/sparse.c: wrong allocation
for mem_section") made it in after rc6. I am still wondering why
83e3c48729 ("mm/sparsemem: Allocate mem_section at runtime for
CONFIG_SPARSEMEM_EXTREME=y") made it into the stable tree in the first
place.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

On Sun, Jan 07, 2018 at 02:23:09PM +0100, Michal Hocko wrote:
> On Sun 07-01-18 13:44:02, Mike Galbraith wrote:
> > On Sun, 2018-01-07 at 11:18 +0100, Michal Hocko wrote:
> > > On Sun 07-01-18 10:11:15, Greg KH wrote:
> > > > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > > > 
> > > > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > > > >  Forward looking wreckage is par for the kdump course, but...
> > > > 
> > > > Is it also broken in Linus's tree with this patch?  Or is there an
> > > > add-on patch that I should apply to 4.14 to resolve this issue there?
> > > 
> > > This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> > > I guess.
> > 
> > That won't unbreak kdump, else master wouldn't be broken.  I don't care
> > deeply, or know if anyone else does, I'm just reporting it because I
> > met it and chased it down.
> 
> OK, I didn't notice that d8cfbbfa0f7 ("mm/sparse.c: wrong allocation
> for mem_section") made it in after rc6. I am still wondering why
> 83e3c48729 ("mm/sparsemem: Allocate mem_section at runtime for
> CONFIG_SPARSEMEM_EXTREME=y") made it into the stable tree in the first
> place.

It was part of the prep for the KTPI code from what I can tell.  If you
think it should be reverted, just let me know and I'll be glad to do so.

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Mike Galbraith]

On Mon, 2018-01-08 at 08:53 +0100, Greg Kroah-Hartman wrote:
> On Sun, Jan 07, 2018 at 02:23:09PM +0100, Michal Hocko wrote:
> > On Sun 07-01-18 13:44:02, Mike Galbraith wrote:
> > > On Sun, 2018-01-07 at 11:18 +0100, Michal Hocko wrote:
> > > > On Sun 07-01-18 10:11:15, Greg KH wrote:
> > > > > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > > > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > > > > 
> > > > > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > > > > >  Forward looking wreckage is par for the kdump course, but...
> > > > > 
> > > > > Is it also broken in Linus's tree with this patch?  Or is there an
> > > > > add-on patch that I should apply to 4.14 to resolve this issue there?
> > > > 
> > > > This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> > > > I guess.
> > > 
> > > That won't unbreak kdump, else master wouldn't be broken.  I don't care
> > > deeply, or know if anyone else does, I'm just reporting it because I
> > > met it and chased it down.
> > 
> > OK, I didn't notice that d8cfbbfa0f7 ("mm/sparse.c: wrong allocation
> > for mem_section") made it in after rc6. I am still wondering why
> > 83e3c48729 ("mm/sparsemem: Allocate mem_section at runtime for
> > CONFIG_SPARSEMEM_EXTREME=y") made it into the stable tree in the first
> > place.
> 
> It was part of the prep for the KTPI code from what I can tell.  If you
> think it should be reverted, just let me know and I'll be glad to do so.

No preference here.  I have to patch master regardless if I want kdump
to work while I patiently wait for userspace to get fixed up (either
that or use time I don't have to go fix it up myself).

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

On Mon, Jan 08, 2018 at 09:15:33AM +0100, Mike Galbraith wrote:
> On Mon, 2018-01-08 at 08:53 +0100, Greg Kroah-Hartman wrote:
> > On Sun, Jan 07, 2018 at 02:23:09PM +0100, Michal Hocko wrote:
> > > On Sun 07-01-18 13:44:02, Mike Galbraith wrote:
> > > > On Sun, 2018-01-07 at 11:18 +0100, Michal Hocko wrote:
> > > > > On Sun 07-01-18 10:11:15, Greg KH wrote:
> > > > > > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > > > > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > > > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > > > > > 
> > > > > > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > > > > > >  Forward looking wreckage is par for the kdump course, but...
> > > > > > 
> > > > > > Is it also broken in Linus's tree with this patch?  Or is there an
> > > > > > add-on patch that I should apply to 4.14 to resolve this issue there?
> > > > > 
> > > > > This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> > > > > I guess.
> > > > 
> > > > That won't unbreak kdump, else master wouldn't be broken.  I don't care
> > > > deeply, or know if anyone else does, I'm just reporting it because I
> > > > met it and chased it down.
> > > 
> > > OK, I didn't notice that d8cfbbfa0f7 ("mm/sparse.c: wrong allocation
> > > for mem_section") made it in after rc6. I am still wondering why
> > > 83e3c48729 ("mm/sparsemem: Allocate mem_section at runtime for
> > > CONFIG_SPARSEMEM_EXTREME=y") made it into the stable tree in the first
> > > place.
> > 
> > It was part of the prep for the KTPI code from what I can tell.  If you
> > think it should be reverted, just let me know and I'll be glad to do so.
> 
> No preference here.  I have to patch master regardless if I want kdump
> to work while I patiently wait for userspace to get fixed up (either
> that or use time I don't have to go fix it up myself).

I'll stay "bug compatible" for the time being.  If you do fix this up,
can you add a cc: stable tag in your patch so I can pick it up when it
gets merged?

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Mike Galbraith]

On Mon, 2018-01-08 at 09:33 +0100, Greg Kroah-Hartman wrote:
> On Mon, Jan 08, 2018 at 09:15:33AM +0100, Mike Galbraith wrote:
> 
> > > It was part of the prep for the KTPI code from what I can tell.  If you
> > > think it should be reverted, just let me know and I'll be glad to do so.
> > 
> > No preference here.  I have to patch master regardless if I want kdump
> > to work while I patiently wait for userspace to get fixed up (either
> > that or use time I don't have to go fix it up myself).
> 
> I'll stay "bug compatible" for the time being.  If you do fix this up,
> can you add a cc: stable tag in your patch so I can pick it up when it
> gets merged?

Userspace (makedumpfile) will have to adapt, not the kernel. Meanwhile
I carry reverts, making kernels, kdump and myself all happy campers.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Michal Hocko]

On Mon 08-01-18 08:53:08, Greg KH wrote:
> On Sun, Jan 07, 2018 at 02:23:09PM +0100, Michal Hocko wrote:
> > On Sun 07-01-18 13:44:02, Mike Galbraith wrote:
> > > On Sun, 2018-01-07 at 11:18 +0100, Michal Hocko wrote:
> > > > On Sun 07-01-18 10:11:15, Greg KH wrote:
> > > > > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > > > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > > > > 
> > > > > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > > > > >  Forward looking wreckage is par for the kdump course, but...
> > > > > 
> > > > > Is it also broken in Linus's tree with this patch?  Or is there an
> > > > > add-on patch that I should apply to 4.14 to resolve this issue there?
> > > > 
> > > > This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> > > > I guess.
> > > 
> > > That won't unbreak kdump, else master wouldn't be broken.  I don't care
> > > deeply, or know if anyone else does, I'm just reporting it because I
> > > met it and chased it down.
> > 
> > OK, I didn't notice that d8cfbbfa0f7 ("mm/sparse.c: wrong allocation
> > for mem_section") made it in after rc6. I am still wondering why
> > 83e3c48729 ("mm/sparsemem: Allocate mem_section at runtime for
> > CONFIG_SPARSEMEM_EXTREME=y") made it into the stable tree in the first
> > place.
> 
> It was part of the prep for the KTPI code from what I can tell.

I do not see a direct relation, to be honest. It is more related to
5-level page tables but I might be missing some subtle relation.

> If you
> think it should be reverted, just let me know and I'll be glad to do so.

This seems to be affecting Linus tree as well so it needs to get
resolved. I would suggest reverting in stable for the mean time.
If you really need it in the stable tree then you can pull it back later
with all the follow up fixes.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

On Mon, Jan 08, 2018 at 09:47:23AM +0100, Michal Hocko wrote:
> On Mon 08-01-18 08:53:08, Greg KH wrote:
> > On Sun, Jan 07, 2018 at 02:23:09PM +0100, Michal Hocko wrote:
> > > On Sun 07-01-18 13:44:02, Mike Galbraith wrote:
> > > > On Sun, 2018-01-07 at 11:18 +0100, Michal Hocko wrote:
> > > > > On Sun 07-01-18 10:11:15, Greg KH wrote:
> > > > > > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > > > > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > > > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > > > > > 
> > > > > > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > > > > > >  Forward looking wreckage is par for the kdump course, but...
> > > > > > 
> > > > > > Is it also broken in Linus's tree with this patch?  Or is there an
> > > > > > add-on patch that I should apply to 4.14 to resolve this issue there?
> > > > > 
> > > > > This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> > > > > I guess.
> > > > 
> > > > That won't unbreak kdump, else master wouldn't be broken.  I don't care
> > > > deeply, or know if anyone else does, I'm just reporting it because I
> > > > met it and chased it down.
> > > 
> > > OK, I didn't notice that d8cfbbfa0f7 ("mm/sparse.c: wrong allocation
> > > for mem_section") made it in after rc6. I am still wondering why
> > > 83e3c48729 ("mm/sparsemem: Allocate mem_section at runtime for
> > > CONFIG_SPARSEMEM_EXTREME=y") made it into the stable tree in the first
> > > place.
> > 
> > It was part of the prep for the KTPI code from what I can tell.
> 
> I do not see a direct relation, to be honest. It is more related to
> 5-level page tables but I might be missing some subtle relation.
> 
> > If you
> > think it should be reverted, just let me know and I'll be glad to do so.
> 
> This seems to be affecting Linus tree as well so it needs to get
> resolved. I would suggest reverting in stable for the mean time.
> If you really need it in the stable tree then you can pull it back later
> with all the follow up fixes.

Ok, I've now reverted it, thanks.

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Greg Kroah-Hartman]

On Mon, Jan 08, 2018 at 10:10:44AM +0100, Greg Kroah-Hartman wrote:
> On Mon, Jan 08, 2018 at 09:47:23AM +0100, Michal Hocko wrote:
> > On Mon 08-01-18 08:53:08, Greg KH wrote:
> > > On Sun, Jan 07, 2018 at 02:23:09PM +0100, Michal Hocko wrote:
> > > > On Sun 07-01-18 13:44:02, Mike Galbraith wrote:
> > > > > On Sun, 2018-01-07 at 11:18 +0100, Michal Hocko wrote:
> > > > > > On Sun 07-01-18 10:11:15, Greg KH wrote:
> > > > > > > On Sun, Jan 07, 2018 at 06:14:22AM +0100, Mike Galbraith wrote:
> > > > > > > > On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > > > > > > > > 4.14-stable review patch.  If anyone has any objections, please let me know.
> > > > > > > > 
> > > > > > > > FYI, this broke kdump, or rather the makedumpfile part thereof.
> > > > > > > >  Forward looking wreckage is par for the kdump course, but...
> > > > > > > 
> > > > > > > Is it also broken in Linus's tree with this patch?  Or is there an
> > > > > > > add-on patch that I should apply to 4.14 to resolve this issue there?
> > > > > > 
> > > > > > This one http://lkml.kernel.org/r/1513932498-20350-1-git-send-email-bhe@redhat.com
> > > > > > I guess.
> > > > > 
> > > > > That won't unbreak kdump, else master wouldn't be broken.  I don't care
> > > > > deeply, or know if anyone else does, I'm just reporting it because I
> > > > > met it and chased it down.
> > > > 
> > > > OK, I didn't notice that d8cfbbfa0f7 ("mm/sparse.c: wrong allocation
> > > > for mem_section") made it in after rc6. I am still wondering why
> > > > 83e3c48729 ("mm/sparsemem: Allocate mem_section at runtime for
> > > > CONFIG_SPARSEMEM_EXTREME=y") made it into the stable tree in the first
> > > > place.
> > > 
> > > It was part of the prep for the KTPI code from what I can tell.
> > 
> > I do not see a direct relation, to be honest. It is more related to
> > 5-level page tables but I might be missing some subtle relation.
> > 
> > > If you
> > > think it should be reverted, just let me know and I'll be glad to do so.
> > 
> > This seems to be affecting Linus tree as well so it needs to get
> > resolved. I would suggest reverting in stable for the mean time.
> > If you really need it in the stable tree then you can pull it back later
> > with all the follow up fixes.
> 
> Ok, I've now reverted it, thanks.

Nope, it breaks the build when reverted, I'm dropping that revert now.

It's as if the x86 maintainers actually knew what they were doing in
asking for this to be backported :)

thanks,

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Ingo Molnar]

hi Kirill,

As Mike reported it below, your 5-level paging related upstream commit 
83e3c48729d9 and all its followup fixes:

 83e3c48729d9: mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y
 629a359bdb0e: mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
 d09cfbbfa0f7: mm/sparse.c: wrong allocation for mem_section

... still breaks kexec - and that now regresses -stable as well.

Given that 5-level paging now syntactically depends on having this commit, if we 
fully revert this then we'll have to disable 5-level paging as well.

Thanks,

	Ingo

* Mike Galbraith <efault@gmx.de> wrote:

> On Fri, 2017-12-22 at 09:45 +0100, Greg Kroah-Hartman wrote:
> > 4.14-stable review patch.  If anyone has any objections, please let me know.
> 
> FYI, this broke kdump, or rather the makedumpfile part thereof.
>  Forward looking wreckage is par for the kdump course, but...
> 
> > ------------------
> > 
> > From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > 
> > commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4 upstream.
> > 
> > Size of the mem_section[] array depends on the size of the physical address space.
> > 
> > In preparation for boot-time switching between paging modes on x86-64
> > we need to make the allocation of mem_section[] dynamic, because otherwise
> > we waste a lot of RAM: with CONFIG_NODE_SHIFT=10, mem_section[] size is 32kB
> > for 4-level paging and 2MB for 5-level paging mode.
> > 
> > The patch allocates the array on the first call to sparse_memory_present_with_active_regions().
> > 
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Andy Lutomirski <luto@amacapital.net>
> > Cc: Borislav Petkov <bp@suse.de>
> > Cc: Cyrill Gorcunov <gorcunov@openvz.org>
> > Cc: Linus Torvalds <torvalds@linux-foundation.org>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: linux-mm@kvack.org
> > Link: http://lkml.kernel.org/r/20170929140821.37654-2-kirill.shutemov@linux.intel.com
> > Signed-off-by: Ingo Molnar <mingo@kernel.org>
> > Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > 
> > ---
> >  include/linux/mmzone.h |    6 +++++-
> >  mm/page_alloc.c        |   10 ++++++++++
> >  mm/sparse.c            |   17 +++++++++++------
> >  3 files changed, 26 insertions(+), 7 deletions(-)
> > 
> > --- a/include/linux/mmzone.h
> > +++ b/include/linux/mmzone.h
> > @@ -1152,13 +1152,17 @@ struct mem_section {
> >  #define SECTION_ROOT_MASK	(SECTIONS_PER_ROOT - 1)
> >  
> >  #ifdef CONFIG_SPARSEMEM_EXTREME
> > -extern struct mem_section *mem_section[NR_SECTION_ROOTS];
> > +extern struct mem_section **mem_section;
> >  #else
> >  extern struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT];
> >  #endif
> >  
> >  static inline struct mem_section *__nr_to_section(unsigned long nr)
> >  {
> > +#ifdef CONFIG_SPARSEMEM_EXTREME
> > +	if (!mem_section)
> > +		return NULL;
> > +#endif
> >  	if (!mem_section[SECTION_NR_TO_ROOT(nr)])
> >  		return NULL;
> >  	return &mem_section[SECTION_NR_TO_ROOT(nr)][nr & SECTION_ROOT_MASK];
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -5651,6 +5651,16 @@ void __init sparse_memory_present_with_a
> >  	unsigned long start_pfn, end_pfn;
> >  	int i, this_nid;
> >  
> > +#ifdef CONFIG_SPARSEMEM_EXTREME
> > +	if (!mem_section) {
> > +		unsigned long size, align;
> > +
> > +		size = sizeof(struct mem_section) * NR_SECTION_ROOTS;
> > +		align = 1 << (INTERNODE_CACHE_SHIFT);
> > +		mem_section = memblock_virt_alloc(size, align);
> > +	}
> > +#endif
> > +
> >  	for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, &this_nid)
> >  		memory_present(this_nid, start_pfn, end_pfn);
> >  }
> > --- a/mm/sparse.c
> > +++ b/mm/sparse.c
> > @@ -23,8 +23,7 @@
> >   * 1) mem_section	- memory sections, mem_map's for valid memory
> >   */
> >  #ifdef CONFIG_SPARSEMEM_EXTREME
> > -struct mem_section *mem_section[NR_SECTION_ROOTS]
> > -	____cacheline_internodealigned_in_smp;
> > +struct mem_section **mem_section;
> >  #else
> >  struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT]
> >  	____cacheline_internodealigned_in_smp;
> > @@ -101,7 +100,7 @@ static inline int sparse_index_init(unsi
> >  int __section_nr(struct mem_section* ms)
> >  {
> >  	unsigned long root_nr;
> > -	struct mem_section* root;
> > +	struct mem_section *root = NULL;
> >  
> >  	for (root_nr = 0; root_nr < NR_SECTION_ROOTS; root_nr++) {
> >  		root = __nr_to_section(root_nr * SECTIONS_PER_ROOT);
> > @@ -112,7 +111,7 @@ int __section_nr(struct mem_section* ms)
> >  		     break;
> >  	}
> >  
> > -	VM_BUG_ON(root_nr == NR_SECTION_ROOTS);
> > +	VM_BUG_ON(!root);
> >  
> >  	return (root_nr * SECTIONS_PER_ROOT) + (ms - root);
> >  }
> > @@ -330,11 +329,17 @@ again:
> >  static void __init check_usemap_section_nr(int nid, unsigned long *usemap)
> >  {
> >  	unsigned long usemap_snr, pgdat_snr;
> > -	static unsigned long old_usemap_snr = NR_MEM_SECTIONS;
> > -	static unsigned long old_pgdat_snr = NR_MEM_SECTIONS;
> > +	static unsigned long old_usemap_snr;
> > +	static unsigned long old_pgdat_snr;
> >  	struct pglist_data *pgdat = NODE_DATA(nid);
> >  	int usemap_nid;
> >  
> > +	/* First call */
> > +	if (!old_usemap_snr) {
> > +		old_usemap_snr = NR_MEM_SECTIONS;
> > +		old_pgdat_snr = NR_MEM_SECTIONS;
> > +	}
> > +
> >  	usemap_snr = pfn_to_section_nr(__pa(usemap) >> PAGE_SHIFT);
> >  	pgdat_snr = pfn_to_section_nr(__pa(pgdat) >> PAGE_SHIFT);
> >  	if (usemap_snr == pgdat_snr)
> > 
> > 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Kirill A. Shutemov]

On Mon, Jan 08, 2018 at 04:04:44PM +0000, Ingo Molnar wrote:
> 
> hi Kirill,
> 
> As Mike reported it below, your 5-level paging related upstream commit 
> 83e3c48729d9 and all its followup fixes:
> 
>  83e3c48729d9: mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y
>  629a359bdb0e: mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
>  d09cfbbfa0f7: mm/sparse.c: wrong allocation for mem_section
> 
> ... still breaks kexec - and that now regresses -stable as well.
> 
> Given that 5-level paging now syntactically depends on having this commit, if we 
> fully revert this then we'll have to disable 5-level paging as well.

Urghh.. Sorry about this.

I'm on vacation right now. Give me a day to sort this out.

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Kirill A. Shutemov]

On Mon, Jan 08, 2018 at 08:46:53PM +0300, Kirill A. Shutemov wrote:
> On Mon, Jan 08, 2018 at 04:04:44PM +0000, Ingo Molnar wrote:
> > 
> > hi Kirill,
> > 
> > As Mike reported it below, your 5-level paging related upstream commit 
> > 83e3c48729d9 and all its followup fixes:
> > 
> >  83e3c48729d9: mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y
> >  629a359bdb0e: mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
> >  d09cfbbfa0f7: mm/sparse.c: wrong allocation for mem_section
> > 
> > ... still breaks kexec - and that now regresses -stable as well.
> > 
> > Given that 5-level paging now syntactically depends on having this commit, if we 
> > fully revert this then we'll have to disable 5-level paging as well.

This *should* help.

Mike, could you test this? (On top of the rest of the fixes.)

Sorry for the mess.

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dave Young]

On 01/09/18 at 03:13am, Kirill A. Shutemov wrote:
> On Mon, Jan 08, 2018 at 08:46:53PM +0300, Kirill A. Shutemov wrote:
> > On Mon, Jan 08, 2018 at 04:04:44PM +0000, Ingo Molnar wrote:
> > > 
> > > hi Kirill,
> > > 
> > > As Mike reported it below, your 5-level paging related upstream commit 
> > > 83e3c48729d9 and all its followup fixes:
> > > 
> > >  83e3c48729d9: mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y
> > >  629a359bdb0e: mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
> > >  d09cfbbfa0f7: mm/sparse.c: wrong allocation for mem_section
> > > 
> > > ... still breaks kexec - and that now regresses -stable as well.
> > > 
> > > Given that 5-level paging now syntactically depends on having this commit, if we 
> > > fully revert this then we'll have to disable 5-level paging as well.
> 
> This *should* help.
> 
> Mike, could you test this? (On top of the rest of the fixes.)
> 
> Sorry for the mess.
> 
> From 100fd567754f1457be94732046aefca204c842d2 Mon Sep 17 00:00:00 2001
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Date: Tue, 9 Jan 2018 02:55:47 +0300
> Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> 
> Depending on configuration mem_section can now be an array or a pointer
> to an array allocated dynamically. In most cases, we can continue to refer
> to it as 'mem_section' regardless of what it is.
> 
> But there's one exception: '&mem_section' means "address of the array" if
> mem_section is an array, but if mem_section is a pointer, it would mean
> "address of the pointer".
> 
> We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> writes down address of pointer into vmcoreinfo, not array as we wanted.
> 
> Let's introduce VMCOREINFO_ARRAY() that would handle the situation
> correctly for both cases.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
> ---
>  include/linux/crash_core.h | 2 ++
>  kernel/crash_core.c        | 2 +-
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 06097ef30449..83ae04950269 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
>  	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
>  #define VMCOREINFO_SYMBOL(name) \
>  	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> +#define VMCOREINFO_ARRAY(name) \

Thanks for the patch, I have a similar patch but makedumpfile maintainer
is looking at a userspace fix instead.

As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.

> +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
>  #define VMCOREINFO_SIZE(name) \
>  	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
>  			      (unsigned long)sizeof(name))
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index b3663896278e..d4122a837477 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
>  	VMCOREINFO_SYMBOL(contig_page_data);
>  #endif
>  #ifdef CONFIG_SPARSEMEM
> -	VMCOREINFO_SYMBOL(mem_section);
> +	VMCOREINFO_ARRAY(mem_section);
>  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>  	VMCOREINFO_STRUCT_SIZE(mem_section);
>  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> -- 
>  Kirill A. Shutemov

Thanks
Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

On 01/09/18 at 09:09am, Dave Young wrote:
> On 01/09/18 at 03:13am, Kirill A. Shutemov wrote:
> > On Mon, Jan 08, 2018 at 08:46:53PM +0300, Kirill A. Shutemov wrote:
> > > On Mon, Jan 08, 2018 at 04:04:44PM +0000, Ingo Molnar wrote:
> > > > 
> > > > hi Kirill,
> > > > 
> > > > As Mike reported it below, your 5-level paging related upstream commit 
> > > > 83e3c48729d9 and all its followup fixes:
> > > > 
> > > >  83e3c48729d9: mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y
> > > >  629a359bdb0e: mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
> > > >  d09cfbbfa0f7: mm/sparse.c: wrong allocation for mem_section
> > > > 
> > > > ... still breaks kexec - and that now regresses -stable as well.
> > > > 
> > > > Given that 5-level paging now syntactically depends on having this commit, if we 
> > > > fully revert this then we'll have to disable 5-level paging as well.
> > 
> > This *should* help.
> > 
> > Mike, could you test this? (On top of the rest of the fixes.)
> > 
> > Sorry for the mess.
> > 
> > From 100fd567754f1457be94732046aefca204c842d2 Mon Sep 17 00:00:00 2001
> > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> > Date: Tue, 9 Jan 2018 02:55:47 +0300
> > Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> > 
> > Depending on configuration mem_section can now be an array or a pointer
> > to an array allocated dynamically. In most cases, we can continue to refer
> > to it as 'mem_section' regardless of what it is.
> > 
> > But there's one exception: '&mem_section' means "address of the array" if
> > mem_section is an array, but if mem_section is a pointer, it would mean
> > "address of the pointer".
> > 
> > We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> > writes down address of pointer into vmcoreinfo, not array as we wanted.
> > 
> > Let's introduce VMCOREINFO_ARRAY() that would handle the situation
> > correctly for both cases.
> > 
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
> > ---
> >  include/linux/crash_core.h | 2 ++
> >  kernel/crash_core.c        | 2 +-
> >  2 files changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> > index 06097ef30449..83ae04950269 100644
> > --- a/include/linux/crash_core.h
> > +++ b/include/linux/crash_core.h
> > @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
> >  	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
> >  #define VMCOREINFO_SYMBOL(name) \
> >  	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> > +#define VMCOREINFO_ARRAY(name) \
> 
> Thanks for the patch, I have a similar patch but makedumpfile maintainer
> is looking at a userspace fix instead.

Seems we should add lkml to CC next time so that people can watch it.

> As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.

I still think using vmcoreinfo_append_str is better. Unless we replace
all array variables with the newly added macro.

vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
                                (unsigned long)mem_section);
> 
> > +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
> >  #define VMCOREINFO_SIZE(name) \
> >  	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
> >  			      (unsigned long)sizeof(name))
> > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> > index b3663896278e..d4122a837477 100644
> > --- a/kernel/crash_core.c
> > +++ b/kernel/crash_core.c
> > @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
> >  	VMCOREINFO_SYMBOL(contig_page_data);
> >  #endif
> >  #ifdef CONFIG_SPARSEMEM
> > -	VMCOREINFO_SYMBOL(mem_section);
> > +	VMCOREINFO_ARRAY(mem_section);
> >  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
> >  	VMCOREINFO_STRUCT_SIZE(mem_section);
> >  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> > -- 
> >  Kirill A. Shutemov
> 
> Thanks
> Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dave Young]

On 01/09/18 at 01:41pm, Baoquan He wrote:
> On 01/09/18 at 09:09am, Dave Young wrote:
> > On 01/09/18 at 03:13am, Kirill A. Shutemov wrote:
> > > On Mon, Jan 08, 2018 at 08:46:53PM +0300, Kirill A. Shutemov wrote:
> > > > On Mon, Jan 08, 2018 at 04:04:44PM +0000, Ingo Molnar wrote:
> > > > > 
> > > > > hi Kirill,
> > > > > 
> > > > > As Mike reported it below, your 5-level paging related upstream commit 
> > > > > 83e3c48729d9 and all its followup fixes:
> > > > > 
> > > > >  83e3c48729d9: mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y
> > > > >  629a359bdb0e: mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
> > > > >  d09cfbbfa0f7: mm/sparse.c: wrong allocation for mem_section
> > > > > 
> > > > > ... still breaks kexec - and that now regresses -stable as well.
> > > > > 
> > > > > Given that 5-level paging now syntactically depends on having this commit, if we 
> > > > > fully revert this then we'll have to disable 5-level paging as well.
> > > 
> > > This *should* help.
> > > 
> > > Mike, could you test this? (On top of the rest of the fixes.)
> > > 
> > > Sorry for the mess.
> > > 
> > > From 100fd567754f1457be94732046aefca204c842d2 Mon Sep 17 00:00:00 2001
> > > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> > > Date: Tue, 9 Jan 2018 02:55:47 +0300
> > > Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> > > 
> > > Depending on configuration mem_section can now be an array or a pointer
> > > to an array allocated dynamically. In most cases, we can continue to refer
> > > to it as 'mem_section' regardless of what it is.
> > > 
> > > But there's one exception: '&mem_section' means "address of the array" if
> > > mem_section is an array, but if mem_section is a pointer, it would mean
> > > "address of the pointer".
> > > 
> > > We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> > > writes down address of pointer into vmcoreinfo, not array as we wanted.
> > > 
> > > Let's introduce VMCOREINFO_ARRAY() that would handle the situation
> > > correctly for both cases.
> > > 
> > > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
> > > ---
> > >  include/linux/crash_core.h | 2 ++
> > >  kernel/crash_core.c        | 2 +-
> > >  2 files changed, 3 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> > > index 06097ef30449..83ae04950269 100644
> > > --- a/include/linux/crash_core.h
> > > +++ b/include/linux/crash_core.h
> > > @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
> > >  	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
> > >  #define VMCOREINFO_SYMBOL(name) \
> > >  	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> > > +#define VMCOREINFO_ARRAY(name) \
> > 
> > Thanks for the patch, I have a similar patch but makedumpfile maintainer
> > is looking at a userspace fix instead.
> 
> Seems we should add lkml to CC next time so that people can watch it.

Yes, agreed.

> 
> > As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.
> 
> I still think using vmcoreinfo_append_str is better. Unless we replace
> all array variables with the newly added macro.
> 
> vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
>                                 (unsigned long)mem_section);

I have no strong opinion, either change all array uses or just introduce
the macro and start to use it from now on if we have similar array
symbols.

> > 
> > > +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
> > >  #define VMCOREINFO_SIZE(name) \
> > >  	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
> > >  			      (unsigned long)sizeof(name))
> > > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> > > index b3663896278e..d4122a837477 100644
> > > --- a/kernel/crash_core.c
> > > +++ b/kernel/crash_core.c
> > > @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
> > >  	VMCOREINFO_SYMBOL(contig_page_data);
> > >  #endif
> > >  #ifdef CONFIG_SPARSEMEM
> > > -	VMCOREINFO_SYMBOL(mem_section);
> > > +	VMCOREINFO_ARRAY(mem_section);
> > >  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
> > >  	VMCOREINFO_STRUCT_SIZE(mem_section);
> > >  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> > > -- 
> > >  Kirill A. Shutemov
> > 
> > Thanks
> > Dave

Thanks
Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Kirill A. Shutemov]

On Tue, Jan 09, 2018 at 03:24:40PM +0800, Dave Young wrote:
> On 01/09/18 at 01:41pm, Baoquan He wrote:
> > On 01/09/18 at 09:09am, Dave Young wrote:
> > 
> > > As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.

Yep, that's better.

> > I still think using vmcoreinfo_append_str is better. Unless we replace
> > all array variables with the newly added macro.
> > 
> > vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
> >                                 (unsigned long)mem_section);
> 
> I have no strong opinion, either change all array uses or just introduce
> the macro and start to use it from now on if we have similar array
> symbols.

Do you need some action on my side or will you folks take care about this?

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dave Young]

On Tue, Jan 09, 2018 at 12:05:52PM +0300, Kirill A. Shutemov wrote:
> On Tue, Jan 09, 2018 at 03:24:40PM +0800, Dave Young wrote:
> > On 01/09/18 at 01:41pm, Baoquan He wrote:
> > > On 01/09/18 at 09:09am, Dave Young wrote:
> > > 
> > > > As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.
> 
> Yep, that's better.
> 
> > > I still think using vmcoreinfo_append_str is better. Unless we replace
> > > all array variables with the newly added macro.
> > > 
> > > vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
> > >                                 (unsigned long)mem_section);
> > 
> > I have no strong opinion, either change all array uses or just introduce
> > the macro and start to use it from now on if we have similar array
> > symbols.
> 
> Do you need some action on my side or will you folks take care about this?

I think Baoquan was suggesting to update all array users in current code, if you can check every VMCOREINFO_SYMBOL and update all the arrays he will be happy. But if can not do it easily I'm fine with a VMCOREINFO_SYMBOL_ARRAY changes only now, we kdump people can do it later as well. 

> 
> -- 
>  Kirill A. Shutemov

Thanks
Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Kirill A. Shutemov]

On Wed, Jan 10, 2018 at 03:08:04AM +0000, Dave Young wrote:
> On Tue, Jan 09, 2018 at 12:05:52PM +0300, Kirill A. Shutemov wrote:
> > On Tue, Jan 09, 2018 at 03:24:40PM +0800, Dave Young wrote:
> > > On 01/09/18 at 01:41pm, Baoquan He wrote:
> > > > On 01/09/18 at 09:09am, Dave Young wrote:
> > > > 
> > > > > As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.
> > 
> > Yep, that's better.
> > 
> > > > I still think using vmcoreinfo_append_str is better. Unless we replace
> > > > all array variables with the newly added macro.
> > > > 
> > > > vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
> > > >                                 (unsigned long)mem_section);
> > > 
> > > I have no strong opinion, either change all array uses or just introduce
> > > the macro and start to use it from now on if we have similar array
> > > symbols.
> > 
> > Do you need some action on my side or will you folks take care about this?
> 
> I think Baoquan was suggesting to update all array users in current
> code, if you can check every VMCOREINFO_SYMBOL and update all the arrays
> he will be happy. But if can not do it easily I'm fine with a
> VMCOREINFO_SYMBOL_ARRAY changes only now, we kdump people can do it
> later as well. 

It seems it's the only array we have there. swapper_pg_dir is a potential
candidate, but it's 'unsigned long' on arm.

Below it patch with corrected macro name.

Please, consider applying.

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

On 01/10/18 at 02:16pm, Kirill A. Shutemov wrote:
> On Wed, Jan 10, 2018 at 03:08:04AM +0000, Dave Young wrote:
> > On Tue, Jan 09, 2018 at 12:05:52PM +0300, Kirill A. Shutemov wrote:
> > > On Tue, Jan 09, 2018 at 03:24:40PM +0800, Dave Young wrote:
> > > > On 01/09/18 at 01:41pm, Baoquan He wrote:
> > > > > On 01/09/18 at 09:09am, Dave Young wrote:
> > > > > 
> > > > > > As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.
> > > 
> > > Yep, that's better.
> > > 
> > > > > I still think using vmcoreinfo_append_str is better. Unless we replace
> > > > > all array variables with the newly added macro.
> > > > > 
> > > > > vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
> > > > >                                 (unsigned long)mem_section);
> > > > 
> > > > I have no strong opinion, either change all array uses or just introduce
> > > > the macro and start to use it from now on if we have similar array
> > > > symbols.
> > > 
> > > Do you need some action on my side or will you folks take care about this?
> > 
> > I think Baoquan was suggesting to update all array users in current
> > code, if you can check every VMCOREINFO_SYMBOL and update all the arrays
> > he will be happy. But if can not do it easily I'm fine with a
> > VMCOREINFO_SYMBOL_ARRAY changes only now, we kdump people can do it
> > later as well. 
> 
> It seems it's the only array we have there. swapper_pg_dir is a potential
> candidate, but it's 'unsigned long' on arm.
> 
> Below it patch with corrected macro name.
> 
> Please, consider applying.
> 
> From 70f3a84b97f2de98d1364f7b10b7a42a1d8e9968 Mon Sep 17 00:00:00 2001
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Date: Tue, 9 Jan 2018 02:55:47 +0300
> Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> 
> Depending on configuration mem_section can now be an array or a pointer
> to an array allocated dynamically. In most cases, we can continue to refer
> to it as 'mem_section' regardless of what it is.
> 
> But there's one exception: '&mem_section' means "address of the array" if
> mem_section is an array, but if mem_section is a pointer, it would mean
> "address of the pointer".
> 
> We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> writes down address of pointer into vmcoreinfo, not array as we wanted.
> 
> Let's introduce VMCOREINFO_SYMBOL_ARRAY() that would handle the
> situation correctly for both cases.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")

Ack it, thanks.

Acked-by: Baoquan He <bhe@redhat.com>

> ---
>  include/linux/crash_core.h | 2 ++
>  kernel/crash_core.c        | 2 +-
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 06097ef30449..b511f6d24b42 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
>  	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
>  #define VMCOREINFO_SYMBOL(name) \
>  	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> +#define VMCOREINFO_SYMBOL_ARRAY(name) \
> +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
>  #define VMCOREINFO_SIZE(name) \
>  	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
>  			      (unsigned long)sizeof(name))
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index b3663896278e..4f63597c824d 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
>  	VMCOREINFO_SYMBOL(contig_page_data);
>  #endif
>  #ifdef CONFIG_SPARSEMEM
> -	VMCOREINFO_SYMBOL(mem_section);
> +	VMCOREINFO_SYMBOL_ARRAY(mem_section);
>  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>  	VMCOREINFO_STRUCT_SIZE(mem_section);
>  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> -- 
>  Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dave Young]

On 01/10/18 at 02:16pm, Kirill A. Shutemov wrote:
> On Wed, Jan 10, 2018 at 03:08:04AM +0000, Dave Young wrote:
> > On Tue, Jan 09, 2018 at 12:05:52PM +0300, Kirill A. Shutemov wrote:
> > > On Tue, Jan 09, 2018 at 03:24:40PM +0800, Dave Young wrote:
> > > > On 01/09/18 at 01:41pm, Baoquan He wrote:
> > > > > On 01/09/18 at 09:09am, Dave Young wrote:
> > > > > 
> > > > > > As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.
> > > 
> > > Yep, that's better.
> > > 
> > > > > I still think using vmcoreinfo_append_str is better. Unless we replace
> > > > > all array variables with the newly added macro.
> > > > > 
> > > > > vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
> > > > >                                 (unsigned long)mem_section);
> > > > 
> > > > I have no strong opinion, either change all array uses or just introduce
> > > > the macro and start to use it from now on if we have similar array
> > > > symbols.
> > > 
> > > Do you need some action on my side or will you folks take care about this?
> > 
> > I think Baoquan was suggesting to update all array users in current
> > code, if you can check every VMCOREINFO_SYMBOL and update all the arrays
> > he will be happy. But if can not do it easily I'm fine with a
> > VMCOREINFO_SYMBOL_ARRAY changes only now, we kdump people can do it
> > later as well. 
> 
> It seems it's the only array we have there. swapper_pg_dir is a potential
> candidate, but it's 'unsigned long' on arm.
> 
> Below it patch with corrected macro name.
> 
> Please, consider applying.
> 
> From 70f3a84b97f2de98d1364f7b10b7a42a1d8e9968 Mon Sep 17 00:00:00 2001
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Date: Tue, 9 Jan 2018 02:55:47 +0300
> Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> 
> Depending on configuration mem_section can now be an array or a pointer
> to an array allocated dynamically. In most cases, we can continue to refer
> to it as 'mem_section' regardless of what it is.
> 
> But there's one exception: '&mem_section' means "address of the array" if
> mem_section is an array, but if mem_section is a pointer, it would mean
> "address of the pointer".
> 
> We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> writes down address of pointer into vmcoreinfo, not array as we wanted.
> 
> Let's introduce VMCOREINFO_SYMBOL_ARRAY() that would handle the
> situation correctly for both cases.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
> ---
>  include/linux/crash_core.h | 2 ++
>  kernel/crash_core.c        | 2 +-
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 06097ef30449..b511f6d24b42 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
>  	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
>  #define VMCOREINFO_SYMBOL(name) \
>  	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> +#define VMCOREINFO_SYMBOL_ARRAY(name) \
> +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
>  #define VMCOREINFO_SIZE(name) \
>  	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
>  			      (unsigned long)sizeof(name))
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index b3663896278e..4f63597c824d 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
>  	VMCOREINFO_SYMBOL(contig_page_data);
>  #endif
>  #ifdef CONFIG_SPARSEMEM
> -	VMCOREINFO_SYMBOL(mem_section);
> +	VMCOREINFO_SYMBOL_ARRAY(mem_section);
>  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>  	VMCOREINFO_STRUCT_SIZE(mem_section);
>  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> -- 
>  Kirill A. Shutemov


Acked-by: Dave Young <dyoung@redhat.com>

If stable kernel took the mem section commits, then should also cc
stable.  Andrew, can you help to make this in 4.15?

Thanks
Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Omar Sandoval]

On Fri, Jan 12, 2018 at 08:55:49AM +0800, Dave Young wrote:
> On 01/10/18 at 02:16pm, Kirill A. Shutemov wrote:
> > On Wed, Jan 10, 2018 at 03:08:04AM +0000, Dave Young wrote:
> > > On Tue, Jan 09, 2018 at 12:05:52PM +0300, Kirill A. Shutemov wrote:
> > > > On Tue, Jan 09, 2018 at 03:24:40PM +0800, Dave Young wrote:
> > > > > On 01/09/18 at 01:41pm, Baoquan He wrote:
> > > > > > On 01/09/18 at 09:09am, Dave Young wrote:
> > > > > > 
> > > > > > > As for the macro name, VMCOREINFO_SYMBOL_ARRAY sounds better.
> > > > 
> > > > Yep, that's better.
> > > > 
> > > > > > I still think using vmcoreinfo_append_str is better. Unless we replace
> > > > > > all array variables with the newly added macro.
> > > > > > 
> > > > > > vmcoreinfo_append_str("SYMBOL(mem_section)=%lx\n",
> > > > > >                                 (unsigned long)mem_section);
> > > > > 
> > > > > I have no strong opinion, either change all array uses or just introduce
> > > > > the macro and start to use it from now on if we have similar array
> > > > > symbols.
> > > > 
> > > > Do you need some action on my side or will you folks take care about this?
> > > 
> > > I think Baoquan was suggesting to update all array users in current
> > > code, if you can check every VMCOREINFO_SYMBOL and update all the arrays
> > > he will be happy. But if can not do it easily I'm fine with a
> > > VMCOREINFO_SYMBOL_ARRAY changes only now, we kdump people can do it
> > > later as well. 
> > 
> > It seems it's the only array we have there. swapper_pg_dir is a potential
> > candidate, but it's 'unsigned long' on arm.
> > 
> > Below it patch with corrected macro name.
> > 
> > Please, consider applying.
> > 
> > From 70f3a84b97f2de98d1364f7b10b7a42a1d8e9968 Mon Sep 17 00:00:00 2001
> > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> > Date: Tue, 9 Jan 2018 02:55:47 +0300
> > Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> > 
> > Depending on configuration mem_section can now be an array or a pointer
> > to an array allocated dynamically. In most cases, we can continue to refer
> > to it as 'mem_section' regardless of what it is.
> > 
> > But there's one exception: '&mem_section' means "address of the array" if
> > mem_section is an array, but if mem_section is a pointer, it would mean
> > "address of the pointer".
> > 
> > We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> > writes down address of pointer into vmcoreinfo, not array as we wanted.
> > 
> > Let's introduce VMCOREINFO_SYMBOL_ARRAY() that would handle the
> > situation correctly for both cases.
> > 
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
> > ---
> >  include/linux/crash_core.h | 2 ++
> >  kernel/crash_core.c        | 2 +-
> >  2 files changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> > index 06097ef30449..b511f6d24b42 100644
> > --- a/include/linux/crash_core.h
> > +++ b/include/linux/crash_core.h
> > @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
> >  	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
> >  #define VMCOREINFO_SYMBOL(name) \
> >  	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> > +#define VMCOREINFO_SYMBOL_ARRAY(name) \
> > +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
> >  #define VMCOREINFO_SIZE(name) \
> >  	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
> >  			      (unsigned long)sizeof(name))
> > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> > index b3663896278e..4f63597c824d 100644
> > --- a/kernel/crash_core.c
> > +++ b/kernel/crash_core.c
> > @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
> >  	VMCOREINFO_SYMBOL(contig_page_data);
> >  #endif
> >  #ifdef CONFIG_SPARSEMEM
> > -	VMCOREINFO_SYMBOL(mem_section);
> > +	VMCOREINFO_SYMBOL_ARRAY(mem_section);
> >  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
> >  	VMCOREINFO_STRUCT_SIZE(mem_section);
> >  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> > -- 
> >  Kirill A. Shutemov
> 
> 
> Acked-by: Dave Young <dyoung@redhat.com>
> 
> If stable kernel took the mem section commits, then should also cc
> stable.  Andrew, can you help to make this in 4.15?
> 
> Thanks
> Dave

Hm, this fix means that the vmlinux symbol table and vmcoreinfo have
different values for mem_section. That seems... odd. I had to patch
makedumpfile to fix the case of an explicit vmlinux being passed on the
command line (which I realized I don't need to do, but it should still
work):

RE: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Atsushi Kumagai]

>Hm, this fix means that the vmlinux symbol table and vmcoreinfo have
>different values for mem_section. That seems... odd. I had to patch
>makedumpfile to fix the case of an explicit vmlinux being passed on the
>command line (which I realized I don't need to do, but it should still
>work):

Looks good to me, I'll merge this into makedumpfile-1.6.4.

Thanks,
Atsushi Kumagai

>From 542a11a8f28b0f0a989abc3adff89da22f44c719 Mon Sep 17 00:00:00 2001
>Message-Id: <542a11a8f28b0f0a989abc3adff89da22f44c719.1515995400.git.osandov@fb.com>
>From: Omar Sandoval <osandov@fb.com>
>Date: Sun, 14 Jan 2018 17:10:30 -0800
>Subject: [PATCH] Fix SPARSEMEM_EXTREME support on Linux v4.15 when passing
> vmlinux
>
>Since kernel commit 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at
>runtime for CONFIG_SPARSEMEM_EXTREME=y"), mem_section is a dynamically
>allocated array of pointers to mem_section instead of a static one
>(i.e., struct mem_section ** instead of struct mem_section * []). This
>adds an extra layer of indirection that breaks makedumpfile, which will
>end up with a bunch of bogus mem_maps.
>
>Since kernel commit a0b1280368d1 ("kdump: write correct address of
>mem_section into vmcoreinfo"), the mem_section symbol in vmcoreinfo
>contains the address of the actual struct mem_section * array instead of
>the address of the pointer in .bss, which gets rid of the extra
>indirection. However, makedumpfile still uses the debugging symbol from
>the vmlinux image. Fix this by allowing symbols from the vmcore to
>override symbols from the vmlinux image. As the comment in initial()
>says, "vmcoreinfo in /proc/vmcore is more reliable than -x/-i option".
>
>Signed-off-by: Omar Sandoval <osandov@fb.com>
>---
> makedumpfile.h | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
>diff --git a/makedumpfile.h b/makedumpfile.h
>index 57cf4d9..d68c798 100644
>--- a/makedumpfile.h
>+++ b/makedumpfile.h
>@@ -274,8 +274,10 @@ do { \
> } while (0)
> #define READ_SYMBOL(str_symbol, symbol) \
> do { \
>-	if (SYMBOL(symbol) == NOT_FOUND_SYMBOL) { \
>-		SYMBOL(symbol) = read_vmcoreinfo_symbol(STR_SYMBOL(str_symbol)); \
>+	unsigned long _tmp_symbol; \
>+	_tmp_symbol = read_vmcoreinfo_symbol(STR_SYMBOL(str_symbol)); \
>+	if (_tmp_symbol != NOT_FOUND_SYMBOL) { \
>+		SYMBOL(symbol) = _tmp_symbol; \
> 		if (SYMBOL(symbol) == INVALID_SYMBOL_DATA) \
> 			return FALSE; \
> 	} \
>--
>2.9.5
>
>
>_______________________________________________
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Mike Galbraith]

On Tue, 2018-01-09 at 03:13 +0300, Kirill A. Shutemov wrote:
> 
> Mike, could you test this? (On top of the rest of the fixes.)

homer:..crash/2018-01-09-04:25 # ll
total 1863604
-rw------- 1 root root      66255 Jan  9 04:25 dmesg.txt
-rw-r--r-- 1 root root        182 Jan  9 04:25 README.txt
-rw-r--r-- 1 root root    2818240 Jan  9 04:25 System.map-4.15.0.gb2cd1df-master
-rw------- 1 root root 1832914928 Jan  9 04:25 vmcore
-rw-r--r-- 1 root root   72514993 Jan  9 04:25 vmlinux-4.15.0.gb2cd1df-master.gz

Yup, all better.

> Sorry for the mess.

(why, developers not installing shiny new bugs is a whole lot worse:)

> From 100fd567754f1457be94732046aefca204c842d2 Mon Sep 17 00:00:00 2001
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Date: Tue, 9 Jan 2018 02:55:47 +0300
> Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> 
> Depending on configuration mem_section can now be an array or a pointer
> to an array allocated dynamically. In most cases, we can continue to refer
> to it as 'mem_section' regardless of what it is.
> 
> But there's one exception: '&mem_section' means "address of the array" if
> mem_section is an array, but if mem_section is a pointer, it would mean
> "address of the pointer".
> 
> We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> writes down address of pointer into vmcoreinfo, not array as we wanted.
> 
> Let's introduce VMCOREINFO_ARRAY() that would handle the situation
> correctly for both cases.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
> ---
>  include/linux/crash_core.h | 2 ++
>  kernel/crash_core.c        | 2 +-
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 06097ef30449..83ae04950269 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
>  	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
>  #define VMCOREINFO_SYMBOL(name) \
>  	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> +#define VMCOREINFO_ARRAY(name) \
> +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
>  #define VMCOREINFO_SIZE(name) \
>  	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
>  			      (unsigned long)sizeof(name))
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index b3663896278e..d4122a837477 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
>  	VMCOREINFO_SYMBOL(contig_page_data);
>  #endif
>  #ifdef CONFIG_SPARSEMEM
> -	VMCOREINFO_SYMBOL(mem_section);
> +	VMCOREINFO_ARRAY(mem_section);
>  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>  	VMCOREINFO_STRUCT_SIZE(mem_section);
>  	VMCOREINFO_OFFSET(mem_section, section_mem_map);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dou Liyang]

Hi All,

I met the makedumpfile failed in the upstream kernel which contained
this patch. Did I missed something else?

In fedora27 host:

[douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+

sadump: does not have partition header
sadump: read dump device as unknown format
sadump: unknown format
LOAD (0)
   phys_start : 1000000
   phys_end   : 2a86000
   virt_start : ffffffff81000000
   virt_end   : ffffffff82a86000
LOAD (1)
   phys_start : 1000
   phys_end   : 9fc00
   virt_start : ffff880000001000
   virt_end   : ffff88000009fc00
LOAD (2)
   phys_start : 100000
   phys_end   : 13000000
   virt_start : ffff880000100000
   virt_end   : ffff880013000000
LOAD (3)
   phys_start : 33000000
   phys_end   : 7ffd7000
   virt_start : ffff880033000000
   virt_end   : ffff88007ffd7000
Linux kdump
page_size    : 4096

max_mapnr    : 7ffd7

Buffer size for the cyclic mode: 131061
The kernel version is not supported.
The makedumpfile operation may be incomplete.

num of NODEs : 1


Memory type  : SPARSEMEM_EX

mem_map (0)
   mem_map    : ffff88007ff26000
   pfn_start  : 0
   pfn_end    : 8000
mem_map (1)
   mem_map    : 0
   pfn_start  : 8000
   pfn_end    : 10000
mem_map (2)
   mem_map    : 0
   pfn_start  : 10000
   pfn_end    : 18000
mem_map (3)
   mem_map    : 0
   pfn_start  : 18000
   pfn_end    : 20000
mem_map (4)
   mem_map    : 0
   pfn_start  : 20000
   pfn_end    : 28000
mem_map (5)
   mem_map    : 0
   pfn_start  : 28000
   pfn_end    : 30000
mem_map (6)
   mem_map    : 0
   pfn_start  : 30000
   pfn_end    : 38000
mem_map (7)
   mem_map    : 0
   pfn_start  : 38000
   pfn_end    : 40000
mem_map (8)
   mem_map    : 0
   pfn_start  : 40000
   pfn_end    : 48000
mem_map (9)
   mem_map    : 0
   pfn_start  : 48000
   pfn_end    : 50000
mem_map (10)
   mem_map    : 0
   pfn_start  : 50000
   pfn_end    : 58000
mem_map (11)
   mem_map    : 0
   pfn_start  : 58000
   pfn_end    : 60000
mem_map (12)
   mem_map    : 0
   pfn_start  : 60000
   pfn_end    : 68000
mem_map (13)
   mem_map    : 0
   pfn_start  : 68000
   pfn_end    : 70000
mem_map (14)
   mem_map    : 0
   pfn_start  : 70000
   pfn_end    : 78000
mem_map (15)
   mem_map    : 0
   pfn_start  : 78000
   pfn_end    : 7ffd7
mmap() is available on the kernel.
Checking for memory holes                         : [100.0 %] | 
         STEP [Checking for memory holes  ] : 0.000060 seconds
__vtop4_x86_64: Can't get a valid pte.
readmem: Can't convert a virtual address(ffff88007ffd7000) to physical 
address.
readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
__exclude_unnecessary_pages: Can't read the buffer of struct page.
create_2nd_bitmap: Can't exclude unnecessary pages.
Checking for memory holes                         : [100.0 %] \ 
         STEP [Checking for memory holes  ] : 0.000010 seconds
Checking for memory holes                         : [100.0 %] - 
         STEP [Checking for memory holes  ] : 0.000004 seconds
__vtop4_x86_64: Can't get a valid pte.
readmem: Can't convert a virtual address(ffff88007ffd7000) to physical 
address.
readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
__exclude_unnecessary_pages: Can't read the buffer of struct page.
create_2nd_bitmap: Can't exclude unnecessary pages.

Thanks,
	dou
At 01/09/2018 11:44 AM, Mike Galbraith wrote:
> On Tue, 2018-01-09 at 03:13 +0300, Kirill A. Shutemov wrote:
>>
>> Mike, could you test this? (On top of the rest of the fixes.)
> 
> homer:..crash/2018-01-09-04:25 # ll
> total 1863604
> -rw------- 1 root root      66255 Jan  9 04:25 dmesg.txt
> -rw-r--r-- 1 root root        182 Jan  9 04:25 README.txt
> -rw-r--r-- 1 root root    2818240 Jan  9 04:25 System.map-4.15.0.gb2cd1df-master
> -rw------- 1 root root 1832914928 Jan  9 04:25 vmcore
> -rw-r--r-- 1 root root   72514993 Jan  9 04:25 vmlinux-4.15.0.gb2cd1df-master.gz
> 
> Yup, all better.
> 
>> Sorry for the mess.
> 
> (why, developers not installing shiny new bugs is a whole lot worse:)
> 
>>  From 100fd567754f1457be94732046aefca204c842d2 Mon Sep 17 00:00:00 2001
>> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
>> Date: Tue, 9 Jan 2018 02:55:47 +0300
>> Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
>>
>> Depending on configuration mem_section can now be an array or a pointer
>> to an array allocated dynamically. In most cases, we can continue to refer
>> to it as 'mem_section' regardless of what it is.
>>
>> But there's one exception: '&mem_section' means "address of the array" if
>> mem_section is an array, but if mem_section is a pointer, it would mean
>> "address of the pointer".
>>
>> We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
>> writes down address of pointer into vmcoreinfo, not array as we wanted.
>>
>> Let's introduce VMCOREINFO_ARRAY() that would handle the situation
>> correctly for both cases.
>>
>> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>> Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
>> ---
>>   include/linux/crash_core.h | 2 ++
>>   kernel/crash_core.c        | 2 +-
>>   2 files changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
>> index 06097ef30449..83ae04950269 100644
>> --- a/include/linux/crash_core.h
>> +++ b/include/linux/crash_core.h
>> @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
>>   	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
>>   #define VMCOREINFO_SYMBOL(name) \
>>   	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
>> +#define VMCOREINFO_ARRAY(name) \
>> +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
>>   #define VMCOREINFO_SIZE(name) \
>>   	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
>>   			      (unsigned long)sizeof(name))
>> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
>> index b3663896278e..d4122a837477 100644
>> --- a/kernel/crash_core.c
>> +++ b/kernel/crash_core.c
>> @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
>>   	VMCOREINFO_SYMBOL(contig_page_data);
>>   #endif
>>   #ifdef CONFIG_SPARSEMEM
>> -	VMCOREINFO_SYMBOL(mem_section);
>> +	VMCOREINFO_ARRAY(mem_section);
>>   	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>>   	VMCOREINFO_STRUCT_SIZE(mem_section);
>>   	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> 
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
> 
> 
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Kirill A. Shutemov]

On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
> Hi All,
> 
> I met the makedumpfile failed in the upstream kernel which contained
> this patch. Did I missed something else?

None I'm aware of.

Is there a reason to suspect that the issue is related to the bug this patch
fixed?

-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Mike Galbraith]

On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
> On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
> > Hi All,
> > 
> > I met the makedumpfile failed in the upstream kernel which contained
> > this patch. Did I missed something else?
> 
> None I'm aware of.
> 
> Is there a reason to suspect that the issue is related to the bug this patch
> fixed?

Still works fine for me with .today.  Box is only 16GB desktop box though.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dou Liyang]

Hi Kirill,Mike

At 02/07/2018 06:45 PM, Mike Galbraith wrote:
> On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
>> On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
>>> Hi All,
>>>
>>> I met the makedumpfile failed in the upstream kernel which contained
>>> this patch. Did I missed something else?
>>
>> None I'm aware of.
>>
>> Is there a reason to suspect that the issue is related to the bug this patch
>> fixed?
> 

I did a contrastive test by my colleagues Indoh's suggestion.

Revert your two commits:

commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Date:   Fri Sep 29 17:08:16 2017 +0300

commit 629a359bdb0e0652a8227b4ff3125431995fec6e
Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Date:   Tue Nov 7 11:33:37 2017 +0300

...and keep others unchanged, the makedumpfile works well.

> Still works fine for me with .today.  Box is only 16GB desktop box though.
> 
Btw, In the upstream kernel which contained this patch, I did two tests:

  1) use the makedumpfile as core_collector in /etc/kdump.conf, then
trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
makedumpfile works well and I can get the vmcore file.

      ......It is OK

  2) use cp as core_collector, do the same operation to get the vmcore 
file. then use makedumpfile to do like above:

     [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+

     ......It causes makedumpfile failed.


Thanks,
	dou.

> 	-Mike
> 
> 
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

On 02/07/18 at 08:00pm, Dou Liyang wrote:
> Hi Kirill,Mike
> 
> At 02/07/2018 06:45 PM, Mike Galbraith wrote:
> > On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
> > > On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
> > > > Hi All,
> > > > 
> > > > I met the makedumpfile failed in the upstream kernel which contained
> > > > this patch. Did I missed something else?
> > > 
> > > None I'm aware of.
> > > 
> > > Is there a reason to suspect that the issue is related to the bug this patch
> > > fixed?
> > 
> 
> I did a contrastive test by my colleagues Indoh's suggestion.
> 
> Revert your two commits:
> 
> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Date:   Fri Sep 29 17:08:16 2017 +0300
> 
> commit 629a359bdb0e0652a8227b4ff3125431995fec6e
> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Date:   Tue Nov 7 11:33:37 2017 +0300
> 
> ...and keep others unchanged, the makedumpfile works well.
> 
> > Still works fine for me with .today.  Box is only 16GB desktop box though.
> > 
> Btw, In the upstream kernel which contained this patch, I did two tests:
> 
>  1) use the makedumpfile as core_collector in /etc/kdump.conf, then
> trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
> makedumpfile works well and I can get the vmcore file.
> 
>      ......It is OK
> 
>  2) use cp as core_collector, do the same operation to get the vmcore file.
> then use makedumpfile to do like above:
> 
>     [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
> vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+

Oh, then please ignore my previous comment. Adding '-D' can give more
debugging message.

> 
>     ......It causes makedumpfile failed.
> 
> 
> Thanks,
> 	dou.
> 
> > 	-Mike
> > 
> > 
> > 
> 
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dou Liyang]

Hi Baoquan,

At 02/07/2018 08:08 PM, Baoquan He wrote:
> On 02/07/18 at 08:00pm, Dou Liyang wrote:
>> Hi Kirill,Mike
>>
>> At 02/07/2018 06:45 PM, Mike Galbraith wrote:
>>> On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
>>>> On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
>>>>> Hi All,
>>>>>
>>>>> I met the makedumpfile failed in the upstream kernel which contained
>>>>> this patch. Did I missed something else?
>>>>
>>>> None I'm aware of.
>>>>
>>>> Is there a reason to suspect that the issue is related to the bug this patch
>>>> fixed?
>>>
>>
>> I did a contrastive test by my colleagues Indoh's suggestion.
>>
>> Revert your two commits:
>>
>> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
>> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>> Date:   Fri Sep 29 17:08:16 2017 +0300
>>
>> commit 629a359bdb0e0652a8227b4ff3125431995fec6e
>> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>> Date:   Tue Nov 7 11:33:37 2017 +0300
>>
>> ...and keep others unchanged, the makedumpfile works well.
>>
>>> Still works fine for me with .today.  Box is only 16GB desktop box though.
>>>
>> Btw, In the upstream kernel which contained this patch, I did two tests:
>>
>>   1) use the makedumpfile as core_collector in /etc/kdump.conf, then
>> trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
>> makedumpfile works well and I can get the vmcore file.
>>
>>       ......It is OK
>>
>>   2) use cp as core_collector, do the same operation to get the vmcore file.
>> then use makedumpfile to do like above:
>>
>>      [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
>> vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+
> 
> Oh, then please ignore my previous comment. Adding '-D' can give more
> debugging message.

I added '-D', Just like before, no more debugging message:

BTW, I use crash to analyze the vmcore file created by 'cp' command.

    ./crash ../makedumpfile/code/vmcore_4.15+_from_cp_command 
../makedumpfile/code/vmlinux_4.15+

the crash works well, It's so interesting.

Thanks,
	dou.

The debugging message with '-D':

[douly@localhost code]$ ./makedumpfile -D -d 31 --message-level 31 -x 
vmlinux_4.15+  vmcore_4.15+_from_cp_command vmcore_4.15+
sadump: does not have partition header
sadump: read dump device as unknown format
sadump: unknown format
LOAD (0)
   phys_start : 1000000
   phys_end   : 2a86000
   virt_start : ffffffff81000000
   virt_end   : ffffffff82a86000
LOAD (1)
   phys_start : 1000
   phys_end   : 9fc00
   virt_start : ffff880000001000
   virt_end   : ffff88000009fc00
LOAD (2)
   phys_start : 100000
   phys_end   : 13000000
   virt_start : ffff880000100000
   virt_end   : ffff880013000000
LOAD (3)
   phys_start : 33000000
   phys_end   : 7ffd7000
   virt_start : ffff880033000000
   virt_end   : ffff88007ffd7000
Linux kdump
page_size    : 4096

max_mapnr    : 7ffd7

Buffer size for the cyclic mode: 131061
The kernel version is not supported.
The makedumpfile operation may be incomplete.

num of NODEs : 1


Memory type  : SPARSEMEM_EX

mem_map (0)
   mem_map    : ffff88007ff26000
   pfn_start  : 0
   pfn_end    : 8000
mem_map (1)
   mem_map    : 0
   pfn_start  : 8000
   pfn_end    : 10000
mem_map (2)
   mem_map    : 0
   pfn_start  : 10000
   pfn_end    : 18000
mem_map (3)
   mem_map    : 0
   pfn_start  : 18000
   pfn_end    : 20000
mem_map (4)
   mem_map    : 0
   pfn_start  : 20000
   pfn_end    : 28000
mem_map (5)
   mem_map    : 0
   pfn_start  : 28000
   pfn_end    : 30000
mem_map (6)
   mem_map    : 0
   pfn_start  : 30000
   pfn_end    : 38000
mem_map (7)
   mem_map    : 0
   pfn_start  : 38000
   pfn_end    : 40000
mem_map (8)
   mem_map    : 0
   pfn_start  : 40000
   pfn_end    : 48000
mem_map (9)
   mem_map    : 0
   pfn_start  : 48000
   pfn_end    : 50000
mem_map (10)
   mem_map    : 0
   pfn_start  : 50000
   pfn_end    : 58000
mem_map (11)
   mem_map    : 0
   pfn_start  : 58000
   pfn_end    : 60000
mem_map (12)
   mem_map    : 0
   pfn_start  : 60000
   pfn_end    : 68000
mem_map (13)
   mem_map    : 0
   pfn_start  : 68000
   pfn_end    : 70000
mem_map (14)
   mem_map    : 0
   pfn_start  : 70000
   pfn_end    : 78000
mem_map (15)
   mem_map    : 0
   pfn_start  : 78000
   pfn_end    : 7ffd7
mmap() is available on the kernel.
Checking for memory holes                         : [100.0 %] | 
         STEP [Checking for memory holes  ] : 0.000014 seconds
__vtop4_x86_64: Can't get a valid pte.
readmem: Can't convert a virtual address(ffff88007ffd7000) to physical 
address.
readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
__exclude_unnecessary_pages: Can't read the buffer of struct page.
create_2nd_bitmap: Can't exclude unnecessary pages.
Checking for memory holes                         : [100.0 %] \ 
         STEP [Checking for memory holes  ] : 0.000006 seconds
Checking for memory holes                         : [100.0 %] - 
         STEP [Checking for memory holes  ] : 0.000004 seconds
__vtop4_x86_64: Can't get a valid pte.
readmem: Can't convert a virtual address(ffff88007ffd7000) to physical 
address.
readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
__exclude_unnecessary_pages: Can't read the buffer of struct page.
create_2nd_bitmap: Can't exclude unnecessary pages.

makedumpfile Failed.

> 
>>
>>      ......It causes makedumpfile failed.
>>
>>
>> Thanks,
>> 	dou.
>>
>>> 	-Mike
>>>
>>>
>>>
>>
>>
> 
> 
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

On 02/07/18 at 08:17pm, Dou Liyang wrote:
> Hi Baoquan,
> 
> At 02/07/2018 08:08 PM, Baoquan He wrote:
> > On 02/07/18 at 08:00pm, Dou Liyang wrote:
> > > Hi Kirill,Mike
> > > 
> > > At 02/07/2018 06:45 PM, Mike Galbraith wrote:
> > > > On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
> > > > > On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
> > > > > > Hi All,
> > > > > > 
> > > > > > I met the makedumpfile failed in the upstream kernel which contained
> > > > > > this patch. Did I missed something else?
> > > > > 
> > > > > None I'm aware of.
> > > > > 
> > > > > Is there a reason to suspect that the issue is related to the bug this patch
> > > > > fixed?
> > > > 
> > > 
> > > I did a contrastive test by my colleagues Indoh's suggestion.
> > > 
> > > Revert your two commits:
> > > 
> > > commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
> > > Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > Date:   Fri Sep 29 17:08:16 2017 +0300
> > > 
> > > commit 629a359bdb0e0652a8227b4ff3125431995fec6e
> > > Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > Date:   Tue Nov 7 11:33:37 2017 +0300
> > > 
> > > ...and keep others unchanged, the makedumpfile works well.
> > > 
> > > > Still works fine for me with .today.  Box is only 16GB desktop box though.
> > > > 
> > > Btw, In the upstream kernel which contained this patch, I did two tests:
> > > 
> > >   1) use the makedumpfile as core_collector in /etc/kdump.conf, then
> > > trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
> > > makedumpfile works well and I can get the vmcore file.
> > > 
> > >       ......It is OK
> > > 
> > >   2) use cp as core_collector, do the same operation to get the vmcore file.
> > > then use makedumpfile to do like above:
> > > 
> > >      [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
> > > vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+
> > 
> > Oh, then please ignore my previous comment. Adding '-D' can give more
> > debugging message.
> 
> I added '-D', Just like before, no more debugging message:
> 
> BTW, I use crash to analyze the vmcore file created by 'cp' command.
> 
>    ./crash ../makedumpfile/code/vmcore_4.15+_from_cp_command
> ../makedumpfile/code/vmlinux_4.15+
> 
> the crash works well, It's so interesting.
> 
> Thanks,
> 	dou.
> 
> The debugging message with '-D':

And what's the debugging printing when trigger crash by sysrq?

> 
> [douly@localhost code]$ ./makedumpfile -D -d 31 --message-level 31 -x
> vmlinux_4.15+  vmcore_4.15+_from_cp_command vmcore_4.15+
> sadump: does not have partition header
> sadump: read dump device as unknown format
> sadump: unknown format
> LOAD (0)
>   phys_start : 1000000
>   phys_end   : 2a86000
>   virt_start : ffffffff81000000
>   virt_end   : ffffffff82a86000
> LOAD (1)
>   phys_start : 1000
>   phys_end   : 9fc00
>   virt_start : ffff880000001000
>   virt_end   : ffff88000009fc00
> LOAD (2)
>   phys_start : 100000
>   phys_end   : 13000000
>   virt_start : ffff880000100000
>   virt_end   : ffff880013000000
> LOAD (3)
>   phys_start : 33000000
>   phys_end   : 7ffd7000
>   virt_start : ffff880033000000
>   virt_end   : ffff88007ffd7000
> Linux kdump
> page_size    : 4096
> 
> max_mapnr    : 7ffd7
> 
> Buffer size for the cyclic mode: 131061
> The kernel version is not supported.
> The makedumpfile operation may be incomplete.
> 
> num of NODEs : 1
> 
> 
> Memory type  : SPARSEMEM_EX
> 
> mem_map (0)
>   mem_map    : ffff88007ff26000
>   pfn_start  : 0
>   pfn_end    : 8000
> mem_map (1)
>   mem_map    : 0
>   pfn_start  : 8000
>   pfn_end    : 10000
> mem_map (2)
>   mem_map    : 0
>   pfn_start  : 10000
>   pfn_end    : 18000
> mem_map (3)
>   mem_map    : 0
>   pfn_start  : 18000
>   pfn_end    : 20000
> mem_map (4)
>   mem_map    : 0
>   pfn_start  : 20000
>   pfn_end    : 28000
> mem_map (5)
>   mem_map    : 0
>   pfn_start  : 28000
>   pfn_end    : 30000
> mem_map (6)
>   mem_map    : 0
>   pfn_start  : 30000
>   pfn_end    : 38000
> mem_map (7)
>   mem_map    : 0
>   pfn_start  : 38000
>   pfn_end    : 40000
> mem_map (8)
>   mem_map    : 0
>   pfn_start  : 40000
>   pfn_end    : 48000
> mem_map (9)
>   mem_map    : 0
>   pfn_start  : 48000
>   pfn_end    : 50000
> mem_map (10)
>   mem_map    : 0
>   pfn_start  : 50000
>   pfn_end    : 58000
> mem_map (11)
>   mem_map    : 0
>   pfn_start  : 58000
>   pfn_end    : 60000
> mem_map (12)
>   mem_map    : 0
>   pfn_start  : 60000
>   pfn_end    : 68000
> mem_map (13)
>   mem_map    : 0
>   pfn_start  : 68000
>   pfn_end    : 70000
> mem_map (14)
>   mem_map    : 0
>   pfn_start  : 70000
>   pfn_end    : 78000
> mem_map (15)
>   mem_map    : 0
>   pfn_start  : 78000
>   pfn_end    : 7ffd7
> mmap() is available on the kernel.
> Checking for memory holes                         : [100.0 %] |         STEP
> [Checking for memory holes  ] : 0.000014 seconds
> __vtop4_x86_64: Can't get a valid pte.
> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
> address.
> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
> __exclude_unnecessary_pages: Can't read the buffer of struct page.
> create_2nd_bitmap: Can't exclude unnecessary pages.
> Checking for memory holes                         : [100.0 %] \         STEP
> [Checking for memory holes  ] : 0.000006 seconds
> Checking for memory holes                         : [100.0 %] -         STEP
> [Checking for memory holes  ] : 0.000004 seconds
> __vtop4_x86_64: Can't get a valid pte.
> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
> address.
> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
> __exclude_unnecessary_pages: Can't read the buffer of struct page.
> create_2nd_bitmap: Can't exclude unnecessary pages.
> 
> makedumpfile Failed.
> 
> > 
> > > 
> > >      ......It causes makedumpfile failed.
> > > 
> > > 
> > > Thanks,
> > > 	dou.
> > > 
> > > > 	-Mike
> > > > 
> > > > 
> > > > 
> > > 
> > > 
> > 
> > 
> > 
> 
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dou Liyang]

At 02/07/2018 08:27 PM, Baoquan He wrote:
> On 02/07/18 at 08:17pm, Dou Liyang wrote:
>> Hi Baoquan,
>>
>> At 02/07/2018 08:08 PM, Baoquan He wrote:
>>> On 02/07/18 at 08:00pm, Dou Liyang wrote:
>>>> Hi Kirill,Mike
>>>>
>>>> At 02/07/2018 06:45 PM, Mike Galbraith wrote:
>>>>> On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
>>>>>> On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I met the makedumpfile failed in the upstream kernel which contained
>>>>>>> this patch. Did I missed something else?
>>>>>>
>>>>>> None I'm aware of.
>>>>>>
>>>>>> Is there a reason to suspect that the issue is related to the bug this patch
>>>>>> fixed?
>>>>>
>>>>
>>>> I did a contrastive test by my colleagues Indoh's suggestion.
>>>>
>>>> Revert your two commits:
>>>>
>>>> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
>>>> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>>>> Date:   Fri Sep 29 17:08:16 2017 +0300
>>>>
>>>> commit 629a359bdb0e0652a8227b4ff3125431995fec6e
>>>> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>>>> Date:   Tue Nov 7 11:33:37 2017 +0300
>>>>
>>>> ...and keep others unchanged, the makedumpfile works well.
>>>>
>>>>> Still works fine for me with .today.  Box is only 16GB desktop box though.
>>>>>
>>>> Btw, In the upstream kernel which contained this patch, I did two tests:
>>>>
>>>>    1) use the makedumpfile as core_collector in /etc/kdump.conf, then
>>>> trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
>>>> makedumpfile works well and I can get the vmcore file.
>>>>
>>>>        ......It is OK
>>>>
>>>>    2) use cp as core_collector, do the same operation to get the vmcore file.
>>>> then use makedumpfile to do like above:
>>>>
>>>>       [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
>>>> vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+
>>>
>>> Oh, then please ignore my previous comment. Adding '-D' can give more
>>> debugging message.
>>
>> I added '-D', Just like before, no more debugging message:
>>
>> BTW, I use crash to analyze the vmcore file created by 'cp' command.
>>
>>     ./crash ../makedumpfile/code/vmcore_4.15+_from_cp_command
>> ../makedumpfile/code/vmlinux_4.15+
>>
>> the crash works well, It's so interesting.
>>
>> Thanks,
>> 	dou.
>>
>> The debugging message with '-D':
> 
> And what's the debugging printing when trigger crash by sysrq?
> 

kdump: dump target is /dev/vda2
kdump: saving to /sysroot//var/crash/127.0.0.1-2018-02-07-07:31:56/
[    2.751352] EXT4-fs (vda2): re-mounted. Opts: data=ordered
kdump: saving vmcore-dmesg.txt
kdump: saving vmcore-dmesg.txt complete
kdump: saving vmcore
sadump: does not have partition header
sadump: read dump device as unknown format
sadump: unknown format
LOAD (0)
   phys_start : 1000000
   phys_end   : 2a86000
   virt_start : ffffffff81000000
   virt_end   : ffffffff82a86000
LOAD (1)
   phys_start : 1000
   phys_end   : 9fc00
   virt_start : ffff880000001000
   virt_end   : ffff88000009fc00
LOAD (2)
   phys_start : 100000
   phys_end   : 13000000
   virt_start : ffff880000100000
   virt_end   : ffff880013000000
LOAD (3)
   phys_start : 33000000
   phys_end   : 7ffd7000
   virt_start : ffff880033000000
   virt_end   : ffff88007ffd7000
Linux kdump
page_size    : 4096

max_mapnr    : 7ffd7

Buffer size for the cyclic mode: 131061

num of NODEs : 1


Memory type  : SPARSEMEM_EX

mem_map (0)
   mem_map    : ffffea0000000000
   pfn_start  : 0
   pfn_end    : 8000
mem_map (1)
   mem_map    : ffffea0000200000
   pfn_start  : 8000
   pfn_end    : 10000
mem_map (2)
   mem_map    : ffffea0000400000
   pfn_start  : 10000
   pfn_end    : 18000
mem_map (3)
   mem_map    : ffffea0000600000
   pfn_start  : 18000
   pfn_end    : 20000
mem_map (4)
   mem_map    : ffffea0000800000
   pfn_start  : 20000
   pfn_end    : 28000
mem_map (5)
   mem_map    : ffffea0000a00000
   pfn_start  : 28000
   pfn_end    : 30000
mem_map (6)
   mem_map    : ffffea0000c00000
   pfn_start  : 30000
   pfn_end    : 38000
mem_map (7)
   mem_map    : ffffea0000e00000
   pfn_start  : 38000
   pfn_end    : 40000
mem_map (8)
   mem_map    : ffffea0001000000
   pfn_start  : 40000
   pfn_end    : 48000
mem_map (9)
   mem_map    : ffffea0001200000
   pfn_start  : 48000
   pfn_end    : 50000
mem_map (10)
   mem_map    : ffffea0001400000
   pfn_start  : 50000
   pfn_end    : 58000
mem_map (11)
   mem_map    : ffffea0001600000
   pfn_start  : 58000
   pfn_end    : 60000
mem_map (12)
   mem_map    : ffffea0001800000
   pfn_start  : 60000
   pfn_end    : 68000
mem_map (13)
   mem_map    : ffffea0001a00000
   pfn_start  : 68000
   pfn_end    : 70000
mem_map (14)
   mem_map    : ffffea0001c00000
   pfn_start  : 70000
   pfn_end    : 78000
mem_map (15)
   mem_map    : ffffea0001e00000
   pfn_start  : 78000
   pfn_end    : 7ffd7
mmap() is available on the kernel.
Copying data                                      : [100.0 %] - 
  eta: 0s
Writing erase info...
offset_eraseinfo: 9567fb0, size_eraseinfo: 0
kdump: saving vmcore complete

Thanks,
	dou

>>
>> [douly@localhost code]$ ./makedumpfile -D -d 31 --message-level 31 -x
>> vmlinux_4.15+  vmcore_4.15+_from_cp_command vmcore_4.15+
>> sadump: does not have partition header
>> sadump: read dump device as unknown format
>> sadump: unknown format
>> LOAD (0)
>>    phys_start : 1000000
>>    phys_end   : 2a86000
>>    virt_start : ffffffff81000000
>>    virt_end   : ffffffff82a86000
>> LOAD (1)
>>    phys_start : 1000
>>    phys_end   : 9fc00
>>    virt_start : ffff880000001000
>>    virt_end   : ffff88000009fc00
>> LOAD (2)
>>    phys_start : 100000
>>    phys_end   : 13000000
>>    virt_start : ffff880000100000
>>    virt_end   : ffff880013000000
>> LOAD (3)
>>    phys_start : 33000000
>>    phys_end   : 7ffd7000
>>    virt_start : ffff880033000000
>>    virt_end   : ffff88007ffd7000
>> Linux kdump
>> page_size    : 4096
>>
>> max_mapnr    : 7ffd7
>>
>> Buffer size for the cyclic mode: 131061
>> The kernel version is not supported.
>> The makedumpfile operation may be incomplete.
>>
>> num of NODEs : 1
>>
>>
>> Memory type  : SPARSEMEM_EX
>>
>> mem_map (0)
>>    mem_map    : ffff88007ff26000
>>    pfn_start  : 0
>>    pfn_end    : 8000
>> mem_map (1)
>>    mem_map    : 0
>>    pfn_start  : 8000
>>    pfn_end    : 10000
>> mem_map (2)
>>    mem_map    : 0
>>    pfn_start  : 10000
>>    pfn_end    : 18000
>> mem_map (3)
>>    mem_map    : 0
>>    pfn_start  : 18000
>>    pfn_end    : 20000
>> mem_map (4)
>>    mem_map    : 0
>>    pfn_start  : 20000
>>    pfn_end    : 28000
>> mem_map (5)
>>    mem_map    : 0
>>    pfn_start  : 28000
>>    pfn_end    : 30000
>> mem_map (6)
>>    mem_map    : 0
>>    pfn_start  : 30000
>>    pfn_end    : 38000
>> mem_map (7)
>>    mem_map    : 0
>>    pfn_start  : 38000
>>    pfn_end    : 40000
>> mem_map (8)
>>    mem_map    : 0
>>    pfn_start  : 40000
>>    pfn_end    : 48000
>> mem_map (9)
>>    mem_map    : 0
>>    pfn_start  : 48000
>>    pfn_end    : 50000
>> mem_map (10)
>>    mem_map    : 0
>>    pfn_start  : 50000
>>    pfn_end    : 58000
>> mem_map (11)
>>    mem_map    : 0
>>    pfn_start  : 58000
>>    pfn_end    : 60000
>> mem_map (12)
>>    mem_map    : 0
>>    pfn_start  : 60000
>>    pfn_end    : 68000
>> mem_map (13)
>>    mem_map    : 0
>>    pfn_start  : 68000
>>    pfn_end    : 70000
>> mem_map (14)
>>    mem_map    : 0
>>    pfn_start  : 70000
>>    pfn_end    : 78000
>> mem_map (15)
>>    mem_map    : 0
>>    pfn_start  : 78000
>>    pfn_end    : 7ffd7
>> mmap() is available on the kernel.
>> Checking for memory holes                         : [100.0 %] |         STEP
>> [Checking for memory holes  ] : 0.000014 seconds
>> __vtop4_x86_64: Can't get a valid pte.
>> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
>> address.
>> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
>> __exclude_unnecessary_pages: Can't read the buffer of struct page.
>> create_2nd_bitmap: Can't exclude unnecessary pages.
>> Checking for memory holes                         : [100.0 %] \         STEP
>> [Checking for memory holes  ] : 0.000006 seconds
>> Checking for memory holes                         : [100.0 %] -         STEP
>> [Checking for memory holes  ] : 0.000004 seconds
>> __vtop4_x86_64: Can't get a valid pte.
>> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
>> address.
>> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
>> __exclude_unnecessary_pages: Can't read the buffer of struct page.
>> create_2nd_bitmap: Can't exclude unnecessary pages.
>>
>> makedumpfile Failed.
>>
>>>
>>>>
>>>>       ......It causes makedumpfile failed.
>>>>
>>>>
>>>> Thanks,
>>>> 	dou.
>>>>
>>>>> 	-Mike
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
> 
> 
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

On 02/07/18 at 08:34pm, Dou Liyang wrote:
> 
> 
> At 02/07/2018 08:27 PM, Baoquan He wrote:
> > On 02/07/18 at 08:17pm, Dou Liyang wrote:
> > > Hi Baoquan,
> > > 
> > > At 02/07/2018 08:08 PM, Baoquan He wrote:
> > > > On 02/07/18 at 08:00pm, Dou Liyang wrote:
> > > > > Hi Kirill,Mike
> > > > > 
> > > > > At 02/07/2018 06:45 PM, Mike Galbraith wrote:
> > > > > > On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
> > > > > > > On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
> > > > > > > > Hi All,
> > > > > > > > 
> > > > > > > > I met the makedumpfile failed in the upstream kernel which contained
> > > > > > > > this patch. Did I missed something else?
> > > > > > > 
> > > > > > > None I'm aware of.
> > > > > > > 
> > > > > > > Is there a reason to suspect that the issue is related to the bug this patch
> > > > > > > fixed?
> > > > > > 
> > > > > 
> > > > > I did a contrastive test by my colleagues Indoh's suggestion.

OK, I may get the reason. kaslr is enabled, right? You can try to
disable kaslr and try them again. Because phys_base and kaslr_offset are
got from vmlinux, while these are generated at compiling time. Just a
guess.

> > > > > 
> > > > > Revert your two commits:
> > > > > 
> > > > > commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
> > > > > Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > > > Date:   Fri Sep 29 17:08:16 2017 +0300
> > > > > 
> > > > > commit 629a359bdb0e0652a8227b4ff3125431995fec6e
> > > > > Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > > > Date:   Tue Nov 7 11:33:37 2017 +0300
> > > > > 
> > > > > ...and keep others unchanged, the makedumpfile works well.
> > > > > 
> > > > > > Still works fine for me with .today.  Box is only 16GB desktop box though.
> > > > > > 
> > > > > Btw, In the upstream kernel which contained this patch, I did two tests:
> > > > > 
> > > > >    1) use the makedumpfile as core_collector in /etc/kdump.conf, then
> > > > > trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
> > > > > makedumpfile works well and I can get the vmcore file.
> > > > > 
> > > > >        ......It is OK
> > > > > 
> > > > >    2) use cp as core_collector, do the same operation to get the vmcore file.
> > > > > then use makedumpfile to do like above:
> > > > > 
> > > > >       [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
> > > > > vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+
> > > > 
> > > > Oh, then please ignore my previous comment. Adding '-D' can give more
> > > > debugging message.
> > > 
> > > I added '-D', Just like before, no more debugging message:
> > > 
> > > BTW, I use crash to analyze the vmcore file created by 'cp' command.
> > > 
> > >     ./crash ../makedumpfile/code/vmcore_4.15+_from_cp_command
> > > ../makedumpfile/code/vmlinux_4.15+
> > > 
> > > the crash works well, It's so interesting.
> > > 
> > > Thanks,
> > > 	dou.
> > > 
> > > The debugging message with '-D':
> > 
> > And what's the debugging printing when trigger crash by sysrq?
> > 
> 
> kdump: dump target is /dev/vda2
> kdump: saving to /sysroot//var/crash/127.0.0.1-2018-02-07-07:31:56/
> [    2.751352] EXT4-fs (vda2): re-mounted. Opts: data=ordered
> kdump: saving vmcore-dmesg.txt
> kdump: saving vmcore-dmesg.txt complete
> kdump: saving vmcore
> sadump: does not have partition header
> sadump: read dump device as unknown format
> sadump: unknown format
> LOAD (0)
>   phys_start : 1000000
>   phys_end   : 2a86000
>   virt_start : ffffffff81000000
>   virt_end   : ffffffff82a86000
> LOAD (1)
>   phys_start : 1000
>   phys_end   : 9fc00
>   virt_start : ffff880000001000
>   virt_end   : ffff88000009fc00
> LOAD (2)
>   phys_start : 100000
>   phys_end   : 13000000
>   virt_start : ffff880000100000
>   virt_end   : ffff880013000000
> LOAD (3)
>   phys_start : 33000000
>   phys_end   : 7ffd7000
>   virt_start : ffff880033000000
>   virt_end   : ffff88007ffd7000
> Linux kdump
> page_size    : 4096
> 
> max_mapnr    : 7ffd7
> 
> Buffer size for the cyclic mode: 131061
> 
> num of NODEs : 1
> 
> 
> Memory type  : SPARSEMEM_EX
> 
> mem_map (0)
>   mem_map    : ffffea0000000000
>   pfn_start  : 0
>   pfn_end    : 8000
> mem_map (1)
>   mem_map    : ffffea0000200000
>   pfn_start  : 8000
>   pfn_end    : 10000
> mem_map (2)
>   mem_map    : ffffea0000400000
>   pfn_start  : 10000
>   pfn_end    : 18000
> mem_map (3)
>   mem_map    : ffffea0000600000
>   pfn_start  : 18000
>   pfn_end    : 20000
> mem_map (4)
>   mem_map    : ffffea0000800000
>   pfn_start  : 20000
>   pfn_end    : 28000
> mem_map (5)
>   mem_map    : ffffea0000a00000
>   pfn_start  : 28000
>   pfn_end    : 30000
> mem_map (6)
>   mem_map    : ffffea0000c00000
>   pfn_start  : 30000
>   pfn_end    : 38000
> mem_map (7)
>   mem_map    : ffffea0000e00000
>   pfn_start  : 38000
>   pfn_end    : 40000
> mem_map (8)
>   mem_map    : ffffea0001000000
>   pfn_start  : 40000
>   pfn_end    : 48000
> mem_map (9)
>   mem_map    : ffffea0001200000
>   pfn_start  : 48000
>   pfn_end    : 50000
> mem_map (10)
>   mem_map    : ffffea0001400000
>   pfn_start  : 50000
>   pfn_end    : 58000
> mem_map (11)
>   mem_map    : ffffea0001600000
>   pfn_start  : 58000
>   pfn_end    : 60000
> mem_map (12)
>   mem_map    : ffffea0001800000
>   pfn_start  : 60000
>   pfn_end    : 68000
> mem_map (13)
>   mem_map    : ffffea0001a00000
>   pfn_start  : 68000
>   pfn_end    : 70000
> mem_map (14)
>   mem_map    : ffffea0001c00000
>   pfn_start  : 70000
>   pfn_end    : 78000
> mem_map (15)
>   mem_map    : ffffea0001e00000
>   pfn_start  : 78000
>   pfn_end    : 7ffd7
> mmap() is available on the kernel.
> Copying data                                      : [100.0 %] -  eta: 0s
> Writing erase info...
> offset_eraseinfo: 9567fb0, size_eraseinfo: 0
> kdump: saving vmcore complete
> 
> Thanks,
> 	dou
> 
> > > 
> > > [douly@localhost code]$ ./makedumpfile -D -d 31 --message-level 31 -x
> > > vmlinux_4.15+  vmcore_4.15+_from_cp_command vmcore_4.15+
> > > sadump: does not have partition header
> > > sadump: read dump device as unknown format
> > > sadump: unknown format
> > > LOAD (0)
> > >    phys_start : 1000000
> > >    phys_end   : 2a86000
> > >    virt_start : ffffffff81000000
> > >    virt_end   : ffffffff82a86000
> > > LOAD (1)
> > >    phys_start : 1000
> > >    phys_end   : 9fc00
> > >    virt_start : ffff880000001000
> > >    virt_end   : ffff88000009fc00
> > > LOAD (2)
> > >    phys_start : 100000
> > >    phys_end   : 13000000
> > >    virt_start : ffff880000100000
> > >    virt_end   : ffff880013000000
> > > LOAD (3)
> > >    phys_start : 33000000
> > >    phys_end   : 7ffd7000
> > >    virt_start : ffff880033000000
> > >    virt_end   : ffff88007ffd7000
> > > Linux kdump
> > > page_size    : 4096
> > > 
> > > max_mapnr    : 7ffd7
> > > 
> > > Buffer size for the cyclic mode: 131061
> > > The kernel version is not supported.
> > > The makedumpfile operation may be incomplete.
> > > 
> > > num of NODEs : 1
> > > 
> > > 
> > > Memory type  : SPARSEMEM_EX
> > > 
> > > mem_map (0)
> > >    mem_map    : ffff88007ff26000
> > >    pfn_start  : 0
> > >    pfn_end    : 8000
> > > mem_map (1)
> > >    mem_map    : 0
> > >    pfn_start  : 8000
> > >    pfn_end    : 10000
> > > mem_map (2)
> > >    mem_map    : 0
> > >    pfn_start  : 10000
> > >    pfn_end    : 18000
> > > mem_map (3)
> > >    mem_map    : 0
> > >    pfn_start  : 18000
> > >    pfn_end    : 20000
> > > mem_map (4)
> > >    mem_map    : 0
> > >    pfn_start  : 20000
> > >    pfn_end    : 28000
> > > mem_map (5)
> > >    mem_map    : 0
> > >    pfn_start  : 28000
> > >    pfn_end    : 30000
> > > mem_map (6)
> > >    mem_map    : 0
> > >    pfn_start  : 30000
> > >    pfn_end    : 38000
> > > mem_map (7)
> > >    mem_map    : 0
> > >    pfn_start  : 38000
> > >    pfn_end    : 40000
> > > mem_map (8)
> > >    mem_map    : 0
> > >    pfn_start  : 40000
> > >    pfn_end    : 48000
> > > mem_map (9)
> > >    mem_map    : 0
> > >    pfn_start  : 48000
> > >    pfn_end    : 50000
> > > mem_map (10)
> > >    mem_map    : 0
> > >    pfn_start  : 50000
> > >    pfn_end    : 58000
> > > mem_map (11)
> > >    mem_map    : 0
> > >    pfn_start  : 58000
> > >    pfn_end    : 60000
> > > mem_map (12)
> > >    mem_map    : 0
> > >    pfn_start  : 60000
> > >    pfn_end    : 68000
> > > mem_map (13)
> > >    mem_map    : 0
> > >    pfn_start  : 68000
> > >    pfn_end    : 70000
> > > mem_map (14)
> > >    mem_map    : 0
> > >    pfn_start  : 70000
> > >    pfn_end    : 78000
> > > mem_map (15)
> > >    mem_map    : 0
> > >    pfn_start  : 78000
> > >    pfn_end    : 7ffd7
> > > mmap() is available on the kernel.
> > > Checking for memory holes                         : [100.0 %] |         STEP
> > > [Checking for memory holes  ] : 0.000014 seconds
> > > __vtop4_x86_64: Can't get a valid pte.
> > > readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
> > > address.
> > > readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
> > > __exclude_unnecessary_pages: Can't read the buffer of struct page.
> > > create_2nd_bitmap: Can't exclude unnecessary pages.
> > > Checking for memory holes                         : [100.0 %] \         STEP
> > > [Checking for memory holes  ] : 0.000006 seconds
> > > Checking for memory holes                         : [100.0 %] -         STEP
> > > [Checking for memory holes  ] : 0.000004 seconds
> > > __vtop4_x86_64: Can't get a valid pte.
> > > readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
> > > address.
> > > readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
> > > __exclude_unnecessary_pages: Can't read the buffer of struct page.
> > > create_2nd_bitmap: Can't exclude unnecessary pages.
> > > 
> > > makedumpfile Failed.
> > > 
> > > > 
> > > > > 
> > > > >       ......It causes makedumpfile failed.
> > > > > 
> > > > > 
> > > > > Thanks,
> > > > > 	dou.
> > > > > 
> > > > > > 	-Mike
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > 
> > 
> > 
> > 
> 
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dou Liyang]

Hi Baoquan,

At 02/07/2018 08:45 PM, Baoquan He wrote:
> On 02/07/18 at 08:34pm, Dou Liyang wrote:
>>
>>
>> At 02/07/2018 08:27 PM, Baoquan He wrote:
>>> On 02/07/18 at 08:17pm, Dou Liyang wrote:
>>>> Hi Baoquan,
>>>>
>>>> At 02/07/2018 08:08 PM, Baoquan He wrote:
>>>>> On 02/07/18 at 08:00pm, Dou Liyang wrote:
>>>>>> Hi Kirill,Mike
>>>>>>
>>>>>> At 02/07/2018 06:45 PM, Mike Galbraith wrote:
>>>>>>> On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
>>>>>>>> On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I met the makedumpfile failed in the upstream kernel which contained
>>>>>>>>> this patch. Did I missed something else?
>>>>>>>>
>>>>>>>> None I'm aware of.
>>>>>>>>
>>>>>>>> Is there a reason to suspect that the issue is related to the bug this patch
>>>>>>>> fixed?
>>>>>>>
>>>>>>
>>>>>> I did a contrastive test by my colleagues Indoh's suggestion.
> 
> OK, I may get the reason. kaslr is enabled, right? You can try to

I add 'nokaslr' to disable the KASLR feature.

# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.15.0+ 
root=UUID=10f10326-c923-4098-86aa-afed5c54ee0b ro crashkernel=512M rhgb 
console=tty0 console=ttyS0 nokaslr LANG=en_US.UTF-8

> disable kaslr and try them again. Because phys_base and kaslr_offset are
> got from vmlinux, while these are generated at compiling time. Just a
> guess.
> 

Oh, I will recompile the kernel with KASLR disabled in .config.


Thanks,
	dou.
>>>>>>
>>>>>> Revert your two commits:
>>>>>>
>>>>>> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
>>>>>> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>>>>>> Date:   Fri Sep 29 17:08:16 2017 +0300
>>>>>>
>>>>>> commit 629a359bdb0e0652a8227b4ff3125431995fec6e
>>>>>> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>>>>>> Date:   Tue Nov 7 11:33:37 2017 +0300
>>>>>>
>>>>>> ...and keep others unchanged, the makedumpfile works well.
>>>>>>
>>>>>>> Still works fine for me with .today.  Box is only 16GB desktop box though.
>>>>>>>
>>>>>> Btw, In the upstream kernel which contained this patch, I did two tests:
>>>>>>
>>>>>>     1) use the makedumpfile as core_collector in /etc/kdump.conf, then
>>>>>> trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
>>>>>> makedumpfile works well and I can get the vmcore file.
>>>>>>
>>>>>>         ......It is OK
>>>>>>
>>>>>>     2) use cp as core_collector, do the same operation to get the vmcore file.
>>>>>> then use makedumpfile to do like above:
>>>>>>
>>>>>>        [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
>>>>>> vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+
>>>>>
>>>>> Oh, then please ignore my previous comment. Adding '-D' can give more
>>>>> debugging message.
>>>>
>>>> I added '-D', Just like before, no more debugging message:
>>>>
>>>> BTW, I use crash to analyze the vmcore file created by 'cp' command.
>>>>
>>>>      ./crash ../makedumpfile/code/vmcore_4.15+_from_cp_command
>>>> ../makedumpfile/code/vmlinux_4.15+
>>>>
>>>> the crash works well, It's so interesting.
>>>>
>>>> Thanks,
>>>> 	dou.
>>>>
>>>> The debugging message with '-D':
>>>
>>> And what's the debugging printing when trigger crash by sysrq?
>>>
>>
>> kdump: dump target is /dev/vda2
>> kdump: saving to /sysroot//var/crash/127.0.0.1-2018-02-07-07:31:56/
>> [    2.751352] EXT4-fs (vda2): re-mounted. Opts: data=ordered
>> kdump: saving vmcore-dmesg.txt
>> kdump: saving vmcore-dmesg.txt complete
>> kdump: saving vmcore
>> sadump: does not have partition header
>> sadump: read dump device as unknown format
>> sadump: unknown format
>> LOAD (0)
>>    phys_start : 1000000
>>    phys_end   : 2a86000
>>    virt_start : ffffffff81000000
>>    virt_end   : ffffffff82a86000
>> LOAD (1)
>>    phys_start : 1000
>>    phys_end   : 9fc00
>>    virt_start : ffff880000001000
>>    virt_end   : ffff88000009fc00
>> LOAD (2)
>>    phys_start : 100000
>>    phys_end   : 13000000
>>    virt_start : ffff880000100000
>>    virt_end   : ffff880013000000
>> LOAD (3)
>>    phys_start : 33000000
>>    phys_end   : 7ffd7000
>>    virt_start : ffff880033000000
>>    virt_end   : ffff88007ffd7000
>> Linux kdump
>> page_size    : 4096
>>
>> max_mapnr    : 7ffd7
>>
>> Buffer size for the cyclic mode: 131061
>>
>> num of NODEs : 1
>>
>>
>> Memory type  : SPARSEMEM_EX
>>
>> mem_map (0)
>>    mem_map    : ffffea0000000000
>>    pfn_start  : 0
>>    pfn_end    : 8000
>> mem_map (1)
>>    mem_map    : ffffea0000200000
>>    pfn_start  : 8000
>>    pfn_end    : 10000
>> mem_map (2)
>>    mem_map    : ffffea0000400000
>>    pfn_start  : 10000
>>    pfn_end    : 18000
>> mem_map (3)
>>    mem_map    : ffffea0000600000
>>    pfn_start  : 18000
>>    pfn_end    : 20000
>> mem_map (4)
>>    mem_map    : ffffea0000800000
>>    pfn_start  : 20000
>>    pfn_end    : 28000
>> mem_map (5)
>>    mem_map    : ffffea0000a00000
>>    pfn_start  : 28000
>>    pfn_end    : 30000
>> mem_map (6)
>>    mem_map    : ffffea0000c00000
>>    pfn_start  : 30000
>>    pfn_end    : 38000
>> mem_map (7)
>>    mem_map    : ffffea0000e00000
>>    pfn_start  : 38000
>>    pfn_end    : 40000
>> mem_map (8)
>>    mem_map    : ffffea0001000000
>>    pfn_start  : 40000
>>    pfn_end    : 48000
>> mem_map (9)
>>    mem_map    : ffffea0001200000
>>    pfn_start  : 48000
>>    pfn_end    : 50000
>> mem_map (10)
>>    mem_map    : ffffea0001400000
>>    pfn_start  : 50000
>>    pfn_end    : 58000
>> mem_map (11)
>>    mem_map    : ffffea0001600000
>>    pfn_start  : 58000
>>    pfn_end    : 60000
>> mem_map (12)
>>    mem_map    : ffffea0001800000
>>    pfn_start  : 60000
>>    pfn_end    : 68000
>> mem_map (13)
>>    mem_map    : ffffea0001a00000
>>    pfn_start  : 68000
>>    pfn_end    : 70000
>> mem_map (14)
>>    mem_map    : ffffea0001c00000
>>    pfn_start  : 70000
>>    pfn_end    : 78000
>> mem_map (15)
>>    mem_map    : ffffea0001e00000
>>    pfn_start  : 78000
>>    pfn_end    : 7ffd7
>> mmap() is available on the kernel.
>> Copying data                                      : [100.0 %] -  eta: 0s
>> Writing erase info...
>> offset_eraseinfo: 9567fb0, size_eraseinfo: 0
>> kdump: saving vmcore complete
>>
>> Thanks,
>> 	dou
>>
>>>>
>>>> [douly@localhost code]$ ./makedumpfile -D -d 31 --message-level 31 -x
>>>> vmlinux_4.15+  vmcore_4.15+_from_cp_command vmcore_4.15+
>>>> sadump: does not have partition header
>>>> sadump: read dump device as unknown format
>>>> sadump: unknown format
>>>> LOAD (0)
>>>>     phys_start : 1000000
>>>>     phys_end   : 2a86000
>>>>     virt_start : ffffffff81000000
>>>>     virt_end   : ffffffff82a86000
>>>> LOAD (1)
>>>>     phys_start : 1000
>>>>     phys_end   : 9fc00
>>>>     virt_start : ffff880000001000
>>>>     virt_end   : ffff88000009fc00
>>>> LOAD (2)
>>>>     phys_start : 100000
>>>>     phys_end   : 13000000
>>>>     virt_start : ffff880000100000
>>>>     virt_end   : ffff880013000000
>>>> LOAD (3)
>>>>     phys_start : 33000000
>>>>     phys_end   : 7ffd7000
>>>>     virt_start : ffff880033000000
>>>>     virt_end   : ffff88007ffd7000
>>>> Linux kdump
>>>> page_size    : 4096
>>>>
>>>> max_mapnr    : 7ffd7
>>>>
>>>> Buffer size for the cyclic mode: 131061
>>>> The kernel version is not supported.
>>>> The makedumpfile operation may be incomplete.
>>>>
>>>> num of NODEs : 1
>>>>
>>>>
>>>> Memory type  : SPARSEMEM_EX
>>>>
>>>> mem_map (0)
>>>>     mem_map    : ffff88007ff26000
>>>>     pfn_start  : 0
>>>>     pfn_end    : 8000
>>>> mem_map (1)
>>>>     mem_map    : 0
>>>>     pfn_start  : 8000
>>>>     pfn_end    : 10000
>>>> mem_map (2)
>>>>     mem_map    : 0
>>>>     pfn_start  : 10000
>>>>     pfn_end    : 18000
>>>> mem_map (3)
>>>>     mem_map    : 0
>>>>     pfn_start  : 18000
>>>>     pfn_end    : 20000
>>>> mem_map (4)
>>>>     mem_map    : 0
>>>>     pfn_start  : 20000
>>>>     pfn_end    : 28000
>>>> mem_map (5)
>>>>     mem_map    : 0
>>>>     pfn_start  : 28000
>>>>     pfn_end    : 30000
>>>> mem_map (6)
>>>>     mem_map    : 0
>>>>     pfn_start  : 30000
>>>>     pfn_end    : 38000
>>>> mem_map (7)
>>>>     mem_map    : 0
>>>>     pfn_start  : 38000
>>>>     pfn_end    : 40000
>>>> mem_map (8)
>>>>     mem_map    : 0
>>>>     pfn_start  : 40000
>>>>     pfn_end    : 48000
>>>> mem_map (9)
>>>>     mem_map    : 0
>>>>     pfn_start  : 48000
>>>>     pfn_end    : 50000
>>>> mem_map (10)
>>>>     mem_map    : 0
>>>>     pfn_start  : 50000
>>>>     pfn_end    : 58000
>>>> mem_map (11)
>>>>     mem_map    : 0
>>>>     pfn_start  : 58000
>>>>     pfn_end    : 60000
>>>> mem_map (12)
>>>>     mem_map    : 0
>>>>     pfn_start  : 60000
>>>>     pfn_end    : 68000
>>>> mem_map (13)
>>>>     mem_map    : 0
>>>>     pfn_start  : 68000
>>>>     pfn_end    : 70000
>>>> mem_map (14)
>>>>     mem_map    : 0
>>>>     pfn_start  : 70000
>>>>     pfn_end    : 78000
>>>> mem_map (15)
>>>>     mem_map    : 0
>>>>     pfn_start  : 78000
>>>>     pfn_end    : 7ffd7
>>>> mmap() is available on the kernel.
>>>> Checking for memory holes                         : [100.0 %] |         STEP
>>>> [Checking for memory holes  ] : 0.000014 seconds
>>>> __vtop4_x86_64: Can't get a valid pte.
>>>> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
>>>> address.
>>>> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
>>>> __exclude_unnecessary_pages: Can't read the buffer of struct page.
>>>> create_2nd_bitmap: Can't exclude unnecessary pages.
>>>> Checking for memory holes                         : [100.0 %] \         STEP
>>>> [Checking for memory holes  ] : 0.000006 seconds
>>>> Checking for memory holes                         : [100.0 %] -         STEP
>>>> [Checking for memory holes  ] : 0.000004 seconds
>>>> __vtop4_x86_64: Can't get a valid pte.
>>>> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
>>>> address.
>>>> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
>>>> __exclude_unnecessary_pages: Can't read the buffer of struct page.
>>>> create_2nd_bitmap: Can't exclude unnecessary pages.
>>>>
>>>> makedumpfile Failed.
>>>>
>>>>>
>>>>>>
>>>>>>        ......It causes makedumpfile failed.
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> 	dou.
>>>>>>
>>>>>>> 	-Mike
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
> 
> 
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

On 02/08/18 at 09:14am, Dou Liyang wrote:
> Hi Baoquan,
> 
> At 02/07/2018 08:45 PM, Baoquan He wrote:
> > On 02/07/18 at 08:34pm, Dou Liyang wrote:
> > > 
> > > 
> > > At 02/07/2018 08:27 PM, Baoquan He wrote:
> > > > On 02/07/18 at 08:17pm, Dou Liyang wrote:
> > > > > Hi Baoquan,
> > > > > 
> > > > > At 02/07/2018 08:08 PM, Baoquan He wrote:
> > > > > > On 02/07/18 at 08:00pm, Dou Liyang wrote:
> > > > > > > Hi Kirill,Mike
> > > > > > > 
> > > > > > > At 02/07/2018 06:45 PM, Mike Galbraith wrote:
> > > > > > > > On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
> > > > > > > > > On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
> > > > > > > > > > Hi All,
> > > > > > > > > > 
> > > > > > > > > > I met the makedumpfile failed in the upstream kernel which contained
> > > > > > > > > > this patch. Did I missed something else?
> > > > > > > > > 
> > > > > > > > > None I'm aware of.
> > > > > > > > > 
> > > > > > > > > Is there a reason to suspect that the issue is related to the bug this patch
> > > > > > > > > fixed?
> > > > > > > > 
> > > > > > > 
> > > > > > > I did a contrastive test by my colleagues Indoh's suggestion.
> > 
> > OK, I may get the reason. kaslr is enabled, right? You can try to
> 
> I add 'nokaslr' to disable the KASLR feature.
    ~~~added??
> 
> # cat /proc/cmdline
> BOOT_IMAGE=/vmlinuz-4.15.0+ root=UUID=10f10326-c923-4098-86aa-afed5c54ee0b
> ro crashkernel=512M rhgb console=tty0 console=ttyS0 nokaslr LANG=en_US.UTF-8
> 
> > disable kaslr and try them again. Because phys_base and kaslr_offset are
> > got from vmlinux, while these are generated at compiling time. Just a
> > guess.
> > 
> 
> Oh, I will recompile the kernel with KASLR disabled in .config.

Then it's not what I guessed. Need debug makedumpfile since using
vmlinux is another code path, few people use it usually.

> 
> 
> Thanks,
> 	dou.
> > > > > > > 
> > > > > > > Revert your two commits:
> > > > > > > 
> > > > > > > commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
> > > > > > > Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > > > > > Date:   Fri Sep 29 17:08:16 2017 +0300
> > > > > > > 
> > > > > > > commit 629a359bdb0e0652a8227b4ff3125431995fec6e
> > > > > > > Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > > > > > Date:   Tue Nov 7 11:33:37 2017 +0300
> > > > > > > 
> > > > > > > ...and keep others unchanged, the makedumpfile works well.
> > > > > > > 
> > > > > > > > Still works fine for me with .today.  Box is only 16GB desktop box though.
> > > > > > > > 
> > > > > > > Btw, In the upstream kernel which contained this patch, I did two tests:
> > > > > > > 
> > > > > > >     1) use the makedumpfile as core_collector in /etc/kdump.conf, then
> > > > > > > trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
> > > > > > > makedumpfile works well and I can get the vmcore file.
> > > > > > > 
> > > > > > >         ......It is OK
> > > > > > > 
> > > > > > >     2) use cp as core_collector, do the same operation to get the vmcore file.
> > > > > > > then use makedumpfile to do like above:
> > > > > > > 
> > > > > > >        [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
> > > > > > > vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+
> > > > > > 
> > > > > > Oh, then please ignore my previous comment. Adding '-D' can give more
> > > > > > debugging message.
> > > > > 
> > > > > I added '-D', Just like before, no more debugging message:
> > > > > 
> > > > > BTW, I use crash to analyze the vmcore file created by 'cp' command.
> > > > > 
> > > > >      ./crash ../makedumpfile/code/vmcore_4.15+_from_cp_command
> > > > > ../makedumpfile/code/vmlinux_4.15+
> > > > > 
> > > > > the crash works well, It's so interesting.
> > > > > 
> > > > > Thanks,
> > > > > 	dou.
> > > > > 
> > > > > The debugging message with '-D':
> > > > 
> > > > And what's the debugging printing when trigger crash by sysrq?
> > > > 
> > > 
> > > kdump: dump target is /dev/vda2
> > > kdump: saving to /sysroot//var/crash/127.0.0.1-2018-02-07-07:31:56/
> > > [    2.751352] EXT4-fs (vda2): re-mounted. Opts: data=ordered
> > > kdump: saving vmcore-dmesg.txt
> > > kdump: saving vmcore-dmesg.txt complete
> > > kdump: saving vmcore
> > > sadump: does not have partition header
> > > sadump: read dump device as unknown format
> > > sadump: unknown format
> > > LOAD (0)
> > >    phys_start : 1000000
> > >    phys_end   : 2a86000
> > >    virt_start : ffffffff81000000
> > >    virt_end   : ffffffff82a86000
> > > LOAD (1)
> > >    phys_start : 1000
> > >    phys_end   : 9fc00
> > >    virt_start : ffff880000001000
> > >    virt_end   : ffff88000009fc00
> > > LOAD (2)
> > >    phys_start : 100000
> > >    phys_end   : 13000000
> > >    virt_start : ffff880000100000
> > >    virt_end   : ffff880013000000
> > > LOAD (3)
> > >    phys_start : 33000000
> > >    phys_end   : 7ffd7000
> > >    virt_start : ffff880033000000
> > >    virt_end   : ffff88007ffd7000
> > > Linux kdump
> > > page_size    : 4096
> > > 
> > > max_mapnr    : 7ffd7
> > > 
> > > Buffer size for the cyclic mode: 131061
> > > 
> > > num of NODEs : 1
> > > 
> > > 
> > > Memory type  : SPARSEMEM_EX
> > > 
> > > mem_map (0)
> > >    mem_map    : ffffea0000000000
> > >    pfn_start  : 0
> > >    pfn_end    : 8000
> > > mem_map (1)
> > >    mem_map    : ffffea0000200000
> > >    pfn_start  : 8000
> > >    pfn_end    : 10000
> > > mem_map (2)
> > >    mem_map    : ffffea0000400000
> > >    pfn_start  : 10000
> > >    pfn_end    : 18000
> > > mem_map (3)
> > >    mem_map    : ffffea0000600000
> > >    pfn_start  : 18000
> > >    pfn_end    : 20000
> > > mem_map (4)
> > >    mem_map    : ffffea0000800000
> > >    pfn_start  : 20000
> > >    pfn_end    : 28000
> > > mem_map (5)
> > >    mem_map    : ffffea0000a00000
> > >    pfn_start  : 28000
> > >    pfn_end    : 30000
> > > mem_map (6)
> > >    mem_map    : ffffea0000c00000
> > >    pfn_start  : 30000
> > >    pfn_end    : 38000
> > > mem_map (7)
> > >    mem_map    : ffffea0000e00000
> > >    pfn_start  : 38000
> > >    pfn_end    : 40000
> > > mem_map (8)
> > >    mem_map    : ffffea0001000000
> > >    pfn_start  : 40000
> > >    pfn_end    : 48000
> > > mem_map (9)
> > >    mem_map    : ffffea0001200000
> > >    pfn_start  : 48000
> > >    pfn_end    : 50000
> > > mem_map (10)
> > >    mem_map    : ffffea0001400000
> > >    pfn_start  : 50000
> > >    pfn_end    : 58000
> > > mem_map (11)
> > >    mem_map    : ffffea0001600000
> > >    pfn_start  : 58000
> > >    pfn_end    : 60000
> > > mem_map (12)
> > >    mem_map    : ffffea0001800000
> > >    pfn_start  : 60000
> > >    pfn_end    : 68000
> > > mem_map (13)
> > >    mem_map    : ffffea0001a00000
> > >    pfn_start  : 68000
> > >    pfn_end    : 70000
> > > mem_map (14)
> > >    mem_map    : ffffea0001c00000
> > >    pfn_start  : 70000
> > >    pfn_end    : 78000
> > > mem_map (15)
> > >    mem_map    : ffffea0001e00000
> > >    pfn_start  : 78000
> > >    pfn_end    : 7ffd7
> > > mmap() is available on the kernel.
> > > Copying data                                      : [100.0 %] -  eta: 0s
> > > Writing erase info...
> > > offset_eraseinfo: 9567fb0, size_eraseinfo: 0
> > > kdump: saving vmcore complete
> > > 
> > > Thanks,
> > > 	dou
> > > 
> > > > > 
> > > > > [douly@localhost code]$ ./makedumpfile -D -d 31 --message-level 31 -x
> > > > > vmlinux_4.15+  vmcore_4.15+_from_cp_command vmcore_4.15+
> > > > > sadump: does not have partition header
> > > > > sadump: read dump device as unknown format
> > > > > sadump: unknown format
> > > > > LOAD (0)
> > > > >     phys_start : 1000000
> > > > >     phys_end   : 2a86000
> > > > >     virt_start : ffffffff81000000
> > > > >     virt_end   : ffffffff82a86000
> > > > > LOAD (1)
> > > > >     phys_start : 1000
> > > > >     phys_end   : 9fc00
> > > > >     virt_start : ffff880000001000
> > > > >     virt_end   : ffff88000009fc00
> > > > > LOAD (2)
> > > > >     phys_start : 100000
> > > > >     phys_end   : 13000000
> > > > >     virt_start : ffff880000100000
> > > > >     virt_end   : ffff880013000000
> > > > > LOAD (3)
> > > > >     phys_start : 33000000
> > > > >     phys_end   : 7ffd7000
> > > > >     virt_start : ffff880033000000
> > > > >     virt_end   : ffff88007ffd7000
> > > > > Linux kdump
> > > > > page_size    : 4096
> > > > > 
> > > > > max_mapnr    : 7ffd7
> > > > > 
> > > > > Buffer size for the cyclic mode: 131061
> > > > > The kernel version is not supported.
> > > > > The makedumpfile operation may be incomplete.
> > > > > 
> > > > > num of NODEs : 1
> > > > > 
> > > > > 
> > > > > Memory type  : SPARSEMEM_EX
> > > > > 
> > > > > mem_map (0)
> > > > >     mem_map    : ffff88007ff26000
> > > > >     pfn_start  : 0
> > > > >     pfn_end    : 8000
> > > > > mem_map (1)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 8000
> > > > >     pfn_end    : 10000
> > > > > mem_map (2)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 10000
> > > > >     pfn_end    : 18000
> > > > > mem_map (3)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 18000
> > > > >     pfn_end    : 20000
> > > > > mem_map (4)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 20000
> > > > >     pfn_end    : 28000
> > > > > mem_map (5)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 28000
> > > > >     pfn_end    : 30000
> > > > > mem_map (6)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 30000
> > > > >     pfn_end    : 38000
> > > > > mem_map (7)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 38000
> > > > >     pfn_end    : 40000
> > > > > mem_map (8)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 40000
> > > > >     pfn_end    : 48000
> > > > > mem_map (9)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 48000
> > > > >     pfn_end    : 50000
> > > > > mem_map (10)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 50000
> > > > >     pfn_end    : 58000
> > > > > mem_map (11)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 58000
> > > > >     pfn_end    : 60000
> > > > > mem_map (12)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 60000
> > > > >     pfn_end    : 68000
> > > > > mem_map (13)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 68000
> > > > >     pfn_end    : 70000
> > > > > mem_map (14)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 70000
> > > > >     pfn_end    : 78000
> > > > > mem_map (15)
> > > > >     mem_map    : 0
> > > > >     pfn_start  : 78000
> > > > >     pfn_end    : 7ffd7
> > > > > mmap() is available on the kernel.
> > > > > Checking for memory holes                         : [100.0 %] |         STEP
> > > > > [Checking for memory holes  ] : 0.000014 seconds
> > > > > __vtop4_x86_64: Can't get a valid pte.
> > > > > readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
> > > > > address.
> > > > > readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
> > > > > __exclude_unnecessary_pages: Can't read the buffer of struct page.
> > > > > create_2nd_bitmap: Can't exclude unnecessary pages.
> > > > > Checking for memory holes                         : [100.0 %] \         STEP
> > > > > [Checking for memory holes  ] : 0.000006 seconds
> > > > > Checking for memory holes                         : [100.0 %] -         STEP
> > > > > [Checking for memory holes  ] : 0.000004 seconds
> > > > > __vtop4_x86_64: Can't get a valid pte.
> > > > > readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
> > > > > address.
> > > > > readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
> > > > > __exclude_unnecessary_pages: Can't read the buffer of struct page.
> > > > > create_2nd_bitmap: Can't exclude unnecessary pages.
> > > > > 
> > > > > makedumpfile Failed.
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > >        ......It causes makedumpfile failed.
> > > > > > > 
> > > > > > > 
> > > > > > > Thanks,
> > > > > > > 	dou.
> > > > > > > 
> > > > > > > > 	-Mike
> > > > > > > > 
> > > > > > > > 
> > > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > 
> > 
> > 
> > 
> 
> 
> 
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Dou Liyang]

Hi Baoquan,

At 02/08/2018 09:23 AM, Baoquan He wrote:
> On 02/08/18 at 09:14am, Dou Liyang wrote:
>> Hi Baoquan,
>>
>> At 02/07/2018 08:45 PM, Baoquan He wrote:
>>> On 02/07/18 at 08:34pm, Dou Liyang wrote:
>>>>
>>>>
>>>> At 02/07/2018 08:27 PM, Baoquan He wrote:
>>>>> On 02/07/18 at 08:17pm, Dou Liyang wrote:
>>>>>> Hi Baoquan,
>>>>>>
>>>>>> At 02/07/2018 08:08 PM, Baoquan He wrote:
>>>>>>> On 02/07/18 at 08:00pm, Dou Liyang wrote:
>>>>>>>> Hi Kirill,Mike
>>>>>>>>
>>>>>>>> At 02/07/2018 06:45 PM, Mike Galbraith wrote:
>>>>>>>>> On Wed, 2018-02-07 at 13:41 +0300, Kirill A. Shutemov wrote:
>>>>>>>>>> On Wed, Feb 07, 2018 at 05:25:05PM +0800, Dou Liyang wrote:
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> I met the makedumpfile failed in the upstream kernel which contained
>>>>>>>>>>> this patch. Did I missed something else?
>>>>>>>>>>
>>>>>>>>>> None I'm aware of.
>>>>>>>>>>
>>>>>>>>>> Is there a reason to suspect that the issue is related to the bug this patch
>>>>>>>>>> fixed?
>>>>>>>>>
>>>>>>>>
>>>>>>>> I did a contrastive test by my colleagues Indoh's suggestion.
>>>
>>> OK, I may get the reason. kaslr is enabled, right? You can try to
>>
>> I add 'nokaslr' to disable the KASLR feature.
>      ~~~added??

oops! yes, the kaslr had already disabled by this option when I tested.

>>
>> # cat /proc/cmdline
>> BOOT_IMAGE=/vmlinuz-4.15.0+ root=UUID=10f10326-c923-4098-86aa-afed5c54ee0b
>> ro crashkernel=512M rhgb console=tty0 console=ttyS0 nokaslr LANG=en_US.UTF-8
>>
>>> disable kaslr and try them again. Because phys_base and kaslr_offset are
>>> got from vmlinux, while these are generated at compiling time. Just a
>>> guess.
>>>
>>
>> Oh, I will recompile the kernel with KASLR disabled in .config.
> 
> Then it's not what I guessed. Need debug makedumpfile since using
> vmlinux is another code path, few people use it usually.
> 

Understood, I will try to look into it.

Thanks,
	dou

>>
>>
>> Thanks,
>> 	dou.
>>>>>>>>
>>>>>>>> Revert your two commits:
>>>>>>>>
>>>>>>>> commit 83e3c48729d9ebb7af5a31a504f3fd6aff0348c4
>>>>>>>> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>>>>>>>> Date:   Fri Sep 29 17:08:16 2017 +0300
>>>>>>>>
>>>>>>>> commit 629a359bdb0e0652a8227b4ff3125431995fec6e
>>>>>>>> Author: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>>>>>>>> Date:   Tue Nov 7 11:33:37 2017 +0300
>>>>>>>>
>>>>>>>> ...and keep others unchanged, the makedumpfile works well.
>>>>>>>>
>>>>>>>>> Still works fine for me with .today.  Box is only 16GB desktop box though.
>>>>>>>>>
>>>>>>>> Btw, In the upstream kernel which contained this patch, I did two tests:
>>>>>>>>
>>>>>>>>      1) use the makedumpfile as core_collector in /etc/kdump.conf, then
>>>>>>>> trigger the process of kdump by echo 1 >/proc/sysrq-trigger, the
>>>>>>>> makedumpfile works well and I can get the vmcore file.
>>>>>>>>
>>>>>>>>          ......It is OK
>>>>>>>>
>>>>>>>>      2) use cp as core_collector, do the same operation to get the vmcore file.
>>>>>>>> then use makedumpfile to do like above:
>>>>>>>>
>>>>>>>>         [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
>>>>>>>> vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+
>>>>>>>
>>>>>>> Oh, then please ignore my previous comment. Adding '-D' can give more
>>>>>>> debugging message.
>>>>>>
>>>>>> I added '-D', Just like before, no more debugging message:
>>>>>>
>>>>>> BTW, I use crash to analyze the vmcore file created by 'cp' command.
>>>>>>
>>>>>>       ./crash ../makedumpfile/code/vmcore_4.15+_from_cp_command
>>>>>> ../makedumpfile/code/vmlinux_4.15+
>>>>>>
>>>>>> the crash works well, It's so interesting.
>>>>>>
>>>>>> Thanks,
>>>>>> 	dou.
>>>>>>
>>>>>> The debugging message with '-D':
>>>>>
>>>>> And what's the debugging printing when trigger crash by sysrq?
>>>>>
>>>>
>>>> kdump: dump target is /dev/vda2
>>>> kdump: saving to /sysroot//var/crash/127.0.0.1-2018-02-07-07:31:56/
>>>> [    2.751352] EXT4-fs (vda2): re-mounted. Opts: data=ordered
>>>> kdump: saving vmcore-dmesg.txt
>>>> kdump: saving vmcore-dmesg.txt complete
>>>> kdump: saving vmcore
>>>> sadump: does not have partition header
>>>> sadump: read dump device as unknown format
>>>> sadump: unknown format
>>>> LOAD (0)
>>>>     phys_start : 1000000
>>>>     phys_end   : 2a86000
>>>>     virt_start : ffffffff81000000
>>>>     virt_end   : ffffffff82a86000
>>>> LOAD (1)
>>>>     phys_start : 1000
>>>>     phys_end   : 9fc00
>>>>     virt_start : ffff880000001000
>>>>     virt_end   : ffff88000009fc00
>>>> LOAD (2)
>>>>     phys_start : 100000
>>>>     phys_end   : 13000000
>>>>     virt_start : ffff880000100000
>>>>     virt_end   : ffff880013000000
>>>> LOAD (3)
>>>>     phys_start : 33000000
>>>>     phys_end   : 7ffd7000
>>>>     virt_start : ffff880033000000
>>>>     virt_end   : ffff88007ffd7000
>>>> Linux kdump
>>>> page_size    : 4096
>>>>
>>>> max_mapnr    : 7ffd7
>>>>
>>>> Buffer size for the cyclic mode: 131061
>>>>
>>>> num of NODEs : 1
>>>>
>>>>
>>>> Memory type  : SPARSEMEM_EX
>>>>
>>>> mem_map (0)
>>>>     mem_map    : ffffea0000000000
>>>>     pfn_start  : 0
>>>>     pfn_end    : 8000
>>>> mem_map (1)
>>>>     mem_map    : ffffea0000200000
>>>>     pfn_start  : 8000
>>>>     pfn_end    : 10000
>>>> mem_map (2)
>>>>     mem_map    : ffffea0000400000
>>>>     pfn_start  : 10000
>>>>     pfn_end    : 18000
>>>> mem_map (3)
>>>>     mem_map    : ffffea0000600000
>>>>     pfn_start  : 18000
>>>>     pfn_end    : 20000
>>>> mem_map (4)
>>>>     mem_map    : ffffea0000800000
>>>>     pfn_start  : 20000
>>>>     pfn_end    : 28000
>>>> mem_map (5)
>>>>     mem_map    : ffffea0000a00000
>>>>     pfn_start  : 28000
>>>>     pfn_end    : 30000
>>>> mem_map (6)
>>>>     mem_map    : ffffea0000c00000
>>>>     pfn_start  : 30000
>>>>     pfn_end    : 38000
>>>> mem_map (7)
>>>>     mem_map    : ffffea0000e00000
>>>>     pfn_start  : 38000
>>>>     pfn_end    : 40000
>>>> mem_map (8)
>>>>     mem_map    : ffffea0001000000
>>>>     pfn_start  : 40000
>>>>     pfn_end    : 48000
>>>> mem_map (9)
>>>>     mem_map    : ffffea0001200000
>>>>     pfn_start  : 48000
>>>>     pfn_end    : 50000
>>>> mem_map (10)
>>>>     mem_map    : ffffea0001400000
>>>>     pfn_start  : 50000
>>>>     pfn_end    : 58000
>>>> mem_map (11)
>>>>     mem_map    : ffffea0001600000
>>>>     pfn_start  : 58000
>>>>     pfn_end    : 60000
>>>> mem_map (12)
>>>>     mem_map    : ffffea0001800000
>>>>     pfn_start  : 60000
>>>>     pfn_end    : 68000
>>>> mem_map (13)
>>>>     mem_map    : ffffea0001a00000
>>>>     pfn_start  : 68000
>>>>     pfn_end    : 70000
>>>> mem_map (14)
>>>>     mem_map    : ffffea0001c00000
>>>>     pfn_start  : 70000
>>>>     pfn_end    : 78000
>>>> mem_map (15)
>>>>     mem_map    : ffffea0001e00000
>>>>     pfn_start  : 78000
>>>>     pfn_end    : 7ffd7
>>>> mmap() is available on the kernel.
>>>> Copying data                                      : [100.0 %] -  eta: 0s
>>>> Writing erase info...
>>>> offset_eraseinfo: 9567fb0, size_eraseinfo: 0
>>>> kdump: saving vmcore complete
>>>>
>>>> Thanks,
>>>> 	dou
>>>>
>>>>>>
>>>>>> [douly@localhost code]$ ./makedumpfile -D -d 31 --message-level 31 -x
>>>>>> vmlinux_4.15+  vmcore_4.15+_from_cp_command vmcore_4.15+
>>>>>> sadump: does not have partition header
>>>>>> sadump: read dump device as unknown format
>>>>>> sadump: unknown format
>>>>>> LOAD (0)
>>>>>>      phys_start : 1000000
>>>>>>      phys_end   : 2a86000
>>>>>>      virt_start : ffffffff81000000
>>>>>>      virt_end   : ffffffff82a86000
>>>>>> LOAD (1)
>>>>>>      phys_start : 1000
>>>>>>      phys_end   : 9fc00
>>>>>>      virt_start : ffff880000001000
>>>>>>      virt_end   : ffff88000009fc00
>>>>>> LOAD (2)
>>>>>>      phys_start : 100000
>>>>>>      phys_end   : 13000000
>>>>>>      virt_start : ffff880000100000
>>>>>>      virt_end   : ffff880013000000
>>>>>> LOAD (3)
>>>>>>      phys_start : 33000000
>>>>>>      phys_end   : 7ffd7000
>>>>>>      virt_start : ffff880033000000
>>>>>>      virt_end   : ffff88007ffd7000
>>>>>> Linux kdump
>>>>>> page_size    : 4096
>>>>>>
>>>>>> max_mapnr    : 7ffd7
>>>>>>
>>>>>> Buffer size for the cyclic mode: 131061
>>>>>> The kernel version is not supported.
>>>>>> The makedumpfile operation may be incomplete.
>>>>>>
>>>>>> num of NODEs : 1
>>>>>>
>>>>>>
>>>>>> Memory type  : SPARSEMEM_EX
>>>>>>
>>>>>> mem_map (0)
>>>>>>      mem_map    : ffff88007ff26000
>>>>>>      pfn_start  : 0
>>>>>>      pfn_end    : 8000
>>>>>> mem_map (1)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 8000
>>>>>>      pfn_end    : 10000
>>>>>> mem_map (2)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 10000
>>>>>>      pfn_end    : 18000
>>>>>> mem_map (3)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 18000
>>>>>>      pfn_end    : 20000
>>>>>> mem_map (4)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 20000
>>>>>>      pfn_end    : 28000
>>>>>> mem_map (5)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 28000
>>>>>>      pfn_end    : 30000
>>>>>> mem_map (6)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 30000
>>>>>>      pfn_end    : 38000
>>>>>> mem_map (7)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 38000
>>>>>>      pfn_end    : 40000
>>>>>> mem_map (8)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 40000
>>>>>>      pfn_end    : 48000
>>>>>> mem_map (9)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 48000
>>>>>>      pfn_end    : 50000
>>>>>> mem_map (10)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 50000
>>>>>>      pfn_end    : 58000
>>>>>> mem_map (11)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 58000
>>>>>>      pfn_end    : 60000
>>>>>> mem_map (12)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 60000
>>>>>>      pfn_end    : 68000
>>>>>> mem_map (13)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 68000
>>>>>>      pfn_end    : 70000
>>>>>> mem_map (14)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 70000
>>>>>>      pfn_end    : 78000
>>>>>> mem_map (15)
>>>>>>      mem_map    : 0
>>>>>>      pfn_start  : 78000
>>>>>>      pfn_end    : 7ffd7
>>>>>> mmap() is available on the kernel.
>>>>>> Checking for memory holes                         : [100.0 %] |         STEP
>>>>>> [Checking for memory holes  ] : 0.000014 seconds
>>>>>> __vtop4_x86_64: Can't get a valid pte.
>>>>>> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
>>>>>> address.
>>>>>> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
>>>>>> __exclude_unnecessary_pages: Can't read the buffer of struct page.
>>>>>> create_2nd_bitmap: Can't exclude unnecessary pages.
>>>>>> Checking for memory holes                         : [100.0 %] \         STEP
>>>>>> [Checking for memory holes  ] : 0.000006 seconds
>>>>>> Checking for memory holes                         : [100.0 %] -         STEP
>>>>>> [Checking for memory holes  ] : 0.000004 seconds
>>>>>> __vtop4_x86_64: Can't get a valid pte.
>>>>>> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
>>>>>> address.
>>>>>> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
>>>>>> __exclude_unnecessary_pages: Can't read the buffer of struct page.
>>>>>> create_2nd_bitmap: Can't exclude unnecessary pages.
>>>>>>
>>>>>> makedumpfile Failed.
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>         ......It causes makedumpfile failed.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> 	dou.
>>>>>>>>
>>>>>>>>> 	-Mike
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec
> 
> 
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

On 02/07/18 at 05:25pm, Dou Liyang wrote:
> Hi All,
> 
> I met the makedumpfile failed in the upstream kernel which contained
> this patch. Did I missed something else?

readmem: Can't convert a virtual address(ffff88007ffd7000) to physical

Should not related to this patch. Otherwise your code can't get to that
step. From message, ffff88007ffd7000 is the end of the last mem region,
seems a code bug. You are testing 5-level on makedumpfile, right?

The patches I posted to descrease the memmory cost on mem map allocation
has code bug, Fengguang's test robot sent a mail to me, I have updated
patches, try to write a good patch log. You might also need check the
5-level patches you posted to makedumpfile upstream.

> 
> In fedora27 host:
> 
> [douly@localhost code]$ ./makedumpfile -d 31 --message-level 31 -x
> vmlinux_4.15+ vmcore_4.15+_from_cp_command vmcore_4.15+
> 
> sadump: does not have partition header
> sadump: read dump device as unknown format
> sadump: unknown format
> LOAD (0)
>   phys_start : 1000000
>   phys_end   : 2a86000
>   virt_start : ffffffff81000000
>   virt_end   : ffffffff82a86000
> LOAD (1)
>   phys_start : 1000
>   phys_end   : 9fc00
>   virt_start : ffff880000001000
>   virt_end   : ffff88000009fc00
> LOAD (2)
>   phys_start : 100000
>   phys_end   : 13000000
>   virt_start : ffff880000100000
>   virt_end   : ffff880013000000
> LOAD (3)
>   phys_start : 33000000
>   phys_end   : 7ffd7000
>   virt_start : ffff880033000000
>   virt_end   : ffff88007ffd7000
> Linux kdump
> page_size    : 4096
> 
> max_mapnr    : 7ffd7
> 
> Buffer size for the cyclic mode: 131061
> The kernel version is not supported.
> The makedumpfile operation may be incomplete.
> 
> num of NODEs : 1
> 
> 
> Memory type  : SPARSEMEM_EX
> 
> mem_map (0)
>   mem_map    : ffff88007ff26000
>   pfn_start  : 0
>   pfn_end    : 8000
> mem_map (1)
>   mem_map    : 0
>   pfn_start  : 8000
>   pfn_end    : 10000
> mem_map (2)
>   mem_map    : 0
>   pfn_start  : 10000
>   pfn_end    : 18000
> mem_map (3)
>   mem_map    : 0
>   pfn_start  : 18000
>   pfn_end    : 20000
> mem_map (4)
>   mem_map    : 0
>   pfn_start  : 20000
>   pfn_end    : 28000
> mem_map (5)
>   mem_map    : 0
>   pfn_start  : 28000
>   pfn_end    : 30000
> mem_map (6)
>   mem_map    : 0
>   pfn_start  : 30000
>   pfn_end    : 38000
> mem_map (7)
>   mem_map    : 0
>   pfn_start  : 38000
>   pfn_end    : 40000
> mem_map (8)
>   mem_map    : 0
>   pfn_start  : 40000
>   pfn_end    : 48000
> mem_map (9)
>   mem_map    : 0
>   pfn_start  : 48000
>   pfn_end    : 50000
> mem_map (10)
>   mem_map    : 0
>   pfn_start  : 50000
>   pfn_end    : 58000
> mem_map (11)
>   mem_map    : 0
>   pfn_start  : 58000
>   pfn_end    : 60000
> mem_map (12)
>   mem_map    : 0
>   pfn_start  : 60000
>   pfn_end    : 68000
> mem_map (13)
>   mem_map    : 0
>   pfn_start  : 68000
>   pfn_end    : 70000
> mem_map (14)
>   mem_map    : 0
>   pfn_start  : 70000
>   pfn_end    : 78000
> mem_map (15)
>   mem_map    : 0
>   pfn_start  : 78000
>   pfn_end    : 7ffd7
> mmap() is available on the kernel.
> Checking for memory holes                         : [100.0 %] |         STEP
> [Checking for memory holes  ] : 0.000060 seconds
> __vtop4_x86_64: Can't get a valid pte.
> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
> address.
> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
> __exclude_unnecessary_pages: Can't read the buffer of struct page.
> create_2nd_bitmap: Can't exclude unnecessary pages.
> Checking for memory holes                         : [100.0 %] \         STEP
> [Checking for memory holes  ] : 0.000010 seconds
> Checking for memory holes                         : [100.0 %] -         STEP
> [Checking for memory holes  ] : 0.000004 seconds
> __vtop4_x86_64: Can't get a valid pte.
> readmem: Can't convert a virtual address(ffff88007ffd7000) to physical
> address.
> readmem: type_addr: 0, addr:ffff88007ffd7000, size:32768
> __exclude_unnecessary_pages: Can't read the buffer of struct page.
> create_2nd_bitmap: Can't exclude unnecessary pages.
> 
> Thanks,
> 	dou
> At 01/09/2018 11:44 AM, Mike Galbraith wrote:
> > On Tue, 2018-01-09 at 03:13 +0300, Kirill A. Shutemov wrote:
> > > 
> > > Mike, could you test this? (On top of the rest of the fixes.)
> > 
> > homer:..crash/2018-01-09-04:25 # ll
> > total 1863604
> > -rw------- 1 root root      66255 Jan  9 04:25 dmesg.txt
> > -rw-r--r-- 1 root root        182 Jan  9 04:25 README.txt
> > -rw-r--r-- 1 root root    2818240 Jan  9 04:25 System.map-4.15.0.gb2cd1df-master
> > -rw------- 1 root root 1832914928 Jan  9 04:25 vmcore
> > -rw-r--r-- 1 root root   72514993 Jan  9 04:25 vmlinux-4.15.0.gb2cd1df-master.gz
> > 
> > Yup, all better.
> > 
> > > Sorry for the mess.
> > 
> > (why, developers not installing shiny new bugs is a whole lot worse:)
> > 
> > >  From 100fd567754f1457be94732046aefca204c842d2 Mon Sep 17 00:00:00 2001
> > > From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> > > Date: Tue, 9 Jan 2018 02:55:47 +0300
> > > Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> > > 
> > > Depending on configuration mem_section can now be an array or a pointer
> > > to an array allocated dynamically. In most cases, we can continue to refer
> > > to it as 'mem_section' regardless of what it is.
> > > 
> > > But there's one exception: '&mem_section' means "address of the array" if
> > > mem_section is an array, but if mem_section is a pointer, it would mean
> > > "address of the pointer".
> > > 
> > > We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> > > writes down address of pointer into vmcoreinfo, not array as we wanted.
> > > 
> > > Let's introduce VMCOREINFO_ARRAY() that would handle the situation
> > > correctly for both cases.
> > > 
> > > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
> > > ---
> > >   include/linux/crash_core.h | 2 ++
> > >   kernel/crash_core.c        | 2 +-
> > >   2 files changed, 3 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> > > index 06097ef30449..83ae04950269 100644
> > > --- a/include/linux/crash_core.h
> > > +++ b/include/linux/crash_core.h
> > > @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
> > >   	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
> > >   #define VMCOREINFO_SYMBOL(name) \
> > >   	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> > > +#define VMCOREINFO_ARRAY(name) \
> > > +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
> > >   #define VMCOREINFO_SIZE(name) \
> > >   	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
> > >   			      (unsigned long)sizeof(name))
> > > diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> > > index b3663896278e..d4122a837477 100644
> > > --- a/kernel/crash_core.c
> > > +++ b/kernel/crash_core.c
> > > @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
> > >   	VMCOREINFO_SYMBOL(contig_page_data);
> > >   #endif
> > >   #ifdef CONFIG_SPARSEMEM
> > > -	VMCOREINFO_SYMBOL(mem_section);
> > > +	VMCOREINFO_ARRAY(mem_section);
> > >   	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
> > >   	VMCOREINFO_STRUCT_SIZE(mem_section);
> > >   	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> > 
> > _______________________________________________
> > kexec mailing list
> > kexec@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kexec
> > 
> > 
> > 
> 
> 
> 
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

Hi Kirill,

I setup qemu 2.9.0 to test 5-level on kexec/kdump support. While both
kexec and kdump reset to BIOS immediately after triggering. I saw your
patch adding 5-level paging support for kexec. Wonder if your test
succeeded to jump into kexec/kdump kernel, and what else I need to
make it. By the way, I just tested the latest upstream kernel.

commit 7f6890418 x86/kexec: Add 5-level paging support

[ ~]$ qemu-system-x86_64 --version
QEMU emulator version 2.9.0(qemu-2.9.0-1.fc26.1)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

Thanks
Baoquan

On 01/09/18 at 03:13am, Kirill A. Shutemov wrote:
> On Mon, Jan 08, 2018 at 08:46:53PM +0300, Kirill A. Shutemov wrote:
> > On Mon, Jan 08, 2018 at 04:04:44PM +0000, Ingo Molnar wrote:
> > > 
> > > hi Kirill,
> > > 
> > > As Mike reported it below, your 5-level paging related upstream commit 
> > > 83e3c48729d9 and all its followup fixes:
> > > 
> > >  83e3c48729d9: mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y
> > >  629a359bdb0e: mm/sparsemem: Fix ARM64 boot crash when CONFIG_SPARSEMEM_EXTREME=y
> > >  d09cfbbfa0f7: mm/sparse.c: wrong allocation for mem_section
> > > 
> > > ... still breaks kexec - and that now regresses -stable as well.
> > > 
> > > Given that 5-level paging now syntactically depends on having this commit, if we 
> > > fully revert this then we'll have to disable 5-level paging as well.
> 
> This *should* help.
> 
> Mike, could you test this? (On top of the rest of the fixes.)
> 
> Sorry for the mess.
> 
> From 100fd567754f1457be94732046aefca204c842d2 Mon Sep 17 00:00:00 2001
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Date: Tue, 9 Jan 2018 02:55:47 +0300
> Subject: [PATCH] kdump: Write a correct address of mem_section into vmcoreinfo
> 
> Depending on configuration mem_section can now be an array or a pointer
> to an array allocated dynamically. In most cases, we can continue to refer
> to it as 'mem_section' regardless of what it is.
> 
> But there's one exception: '&mem_section' means "address of the array" if
> mem_section is an array, but if mem_section is a pointer, it would mean
> "address of the pointer".
> 
> We've stepped onto this in kdump code. VMCOREINFO_SYMBOL(mem_section)
> writes down address of pointer into vmcoreinfo, not array as we wanted.
> 
> Let's introduce VMCOREINFO_ARRAY() that would handle the situation
> correctly for both cases.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Fixes: 83e3c48729d9 ("mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y")
> ---
>  include/linux/crash_core.h | 2 ++
>  kernel/crash_core.c        | 2 +-
>  2 files changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/crash_core.h b/include/linux/crash_core.h
> index 06097ef30449..83ae04950269 100644
> --- a/include/linux/crash_core.h
> +++ b/include/linux/crash_core.h
> @@ -42,6 +42,8 @@ phys_addr_t paddr_vmcoreinfo_note(void);
>  	vmcoreinfo_append_str("PAGESIZE=%ld\n", value)
>  #define VMCOREINFO_SYMBOL(name) \
>  	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)&name)
> +#define VMCOREINFO_ARRAY(name) \
> +	vmcoreinfo_append_str("SYMBOL(%s)=%lx\n", #name, (unsigned long)name)
>  #define VMCOREINFO_SIZE(name) \
>  	vmcoreinfo_append_str("SIZE(%s)=%lu\n", #name, \
>  			      (unsigned long)sizeof(name))
> diff --git a/kernel/crash_core.c b/kernel/crash_core.c
> index b3663896278e..d4122a837477 100644
> --- a/kernel/crash_core.c
> +++ b/kernel/crash_core.c
> @@ -410,7 +410,7 @@ static int __init crash_save_vmcoreinfo_init(void)
>  	VMCOREINFO_SYMBOL(contig_page_data);
>  #endif
>  #ifdef CONFIG_SPARSEMEM
> -	VMCOREINFO_SYMBOL(mem_section);
> +	VMCOREINFO_ARRAY(mem_section);
>  	VMCOREINFO_LENGTH(mem_section, NR_SECTION_ROOTS);
>  	VMCOREINFO_STRUCT_SIZE(mem_section);
>  	VMCOREINFO_OFFSET(mem_section, section_mem_map);
> -- 
>  Kirill A. Shutemov
> 
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Kirill A. Shutemov]

On Wed, Jan 17, 2018 at 01:24:54PM +0800, Baoquan He wrote:
> Hi Kirill,
> 
> I setup qemu 2.9.0 to test 5-level on kexec/kdump support. While both
> kexec and kdump reset to BIOS immediately after triggering. I saw your
> patch adding 5-level paging support for kexec. Wonder if your test
> succeeded to jump into kexec/kdump kernel, and what else I need to
> make it. By the way, I just tested the latest upstream kernel.
> 
> commit 7f6890418 x86/kexec: Add 5-level paging support
> 
> [ ~]$ qemu-system-x86_64 --version
> QEMU emulator version 2.9.0(qemu-2.9.0-1.fc26.1)
> Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

Sorry for delay.

I didn't tested it in 5-level paging mode :-/

The patch below helps in my case. Could you test it?

diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocate_kernel_64.S
index 307d3bac5f04..65a98cf2307d 100644
--- a/arch/x86/kernel/relocate_kernel_64.S
+++ b/arch/x86/kernel/relocate_kernel_64.S
@@ -126,8 +126,12 @@ identity_mapped:
        /*
         * Set cr4 to a known state:
         *  - physical address extension enabled
+        *  - 5-level paging, if enabled
         */
        movl    $X86_CR4_PAE, %eax
+#ifdef CONFIG_X86_5LEVEL
+       orl     $X86_CR4_LA57, %eax
+#endif
        movq    %rax, %cr4

        jmp 1f
-- 
 Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 4.14 023/159] mm/sparsemem: Allocate mem_section at runtime for CONFIG_SPARSEMEM_EXTREME=y

[Baoquan He]

On 01/25/18 at 06:50pm, Kirill A. Shutemov wrote:
> On Wed, Jan 17, 2018 at 01:24:54PM +0800, Baoquan He wrote:
> > Hi Kirill,
> > 
> > I setup qemu 2.9.0 to test 5-level on kexec/kdump support. While both
> > kexec and kdump reset to BIOS immediately after triggering. I saw your
> > patch adding 5-level paging support for kexec. Wonder if your test
> > succeeded to jump into kexec/kdump kernel, and what else I need to
> > make it. By the way, I just tested the latest upstream kernel.
> > 
> > commit 7f6890418 x86/kexec: Add 5-level paging support
> > 
> > [ ~]$ qemu-system-x86_64 --version
> > QEMU emulator version 2.9.0(qemu-2.9.0-1.fc26.1)
> > Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> 
> Sorry for delay.
> 
> I didn't tested it in 5-level paging mode :-/
> 
> The patch below helps in my case. Could you test it?

Thanks, Kirill. 

Seems it doesn't work. I have some confusion about the process, will
send you a private mail.

Thanks
Baoquan
> 
> diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocate_kernel_64.S
> index 307d3bac5f04..65a98cf2307d 100644
> --- a/arch/x86/kernel/relocate_kernel_64.S
> +++ b/arch/x86/kernel/relocate_kernel_64.S
> @@ -126,8 +126,12 @@ identity_mapped:
>         /*
>          * Set cr4 to a known state:
>          *  - physical address extension enabled
> +        *  - 5-level paging, if enabled
>          */
>         movl    $X86_CR4_PAE, %eax
> +#ifdef CONFIG_X86_5LEVEL
> +       orl     $X86_CR4_LA57, %eax
> +#endif
>         movq    %rax, %cr4
> 
>         jmp 1f
> -- 
>  Kirill A. Shutemov

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 4.14 024/159] x86/kasan: Use the same shadow offset for 4- and 5-level paging

[Greg Kroah-Hartman]

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Andrey Ryabinin <aryabinin@virtuozzo.com>

commit 12a8cc7fcf54a8575f094be1e99032ec38aa045c upstream.

We are going to support boot-time switching between 4- and 5-level
paging. For KASAN it means we cannot have different KASAN_SHADOW_OFFSET
for different paging modes: the constant is passed to gcc to generate
code and cannot be changed at runtime.

This patch changes KASAN code to use 0xdffffc0000000000 as shadow offset
for both 4- and 5-level paging.

For 5-level paging it means that shadow memory region is not aligned to
PGD boundary anymore and we have to handle unaligned parts of the region
properly.

In addition, we have to exclude paravirt code from KASAN instrumentation
as we now use set_pgd() before KASAN is fully ready.

[kirill.shutemov@linux.intel.com: clenaup, changelog message]
Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@suse.de>
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20170929140821.37654-4-kirill.shutemov@linux.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 Documentation/x86/x86_64/mm.txt |    2 
 arch/x86/Kconfig                |    1 
 arch/x86/kernel/Makefile        |    3 -
 arch/x86/mm/kasan_init_64.c     |  101 +++++++++++++++++++++++++++++++---------
 4 files changed, 83 insertions(+), 24 deletions(-)

--- a/Documentation/x86/x86_64/mm.txt
+++ b/Documentation/x86/x86_64/mm.txt
@@ -34,7 +34,7 @@ ff92000000000000 - ffd1ffffffffffff (=54
 ffd2000000000000 - ffd3ffffffffffff (=49 bits) hole
 ffd4000000000000 - ffd5ffffffffffff (=49 bits) virtual memory map (512TB)
 ... unused hole ...
-ffd8000000000000 - fff7ffffffffffff (=53 bits) kasan shadow memory (8PB)
+ffdf000000000000 - fffffc0000000000 (=53 bits) kasan shadow memory (8PB)
 ... unused hole ...
 ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks
 ... unused hole ...
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -303,7 +303,6 @@ config ARCH_SUPPORTS_DEBUG_PAGEALLOC
 config KASAN_SHADOW_OFFSET
 	hex
 	depends on KASAN
-	default 0xdff8000000000000 if X86_5LEVEL
 	default 0xdffffc0000000000
 
 config HAVE_INTEL_TXT
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -25,7 +25,8 @@ endif
 KASAN_SANITIZE_head$(BITS).o				:= n
 KASAN_SANITIZE_dumpstack.o				:= n
 KASAN_SANITIZE_dumpstack_$(BITS).o			:= n
-KASAN_SANITIZE_stacktrace.o := n
+KASAN_SANITIZE_stacktrace.o				:= n
+KASAN_SANITIZE_paravirt.o				:= n
 
 OBJECT_FILES_NON_STANDARD_relocate_kernel_$(BITS).o	:= y
 OBJECT_FILES_NON_STANDARD_ftrace_$(BITS).o		:= y
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -16,6 +16,8 @@
 
 extern struct range pfn_mapped[E820_MAX_ENTRIES];
 
+static p4d_t tmp_p4d_table[PTRS_PER_P4D] __initdata __aligned(PAGE_SIZE);
+
 static int __init map_range(struct range *range)
 {
 	unsigned long start;
@@ -31,8 +33,10 @@ static void __init clear_pgds(unsigned l
 			unsigned long end)
 {
 	pgd_t *pgd;
+	/* See comment in kasan_init() */
+	unsigned long pgd_end = end & PGDIR_MASK;
 
-	for (; start < end; start += PGDIR_SIZE) {
+	for (; start < pgd_end; start += PGDIR_SIZE) {
 		pgd = pgd_offset_k(start);
 		/*
 		 * With folded p4d, pgd_clear() is nop, use p4d_clear()
@@ -43,29 +47,61 @@ static void __init clear_pgds(unsigned l
 		else
 			pgd_clear(pgd);
 	}
+
+	pgd = pgd_offset_k(start);
+	for (; start < end; start += P4D_SIZE)
+		p4d_clear(p4d_offset(pgd, start));
+}
+
+static inline p4d_t *early_p4d_offset(pgd_t *pgd, unsigned long addr)
+{
+	unsigned long p4d;
+
+	if (!IS_ENABLED(CONFIG_X86_5LEVEL))
+		return (p4d_t *)pgd;
+
+	p4d = __pa_nodebug(pgd_val(*pgd)) & PTE_PFN_MASK;
+	p4d += __START_KERNEL_map - phys_base;
+	return (p4d_t *)p4d + p4d_index(addr);
+}
+
+static void __init kasan_early_p4d_populate(pgd_t *pgd,
+		unsigned long addr,
+		unsigned long end)
+{
+	pgd_t pgd_entry;
+	p4d_t *p4d, p4d_entry;
+	unsigned long next;
+
+	if (pgd_none(*pgd)) {
+		pgd_entry = __pgd(_KERNPG_TABLE | __pa_nodebug(kasan_zero_p4d));
+		set_pgd(pgd, pgd_entry);
+	}
+
+	p4d = early_p4d_offset(pgd, addr);
+	do {
+		next = p4d_addr_end(addr, end);
+
+		if (!p4d_none(*p4d))
+			continue;
+
+		p4d_entry = __p4d(_KERNPG_TABLE | __pa_nodebug(kasan_zero_pud));
+		set_p4d(p4d, p4d_entry);
+	} while (p4d++, addr = next, addr != end && p4d_none(*p4d));
 }
 
 static void __init kasan_map_early_shadow(pgd_t *pgd)
 {
-	int i;
-	unsigned long start = KASAN_SHADOW_START;
+	/* See comment in kasan_init() */
+	unsigned long addr = KASAN_SHADOW_START & PGDIR_MASK;
 	unsigned long end = KASAN_SHADOW_END;
+	unsigned long next;
 
-	for (i = pgd_index(start); start < end; i++) {
-		switch (CONFIG_PGTABLE_LEVELS) {
-		case 4:
-			pgd[i] = __pgd(__pa_nodebug(kasan_zero_pud) |
-					_KERNPG_TABLE);
-			break;
-		case 5:
-			pgd[i] = __pgd(__pa_nodebug(kasan_zero_p4d) |
-					_KERNPG_TABLE);
-			break;
-		default:
-			BUILD_BUG();
-		}
-		start += PGDIR_SIZE;
-	}
+	pgd += pgd_index(addr);
+	do {
+		next = pgd_addr_end(addr, end);
+		kasan_early_p4d_populate(pgd, addr, next);
+	} while (pgd++, addr = next, addr != end);
 }
 
 #ifdef CONFIG_KASAN_INLINE
@@ -102,7 +138,7 @@ void __init kasan_early_init(void)
 	for (i = 0; i < PTRS_PER_PUD; i++)
 		kasan_zero_pud[i] = __pud(pud_val);
 
-	for (i = 0; CONFIG_PGTABLE_LEVELS >= 5 && i < PTRS_PER_P4D; i++)
+	for (i = 0; IS_ENABLED(CONFIG_X86_5LEVEL) && i < PTRS_PER_P4D; i++)
 		kasan_zero_p4d[i] = __p4d(p4d_val);
 
 	kasan_map_early_shadow(early_top_pgt);
@@ -118,12 +154,35 @@ void __init kasan_init(void)
 #endif
 
 	memcpy(early_top_pgt, init_top_pgt, sizeof(early_top_pgt));
+
+	/*
+	 * We use the same shadow offset for 4- and 5-level paging to
+	 * facilitate boot-time switching between paging modes.
+	 * As result in 5-level paging mode KASAN_SHADOW_START and
+	 * KASAN_SHADOW_END are not aligned to PGD boundary.
+	 *
+	 * KASAN_SHADOW_START doesn't share PGD with anything else.
+	 * We claim whole PGD entry to make things easier.
+	 *
+	 * KASAN_SHADOW_END lands in the last PGD entry and it collides with
+	 * bunch of things like kernel code, modules, EFI mapping, etc.
+	 * We need to take extra steps to not overwrite them.
+	 */
+	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
+		void *ptr;
+
+		ptr = (void *)pgd_page_vaddr(*pgd_offset_k(KASAN_SHADOW_END));
+		memcpy(tmp_p4d_table, (void *)ptr, sizeof(tmp_p4d_table));
+		set_pgd(&early_top_pgt[pgd_index(KASAN_SHADOW_END)],
+				__pgd(__pa(tmp_p4d_table) | _KERNPG_TABLE));
+	}
+
 	load_cr3(early_top_pgt);
 	__flush_tlb_all();
 
-	clear_pgds(KASAN_SHADOW_START, KASAN_SHADOW_END);
+	clear_pgds(KASAN_SHADOW_START & PGDIR_MASK, KASAN_SHADOW_END);
 
-	kasan_populate_zero_shadow((void *)KASAN_SHADOW_START,
+	kasan_populate_zero_shadow((void *)(KASAN_SHADOW_START & PGDIR_MASK),
 			kasan_mem_to_shadow((void *)PAGE_OFFSET));
 
 	for (i = 0; i < E820_MAX_ENTRIES; i++) {


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 4.14 025/159] x86/xen: Provide pre-built page tables only for CONFIG_XEN_PV=y and CONFIG_XEN_PVH=y

[Greg Kroah-Hartman]

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>

commit 4375c29985f155d7eb2346615d84e62d1b673682 upstream.

Looks like we only need pre-built page tables in the CONFIG_XEN_PV=y and
CONFIG_XEN_PVH=y cases.

Let's not provide them for other configurations.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@suse.de>
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20170929140821.37654-5-kirill.shutemov@linux.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 arch/x86/kernel/head_64.S |   11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -38,11 +38,12 @@
  *
  */
 
-#define p4d_index(x)	(((x) >> P4D_SHIFT) & (PTRS_PER_P4D-1))
 #define pud_index(x)	(((x) >> PUD_SHIFT) & (PTRS_PER_PUD-1))
 
+#if defined(CONFIG_XEN_PV) || defined(CONFIG_XEN_PVH)
 PGD_PAGE_OFFSET = pgd_index(__PAGE_OFFSET_BASE)
 PGD_START_KERNEL = pgd_index(__START_KERNEL_map)
+#endif
 L3_START_KERNEL = pud_index(__START_KERNEL_map)
 
 	.text
@@ -365,10 +366,7 @@ NEXT_PAGE(early_dynamic_pgts)
 
 	.data
 
-#ifndef CONFIG_XEN
-NEXT_PAGE(init_top_pgt)
-	.fill	512,8,0
-#else
+#if defined(CONFIG_XEN_PV) || defined(CONFIG_XEN_PVH)
 NEXT_PAGE(init_top_pgt)
 	.quad   level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC
 	.org    init_top_pgt + PGD_PAGE_OFFSET*8, 0
@@ -385,6 +383,9 @@ NEXT_PAGE(level2_ident_pgt)
 	 * Don't set NX because code runs from these pages.
 	 */
 	PMDS(0, __PAGE_KERNEL_IDENT_LARGE_EXEC, PTRS_PER_PMD)
+#else
+NEXT_PAGE(init_top_pgt)
+	.fill	512,8,0
 #endif
 
 #ifdef CONFIG_X86_5LEVEL


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 4.14 026/159] x86/xen: Drop 5-level paging support code from the XEN_PV code

[Greg Kroah-Hartman]

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>

commit 773dd2fca581b0a80e5a33332cc8ee67e5a79cba upstream.

It was decided 5-level paging is not going to be supported in XEN_PV.

Let's drop the dead code from the XEN_PV code.

Tested-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@suse.de>
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Link: http://lkml.kernel.org/r/20170929140821.37654-6-kirill.shutemov@linux.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 arch/x86/xen/mmu_pv.c |  159 ++++++++++++++++++--------------------------------
 1 file changed, 60 insertions(+), 99 deletions(-)

--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -449,7 +449,7 @@ __visible pmd_t xen_make_pmd(pmdval_t pm
 }
 PV_CALLEE_SAVE_REGS_THUNK(xen_make_pmd);
 
-#if CONFIG_PGTABLE_LEVELS == 4
+#ifdef CONFIG_X86_64
 __visible pudval_t xen_pud_val(pud_t pud)
 {
 	return pte_mfn_to_pfn(pud.pud);
@@ -538,7 +538,7 @@ static void xen_set_p4d(p4d_t *ptr, p4d_
 
 	xen_mc_issue(PARAVIRT_LAZY_MMU);
 }
-#endif	/* CONFIG_PGTABLE_LEVELS == 4 */
+#endif	/* CONFIG_X86_64 */
 
 static int xen_pmd_walk(struct mm_struct *mm, pmd_t *pmd,
 		int (*func)(struct mm_struct *mm, struct page *, enum pt_level),
@@ -580,21 +580,17 @@ static int xen_p4d_walk(struct mm_struct
 		int (*func)(struct mm_struct *mm, struct page *, enum pt_level),
 		bool last, unsigned long limit)
 {
-	int i, nr, flush = 0;
+	int flush = 0;
+	pud_t *pud;
 
-	nr = last ? p4d_index(limit) + 1 : PTRS_PER_P4D;
-	for (i = 0; i < nr; i++) {
-		pud_t *pud;
 
-		if (p4d_none(p4d[i]))
-			continue;
+	if (p4d_none(*p4d))
+		return flush;
 
-		pud = pud_offset(&p4d[i], 0);
-		if (PTRS_PER_PUD > 1)
-			flush |= (*func)(mm, virt_to_page(pud), PT_PUD);
-		flush |= xen_pud_walk(mm, pud, func,
-				last && i == nr - 1, limit);
-	}
+	pud = pud_offset(p4d, 0);
+	if (PTRS_PER_PUD > 1)
+		flush |= (*func)(mm, virt_to_page(pud), PT_PUD);
+	flush |= xen_pud_walk(mm, pud, func, last, limit);
 	return flush;
 }
 
@@ -644,8 +640,6 @@ static int __xen_pgd_walk(struct mm_stru
 			continue;
 
 		p4d = p4d_offset(&pgd[i], 0);
-		if (PTRS_PER_P4D > 1)
-			flush |= (*func)(mm, virt_to_page(p4d), PT_P4D);
 		flush |= xen_p4d_walk(mm, p4d, func, i == nr - 1, limit);
 	}
 
@@ -1176,22 +1170,14 @@ static void __init xen_cleanmfnmap(unsig
 {
 	pgd_t *pgd;
 	p4d_t *p4d;
-	unsigned int i;
 	bool unpin;
 
 	unpin = (vaddr == 2 * PGDIR_SIZE);
 	vaddr &= PMD_MASK;
 	pgd = pgd_offset_k(vaddr);
 	p4d = p4d_offset(pgd, 0);
-	for (i = 0; i < PTRS_PER_P4D; i++) {
-		if (p4d_none(p4d[i]))
-			continue;
-		xen_cleanmfnmap_p4d(p4d + i, unpin);
-	}
-	if (IS_ENABLED(CONFIG_X86_5LEVEL)) {
-		set_pgd(pgd, __pgd(0));
-		xen_cleanmfnmap_free_pgtbl(p4d, unpin);
-	}
+	if (!p4d_none(*p4d))
+		xen_cleanmfnmap_p4d(p4d, unpin);
 }
 
 static void __init xen_pagetable_p2m_free(void)
@@ -1692,7 +1678,7 @@ static void xen_release_pmd(unsigned lon
 	xen_release_ptpage(pfn, PT_PMD);
 }
 
-#if CONFIG_PGTABLE_LEVELS >= 4
+#ifdef CONFIG_X86_64
 static void xen_alloc_pud(struct mm_struct *mm, unsigned long pfn)
 {
 	xen_alloc_ptpage(mm, pfn, PT_PUD);
@@ -2029,13 +2015,12 @@ static phys_addr_t __init xen_early_virt
  */
 void __init xen_relocate_p2m(void)
 {
-	phys_addr_t size, new_area, pt_phys, pmd_phys, pud_phys, p4d_phys;
+	phys_addr_t size, new_area, pt_phys, pmd_phys, pud_phys;
 	unsigned long p2m_pfn, p2m_pfn_end, n_frames, pfn, pfn_end;
-	int n_pte, n_pt, n_pmd, n_pud, n_p4d, idx_pte, idx_pt, idx_pmd, idx_pud, idx_p4d;
+	int n_pte, n_pt, n_pmd, n_pud, idx_pte, idx_pt, idx_pmd, idx_pud;
 	pte_t *pt;
 	pmd_t *pmd;
 	pud_t *pud;
-	p4d_t *p4d = NULL;
 	pgd_t *pgd;
 	unsigned long *new_p2m;
 	int save_pud;
@@ -2045,11 +2030,7 @@ void __init xen_relocate_p2m(void)
 	n_pt = roundup(size, PMD_SIZE) >> PMD_SHIFT;
 	n_pmd = roundup(size, PUD_SIZE) >> PUD_SHIFT;
 	n_pud = roundup(size, P4D_SIZE) >> P4D_SHIFT;
-	if (PTRS_PER_P4D > 1)
-		n_p4d = roundup(size, PGDIR_SIZE) >> PGDIR_SHIFT;
-	else
-		n_p4d = 0;
-	n_frames = n_pte + n_pt + n_pmd + n_pud + n_p4d;
+	n_frames = n_pte + n_pt + n_pmd + n_pud;
 
 	new_area = xen_find_free_area(PFN_PHYS(n_frames));
 	if (!new_area) {
@@ -2065,76 +2046,56 @@ void __init xen_relocate_p2m(void)
 	 * To avoid any possible virtual address collision, just use
 	 * 2 * PUD_SIZE for the new area.
 	 */
-	p4d_phys = new_area;
-	pud_phys = p4d_phys + PFN_PHYS(n_p4d);
+	pud_phys = new_area;
 	pmd_phys = pud_phys + PFN_PHYS(n_pud);
 	pt_phys = pmd_phys + PFN_PHYS(n_pmd);
 	p2m_pfn = PFN_DOWN(pt_phys) + n_pt;
 
 	pgd = __va(read_cr3_pa());
 	new_p2m = (unsigned long *)(2 * PGDIR_SIZE);
-	idx_p4d = 0;
 	save_pud = n_pud;
-	do {
-		if (n_p4d > 0) {
-			p4d = early_memremap(p4d_phys, PAGE_SIZE);
-			clear_page(p4d);
-			n_pud = min(save_pud, PTRS_PER_P4D);
-		}
-		for (idx_pud = 0; idx_pud < n_pud; idx_pud++) {
-			pud = early_memremap(pud_phys, PAGE_SIZE);
-			clear_page(pud);
-			for (idx_pmd = 0; idx_pmd < min(n_pmd, PTRS_PER_PUD);
-				 idx_pmd++) {
-				pmd = early_memremap(pmd_phys, PAGE_SIZE);
-				clear_page(pmd);
-				for (idx_pt = 0; idx_pt < min(n_pt, PTRS_PER_PMD);
-					 idx_pt++) {
-					pt = early_memremap(pt_phys, PAGE_SIZE);
-					clear_page(pt);
-					for (idx_pte = 0;
-						 idx_pte < min(n_pte, PTRS_PER_PTE);
-						 idx_pte++) {
-						set_pte(pt + idx_pte,
-								pfn_pte(p2m_pfn, PAGE_KERNEL));
-						p2m_pfn++;
-					}
-					n_pte -= PTRS_PER_PTE;
-					early_memunmap(pt, PAGE_SIZE);
-					make_lowmem_page_readonly(__va(pt_phys));
-					pin_pagetable_pfn(MMUEXT_PIN_L1_TABLE,
-							PFN_DOWN(pt_phys));
-					set_pmd(pmd + idx_pt,
-							__pmd(_PAGE_TABLE | pt_phys));
-					pt_phys += PAGE_SIZE;
+	for (idx_pud = 0; idx_pud < n_pud; idx_pud++) {
+		pud = early_memremap(pud_phys, PAGE_SIZE);
+		clear_page(pud);
+		for (idx_pmd = 0; idx_pmd < min(n_pmd, PTRS_PER_PUD);
+				idx_pmd++) {
+			pmd = early_memremap(pmd_phys, PAGE_SIZE);
+			clear_page(pmd);
+			for (idx_pt = 0; idx_pt < min(n_pt, PTRS_PER_PMD);
+					idx_pt++) {
+				pt = early_memremap(pt_phys, PAGE_SIZE);
+				clear_page(pt);
+				for (idx_pte = 0;
+						idx_pte < min(n_pte, PTRS_PER_PTE);
+						idx_pte++) {
+					set_pte(pt + idx_pte,
+							pfn_pte(p2m_pfn, PAGE_KERNEL));
+					p2m_pfn++;
 				}
-				n_pt -= PTRS_PER_PMD;
-				early_memunmap(pmd, PAGE_SIZE);
-				make_lowmem_page_readonly(__va(pmd_phys));
-				pin_pagetable_pfn(MMUEXT_PIN_L2_TABLE,
-						PFN_DOWN(pmd_phys));
-				set_pud(pud + idx_pmd, __pud(_PAGE_TABLE | pmd_phys));
-				pmd_phys += PAGE_SIZE;
+				n_pte -= PTRS_PER_PTE;
+				early_memunmap(pt, PAGE_SIZE);
+				make_lowmem_page_readonly(__va(pt_phys));
+				pin_pagetable_pfn(MMUEXT_PIN_L1_TABLE,
+						PFN_DOWN(pt_phys));
+				set_pmd(pmd + idx_pt,
+						__pmd(_PAGE_TABLE | pt_phys));
+				pt_phys += PAGE_SIZE;
 			}
-			n_pmd -= PTRS_PER_PUD;
-			early_memunmap(pud, PAGE_SIZE);
-			make_lowmem_page_readonly(__va(pud_phys));
-			pin_pagetable_pfn(MMUEXT_PIN_L3_TABLE, PFN_DOWN(pud_phys));
-			if (n_p4d > 0)
-				set_p4d(p4d + idx_pud, __p4d(_PAGE_TABLE | pud_phys));
-			else
-				set_pgd(pgd + 2 + idx_pud, __pgd(_PAGE_TABLE | pud_phys));
-			pud_phys += PAGE_SIZE;
-		}
-		if (n_p4d > 0) {
-			save_pud -= PTRS_PER_P4D;
-			early_memunmap(p4d, PAGE_SIZE);
-			make_lowmem_page_readonly(__va(p4d_phys));
-			pin_pagetable_pfn(MMUEXT_PIN_L4_TABLE, PFN_DOWN(p4d_phys));
-			set_pgd(pgd + 2 + idx_p4d, __pgd(_PAGE_TABLE | p4d_phys));
-			p4d_phys += PAGE_SIZE;
+			n_pt -= PTRS_PER_PMD;
+			early_memunmap(pmd, PAGE_SIZE);
+			make_lowmem_page_readonly(__va(pmd_phys));
+			pin_pagetable_pfn(MMUEXT_PIN_L2_TABLE,
+					PFN_DOWN(pmd_phys));
+			set_pud(pud + idx_pmd, __pud(_PAGE_TABLE | pmd_phys));
+			pmd_phys += PAGE_SIZE;
 		}
-	} while (++idx_p4d < n_p4d);
+		n_pmd -= PTRS_PER_PUD;
+		early_memunmap(pud, PAGE_SIZE);
+		make_lowmem_page_readonly(__va(pud_phys));
+		pin_pagetable_pfn(MMUEXT_PIN_L3_TABLE, PFN_DOWN(pud_phys));
+		set_pgd(pgd + 2 + idx_pud, __pgd(_PAGE_TABLE | pud_phys));
+		pud_phys += PAGE_SIZE;
+	}
 
 	/* Now copy the old p2m info to the new area. */
 	memcpy(new_p2m, xen_p2m_addr, size);
@@ -2361,7 +2322,7 @@ static void __init xen_post_allocator_in
 	pv_mmu_ops.set_pte = xen_set_pte;
 	pv_mmu_ops.set_pmd = xen_set_pmd;
 	pv_mmu_ops.set_pud = xen_set_pud;
-#if CONFIG_PGTABLE_LEVELS >= 4
+#ifdef CONFIG_X86_64
 	pv_mmu_ops.set_p4d = xen_set_p4d;
 #endif
 
@@ -2371,7 +2332,7 @@ static void __init xen_post_allocator_in
 	pv_mmu_ops.alloc_pmd = xen_alloc_pmd;
 	pv_mmu_ops.release_pte = xen_release_pte;
 	pv_mmu_ops.release_pmd = xen_release_pmd;
-#if CONFIG_PGTABLE_LEVELS >= 4
+#ifdef CONFIG_X86_64
 	pv_mmu_ops.alloc_pud = xen_alloc_pud;
 	pv_mmu_ops.release_pud = xen_release_pud;
 #endif
@@ -2435,14 +2396,14 @@ static const struct pv_mmu_ops xen_mmu_o
 	.make_pmd = PV_CALLEE_SAVE(xen_make_pmd),
 	.pmd_val = PV_CALLEE_SAVE(xen_pmd_val),
 
-#if CONFIG_PGTABLE_LEVELS >= 4
+#ifdef CONFIG_X86_64
 	.pud_val = PV_CALLEE_SAVE(xen_pud_val),
 	.make_pud = PV_CALLEE_SAVE(xen_make_pud),
 	.set_p4d = xen_set_p4d_hyper,
 
 	.alloc_pud = xen_alloc_pmd_init,
 	.release_pud = xen_release_pmd_init,
-#endif	/* CONFIG_PGTABLE_LEVELS == 4 */
+#endif	/* CONFIG_X86_64 */
 
 	.activate_mm = xen_activate_mm,
 	.dup_mmap = xen_dup_mmap,


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 4.14 033/159] x86/boot: Relocate definition of the initial state of CR0

[Greg Kroah-Hartman]

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>

commit b0ce5b8c95c83a7b98c679b117e3d6ae6f97154b upstream.

Both head_32.S and head_64.S utilize the same value to initialize the
control register CR0. Also, other parts of the kernel might want to access
this initial definition (e.g., emulation code for User-Mode Instruction
Prevention uses this state to provide a sane dummy value for CR0 when
emulating the smsw instruction). Thus, relocate this definition to a
header file from which it can be conveniently accessed.

Suggested-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Borislav Petkov <bp@suse.de>
Reviewed-by: Andy Lutomirski <luto@kernel.org>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: ricardo.neri@intel.com
Cc: linux-mm@kvack.org
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Huang Rui <ray.huang@amd.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: linux-arch@vger.kernel.org
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Jiri Slaby <jslaby@suse.cz>
Cc: "Ravi V. Shankar" <ravi.v.shankar@intel.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Chen Yucong <slaoub@gmail.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lkml.kernel.org/r/1509135945-13762-3-git-send-email-ricardo.neri-calderon@linux.intel.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 arch/x86/include/uapi/asm/processor-flags.h |    3 +++
 arch/x86/kernel/head_32.S                   |    3 ---
 arch/x86/kernel/head_64.S                   |    3 ---
 3 files changed, 3 insertions(+), 6 deletions(-)

--- a/arch/x86/include/uapi/asm/processor-flags.h
+++ b/arch/x86/include/uapi/asm/processor-flags.h
@@ -152,5 +152,8 @@
 #define CX86_ARR_BASE	0xc4
 #define CX86_RCR_BASE	0xdc
 
+#define CR0_STATE	(X86_CR0_PE | X86_CR0_MP | X86_CR0_ET | \
+			 X86_CR0_NE | X86_CR0_WP | X86_CR0_AM | \
+			 X86_CR0_PG)
 
 #endif /* _UAPI_ASM_X86_PROCESSOR_FLAGS_H */
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -212,9 +212,6 @@ ENTRY(startup_32_smp)
 #endif
 
 .Ldefault_entry:
-#define CR0_STATE	(X86_CR0_PE | X86_CR0_MP | X86_CR0_ET | \
-			 X86_CR0_NE | X86_CR0_WP | X86_CR0_AM | \
-			 X86_CR0_PG)
 	movl $(CR0_STATE & ~X86_CR0_PG),%eax
 	movl %eax,%cr0
 
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -154,9 +154,6 @@ ENTRY(secondary_startup_64)
 1:	wrmsr				/* Make changes effective */
 
 	/* Setup cr0 */
-#define CR0_STATE	(X86_CR0_PE | X86_CR0_MP | X86_CR0_ET | \
-			 X86_CR0_NE | X86_CR0_WP | X86_CR0_AM | \
-			 X86_CR0_PG)
 	movl	$CR0_STATE, %eax
 	/* Make changes effective */
 	movq	%rax, %cr0


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 4.14 097/159] x86/paravirt: Dont patch flush_tlb_single

[Greg Kroah-Hartman]

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Thomas Gleixner <tglx@linutronix.de>

commit a035795499ca1c2bd1928808d1a156eda1420383 upstream.

native_flush_tlb_single() will be changed with the upcoming
PAGE_TABLE_ISOLATION feature. This requires to have more code in
there than INVLPG.

Remove the paravirt patching for it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Borislav Petkov <bpetkov@suse.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Laight <David.Laight@aculab.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Eduardo Valentin <eduval@amazon.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: aliguori@amazon.com
Cc: daniel.gruss@iaik.tugraz.at
Cc: hughd@google.com
Cc: keescook@google.com
Cc: linux-mm@kvack.org
Cc: michael.schwarz@iaik.tugraz.at
Cc: moritz.lipp@iaik.tugraz.at
Cc: richard.fellner@student.tugraz.at
Link: https://lkml.kernel.org/r/20171204150606.828111617@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 arch/x86/kernel/paravirt_patch_64.c |    2 --
 1 file changed, 2 deletions(-)

--- a/arch/x86/kernel/paravirt_patch_64.c
+++ b/arch/x86/kernel/paravirt_patch_64.c
@@ -10,7 +10,6 @@ DEF_NATIVE(pv_irq_ops, save_fl, "pushfq;
 DEF_NATIVE(pv_mmu_ops, read_cr2, "movq %cr2, %rax");
 DEF_NATIVE(pv_mmu_ops, read_cr3, "movq %cr3, %rax");
 DEF_NATIVE(pv_mmu_ops, write_cr3, "movq %rdi, %cr3");
-DEF_NATIVE(pv_mmu_ops, flush_tlb_single, "invlpg (%rdi)");
 DEF_NATIVE(pv_cpu_ops, wbinvd, "wbinvd");
 
 DEF_NATIVE(pv_cpu_ops, usergs_sysret64, "swapgs; sysretq");
@@ -60,7 +59,6 @@ unsigned native_patch(u8 type, u16 clobb
 		PATCH_SITE(pv_mmu_ops, read_cr2);
 		PATCH_SITE(pv_mmu_ops, read_cr3);
 		PATCH_SITE(pv_mmu_ops, write_cr3);
-		PATCH_SITE(pv_mmu_ops, flush_tlb_single);
 		PATCH_SITE(pv_cpu_ops, wbinvd);
 #if defined(CONFIG_PARAVIRT_SPINLOCKS)
 		case PARAVIRT_PATCH(pv_lock_ops.queued_spin_unlock):


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>