[PATCH 0/4] memblock related fixes for -tip

[PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

Please check memblock related patches

[PATCH 1/4] memblock: Fix big size with find_region()
[PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
[PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
[PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges

first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch

Thanks

Yinghai Lu

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/04/2010 02:57 PM, Yinghai Lu wrote:
> Please check memblock related patches
>
> [PATCH 1/4] memblock: Fix big size with find_region()
> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>
> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch

I also need this to avoid an early crash under Xen.  The Xen init code
ends up calling memblock_x86_reserve_range() before
x86_64_start_reservations().

Also in git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
memblock

    J

From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Date: Tue, 21 Sep 2010 17:05:35 -0700
Subject: [PATCH] memblock: allow memblock_init to be called early

The Xen setup code needs to call memblock_x86_reserve_range() very early,
so allow it to initialize the memblock subsystem before doing so.  The
second memblock_init() is ignored.

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 7d46c84..63b83ce 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -30,6 +30,7 @@
 #include <linux/console.h>
 #include <linux/pci.h>
 #include <linux/gfp.h>
+#include <linux/memblock.h>
 
 #include <xen/xen.h>
 #include <xen/interface/xen.h>
@@ -1183,6 +1184,8 @@ asmlinkage void __init xen_start_kernel(void)
 	local_irq_disable();
 	early_boot_irqs_off();
 
+	memblock_init();
+
 	xen_raw_console_write("mapping kernel into physical memory\n");
 	pgd = xen_setup_kernel_pagetable(pgd, xen_start_info->nr_pages);
 
diff --git a/mm/memblock.c b/mm/memblock.c
index 9ad3969..ae8b06c 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -752,6 +752,12 @@ void __init memblock_analyze(void)
 
 void __init memblock_init(void)
 {
+	static int init_done __initdata = 0;
+
+	if (init_done)
+		return;
+	init_done = 1;
+
 	/* Hookup the initial arrays */
 	memblock.memory.regions	= memblock_memory_init_regions;
 	memblock.memory.max		= INIT_MEMBLOCK_REGIONS;

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

On 10/06/2010 03:52 PM, Jeremy Fitzhardinge wrote:
>  On 10/04/2010 02:57 PM, Yinghai Lu wrote:
>> Please check memblock related patches
>>
>> [PATCH 1/4] memblock: Fix big size with find_region()
>> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
>> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
>> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>>
>> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch
> 
> I also need this to avoid an early crash under Xen.  The Xen init code
> ends up calling memblock_x86_reserve_range() before
> x86_64_start_reservations().
> 
> Also in git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
> memblock
> 
>     J
> 
> From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> Date: Tue, 21 Sep 2010 17:05:35 -0700
> Subject: [PATCH] memblock: allow memblock_init to be called early
> 
> The Xen setup code needs to call memblock_x86_reserve_range() very early,
> so allow it to initialize the memblock subsystem before doing so.  The
> second memblock_init() is ignored.
> 
> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> 
> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
> index 7d46c84..63b83ce 100644
> --- a/arch/x86/xen/enlighten.c
> +++ b/arch/x86/xen/enlighten.c
> @@ -30,6 +30,7 @@
>  #include <linux/console.h>
>  #include <linux/pci.h>
>  #include <linux/gfp.h>
> +#include <linux/memblock.h>
>  
>  #include <xen/xen.h>
>  #include <xen/interface/xen.h>
> @@ -1183,6 +1184,8 @@ asmlinkage void __init xen_start_kernel(void)
>  	local_irq_disable();
>  	early_boot_irqs_off();
>  
> +	memblock_init();
> +
>  	xen_raw_console_write("mapping kernel into physical memory\n");
>  	pgd = xen_setup_kernel_pagetable(pgd, xen_start_info->nr_pages);
>  
> diff --git a/mm/memblock.c b/mm/memblock.c
> index 9ad3969..ae8b06c 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -752,6 +752,12 @@ void __init memblock_analyze(void)
>  
>  void __init memblock_init(void)
>  {
> +	static int init_done __initdata = 0;
> +
> +	if (init_done)
> +		return;
> +	init_done = 1;
> +

How about
+	if (memblock.memory.regions)
+		return;	

>  	/* Hookup the initial arrays */
>  	memblock.memory.regions	= memblock_memory_init_regions;
>  	memblock.memory.max		= INIT_MEMBLOCK_REGIONS;
> 

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/06/2010 03:57 PM, Yinghai Lu wrote:
> 
> How about
> +	if (memblock.memory.regions)
> +		return;	
> 

I was just about to ask the same question. ;)

	-hpa

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/06/2010 03:59 PM, H. Peter Anvin wrote:
> On 10/06/2010 03:57 PM, Yinghai Lu wrote:
>> How about
>> +	if (memblock.memory.regions)
>> +		return;	
>>
> I was just about to ask the same question. ;)

I don't mind either way.  I haven't really looked at how memblock works,
so I wasn't sure if that was completely sufficient.

    J

[tip:core/memblock] memblock: Allow memblock_init to be called early

[tip-bot for Jeremy Fitzhardinge]

Commit-ID:  236260b90dd94516982ad67aa6f5449c4c37db7b
Gitweb:     http://git.kernel.org/tip/236260b90dd94516982ad67aa6f5449c4c37db7b
Author:     Jeremy Fitzhardinge <jeremy@goop.org>
AuthorDate: Wed, 6 Oct 2010 15:52:29 -0700
Committer:  H. Peter Anvin <hpa@linux.intel.com>
CommitDate: Mon, 11 Oct 2010 15:59:01 -0700

memblock: Allow memblock_init to be called early

The Xen setup code needs to call memblock_x86_reserve_range() very early,
so allow it to initialize the memblock subsystem before doing so.  The
second memblock_init() is ignored.

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
LKML-Reference: <4CACFDAD.3090900@goop.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
---
 arch/x86/xen/enlighten.c |    3 +++
 mm/memblock.c            |    6 ++++++
 2 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 7d46c84..63b83ce 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -30,6 +30,7 @@
 #include <linux/console.h>
 #include <linux/pci.h>
 #include <linux/gfp.h>
+#include <linux/memblock.h>
 
 #include <xen/xen.h>
 #include <xen/interface/xen.h>
@@ -1183,6 +1184,8 @@ asmlinkage void __init xen_start_kernel(void)
 	local_irq_disable();
 	early_boot_irqs_off();
 
+	memblock_init();
+
 	xen_raw_console_write("mapping kernel into physical memory\n");
 	pgd = xen_setup_kernel_pagetable(pgd, xen_start_info->nr_pages);
 
diff --git a/mm/memblock.c b/mm/memblock.c
index 9ad3969..ae8b06c 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -752,6 +752,12 @@ void __init memblock_analyze(void)
 
 void __init memblock_init(void)
 {
+	static int init_done __initdata = 0;
+
+	if (init_done)
+		return;
+	init_done = 1;
+
 	/* Hookup the initial arrays */
 	memblock.memory.regions	= memblock_memory_init_regions;
 	memblock.memory.max		= INIT_MEMBLOCK_REGIONS;

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/04/2010 02:57 PM, Yinghai Lu wrote:
> Please check memblock related patches
>
> [PATCH 1/4] memblock: Fix big size with find_region()
> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>
> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch


BTW, the memblock changes prevent the kernel from booting under Xen; it
crashes while setting up the linear maps.  I haven't worked out what's
failing yet, aside from bisecting it down to one of a9ce6bc151000 or
72d7c3b33c9808 (they don't compile in isolation so I had to skip them,
but both are likely looking, but unfortunately large, complex and hard
to further subdivide).

I'll look further into this, but just a heads-up for the upcoming merge
window.

    J

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

On 10/12/2010 11:41 AM, Jeremy Fitzhardinge wrote:
>  On 10/04/2010 02:57 PM, Yinghai Lu wrote:
>> Please check memblock related patches
>>
>> [PATCH 1/4] memblock: Fix big size with find_region()
>> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
>> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
>> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>>
>> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch
> 
> 
> BTW, the memblock changes prevent the kernel from booting under Xen; it
> crashes while setting up the linear maps.  I haven't worked out what's
> failing yet, aside from bisecting it down to one of a9ce6bc151000 or
> 72d7c3b33c9808 (they don't compile in isolation so I had to skip them,
> but both are likely looking, but unfortunately large, complex and hard
> to further subdivide).
> 
> I'll look further into this, but just a heads-up for the upcoming merge
> window.
> 

rebase memblock to today's linus tree. it should make your bisecting smoother.

please check

git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock

all fallout fix patches get merged into original commit.

Thanks

Yinghai


504fabf: memblock: Annotate memblock functions with __init_memblock
80c21ff: memblock: Allow memblock_init to be called early
9a319dd: x86, memblock: Remove __memblock_x86_find_in_range_size()
1be11b9: memblock: Fix wraparound in find_region()
11b9e2b: x86-32, memblock: Make add_highpages honor early reserved ranges
0bb390a: memblock: Fix section mismatch warnings
a982d98: x86: Remove old bootmem code
0a75ad2: x86, memblock: Use memblock_memory_size()/memblock_free_memory_size() to get correct dma_reserve
b1df2e5: x86: Remove not used early_res code
c074c30: x86, memblock: Replace e820_/_early string with memblock_
574489b: x86: Use memblock to replace early_res
b5eab27: x86, memblock: Use memblock_debug to control debug message print out
356c9ba: x86, memblock: Add memblock_x86_memory_in_range()
3f14258: x86, memblock: Add memblock_x86_free_memory_in_range()
cbddb3e: x86, memblock: Add memblock_x86_find_in_range_node()
5a28d45: memblock: Add find_memory_core_early()
ebfa1cb: x86, memblock: Add memblock_x86_register_active_regions() and memblock_x86_hole_size()
406d0e5: x86, memblock: Add get_free_all_memory_range()
192950e: x86, memblock: Add memblock_x86_reserve_range/memblock_x86_free_range
71b3bc5: x86, memblock: Add memblock_x86_to_bootmem()
c8bc172: bootmem, x86: Add weak version of reserve_bootmem_generic
2d05b56: x86, memblock: Add memblock_x86_find_in_range_size()
f3a812f: memblock: Add memblock_free/reserve_reserved_regions()
50d999b: memblock: Add memblock_find_in_range()
4e77e4b: memblock: Option for the architecture to put memblock into the .init section
0d54dbd: memblock: Protect memblock.h with CONFIG_HAVE_MEMBLOCK
3459158: memblock: Make MEMBLOCK_ERROR be 0
0b9d3f0: memblock: Export MEMBLOCK_ERROR
07df680: memblock: Improve debug output when resizing the reserve array
fbd2705: memblock: Expose some memblock bits for use by x86
76be0745: memblock: Add debugfs files to dump the arrays content
f48294e: memblock: Make memblock_alloc_try_nid() fallback to MEMBLOCK_ALLOC_ANYWHERE
52561a7: memblock: Separate memblock_alloc_nid() and memblock_alloc_try_nid()
73fcc82: memblock: NUMA allocate can now use early_pfn_map
b047c4b: memblock: Add "start" argument to memblock_find_base()
0dda32f: memblock: Add arch function to control coalescing of memblock memory regions
7cab753: memblock: Add array resizing support
f3a1d96: memblock: Move functions around into a more sensible order
9bc8683: memblock: split memblock_find_base() out of __memblock_alloc_base()
85676a1: memblock: Move memblock_init() to the bottom of the file
92876e4: memblock: Define MEMBLOCK_ERROR internally instead of using ~(phys_addr_t)0
f6f333e: memblock: Make memblock_find_region() out of memblock_alloc_region()
588a69a: memblock: Add debug markers at the end of the array
9638337: memblock: Move memblock arrays to static storage in memblock.c and make their size a variable
06d2939: memblock: Remove memblock_type.size and add memblock.memory_size instead
779bde3: memblock: Remove unused memblock.debug struct member
e8d828c: memblock: Change u64 to phys_addr_t
fa06948: memblock: Remove rmo_size, burry it in arch/powerpc where it belongs
615a990: memblock: Introduce default allocation limit and use it to replace explicit ones
666f73a: memblock: Expose MEMBLOCK_ALLOC_ANYWHERE
3d62472: memblock: Factor the lowest level alloc function
87de4bd: memblock: Remove nid_range argument, arch provides memblock_nid_range() instead
bc3f694: memblock: Remove memblock_find()
1741db7: memblock: Remove obsolete accessors
4193336: memblock/arm: Use new accessors
8a9f9fb: memblock/powerpc: Use new accessors
840def8: memblock/sparc: Use new accessors
3c77156: memblock/sh: Use new accessors
5f91db7: memblock/microblaze: Use new accessors
988660a: memblock: Introduce for_each_memblock() and new accessors
ca8a80d: memblock/arm: Use memblock_region_is_memory() for omap fb
92e9c55: memblock/arm: pfn_valid uses memblock_is_memory()
9d39b1a: memblock: Implement memblock_is_memory and memblock_is_region_memory
c7987d0: memblock: No reason to include asm/memblock.h late
c188b44: memblock: Rename memblock_region to memblock_type and memblock_property to memblock_region


 arch/arm/mm/init.c                       |   37 +-
 arch/arm/plat-omap/fb.c                  |    6 +-
 arch/microblaze/include/asm/memblock.h   |    3 -
 arch/microblaze/mm/init.c                |   30 +-
 arch/powerpc/include/asm/memblock.h      |    7 -
 arch/powerpc/include/asm/mmu.h           |   12 +
 arch/powerpc/kernel/head_40x.S           |    6 +-
 arch/powerpc/kernel/paca.c               |    2 +-
 arch/powerpc/kernel/prom.c               |   15 +-
 arch/powerpc/kernel/rtas.c               |    2 +-
 arch/powerpc/kernel/setup_32.c           |    2 +-
 arch/powerpc/kernel/setup_64.c           |    2 +-
 arch/powerpc/mm/40x_mmu.c                |   17 +-
 arch/powerpc/mm/44x_mmu.c                |   14 +
 arch/powerpc/mm/fsl_booke_mmu.c          |   12 +-
 arch/powerpc/mm/hash_utils_64.c          |   35 +-
 arch/powerpc/mm/init_32.c                |   43 +-
 arch/powerpc/mm/init_64.c                |    1 +
 arch/powerpc/mm/mem.c                    |   94 ++---
 arch/powerpc/mm/numa.c                   |   17 +-
 arch/powerpc/mm/ppc_mmu_32.c             |   18 +-
 arch/powerpc/mm/tlb_nohash.c             |   16 +
 arch/powerpc/platforms/embedded6xx/wii.c |    2 +-
 arch/sh/include/asm/memblock.h           |    2 -
 arch/sh/mm/init.c                        |   17 +-
 arch/sparc/include/asm/memblock.h        |    2 -
 arch/sparc/mm/init_64.c                  |   46 +-
 arch/x86/Kconfig                         |   15 +-
 arch/x86/include/asm/e820.h              |   20 +-
 arch/x86/include/asm/efi.h               |    2 +-
 arch/x86/include/asm/memblock.h          |   23 +
 arch/x86/kernel/acpi/sleep.c             |    9 +-
 arch/x86/kernel/apic/numaq_32.c          |    3 +-
 arch/x86/kernel/check.c                  |   16 +-
 arch/x86/kernel/e820.c                   |  191 ++-----
 arch/x86/kernel/efi.c                    |    5 +-
 arch/x86/kernel/head.c                   |    3 +-
 arch/x86/kernel/head32.c                 |   10 +-
 arch/x86/kernel/head64.c                 |    7 +-
 arch/x86/kernel/mpparse.c                |    5 +-
 arch/x86/kernel/setup.c                  |   87 ++--
 arch/x86/kernel/setup_percpu.c           |    6 -
 arch/x86/kernel/trampoline.c             |   10 +-
 arch/x86/mm/Makefile                     |    2 +
 arch/x86/mm/init.c                       |   10 +-
 arch/x86/mm/init_32.c                    |  119 +----
 arch/x86/mm/init_64.c                    |   67 +---
 arch/x86/mm/k8topology_64.c              |    4 +-
 arch/x86/mm/memblock.c                   |  348 +++++++++++++
 arch/x86/mm/memtest.c                    |    7 +-
 arch/x86/mm/numa_32.c                    |   30 +-
 arch/x86/mm/numa_64.c                    |   84 +---
 arch/x86/mm/srat_32.c                    |    3 +-
 arch/x86/mm/srat_64.c                    |   11 +-
 arch/x86/xen/enlighten.c                 |    3 +
 arch/x86/xen/mmu.c                       |    5 +-
 arch/x86/xen/setup.c                     |    3 +-
 drivers/video/omap2/vram.c               |    8 +-
 include/linux/early_res.h                |   23 -
 include/linux/memblock.h                 |  168 +++++--
 include/linux/mm.h                       |    2 +
 kernel/Makefile                          |    1 -
 kernel/early_res.c                       |  590 ---------------------
 mm/bootmem.c                             |   13 +-
 mm/memblock.c                            |  837 ++++++++++++++++++++----------
 mm/page_alloc.c                          |   86 ++--
 mm/sparse-vmemmap.c                      |   11 -
 67 files changed, 1613 insertions(+), 1694 deletions(-)
 create mode 100644 arch/x86/include/asm/memblock.h
 create mode 100644 arch/x86/mm/memblock.c
 delete mode 100644 include/linux/early_res.h
 delete mode 100644 kernel/early_res.c

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

On 10/12/2010 11:41 AM, Jeremy Fitzhardinge wrote:
>  On 10/04/2010 02:57 PM, Yinghai Lu wrote:
>> Please check memblock related patches
>>
>> [PATCH 1/4] memblock: Fix big size with find_region()
>> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
>> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
>> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>>
>> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch
> 
> 
> BTW, the memblock changes prevent the kernel from booting under Xen; it
> crashes while setting up the linear maps.  I haven't worked out what's
> failing yet, aside from bisecting it down to one of a9ce6bc151000 or
> 72d7c3b33c9808 (they don't compile in isolation so I had to skip them,
> but both are likely looking, but unfortunately large, complex and hard
> to further subdivide).
> 
> I'll look further into this, but just a heads-up for the upcoming merge
> window.
> 

please use 

git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock

to find which commit cause xen broken.

two xen related patch are folded into first converting commit.

Thanks


1e5f012: x86: Remove old bootmem code
f2c0394: x86, memblock: Use memblock_memory_size()/memblock_free_memory_size() to get correct dma_reserve
291e26f: x86: Remove not used early_res code
75cd1e9: x86, memblock: Replace e820_/_early string with memblock_
ce883cf: x86: Use memblock to replace early_res
c9d219e: x86, memblock: Add __get_free_all_memory_range()
b2d0276: x86, memblock: Use memblock_debug to control debug message print out
945eabb: x86, memblock: Add memblock_x86_memory_in_range()
9c6c086: x86, memblock: Add memblock_x86_free_memory_in_range()
2310711: x86, memblock: Add memblock_x86_find_in_range_node()
6b4ffee: memblock: Add find_memory_core_early()
1d168bf: x86, memblock: Add memblock_x86_register_active_regions() and memblock_x86_hole_size()
8d39571: x86, memblock: Add get_free_all_memory_range()
46d5782: x86, memblock: Add memblock_x86_reserve_range/memblock_x86_free_range
c6ade77: x86, memblock: Add memblock_x86_to_bootmem()
4a268b8: bootmem, x86: Add weak version of reserve_bootmem_generic
36d3afc: x86, memblock: Remove __memblock_x86_find_in_range_size()
b352f42: x86, memblock: Add memblock_x86_find_in_range_size()
a4c00b4: memblock: Allow memblock_init to be called early
c795896: memblock: Fix wraparound in find_region()
3ecfe12: memblock: Add memblock_free/reserve_reserved_regions()
17462d5: memblock: Add memblock_find_in_range()
1e833c5: memblock: Option for the architecture to put memblock into the .init section
f8266e2: memblock: Protect memblock.h with CONFIG_HAVE_MEMBLOCK
5b31325: memblock: Make MEMBLOCK_ERROR be 0
df92f4d: memblock: Export MEMBLOCK_ERROR
ddf01ef: memblock: Improve debug output when resizing the reserve array
3647f0b: memblock: Expose some memblock bits for use by x86
0526300: memblock: Add debugfs files to dump the arrays content
6fd15ab: memblock: Make memblock_alloc_try_nid() fallback to MEMBLOCK_ALLOC_ANYWHERE
ab3744c: memblock: Separate memblock_alloc_nid() and memblock_alloc_try_nid()
79263d1: memblock: NUMA allocate can now use early_pfn_map
fa21133: memblock: Add "start" argument to memblock_find_base()
d1d5099: memblock: Add arch function to control coalescing of memblock memory regions
2979e28: memblock: Add array resizing support
de17e60: memblock: Move functions around into a more sensible order
ac239c8: memblock: split memblock_find_base() out of __memblock_alloc_base()
763d5f3: memblock: Move memblock_init() to the bottom of the file
674f2ff: memblock: Define MEMBLOCK_ERROR internally instead of using ~(phys_addr_t)0
cf273c9: memblock: Make memblock_find_region() out of memblock_alloc_region()
5c2eb99: memblock: Add debug markers at the end of the array
68e694c: memblock: Move memblock arrays to static storage in memblock.c and make their size a variable
cbfff22: memblock: Remove memblock_type.size and add memblock.memory_size instead
9ae0df1: memblock: Remove unused memblock.debug struct member
564bb2b: memblock: Change u64 to phys_addr_t
e3c49f3: memblock: Remove rmo_size, burry it in arch/powerpc where it belongs
5f8a4aa: memblock: Introduce default allocation limit and use it to replace explicit ones
3608b50: memblock: Expose MEMBLOCK_ALLOC_ANYWHERE
676caa9: memblock: Factor the lowest level alloc function
19ad550: memblock: Remove nid_range argument, arch provides memblock_nid_range() instead
3dd4d87: memblock: Remove memblock_find()
01f4bff: memblock: Remove obsolete accessors
b01b184: memblock/arm: Use new accessors
6283930: memblock/powerpc: Use new accessors
51e6c6e: memblock/sparc: Use new accessors
7c1720c: memblock/sh: Use new accessors
4967561: memblock/microblaze: Use new accessors
cb3f592: memblock: Introduce for_each_memblock() and new accessors
8ccbd40: memblock/arm: Use memblock_region_is_memory() for omap fb
6617ae1: memblock/arm: pfn_valid uses memblock_is_memory()
b25a6d2: memblock: Implement memblock_is_memory and memblock_is_region_memory
2b77abd: memblock: No reason to include asm/memblock.h late
62c5c92: memblock: Rename memblock_region to memblock_type and memblock_property to memblock_region

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/12/2010 02:12 PM, Yinghai Lu wrote:
> On 10/12/2010 11:41 AM, Jeremy Fitzhardinge wrote:
>>  On 10/04/2010 02:57 PM, Yinghai Lu wrote:
>>> Please check memblock related patches
>>>
>>> [PATCH 1/4] memblock: Fix big size with find_region()
>>> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
>>> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
>>> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>>>
>>> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch
>>
>> BTW, the memblock changes prevent the kernel from booting under Xen; it
>> crashes while setting up the linear maps.  I haven't worked out what's
>> failing yet, aside from bisecting it down to one of a9ce6bc151000 or
>> 72d7c3b33c9808 (they don't compile in isolation so I had to skip them,
>> but both are likely looking, but unfortunately large, complex and hard
>> to further subdivide).
>>
>> I'll look further into this, but just a heads-up for the upcoming merge
>> window.
>>
> please use 
>
> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
>
> to find which commit cause xen broken.
>
> two xen related patch are folded into first converting commit.

Thanks, will do.

    J

> Thanks
>
>
> 1e5f012: x86: Remove old bootmem code
> f2c0394: x86, memblock: Use memblock_memory_size()/memblock_free_memory_size() to get correct dma_reserve
> 291e26f: x86: Remove not used early_res code
> 75cd1e9: x86, memblock: Replace e820_/_early string with memblock_
> ce883cf: x86: Use memblock to replace early_res
> c9d219e: x86, memblock: Add __get_free_all_memory_range()
> b2d0276: x86, memblock: Use memblock_debug to control debug message print out
> 945eabb: x86, memblock: Add memblock_x86_memory_in_range()
> 9c6c086: x86, memblock: Add memblock_x86_free_memory_in_range()
> 2310711: x86, memblock: Add memblock_x86_find_in_range_node()
> 6b4ffee: memblock: Add find_memory_core_early()
> 1d168bf: x86, memblock: Add memblock_x86_register_active_regions() and memblock_x86_hole_size()
> 8d39571: x86, memblock: Add get_free_all_memory_range()
> 46d5782: x86, memblock: Add memblock_x86_reserve_range/memblock_x86_free_range
> c6ade77: x86, memblock: Add memblock_x86_to_bootmem()
> 4a268b8: bootmem, x86: Add weak version of reserve_bootmem_generic
> 36d3afc: x86, memblock: Remove __memblock_x86_find_in_range_size()
> b352f42: x86, memblock: Add memblock_x86_find_in_range_size()
> a4c00b4: memblock: Allow memblock_init to be called early
> c795896: memblock: Fix wraparound in find_region()
> 3ecfe12: memblock: Add memblock_free/reserve_reserved_regions()
> 17462d5: memblock: Add memblock_find_in_range()
> 1e833c5: memblock: Option for the architecture to put memblock into the .init section
> f8266e2: memblock: Protect memblock.h with CONFIG_HAVE_MEMBLOCK
> 5b31325: memblock: Make MEMBLOCK_ERROR be 0
> df92f4d: memblock: Export MEMBLOCK_ERROR
> ddf01ef: memblock: Improve debug output when resizing the reserve array
> 3647f0b: memblock: Expose some memblock bits for use by x86
> 0526300: memblock: Add debugfs files to dump the arrays content
> 6fd15ab: memblock: Make memblock_alloc_try_nid() fallback to MEMBLOCK_ALLOC_ANYWHERE
> ab3744c: memblock: Separate memblock_alloc_nid() and memblock_alloc_try_nid()
> 79263d1: memblock: NUMA allocate can now use early_pfn_map
> fa21133: memblock: Add "start" argument to memblock_find_base()
> d1d5099: memblock: Add arch function to control coalescing of memblock memory regions
> 2979e28: memblock: Add array resizing support
> de17e60: memblock: Move functions around into a more sensible order
> ac239c8: memblock: split memblock_find_base() out of __memblock_alloc_base()
> 763d5f3: memblock: Move memblock_init() to the bottom of the file
> 674f2ff: memblock: Define MEMBLOCK_ERROR internally instead of using ~(phys_addr_t)0
> cf273c9: memblock: Make memblock_find_region() out of memblock_alloc_region()
> 5c2eb99: memblock: Add debug markers at the end of the array
> 68e694c: memblock: Move memblock arrays to static storage in memblock.c and make their size a variable
> cbfff22: memblock: Remove memblock_type.size and add memblock.memory_size instead
> 9ae0df1: memblock: Remove unused memblock.debug struct member
> 564bb2b: memblock: Change u64 to phys_addr_t
> e3c49f3: memblock: Remove rmo_size, burry it in arch/powerpc where it belongs
> 5f8a4aa: memblock: Introduce default allocation limit and use it to replace explicit ones
> 3608b50: memblock: Expose MEMBLOCK_ALLOC_ANYWHERE
> 676caa9: memblock: Factor the lowest level alloc function
> 19ad550: memblock: Remove nid_range argument, arch provides memblock_nid_range() instead
> 3dd4d87: memblock: Remove memblock_find()
> 01f4bff: memblock: Remove obsolete accessors
> b01b184: memblock/arm: Use new accessors
> 6283930: memblock/powerpc: Use new accessors
> 51e6c6e: memblock/sparc: Use new accessors
> 7c1720c: memblock/sh: Use new accessors
> 4967561: memblock/microblaze: Use new accessors
> cb3f592: memblock: Introduce for_each_memblock() and new accessors
> 8ccbd40: memblock/arm: Use memblock_region_is_memory() for omap fb
> 6617ae1: memblock/arm: pfn_valid uses memblock_is_memory()
> b25a6d2: memblock: Implement memblock_is_memory and memblock_is_region_memory
> 2b77abd: memblock: No reason to include asm/memblock.h late
> 62c5c92: memblock: Rename memblock_region to memblock_type and memblock_property to memblock_region
>

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/12/2010 02:12 PM, Yinghai Lu wrote:
> 
> please use 
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
> 
> to find which commit cause xen broken.
> 
> two xen related patch are folded into first converting commit.
> 

Do you have an incremental patch for this?

	-hpa

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/12/2010 02:42 PM, Jeremy Fitzhardinge wrote:
>>
>> two xen related patch are folded into first converting commit.
> 

Yinghai, is this the same bug as the ARM bug (which is assumed fixed by
the patch you sent me), or is this a different bug?

	-hpa

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

On 10/12/2010 02:42 PM, H. Peter Anvin wrote:
> On 10/12/2010 02:12 PM, Yinghai Lu wrote:
>>
>> please use 
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
>>
>> to find which commit cause xen broken.
>>
>> two xen related patch are folded into first converting commit.
>>
> 
> Do you have an incremental patch for this?

you have put those two patches in tip/memblock already.

rebasing the tree, so Jeremy could bisect the tree to root cause why xen still doesn't.

Thanks

Yinghai

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

On 10/12/2010 02:50 PM, H. Peter Anvin wrote:
> On 10/12/2010 02:42 PM, Jeremy Fitzhardinge wrote:
>>>
>>> two xen related patch are folded into first converting commit.
>>
> 
> Yinghai, is this the same bug as the ARM bug (which is assumed fixed by
> the patch you sent me), or is this a different bug?

should be different.

other memblock users still try to converting memblock to bootmem, so they need _pfn functions...

Yinghai

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/12/2010 03:01 PM, Yinghai Lu wrote:
> On 10/12/2010 02:42 PM, H. Peter Anvin wrote:
>> On 10/12/2010 02:12 PM, Yinghai Lu wrote:
>>>
>>> please use 
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
>>>
>>> to find which commit cause xen broken.
>>>
>>> two xen related patch are folded into first converting commit.
>>>
>>
>> Do you have an incremental patch for this?
> 
> you have put those two patches in tip/memblock already.
> 
> rebasing the tree, so Jeremy could bisect the tree to root cause why xen still doesn't.
> 

Oh I see, this is for debugging, and not as a fix.

	-hpa

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/12/2010 02:12 PM, Yinghai Lu wrote:
> On 10/12/2010 11:41 AM, Jeremy Fitzhardinge wrote:
>>  On 10/04/2010 02:57 PM, Yinghai Lu wrote:
>>> Please check memblock related patches
>>>
>>> [PATCH 1/4] memblock: Fix big size with find_region()
>>> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
>>> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
>>> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>>>
>>> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch
>>
>> BTW, the memblock changes prevent the kernel from booting under Xen; it
>> crashes while setting up the linear maps.  I haven't worked out what's
>> failing yet, aside from bisecting it down to one of a9ce6bc151000 or
>> 72d7c3b33c9808 (they don't compile in isolation so I had to skip them,
>> but both are likely looking, but unfortunately large, complex and hard
>> to further subdivide).
>>
>> I'll look further into this, but just a heads-up for the upcoming merge
>> window.
>>
> please use 
>
> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
>
> to find which commit cause xen broken.
>
> two xen related patch are folded into first converting commit.

Note that ce883cfc65c51e24 doesn't compile:

/home/jeremy/git/upstream/arch/x86/kernel/e820.c: In function
‘find_e820_area_size’:
/home/jeremy/git/upstream/arch/x86/kernel/e820.c:764: error: expected
‘;’ before ‘return’

However, I can fix it with the obvious fix, and the Xen boot failure
bisects to this change.

The specific crash is when constructing the initial pagetable, when Xen
tries to remap a newly allocated pagetable page read-only, and finds the
allocated page isn't mapped within the linear map. Since it is in the
middle of creating the linear map, I guess its quite possible that the
allocation order has changed, and it starts allocating pages which are
not yet mapped, whereas before it was allocating already-mapped ones.

I'll see if I can confirm this hypothesis and see if I can work around
it (is poses some problems, because it means that when the pages later
become mapped, I need to make sure they get mapped RO).

J

> Thanks
>
>
> 1e5f012: x86: Remove old bootmem code
> f2c0394: x86, memblock: Use memblock_memory_size()/memblock_free_memory_size() to get correct dma_reserve
> 291e26f: x86: Remove not used early_res code
> 75cd1e9: x86, memblock: Replace e820_/_early string with memblock_
> ce883cf: x86: Use memblock to replace early_res
> c9d219e: x86, memblock: Add __get_free_all_memory_range()
> b2d0276: x86, memblock: Use memblock_debug to control debug message print out
> 945eabb: x86, memblock: Add memblock_x86_memory_in_range()
> 9c6c086: x86, memblock: Add memblock_x86_free_memory_in_range()
> 2310711: x86, memblock: Add memblock_x86_find_in_range_node()
> 6b4ffee: memblock: Add find_memory_core_early()
> 1d168bf: x86, memblock: Add memblock_x86_register_active_regions() and memblock_x86_hole_size()
> 8d39571: x86, memblock: Add get_free_all_memory_range()
> 46d5782: x86, memblock: Add memblock_x86_reserve_range/memblock_x86_free_range
> c6ade77: x86, memblock: Add memblock_x86_to_bootmem()
> 4a268b8: bootmem, x86: Add weak version of reserve_bootmem_generic
> 36d3afc: x86, memblock: Remove __memblock_x86_find_in_range_size()
> b352f42: x86, memblock: Add memblock_x86_find_in_range_size()
> a4c00b4: memblock: Allow memblock_init to be called early
> c795896: memblock: Fix wraparound in find_region()
> 3ecfe12: memblock: Add memblock_free/reserve_reserved_regions()
> 17462d5: memblock: Add memblock_find_in_range()
> 1e833c5: memblock: Option for the architecture to put memblock into the .init section
> f8266e2: memblock: Protect memblock.h with CONFIG_HAVE_MEMBLOCK
> 5b31325: memblock: Make MEMBLOCK_ERROR be 0
> df92f4d: memblock: Export MEMBLOCK_ERROR
> ddf01ef: memblock: Improve debug output when resizing the reserve array
> 3647f0b: memblock: Expose some memblock bits for use by x86
> 0526300: memblock: Add debugfs files to dump the arrays content
> 6fd15ab: memblock: Make memblock_alloc_try_nid() fallback to MEMBLOCK_ALLOC_ANYWHERE
> ab3744c: memblock: Separate memblock_alloc_nid() and memblock_alloc_try_nid()
> 79263d1: memblock: NUMA allocate can now use early_pfn_map
> fa21133: memblock: Add "start" argument to memblock_find_base()
> d1d5099: memblock: Add arch function to control coalescing of memblock memory regions
> 2979e28: memblock: Add array resizing support
> de17e60: memblock: Move functions around into a more sensible order
> ac239c8: memblock: split memblock_find_base() out of __memblock_alloc_base()
> 763d5f3: memblock: Move memblock_init() to the bottom of the file
> 674f2ff: memblock: Define MEMBLOCK_ERROR internally instead of using ~(phys_addr_t)0
> cf273c9: memblock: Make memblock_find_region() out of memblock_alloc_region()
> 5c2eb99: memblock: Add debug markers at the end of the array
> 68e694c: memblock: Move memblock arrays to static storage in memblock.c and make their size a variable
> cbfff22: memblock: Remove memblock_type.size and add memblock.memory_size instead
> 9ae0df1: memblock: Remove unused memblock.debug struct member
> 564bb2b: memblock: Change u64 to phys_addr_t
> e3c49f3: memblock: Remove rmo_size, burry it in arch/powerpc where it belongs
> 5f8a4aa: memblock: Introduce default allocation limit and use it to replace explicit ones
> 3608b50: memblock: Expose MEMBLOCK_ALLOC_ANYWHERE
> 676caa9: memblock: Factor the lowest level alloc function
> 19ad550: memblock: Remove nid_range argument, arch provides memblock_nid_range() instead
> 3dd4d87: memblock: Remove memblock_find()
> 01f4bff: memblock: Remove obsolete accessors
> b01b184: memblock/arm: Use new accessors
> 6283930: memblock/powerpc: Use new accessors
> 51e6c6e: memblock/sparc: Use new accessors
> 7c1720c: memblock/sh: Use new accessors
> 4967561: memblock/microblaze: Use new accessors
> cb3f592: memblock: Introduce for_each_memblock() and new accessors
> 8ccbd40: memblock/arm: Use memblock_region_is_memory() for omap fb
> 6617ae1: memblock/arm: pfn_valid uses memblock_is_memory()
> b25a6d2: memblock: Implement memblock_is_memory and memblock_is_region_memory
> 2b77abd: memblock: No reason to include asm/memblock.h late
> 62c5c92: memblock: Rename memblock_region to memblock_type and memblock_property to memblock_region
>

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

On 10/12/2010 04:37 PM, Jeremy Fitzhardinge wrote:
>  On 10/12/2010 02:12 PM, Yinghai Lu wrote:
>> On 10/12/2010 11:41 AM, Jeremy Fitzhardinge wrote:
>>>  On 10/04/2010 02:57 PM, Yinghai Lu wrote:
>>>> Please check memblock related patches
>>>>
>>>> [PATCH 1/4] memblock: Fix big size with find_region()
>>>> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
>>>> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
>>>> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>>>>
>>>> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch
>>>
>>> BTW, the memblock changes prevent the kernel from booting under Xen; it
>>> crashes while setting up the linear maps.  I haven't worked out what's
>>> failing yet, aside from bisecting it down to one of a9ce6bc151000 or
>>> 72d7c3b33c9808 (they don't compile in isolation so I had to skip them,
>>> but both are likely looking, but unfortunately large, complex and hard
>>> to further subdivide).
>>>
>>> I'll look further into this, but just a heads-up for the upcoming merge
>>> window.
>>>
>> please use 
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
>>
>> to find which commit cause xen broken.
>>
>> two xen related patch are folded into first converting commit.
> 
> Note that ce883cfc65c51e24 doesn't compile:
> 
> /home/jeremy/git/upstream/arch/x86/kernel/e820.c: In function
> ‘find_e820_area_size’:
> /home/jeremy/git/upstream/arch/x86/kernel/e820.c:764: error: expected
> ‘;’ before ‘return’
> 
> However, I can fix it with the obvious fix, and the Xen boot failure
> bisects to this change.
> 
> The specific crash is when constructing the initial pagetable, when Xen
> tries to remap a newly allocated pagetable page read-only, and finds the
> allocated page isn't mapped within the linear map. Since it is in the
> middle of creating the linear map, I guess its quite possible that the
> allocation order has changed, and it starts allocating pages which are
> not yet mapped, whereas before it was allocating already-mapped ones.

you can limit the allocation of pagetable page further ...

xen doesn't honor max_pfn_mapped?

> 
> I'll see if I can confirm this hypothesis and see if I can work around
> it (is poses some problems, because it means that when the pages later
> become mapped, I need to make sure they get mapped RO).

please check following debug patch, it will try find area bottom up...

Subject: [PATCH] x86, memblock: Add x86 version of memblock_find_in_range()

Generic version is going from high to low, and it seems it can not find
right area compact enough.

the x86 version will go from goal to limit and just like the way We used
for early_res

use ARCH_FIND_MEMBLOCK_AREA to select from them.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/x86/Kconfig       |    8 +++++++
 arch/x86/mm/memblock.c |   54 +++++++++++++++++++++++++++++++++++++++++++++++++
 mm/memblock.c          |    2 -
 3 files changed, 63 insertions(+), 1 deletion(-)

Index: linux-2.6/arch/x86/mm/memblock.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/memblock.c
+++ linux-2.6/arch/x86/mm/memblock.c
@@ -346,3 +346,57 @@ u64 __init memblock_x86_hole_size(u64 st
 
 	return end - start - ((u64)ram << PAGE_SHIFT);
 }
+#ifdef CONFIG_ARCH_MEMBLOCK_FIND_AREA
+
+/* Check for already reserved areas */
+static inline bool __init check_with_memblock_reserved(u64 *addrp, u64 size, u64 align)
+{
+	u64 addr = *addrp;
+	bool changed = false;
+	struct memblock_region *r;
+again:
+	for_each_memblock(reserved, r) {
+		if ((addr + size) > r->base && addr < (r->base + r->size)) {
+			addr = round_up(r->base + r->size, align);
+			changed = true;
+			goto again;
+		}
+	}
+
+	if (changed)
+		*addrp = addr;
+
+	return changed;
+}
+
+/*
+ * Find a free area with specified alignment in a specific range from bottom up
+ */
+u64 __init memblock_find_in_range(u64 start, u64 end, u64 size, u64 align)
+{
+	struct memblock_region *r;
+
+	for_each_memblock(memory, r) {
+		u64 ei_start = r->base;
+		u64 ei_last = ei_start + r->size;
+		u64 addr, last;
+
+		addr = round_up(ei_start, align);
+		if (addr < start)
+			addr = round_up(start, align);
+		if (addr >= ei_last)
+			continue;
+		while (check_with_memblock_reserved(&addr, size, align) && addr+size <= ei_last)
+			;
+		last = addr + size;
+		if (last > ei_last)
+			continue;
+		if (last > end)
+			continue;
+
+		return addr;
+	}
+
+	return MEMBLOCK_ERROR;
+}
+#endif
Index: linux-2.6/arch/x86/Kconfig
===================================================================
--- linux-2.6.orig/arch/x86/Kconfig
+++ linux-2.6/arch/x86/Kconfig
@@ -542,6 +542,14 @@ config PARAVIRT_DEBUG
 	  Enable to debug paravirt_ops internals.  Specifically, BUG if
 	  a paravirt_op is missing when it is called.
 
+config ARCH_MEMBLOCK_FIND_AREA
+	default y
+	bool "Use x86 own memblock_find_in_range()"
+	---help---
+	  Use memblock_find_in_range() version instead of generic version, it get free
+	  area up from low.
+	  Generic one try to get free area down from limit.
+
 config NO_BOOTMEM
 	def_bool y
 
Index: linux-2.6/mm/memblock.c
===================================================================
--- linux-2.6.orig/mm/memblock.c
+++ linux-2.6/mm/memblock.c
@@ -170,7 +170,7 @@ static phys_addr_t __init_memblock membl
 /*
  * Find a free area with specified alignment in a specific range.
  */
-u64 __init_memblock memblock_find_in_range(u64 start, u64 end, u64 size, u64 align)
+u64 __init_memblock __weak memblock_find_in_range(u64 start, u64 end, u64 size, u64 align)
 {
 	return memblock_find_base(size, align, start, end);
 }

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/12/2010 10:40 PM, Yinghai Lu wrote:
> On 10/12/2010 04:37 PM, Jeremy Fitzhardinge wrote:
>>  On 10/12/2010 02:12 PM, Yinghai Lu wrote:
>>> On 10/12/2010 11:41 AM, Jeremy Fitzhardinge wrote:
>>>>  On 10/04/2010 02:57 PM, Yinghai Lu wrote:
>>>>> Please check memblock related patches
>>>>>
>>>>> [PATCH 1/4] memblock: Fix big size with find_region()
>>>>> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
>>>>> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
>>>>> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>>>>>
>>>>> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch
>>>> BTW, the memblock changes prevent the kernel from booting under Xen; it
>>>> crashes while setting up the linear maps.  I haven't worked out what's
>>>> failing yet, aside from bisecting it down to one of a9ce6bc151000 or
>>>> 72d7c3b33c9808 (they don't compile in isolation so I had to skip them,
>>>> but both are likely looking, but unfortunately large, complex and hard
>>>> to further subdivide).
>>>>
>>>> I'll look further into this, but just a heads-up for the upcoming merge
>>>> window.
>>>>
>>> please use 
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
>>>
>>> to find which commit cause xen broken.
>>>
>>> two xen related patch are folded into first converting commit.
>> Note that ce883cfc65c51e24 doesn't compile:
>>
>> /home/jeremy/git/upstream/arch/x86/kernel/e820.c: In function
>> ‘find_e820_area_size’:
>> /home/jeremy/git/upstream/arch/x86/kernel/e820.c:764: error: expected
>> ‘;’ before ‘return’
>>
>> However, I can fix it with the obvious fix, and the Xen boot failure
>> bisects to this change.
>>
>> The specific crash is when constructing the initial pagetable, when Xen
>> tries to remap a newly allocated pagetable page read-only, and finds the
>> allocated page isn't mapped within the linear map. Since it is in the
>> middle of creating the linear map, I guess its quite possible that the
>> allocation order has changed, and it starts allocating pages which are
>> not yet mapped, whereas before it was allocating already-mapped ones.
> you can limit the allocation of pagetable page further ...
>
> xen doesn't honor max_pfn_mapped?

What do you mean?  Xen sets max_pfn_mapped when it does the early
mapping of the kernel space, but I don't see how it relates to this
stage of mapping?

>> I'll see if I can confirm this hypothesis and see if I can work around
>> it (is poses some problems, because it means that when the pages later
>> become mapped, I need to make sure they get mapped RO).
> please check following debug patch, it will try find area bottom up...
>
> Subject: [PATCH] x86, memblock: Add x86 version of memblock_find_in_range()
>
> Generic version is going from high to low, and it seems it can not find
> right area compact enough.
>
> the x86 version will go from goal to limit and just like the way We used
> for early_res
>
> use ARCH_FIND_MEMBLOCK_AREA to select from them.

Thanks, that fixes the problem.  I would ideally like to make the the
Xen code independent of the page allocation ordering, but it looks like
it will be very tricky since we effectively make use of the pagetable as
a way of storing one bit of information about each page before there's a
struct page in place.

So this patch looks good to me (but there's no need to make it a
separate config option).

Thanks,
    J

> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  arch/x86/Kconfig       |    8 +++++++
>  arch/x86/mm/memblock.c |   54 +++++++++++++++++++++++++++++++++++++++++++++++++
>  mm/memblock.c          |    2 -
>  3 files changed, 63 insertions(+), 1 deletion(-)
>
> Index: linux-2.6/arch/x86/mm/memblock.c
> ===================================================================
> --- linux-2.6.orig/arch/x86/mm/memblock.c
> +++ linux-2.6/arch/x86/mm/memblock.c
> @@ -346,3 +346,57 @@ u64 __init memblock_x86_hole_size(u64 st
>  
>  	return end - start - ((u64)ram << PAGE_SHIFT);
>  }
> +#ifdef CONFIG_ARCH_MEMBLOCK_FIND_AREA
> +
> +/* Check for already reserved areas */
> +static inline bool __init check_with_memblock_reserved(u64 *addrp, u64 size, u64 align)
> +{
> +	u64 addr = *addrp;
> +	bool changed = false;
> +	struct memblock_region *r;
> +again:
> +	for_each_memblock(reserved, r) {
> +		if ((addr + size) > r->base && addr < (r->base + r->size)) {
> +			addr = round_up(r->base + r->size, align);
> +			changed = true;
> +			goto again;
> +		}
> +	}
> +
> +	if (changed)
> +		*addrp = addr;
> +
> +	return changed;
> +}
> +
> +/*
> + * Find a free area with specified alignment in a specific range from bottom up
> + */
> +u64 __init memblock_find_in_range(u64 start, u64 end, u64 size, u64 align)
> +{
> +	struct memblock_region *r;
> +
> +	for_each_memblock(memory, r) {
> +		u64 ei_start = r->base;
> +		u64 ei_last = ei_start + r->size;
> +		u64 addr, last;
> +
> +		addr = round_up(ei_start, align);
> +		if (addr < start)
> +			addr = round_up(start, align);
> +		if (addr >= ei_last)
> +			continue;
> +		while (check_with_memblock_reserved(&addr, size, align) && addr+size <= ei_last)
> +			;
> +		last = addr + size;
> +		if (last > ei_last)
> +			continue;
> +		if (last > end)
> +			continue;
> +
> +		return addr;
> +	}
> +
> +	return MEMBLOCK_ERROR;
> +}
> +#endif
> Index: linux-2.6/arch/x86/Kconfig
> ===================================================================
> --- linux-2.6.orig/arch/x86/Kconfig
> +++ linux-2.6/arch/x86/Kconfig
> @@ -542,6 +542,14 @@ config PARAVIRT_DEBUG
>  	  Enable to debug paravirt_ops internals.  Specifically, BUG if
>  	  a paravirt_op is missing when it is called.
>  
> +config ARCH_MEMBLOCK_FIND_AREA
> +	default y
> +	bool "Use x86 own memblock_find_in_range()"
> +	---help---
> +	  Use memblock_find_in_range() version instead of generic version, it get free
> +	  area up from low.
> +	  Generic one try to get free area down from limit.
> +
>  config NO_BOOTMEM
>  	def_bool y
>  
> Index: linux-2.6/mm/memblock.c
> ===================================================================
> --- linux-2.6.orig/mm/memblock.c
> +++ linux-2.6/mm/memblock.c
> @@ -170,7 +170,7 @@ static phys_addr_t __init_memblock membl
>  /*
>   * Find a free area with specified alignment in a specific range.
>   */
> -u64 __init_memblock memblock_find_in_range(u64 start, u64 end, u64 size, u64 align)
> +u64 __init_memblock __weak memblock_find_in_range(u64 start, u64 end, u64 size, u64 align)
>  {
>  	return memblock_find_base(size, align, start, end);
>  }
>

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

On 10/13/2010 09:31 AM, Jeremy Fitzhardinge wrote:
>  On 10/12/2010 10:40 PM, Yinghai Lu wrote:
>> On 10/12/2010 04:37 PM, Jeremy Fitzhardinge wrote:
>>>  On 10/12/2010 02:12 PM, Yinghai Lu wrote:
>>>> On 10/12/2010 11:41 AM, Jeremy Fitzhardinge wrote:
>>>>>  On 10/04/2010 02:57 PM, Yinghai Lu wrote:
>>>>>> Please check memblock related patches
>>>>>>
>>>>>> [PATCH 1/4] memblock: Fix big size with find_region()
>>>>>> [PATCH -v5 2/4] x86, memblock: Fix crashkernel allocation
>>>>>> [PATCH 3/4] x86, memblock: Remove __memblock_x86_find_in_range_size()
>>>>>> [PATCH 4/4] x86, mm, memblock, 32bit: Make add_highpages honor early reserved ranges
>>>>>>
>>>>>> first one should get into core/memblock branch, and others should be in x86/mm/memeblock branch
>>>>> BTW, the memblock changes prevent the kernel from booting under Xen; it
>>>>> crashes while setting up the linear maps.  I haven't worked out what's
>>>>> failing yet, aside from bisecting it down to one of a9ce6bc151000 or
>>>>> 72d7c3b33c9808 (they don't compile in isolation so I had to skip them,
>>>>> but both are likely looking, but unfortunately large, complex and hard
>>>>> to further subdivide).
>>>>>
>>>>> I'll look further into this, but just a heads-up for the upcoming merge
>>>>> window.
>>>>>
>>>> please use 
>>>>
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
>>>>
>>>> to find which commit cause xen broken.
>>>>
>>>> two xen related patch are folded into first converting commit.
>>> Note that ce883cfc65c51e24 doesn't compile:
>>>
>>> /home/jeremy/git/upstream/arch/x86/kernel/e820.c: In function
>>> ‘find_e820_area_size’:
>>> /home/jeremy/git/upstream/arch/x86/kernel/e820.c:764: error: expected
>>> ‘;’ before ‘return’
>>>
>>> However, I can fix it with the obvious fix, and the Xen boot failure
>>> bisects to this change.
>>>
>>> The specific crash is when constructing the initial pagetable, when Xen
>>> tries to remap a newly allocated pagetable page read-only, and finds the
>>> allocated page isn't mapped within the linear map. Since it is in the
>>> middle of creating the linear map, I guess its quite possible that the
>>> allocation order has changed, and it starts allocating pages which are
>>> not yet mapped, whereas before it was allocating already-mapped ones.
>> you can limit the allocation of pagetable page further ...
>>
>> xen doesn't honor max_pfn_mapped?
> 
> What do you mean?  Xen sets max_pfn_mapped when it does the early
> mapping of the kernel space, but I don't see how it relates to this
> stage of mapping?

We are using max_pfn_mapped to upper limit to find range that can be used as early stage.

so maybe you have max_pfn_mapped set too early and too big?

> 
>>> I'll see if I can confirm this hypothesis and see if I can work around
>>> it (is poses some problems, because it means that when the pages later
>>> become mapped, I need to make sure they get mapped RO).
>> please check following debug patch, it will try find area bottom up...
>>
>> Subject: [PATCH] x86, memblock: Add x86 version of memblock_find_in_range()
>>
>> Generic version is going from high to low, and it seems it can not find
>> right area compact enough.
>>
>> the x86 version will go from goal to limit and just like the way We used
>> for early_res
>>
>> use ARCH_FIND_MEMBLOCK_AREA to select from them.
> 
> Thanks, that fixes the problem.  I would ideally like to make the the
> Xen code independent of the page allocation ordering, but it looks like
> it will be very tricky since we effectively make use of the pagetable as
> a way of storing one bit of information about each page before there's a
> struct page in place.
> 
> So this patch looks good to me (but there's no need to make it a
> separate config option).

that is for debug purpose only. We really want to do top-down than bottom-up.

Thanks

Yinghai

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/13/2010 09:31 AM, Jeremy Fitzhardinge wrote:
>>
>> use ARCH_FIND_MEMBLOCK_AREA to select from them.
> 
> Thanks, that fixes the problem.  I would ideally like to make the the
> Xen code independent of the page allocation ordering, but it looks like
> it will be very tricky since we effectively make use of the pagetable as
> a way of storing one bit of information about each page before there's a
> struct page in place.
> 
> So this patch looks good to me (but there's no need to make it a
> separate config option).
> 

There isn't per se, but I have repeatedly expressed unhappiness about
x86 having a completely different allocation policy -- worse, bottom-up
is the absolutely worst possible allocation policy since low-address
memory is a precious resource for all kinds of odd requirements
(trampoline pages, ZONE_DMA, ZONE_DMA32 and so on.)

Furthermore, I really, really disapprove of interfaces which carry
hidden semantics, such as allocation order.

I have repeatedly asked that we do *not* do this on x86 if we're going
to go to a memblock-everywhere configuration.

Now, if Xen needs it, there are few options that I can see in the short
term, neither of which makes me happy -- I would appreciate

a) Add an explicit interface to allocate bottoms-up, and have Xen use it
because it needs it.  This is appropriate if (and only if) the
allocations in Xen aren't underneath a bunch of extra layers.

b) Add it as a config option and make Xen select it (or depend on it).
Unfortunately this probably would mean using this option on any
paravirtualized kernel too, which would effectively mean Xen is
"infecting" the entire rest of the x86 architecture with a pessimal
allocation policy, which is extremely unfortunate.

c) Just accept it for now with the intent of getting rid of it as soon
as possible.  I'd be fine pushing this for 2.6.37, but I'd like to get a
reasonably firm commitment try to come up with something better within
the next kernel cycle.

Opinions?

	-hpa

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/13/2010 11:20 AM, H. Peter Anvin wrote:
> On 10/13/2010 09:31 AM, Jeremy Fitzhardinge wrote:
>>> use ARCH_FIND_MEMBLOCK_AREA to select from them.
>> Thanks, that fixes the problem.  I would ideally like to make the the
>> Xen code independent of the page allocation ordering, but it looks like
>> it will be very tricky since we effectively make use of the pagetable as
>> a way of storing one bit of information about each page before there's a
>> struct page in place.
>>
>> So this patch looks good to me (but there's no need to make it a
>> separate config option).
>>
> There isn't per se, but I have repeatedly expressed unhappiness about
> x86 having a completely different allocation policy -- worse, bottom-up
> is the absolutely worst possible allocation policy since low-address
> memory is a precious resource for all kinds of odd requirements
> (trampoline pages, ZONE_DMA, ZONE_DMA32 and so on.)
>
> Furthermore, I really, really disapprove of interfaces which carry
> hidden semantics, such as allocation order.

Me too.  I'd like to fix the Xen code to handle any pages.

> I have repeatedly asked that we do *not* do this on x86 if we're going
> to go to a memblock-everywhere configuration.
>
> Now, if Xen needs it, there are few options that I can see in the short
> term, neither of which makes me happy -- I would appreciate
>
> a) Add an explicit interface to allocate bottoms-up, and have Xen use it
> because it needs it.  This is appropriate if (and only if) the
> allocations in Xen aren't underneath a bunch of extra layers.

The allocation is done in find_early_table_space() in x86/mm/init.c, so
the allocation call itself can't be easily replaced, but I suppose some
of the parameters could be global and tweaked by Xen code, but that's
pretty ugly.

> c) Just accept it for now with the intent of getting rid of it as soon
> as possible.  I'd be fine pushing this for 2.6.37, but I'd like to get a
> reasonably firm commitment try to come up with something better within
> the next kernel cycle.
>
> Opinions?

I'm looking at ways of avoiding the dependency on bottom-up allocation
at the moment.  It looks like it's OK if I can assume that the pagetable
is being allocated out of pages in the range
e820_table_start-e820_table_end (though one hopes those will get renamed
to something a bit more meaningful).

I should know how its going to turn out later today.

    J

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/13/2010 01:03 PM, Jeremy Fitzhardinge wrote:
> 
> I'm looking at ways of avoiding the dependency on bottom-up allocation
> at the moment.  It looks like it's OK if I can assume that the pagetable
> is being allocated out of pages in the range
> e820_table_start-e820_table_end (though one hopes those will get renamed
> to something a bit more meaningful).
> 
> I should know how its going to turn out later today.
> 

Thanks, I appreciate it.

	-hpa

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai Lu]

On 10/13/2010 01:03 PM, Jeremy Fitzhardinge wrote:
>> a) Add an explicit interface to allocate bottoms-up, and have Xen use it
>> because it needs it.  This is appropriate if (and only if) the
>> allocations in Xen aren't underneath a bunch of extra layers.
> 
> The allocation is done in find_early_table_space() in x86/mm/init.c, so
> the allocation call itself can't be easily replaced, but I suppose some
> of the parameters could be global and tweaked by Xen code, but that's
> pretty ugly.

we already have
        base = memblock_find_in_range(start, max_pfn_mapped<<PAGE_SHIFT,
                                        tables, PAGE_SIZE);

so looks like xen have set max_pfn_mapped wrong?


Yinghai

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/13/2010 02:03 PM, H. Peter Anvin wrote:
> On 10/13/2010 01:03 PM, Jeremy Fitzhardinge wrote:
>> I'm looking at ways of avoiding the dependency on bottom-up allocation
>> at the moment.  It looks like it's OK if I can assume that the pagetable
>> is being allocated out of pages in the range
>> e820_table_start-e820_table_end (though one hopes those will get renamed
>> to something a bit more meaningful).
>>
>> I should know how its going to turn out later today.
>>
> Thanks, I appreciate it.

This seems to do the trick:

From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Date: Wed, 13 Oct 2010 14:21:55 -0700
Subject: [PATCH] xen: cope with unmapped pages when initializing kernel pagetable

Xen requires that all pages containing pagetable entries to be mapped
read-only.  If pages used for the initial pagetable are already mapped
then we can change the mapping to RO.  However, if they are initially
unmapped, we need to make sure that when they are later mapped, they
are also mapped RO.

We do this by knowing that the kernel pagetable memory is pre-allocated
in the range e820_table_start - e820_table_end, so any pfn within this
range should be mapped read-only.  However, the pagetable setup code
early_ioremaps the pages to write their entries, so we must make sure
that mappings created in the early_ioremap fixmap area are mapped RW.
(Those mappings are removed before the pages are presented to Xen
as pagetable pages.)

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 30a3e97..66aee6c 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -348,6 +348,7 @@ extern void __iomem *early_memremap(resource_size_t phys_addr,
 				    unsigned long size);
 extern void early_iounmap(void __iomem *addr, unsigned long size);
 extern void fixup_early_ioremap(void);
+extern bool is_early_ioremap_ptep(pte_t *ptep);
 
 #define IO_SPACE_LIMIT 0xffff
 
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 3ba6e06..0369843 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -362,6 +362,11 @@ static inline pte_t * __init early_ioremap_pte(unsigned long addr)
 	return &bm_pte[pte_index(addr)];
 }
 
+bool __init is_early_ioremap_ptep(pte_t *ptep)
+{
+	return ptep >= &bm_pte[0] && ptep < &bm_pte[PAGE_SIZE/sizeof(pte_t)];
+}
+
 static unsigned long slot_virt[FIX_BTMAPS_SLOTS] __initdata;
 
 void __init early_ioremap_init(void)
diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 4fe04ac..7d55e9e 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -56,6 +56,7 @@
 #include <asm/e820.h>
 #include <asm/linkage.h>
 #include <asm/page.h>
+#include <asm/init.h>
 
 #include <asm/xen/hypercall.h>
 #include <asm/xen/hypervisor.h>
@@ -360,7 +361,8 @@ void make_lowmem_page_readonly(void *vaddr)
 	unsigned int level;
 
 	pte = lookup_address(address, &level);
-	BUG_ON(pte == NULL);
+	if (pte == NULL)
+		return;		/* vaddr missing */
 
 	ptev = pte_wrprotect(*pte);
 
@@ -375,7 +377,8 @@ void make_lowmem_page_readwrite(void *vaddr)
 	unsigned int level;
 
 	pte = lookup_address(address, &level);
-	BUG_ON(pte == NULL);
+	if (pte == NULL)
+		return;		/* vaddr missing */
 
 	ptev = pte_mkwrite(*pte);
 
@@ -1509,13 +1512,25 @@ static void xen_pgd_free(struct mm_struct *mm, pgd_t *pgd)
 #endif
 }
 
-#ifdef CONFIG_X86_32
 static __init pte_t mask_rw_pte(pte_t *ptep, pte_t pte)
 {
+	unsigned long pfn = pte_pfn(pte);
+
+#ifdef CONFIG_X86_32
 	/* If there's an existing pte, then don't allow _PAGE_RW to be set */
 	if (pte_val_ma(*ptep) & _PAGE_PRESENT)
 		pte = __pte_ma(((pte_val_ma(*ptep) & _PAGE_RW) | ~_PAGE_RW) &
 			       pte_val_ma(pte));
+#endif
+
+	/*
+	 * If the new pfn is within the range of the newly allocated
+	 * kernel pagetable, and it isn't being mapped into an
+	 * early_ioremap fixmap slot, make sure it is RO.
+	 */
+	if (!is_early_ioremap_ptep(ptep) &&
+	    pfn >= e820_table_start && pfn < e820_table_end)
+		pte = pte_wrprotect(pte);
 
 	return pte;
 }
@@ -1528,7 +1543,6 @@ static __init void xen_set_pte_init(pte_t *ptep, pte_t pte)
 
 	xen_set_pte(ptep, pte);
 }
-#endif
 
 static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
 {
@@ -1973,11 +1987,7 @@ static const struct pv_mmu_ops xen_mmu_ops __initdata = {
 	.alloc_pmd_clone = paravirt_nop,
 	.release_pmd = xen_release_pmd_init,
 
-#ifdef CONFIG_X86_64
-	.set_pte = xen_set_pte,
-#else
 	.set_pte = xen_set_pte_init,
-#endif
 	.set_pte_at = xen_set_pte_at,
 	.set_pmd = xen_set_pmd_hyper,
 

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/13/2010 03:06 PM, Yinghai Lu wrote:
> On 10/13/2010 01:03 PM, Jeremy Fitzhardinge wrote:
>>> a) Add an explicit interface to allocate bottoms-up, and have Xen use it
>>> because it needs it.  This is appropriate if (and only if) the
>>> allocations in Xen aren't underneath a bunch of extra layers.
>> The allocation is done in find_early_table_space() in x86/mm/init.c, so
>> the allocation call itself can't be easily replaced, but I suppose some
>> of the parameters could be global and tweaked by Xen code, but that's
>> pretty ugly.
> we already have
>         base = memblock_find_in_range(start, max_pfn_mapped<<PAGE_SHIFT,
>                                         tables, PAGE_SIZE);
>
> so looks like xen have set max_pfn_mapped wrong?

Hm, I'll double check.  But the x86-64 code, at least, doesn't seem to
care whether the allocated memory is mapped or not, since it does
early_ioremap on each page before initializing it anyway.

    J

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/13/2010 04:02 PM, Jeremy Fitzhardinge wrote:
> 
> This seems to do the trick:
> 
> From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> Date: Wed, 13 Oct 2010 14:21:55 -0700
> Subject: [PATCH] xen: cope with unmapped pages when initializing kernel pagetable
> 
> Xen requires that all pages containing pagetable entries to be mapped
> read-only.  If pages used for the initial pagetable are already mapped
> then we can change the mapping to RO.  However, if they are initially
> unmapped, we need to make sure that when they are later mapped, they
> are also mapped RO.
> 
> We do this by knowing that the kernel pagetable memory is pre-allocated
> in the range e820_table_start - e820_table_end, so any pfn within this
> range should be mapped read-only.  However, the pagetable setup code
> early_ioremaps the pages to write their entries, so we must make sure
> that mappings created in the early_ioremap fixmap area are mapped RW.
> (Those mappings are removed before the pages are presented to Xen
> as pagetable pages.)
> 
> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> 

Seems both clean and The Right Thing[TM].

	-hpa

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/13/2010 03:06 PM, Yinghai Lu wrote:
> On 10/13/2010 01:03 PM, Jeremy Fitzhardinge wrote:
>>> a) Add an explicit interface to allocate bottoms-up, and have Xen use it
>>> because it needs it.  This is appropriate if (and only if) the
>>> allocations in Xen aren't underneath a bunch of extra layers.
>> The allocation is done in find_early_table_space() in x86/mm/init.c, so
>> the allocation call itself can't be easily replaced, but I suppose some
>> of the parameters could be global and tweaked by Xen code, but that's
>> pretty ugly.
> we already have
>         base = memblock_find_in_range(start, max_pfn_mapped<<PAGE_SHIFT,
>                                         tables, PAGE_SIZE);
>
> so looks like xen have set max_pfn_mapped wrong?

The Xen code is setting the max_pfn_mapped correctly, but it is just
being overridden by:

	max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;

in setup_arch() - and KERNEL_IMAGE_SIZE is hard-coded to 512MB...

How is this correct?  Does kernel/head_64.S map everything up to 512MB
or something?

    J

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/13/2010 04:14 PM, Jeremy Fitzhardinge wrote:
> 
> The Xen code is setting the max_pfn_mapped correctly, but it is just
> being overridden by:
> 
> 	max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
> 
> in setup_arch() - and KERNEL_IMAGE_SIZE is hard-coded to 512MB...
> 
> How is this correct?  Does kernel/head_64.S map everything up to 512MB
> or something?
> 

Yes:
        /*
         * 512 MB kernel mapping. We spend a full page on this pagetable
         * anyway.
         *
         * The kernel code+data+bss must not be bigger than that.
         *
         * (NOTE: at +512MB starts the module area, see MODULES_VADDR.
         *  If you want to increase this then increase MODULES_VADDR
         *  too.)
         */
        PMDS(0, __PAGE_KERNEL_LARGE_EXEC,
                KERNEL_IMAGE_SIZE/PMD_SIZE)

This is, however, wrong in the sense that it obviously shouldn't be
getting executed on the Xen codepath.

	-hpa

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/13/2010 04:18 PM, H. Peter Anvin wrote:
> On 10/13/2010 04:14 PM, Jeremy Fitzhardinge wrote:
>> The Xen code is setting the max_pfn_mapped correctly, but it is just
>> being overridden by:
>>
>> 	max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
>>
>> in setup_arch() - and KERNEL_IMAGE_SIZE is hard-coded to 512MB...
>>
>> How is this correct?  Does kernel/head_64.S map everything up to 512MB
>> or something?
>>
> Yes:
>         /*
>          * 512 MB kernel mapping. We spend a full page on this pagetable
>          * anyway.
>          *
>          * The kernel code+data+bss must not be bigger than that.
>          *
>          * (NOTE: at +512MB starts the module area, see MODULES_VADDR.
>          *  If you want to increase this then increase MODULES_VADDR
>          *  too.)
>          */
>         PMDS(0, __PAGE_KERNEL_LARGE_EXEC,
>                 KERNEL_IMAGE_SIZE/PMD_SIZE)
>
> This is, however, wrong in the sense that it obviously shouldn't be
> getting executed on the Xen codepath.

I see.  head_32.S sets max_pfn_mapped for itself, so head_64.S should
probably do that too.  Patch below, but I haven't checked to see if it
fixes the original problem (but I don't see why the pagetable memory
allocation should be limited to the mapped region, especially if you
want to put it as high as possible, especially since the mm/init_64.c
code doesn't expect it to be mapped).

    J

From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Date: Wed, 13 Oct 2010 16:31:11 -0700
Subject: [PATCH] x86-64: don't force max_pfn_mapped to 512MB

head_64.S maps up to 512MB, but the Xen entry path doesn't necessarily
map that high.  Set max_pfn_mapped in head_64.S accordingly.  (The Xen
code already sets max_pfn_mapped correctly.)

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 239046b..b75c6f4 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -141,6 +141,8 @@ ident_complete:
 	addq	%rbp, trampoline_level4_pgt + (511*8)(%rip)
 #endif
 
+	movq    $KERNEL_IMAGE_SIZE / PAGE_SIZE, max_pfn_mapped(%rip)
+
 	/* Due to ENTRY(), sometimes the empty space gets filled with
 	 * zeros. Better take a jmp than relying on empty space being
 	 * filled with 0x90 (nop)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index b11a238..c3cebfe 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -932,7 +932,6 @@ void __init setup_arch(char **cmdline_p)
 		max_low_pfn = max_pfn;
 
 	high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1;
-	max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
 #endif
 
 	/*

Re: [PATCH 0/4] memblock related fixes for -tip

[Yinghai]

On Oct 13, 2010, at 4:34 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:

> On 10/13/2010 04:18 PM, H. Peter Anvin wrote:
>> On 10/13/2010 04:14 PM, Jeremy Fitzhardinge wrote:
>>> The Xen code is setting the max_pfn_mapped correctly, but it is just
>>> being overridden by:
>>> 
>>>    max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
>>> 
>>> in setup_arch() - and KERNEL_IMAGE_SIZE is hard-coded to 512MB...
>>> 
>>> How is this correct?  Does kernel/head_64.S map everything up to 512MB
>>> or something?
>>> 
>> Yes:
>>        /*
>>         * 512 MB kernel mapping. We spend a full page on this pagetable
>>         * anyway.
>>         *
>>         * The kernel code+data+bss must not be bigger than that.
>>         *
>>         * (NOTE: at +512MB starts the module area, see MODULES_VADDR.
>>         *  If you want to increase this then increase MODULES_VADDR
>>         *  too.)
>>         */
>>        PMDS(0, __PAGE_KERNEL_LARGE_EXEC,
>>                KERNEL_IMAGE_SIZE/PMD_SIZE)
>> 
>> This is, however, wrong in the sense that it obviously shouldn't be
>> getting executed on the Xen codepath.
> 
> I see.  head_32.S sets max_pfn_mapped for itself, so head_64.S should
> probably do that too.  Patch below, but I haven't checked to see if it
> fixes the original problem (but I don't see why the pagetable memory
> allocation should be limited to the mapped region, especially if you
> want to put it as high as possible, especially since the mm/init_64.c
> code doesn't expect it to be mapped).
> 
>    J
> 
> From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> Date: Wed, 13 Oct 2010 16:31:11 -0700
> Subject: [PATCH] x86-64: don't force max_pfn_mapped to 512MB
> 
> head_64.S maps up to 512MB, but the Xen entry path doesn't necessarily
> map that high.  Set max_pfn_mapped in head_64.S accordingly.  (The Xen
> code already sets max_pfn_mapped correctly.)
> 
> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> 
> diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
> index 239046b..b75c6f4 100644
> --- a/arch/x86/kernel/head_64.S
> +++ b/arch/x86/kernel/head_64.S
> @@ -141,6 +141,8 @@ ident_complete:
>    addq    %rbp, trampoline_level4_pgt + (511*8)(%rip)
> #endif
> 
> +    movq    $KERNEL_IMAGE_SIZE / PAGE_SIZE, max_pfn_mapped(%rip)
> +
>    /* Due to ENTRY(), sometimes the empty space gets filled with
>     * zeros. Better take a jmp than relying on empty space being
>     * filled with 0x90 (nop)
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index b11a238..c3cebfe 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -932,7 +932,6 @@ void __init setup_arch(char **cmdline_p)
>        max_low_pfn = max_pfn;
> 
>    high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1;
> -    max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
> #endif
> 
>    /*
> 

I like this one, can you check if only this one is needed ?

Thanks

Yinghai

Re: [PATCH 0/4] memblock related fixes for -tip

[Jeremy Fitzhardinge]

 On 10/13/2010 05:08 PM, Yinghai wrote:
>
> On Oct 13, 2010, at 4:34 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>
>> On 10/13/2010 04:18 PM, H. Peter Anvin wrote:
>>> On 10/13/2010 04:14 PM, Jeremy Fitzhardinge wrote:
>>>> The Xen code is setting the max_pfn_mapped correctly, but it is just
>>>> being overridden by:
>>>>
>>>>    max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
>>>>
>>>> in setup_arch() - and KERNEL_IMAGE_SIZE is hard-coded to 512MB...
>>>>
>>>> How is this correct?  Does kernel/head_64.S map everything up to 512MB
>>>> or something?
>>>>
>>> Yes:
>>>        /*
>>>         * 512 MB kernel mapping. We spend a full page on this pagetable
>>>         * anyway.
>>>         *
>>>         * The kernel code+data+bss must not be bigger than that.
>>>         *
>>>         * (NOTE: at +512MB starts the module area, see MODULES_VADDR.
>>>         *  If you want to increase this then increase MODULES_VADDR
>>>         *  too.)
>>>         */
>>>        PMDS(0, __PAGE_KERNEL_LARGE_EXEC,
>>>                KERNEL_IMAGE_SIZE/PMD_SIZE)
>>>
>>> This is, however, wrong in the sense that it obviously shouldn't be
>>> getting executed on the Xen codepath.
>> I see.  head_32.S sets max_pfn_mapped for itself, so head_64.S should
>> probably do that too.  Patch below, but I haven't checked to see if it
>> fixes the original problem (but I don't see why the pagetable memory
>> allocation should be limited to the mapped region, especially if you
>> want to put it as high as possible, especially since the mm/init_64.c
>> code doesn't expect it to be mapped).
>>
>>    J
>>
>> From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
>> Date: Wed, 13 Oct 2010 16:31:11 -0700
>> Subject: [PATCH] x86-64: don't force max_pfn_mapped to 512MB
>>
>> head_64.S maps up to 512MB, but the Xen entry path doesn't necessarily
>> map that high.  Set max_pfn_mapped in head_64.S accordingly.  (The Xen
>> code already sets max_pfn_mapped correctly.)
>>
>> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
>>
>> diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
>> index 239046b..b75c6f4 100644
>> --- a/arch/x86/kernel/head_64.S
>> +++ b/arch/x86/kernel/head_64.S
>> @@ -141,6 +141,8 @@ ident_complete:
>>    addq    %rbp, trampoline_level4_pgt + (511*8)(%rip)
>> #endif
>>
>> +    movq    $KERNEL_IMAGE_SIZE / PAGE_SIZE, max_pfn_mapped(%rip)
>> +
>>    /* Due to ENTRY(), sometimes the empty space gets filled with
>>     * zeros. Better take a jmp than relying on empty space being
>>     * filled with 0x90 (nop)
>> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
>> index b11a238..c3cebfe 100644
>> --- a/arch/x86/kernel/setup.c
>> +++ b/arch/x86/kernel/setup.c
>> @@ -932,7 +932,6 @@ void __init setup_arch(char **cmdline_p)
>>        max_low_pfn = max_pfn;
>>
>>    high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1;
>> -    max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
>> #endif
>>
>>    /*
>>
> I like this one, can you check if only this one is needed ?

It is sufficient, but I'd prefer to have both.  I don't think limiting
the pagetable allocation to mapped memory is necessarily correct, and
there's no harm in making the Xen code more robust (plus it cleans up a
couple of #ifdefs).

    J

Re: [PATCH 0/4] memblock related fixes for -tip

[H. Peter Anvin]

On 10/13/2010 05:24 PM, Jeremy Fitzhardinge wrote:
>>>
>> I like this one, can you check if only this one is needed ?
> 
> It is sufficient, but I'd prefer to have both.  I don't think limiting
> the pagetable allocation to mapped memory is necessarily correct, and
> there's no harm in making the Xen code more robust (plus it cleans up a
> couple of #ifdefs).
> 

Yes, I fully agree.  I have put both patches in my tree and will push
them out shortly.

	-hpa

[tip:core/memblock] xen: Cope with unmapped pages when initializing kernel pagetable

[tip-bot for Jeremy Fitzhardinge]

Commit-ID:  fef5ba797991f9335bcfc295942b684f9bf613a1
Gitweb:     http://git.kernel.org/tip/fef5ba797991f9335bcfc295942b684f9bf613a1
Author:     Jeremy Fitzhardinge <jeremy@goop.org>
AuthorDate: Wed, 13 Oct 2010 16:02:24 -0700
Committer:  H. Peter Anvin <hpa@linux.intel.com>
CommitDate: Wed, 13 Oct 2010 16:07:13 -0700

xen: Cope with unmapped pages when initializing kernel pagetable

Xen requires that all pages containing pagetable entries to be mapped
read-only.  If pages used for the initial pagetable are already mapped
then we can change the mapping to RO.  However, if they are initially
unmapped, we need to make sure that when they are later mapped, they
are also mapped RO.

We do this by knowing that the kernel pagetable memory is pre-allocated
in the range e820_table_start - e820_table_end, so any pfn within this
range should be mapped read-only.  However, the pagetable setup code
early_ioremaps the pages to write their entries, so we must make sure
that mappings created in the early_ioremap fixmap area are mapped RW.
(Those mappings are removed before the pages are presented to Xen
as pagetable pages.)

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
LKML-Reference: <4CB63A80.8060702@goop.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
---
 arch/x86/include/asm/io.h |    1 +
 arch/x86/mm/ioremap.c     |    5 +++++
 arch/x86/xen/mmu.c        |   26 ++++++++++++++++++--------
 3 files changed, 24 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 30a3e97..66aee6c 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -348,6 +348,7 @@ extern void __iomem *early_memremap(resource_size_t phys_addr,
 				    unsigned long size);
 extern void early_iounmap(void __iomem *addr, unsigned long size);
 extern void fixup_early_ioremap(void);
+extern bool is_early_ioremap_ptep(pte_t *ptep);
 
 #define IO_SPACE_LIMIT 0xffff
 
diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
index 3ba6e06..0369843 100644
--- a/arch/x86/mm/ioremap.c
+++ b/arch/x86/mm/ioremap.c
@@ -362,6 +362,11 @@ static inline pte_t * __init early_ioremap_pte(unsigned long addr)
 	return &bm_pte[pte_index(addr)];
 }
 
+bool __init is_early_ioremap_ptep(pte_t *ptep)
+{
+	return ptep >= &bm_pte[0] && ptep < &bm_pte[PAGE_SIZE/sizeof(pte_t)];
+}
+
 static unsigned long slot_virt[FIX_BTMAPS_SLOTS] __initdata;
 
 void __init early_ioremap_init(void)
diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 4fe04ac..7d55e9e 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -56,6 +56,7 @@
 #include <asm/e820.h>
 #include <asm/linkage.h>
 #include <asm/page.h>
+#include <asm/init.h>
 
 #include <asm/xen/hypercall.h>
 #include <asm/xen/hypervisor.h>
@@ -360,7 +361,8 @@ void make_lowmem_page_readonly(void *vaddr)
 	unsigned int level;
 
 	pte = lookup_address(address, &level);
-	BUG_ON(pte == NULL);
+	if (pte == NULL)
+		return;		/* vaddr missing */
 
 	ptev = pte_wrprotect(*pte);
 
@@ -375,7 +377,8 @@ void make_lowmem_page_readwrite(void *vaddr)
 	unsigned int level;
 
 	pte = lookup_address(address, &level);
-	BUG_ON(pte == NULL);
+	if (pte == NULL)
+		return;		/* vaddr missing */
 
 	ptev = pte_mkwrite(*pte);
 
@@ -1509,13 +1512,25 @@ static void xen_pgd_free(struct mm_struct *mm, pgd_t *pgd)
 #endif
 }
 
-#ifdef CONFIG_X86_32
 static __init pte_t mask_rw_pte(pte_t *ptep, pte_t pte)
 {
+	unsigned long pfn = pte_pfn(pte);
+
+#ifdef CONFIG_X86_32
 	/* If there's an existing pte, then don't allow _PAGE_RW to be set */
 	if (pte_val_ma(*ptep) & _PAGE_PRESENT)
 		pte = __pte_ma(((pte_val_ma(*ptep) & _PAGE_RW) | ~_PAGE_RW) &
 			       pte_val_ma(pte));
+#endif
+
+	/*
+	 * If the new pfn is within the range of the newly allocated
+	 * kernel pagetable, and it isn't being mapped into an
+	 * early_ioremap fixmap slot, make sure it is RO.
+	 */
+	if (!is_early_ioremap_ptep(ptep) &&
+	    pfn >= e820_table_start && pfn < e820_table_end)
+		pte = pte_wrprotect(pte);
 
 	return pte;
 }
@@ -1528,7 +1543,6 @@ static __init void xen_set_pte_init(pte_t *ptep, pte_t pte)
 
 	xen_set_pte(ptep, pte);
 }
-#endif
 
 static void pin_pagetable_pfn(unsigned cmd, unsigned long pfn)
 {
@@ -1973,11 +1987,7 @@ static const struct pv_mmu_ops xen_mmu_ops __initdata = {
 	.alloc_pmd_clone = paravirt_nop,
 	.release_pmd = xen_release_pmd_init,
 
-#ifdef CONFIG_X86_64
-	.set_pte = xen_set_pte,
-#else
 	.set_pte = xen_set_pte_init,
-#endif
 	.set_pte_at = xen_set_pte_at,
 	.set_pmd = xen_set_pmd_hyper,
 

[tip:core/memblock] x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

[tip-bot for Jeremy Fitzhardinge]

Commit-ID:  859ee4f57fb4e91de8439496f62eb996f4a28ca8
Gitweb:     http://git.kernel.org/tip/859ee4f57fb4e91de8439496f62eb996f4a28ca8
Author:     Jeremy Fitzhardinge <jeremy@goop.org>
AuthorDate: Wed, 13 Oct 2010 16:34:15 -0700
Committer:  H. Peter Anvin <hpa@linux.intel.com>
CommitDate: Wed, 13 Oct 2010 17:09:59 -0700

x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

head_64.S maps up to 512 MiB, but that is not necessarity true for
other entry paths, such as Xen.

Thus, co-locate the setting of max_pfn_mapped with the code to
actually set up the page tables in head_64.S.  The 32-bit code is
already so co-located.  (The Xen code already sets max_pfn_mapped
correctly for its own use case.)

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
LKML-Reference: <4CB641F7.9040103@goop.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
---
 arch/x86/kernel/head_64.S |    2 ++
 arch/x86/kernel/setup.c   |    1 -
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 239046b..b75c6f4 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -141,6 +141,8 @@ ident_complete:
 	addq	%rbp, trampoline_level4_pgt + (511*8)(%rip)
 #endif
 
+	movq    $KERNEL_IMAGE_SIZE / PAGE_SIZE, max_pfn_mapped(%rip)
+
 	/* Due to ENTRY(), sometimes the empty space gets filled with
 	 * zeros. Better take a jmp than relying on empty space being
 	 * filled with 0x90 (nop)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index b11a238..c3cebfe 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -932,7 +932,6 @@ void __init setup_arch(char **cmdline_p)
 		max_low_pfn = max_pfn;
 
 	high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1;
-	max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
 #endif
 
 	/*

Re: [tip:core/memblock] x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

[Ingo Molnar]

* tip-bot for Jeremy Fitzhardinge <jeremy@goop.org> wrote:

> Commit-ID:  859ee4f57fb4e91de8439496f62eb996f4a28ca8
> Gitweb:     http://git.kernel.org/tip/859ee4f57fb4e91de8439496f62eb996f4a28ca8
> Author:     Jeremy Fitzhardinge <jeremy@goop.org>
> AuthorDate: Wed, 13 Oct 2010 16:34:15 -0700
> Committer:  H. Peter Anvin <hpa@linux.intel.com>
> CommitDate: Wed, 13 Oct 2010 17:09:59 -0700
> 
> x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S
> 
> head_64.S maps up to 512 MiB, but that is not necessarity true for
> other entry paths, such as Xen.

These two commits are a boot crasher on x86 defconfig (64-bit):

[    0.000000] init_memory_mapping: 0000000000000000-000000003fff0000
[    0.000000]  0000000000 - 003fe00000 page 2M
[    0.000000]  003fe00000 - 003fff0000 page 4k
[    0.000000] Kernel panic - not syncing: Cannot find space for the kernel page tables
[    0.000000] Pid: 0, comm: swapper Not tainted 2.6.36-rc7-tip+ #49192
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff8156c73f>] panic+0x8c/0x192
[    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
[    0.000000]  [<ffffffff8154ec40>] init_memory_mapping+0x372/0x506
[    0.000000]  [<ffffffff819053ed>] ? memblock_x86_reserve_range+0x7c/0x83
[    0.000000]  [<ffffffff818f2f3d>] setup_arch+0x5a3/0xa68
[    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
[    0.000000]  [<ffffffff81061135>] ? clockevents_register_notifier+0x3e/0x4a
[    0.000000]  [<ffffffff818ef9e1>] start_kernel+0x83/0x36e
[    0.000000]  [<ffffffff818ef2a8>] x86_64_start_reservations+0xb8/0xbc
[    0.000000]  [<ffffffff818ef393>] x86_64_start_kernel+0xe7/0xee

Thanks,

	Ingo

Re: [tip:core/memblock] x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

[Ingo Molnar]

* Ingo Molnar <mingo@elte.hu> wrote:

> These two commits are a boot crasher on x86 defconfig (64-bit):

Below is the full bootlog, with memory layout info.

Thanks,

	Ingo

---------------->

  Booting 'test-64 (test-64)'

root (hd0,5)
 Filesystem type is ext2fs, partition type 0x83
kernel /boot/bzImage-64 root=/dev/sda6 earlyprintk=ttyS0,115200 console=ttyS0,1
15200 debug initcall_debug sysrq_always_enabled ignore_loglevel selinux=0 nmi_w
atchdog=0 panic=1 3
   [Linux-bzImage, setup=0x3400, size=0x402be0]

early console in setup code
early console in decompress_kernel

Decompressing Linux... Parsing ELF... done.
Booting the kernel.
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.36-rc7-tip+ (mingo@sirius) (gcc version 4.4.4 20100514 (Red Hat 4.4.4-3) (GCC) ) #49192 SMP Thu Oct 14 09:00:39 CEST 2010
[    0.000000] Command line: root=/dev/sda6 earlyprintk=ttyS0,115200 console=ttyS0,115200 debug initcall_debug sysrq_always_enabled ignore_loglevel selinux=0 nmi_watchdog=0 panic=1 3
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
[    0.000000]  BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 000000003fff0000 (usable)
[    0.000000]  BIOS-e820: 000000003fff0000 - 000000003fff3000 (ACPI NVS)
[    0.000000]  BIOS-e820: 000000003fff3000 - 0000000040000000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
[    0.000000] bootconsole [earlyser0] enabled
[    0.000000] debug: ignoring loglevel setting.
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.3 present.
[    0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
[    0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x3fff0 max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: uncachable
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-BFFFF uncachable
[    0.000000]   C0000-C7FFF write-protect
[    0.000000]   C8000-FFFFF uncachable
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 0000000000 mask FFC0000000 write-back
[    0.000000]   1 disabled
[    0.000000]   2 disabled
[    0.000000]   3 disabled
[    0.000000]   4 disabled
[    0.000000]   5 disabled
[    0.000000]   6 disabled
[    0.000000]   7 disabled
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] found SMP MP-table at [ffff8800000f5680] f5680
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] initial memory mapped : 0 - 00000000
[    0.000000] init_memory_mapping: 0000000000000000-000000003fff0000
[    0.000000]  0000000000 - 003fe00000 page 2M
[    0.000000]  003fe00000 - 003fff0000 page 4k
[    0.000000] Kernel panic - not syncing: Cannot find space for the kernel page tables
[    0.000000] Pid: 0, comm: swapper Not tainted 2.6.36-rc7-tip+ #49192
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff8156c73f>] panic+0x8c/0x192
[    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
[    0.000000]  [<ffffffff8154ec40>] init_memory_mapping+0x372/0x506
[    0.000000]  [<ffffffff819053ed>] ? memblock_x86_reserve_range+0x7c/0x83
[    0.000000]  [<ffffffff818f2f3d>] setup_arch+0x5a3/0xa68
[    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
[    0.000000]  [<ffffffff81061135>] ? clockevents_register_notifier+0x3e/0x4a
[    0.000000]  [<ffffffff818ef9e1>] start_kernel+0x83/0x36e
[    0.000000]  [<ffffffff818ef2a8>] x86_64_start_reservations+0xb8/0xbc
[    0.000000]  [<ffffffff818ef393>] x86_64_start_kernel+0xe7/0xee

Re: [tip:core/memblock] x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

[Ingo Molnar]

* Ingo Molnar <mingo@elte.hu> wrote:

> * tip-bot for Jeremy Fitzhardinge <jeremy@goop.org> wrote:
> 
> > Commit-ID:  859ee4f57fb4e91de8439496f62eb996f4a28ca8
> > Gitweb:     http://git.kernel.org/tip/859ee4f57fb4e91de8439496f62eb996f4a28ca8
> > Author:     Jeremy Fitzhardinge <jeremy@goop.org>
> > AuthorDate: Wed, 13 Oct 2010 16:34:15 -0700
> > Committer:  H. Peter Anvin <hpa@linux.intel.com>
> > CommitDate: Wed, 13 Oct 2010 17:09:59 -0700
> > 
> > x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S
> > 
> > head_64.S maps up to 512 MiB, but that is not necessarity true for
> > other entry paths, such as Xen.
> 
> These two commits are a boot crasher on x86 defconfig (64-bit):

Caused by this one of the two:

  859ee4f: x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

	Ingo

Re: [tip:core/memblock] x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

[Yinghai Lu]

On 10/13/2010 10:57 PM, Ingo Molnar wrote:
> 
> * tip-bot for Jeremy Fitzhardinge <jeremy@goop.org> wrote:
> 
>> Commit-ID:  859ee4f57fb4e91de8439496f62eb996f4a28ca8
>> Gitweb:     http://git.kernel.org/tip/859ee4f57fb4e91de8439496f62eb996f4a28ca8
>> Author:     Jeremy Fitzhardinge <jeremy@goop.org>
>> AuthorDate: Wed, 13 Oct 2010 16:34:15 -0700
>> Committer:  H. Peter Anvin <hpa@linux.intel.com>
>> CommitDate: Wed, 13 Oct 2010 17:09:59 -0700
>>
>> x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S
>>
>> head_64.S maps up to 512 MiB, but that is not necessarity true for
>> other entry paths, such as Xen.
> 
> These two commits are a boot crasher on x86 defconfig (64-bit):
> 
> [    0.000000] init_memory_mapping: 0000000000000000-000000003fff0000
> [    0.000000]  0000000000 - 003fe00000 page 2M
> [    0.000000]  003fe00000 - 003fff0000 page 4k
> [    0.000000] Kernel panic - not syncing: Cannot find space for the kernel page tables
> [    0.000000] Pid: 0, comm: swapper Not tainted 2.6.36-rc7-tip+ #49192
> [    0.000000] Call Trace:
> [    0.000000]  [<ffffffff8156c73f>] panic+0x8c/0x192
> [    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
> [    0.000000]  [<ffffffff8154ec40>] init_memory_mapping+0x372/0x506
> [    0.000000]  [<ffffffff819053ed>] ? memblock_x86_reserve_range+0x7c/0x83
> [    0.000000]  [<ffffffff818f2f3d>] setup_arch+0x5a3/0xa68
> [    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
> [    0.000000]  [<ffffffff81061135>] ? clockevents_register_notifier+0x3e/0x4a
> [    0.000000]  [<ffffffff818ef9e1>] start_kernel+0x83/0x36e
> [    0.000000]  [<ffffffff818ef2a8>] x86_64_start_reservations+0xb8/0xbc
> [    0.000000]  [<ffffffff818ef393>] x86_64_start_kernel+0xe7/0xee

max_pfn_mapped is in .bss section. so it get cleared.

will have a patch to move clear_bss from head64.c to head_64.S

Thanks

Yinghai

Re: [tip:core/memblock] x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

[Ingo Molnar]

* Yinghai Lu <yinghai@kernel.org> wrote:

> On 10/13/2010 10:57 PM, Ingo Molnar wrote:
> > 
> > * tip-bot for Jeremy Fitzhardinge <jeremy@goop.org> wrote:
> > 
> >> Commit-ID:  859ee4f57fb4e91de8439496f62eb996f4a28ca8
> >> Gitweb:     http://git.kernel.org/tip/859ee4f57fb4e91de8439496f62eb996f4a28ca8
> >> Author:     Jeremy Fitzhardinge <jeremy@goop.org>
> >> AuthorDate: Wed, 13 Oct 2010 16:34:15 -0700
> >> Committer:  H. Peter Anvin <hpa@linux.intel.com>
> >> CommitDate: Wed, 13 Oct 2010 17:09:59 -0700
> >>
> >> x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S
> >>
> >> head_64.S maps up to 512 MiB, but that is not necessarity true for
> >> other entry paths, such as Xen.
> > 
> > These two commits are a boot crasher on x86 defconfig (64-bit):
> > 
> > [    0.000000] init_memory_mapping: 0000000000000000-000000003fff0000
> > [    0.000000]  0000000000 - 003fe00000 page 2M
> > [    0.000000]  003fe00000 - 003fff0000 page 4k
> > [    0.000000] Kernel panic - not syncing: Cannot find space for the kernel page tables
> > [    0.000000] Pid: 0, comm: swapper Not tainted 2.6.36-rc7-tip+ #49192
> > [    0.000000] Call Trace:
> > [    0.000000]  [<ffffffff8156c73f>] panic+0x8c/0x192
> > [    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
> > [    0.000000]  [<ffffffff8154ec40>] init_memory_mapping+0x372/0x506
> > [    0.000000]  [<ffffffff819053ed>] ? memblock_x86_reserve_range+0x7c/0x83
> > [    0.000000]  [<ffffffff818f2f3d>] setup_arch+0x5a3/0xa68
> > [    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
> > [    0.000000]  [<ffffffff81061135>] ? clockevents_register_notifier+0x3e/0x4a
> > [    0.000000]  [<ffffffff818ef9e1>] start_kernel+0x83/0x36e
> > [    0.000000]  [<ffffffff818ef2a8>] x86_64_start_reservations+0xb8/0xbc
> > [    0.000000]  [<ffffffff818ef393>] x86_64_start_kernel+0xe7/0xee
> 
> max_pfn_mapped is in .bss section. so it get cleared.
> 
> will have a patch to move clear_bss from head64.c to head_64.S

Ok, great - thanks!

	Ingo

Re: [tip:core/memblock] x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

[Yinghai Lu]

On 10/13/2010 11:37 PM, Ingo Molnar wrote:
> 
> * Yinghai Lu <yinghai@kernel.org> wrote:
> 
>> On 10/13/2010 10:57 PM, Ingo Molnar wrote:
>>>
>>> * tip-bot for Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>>>
>>>> Commit-ID:  859ee4f57fb4e91de8439496f62eb996f4a28ca8
>>>> Gitweb:     http://git.kernel.org/tip/859ee4f57fb4e91de8439496f62eb996f4a28ca8
>>>> Author:     Jeremy Fitzhardinge <jeremy@goop.org>
>>>> AuthorDate: Wed, 13 Oct 2010 16:34:15 -0700
>>>> Committer:  H. Peter Anvin <hpa@linux.intel.com>
>>>> CommitDate: Wed, 13 Oct 2010 17:09:59 -0700
>>>>
>>>> x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S
>>>>
>>>> head_64.S maps up to 512 MiB, but that is not necessarity true for
>>>> other entry paths, such as Xen.
>>>

please fold following patch into 859ee4f57fb4e91de8439496f62eb996f4a28ca8

Thanks

Yinghai

[PATCH] x86_64: Set max_pfn_mapped in x86_64_start_kernel

Ingo found:
[    0.000000] init_memory_mapping: 0000000000000000-000000003fff0000
[    0.000000]  0000000000 - 003fe00000 page 2M
[    0.000000]  003fe00000 - 003fff0000 page 4k
[    0.000000] Kernel panic - not syncing: Cannot find space for the kernel page tables
[    0.000000] Pid: 0, comm: swapper Not tainted 2.6.36-rc7-tip+ #49192
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff8156c73f>] panic+0x8c/0x192
[    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
[    0.000000]  [<ffffffff8154ec40>] init_memory_mapping+0x372/0x506
[    0.000000]  [<ffffffff819053ed>] ? memblock_x86_reserve_range+0x7c/0x83
[    0.000000]  [<ffffffff818f2f3d>] setup_arch+0x5a3/0xa68
[    0.000000]  [<ffffffff8156c881>] ? printk+0x3c/0x3e
[    0.000000]  [<ffffffff81061135>] ? clockevents_register_notifier+0x3e/0x4a
[    0.000000]  [<ffffffff818ef9e1>] start_kernel+0x83/0x36e
[    0.000000]  [<ffffffff818ef2a8>] x86_64_start_reservations+0xb8/0xbc
[    0.000000]  [<ffffffff818ef393>] x86_64_start_kernel+0xe7/0xee

caused by:
|  859ee4f: x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

it turns out max_pfn_mapped is in .bss section.

So We need to set that after bss get cleared.

that is safe because xen don't call x86_64_start_kernel()

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/kernel/head64.c  |    2 ++
 arch/x86/kernel/head_64.S |    2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

Index: linux-2.6/arch/x86/kernel/head64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/head64.c
+++ linux-2.6/arch/x86/kernel/head64.c
@@ -80,6 +80,8 @@ void __init x86_64_start_kernel(char * r
 	/* Cleanup the over mapped high alias */
 	cleanup_highmap();
 
+	max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
+
 	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++) {
 #ifdef CONFIG_EARLY_PRINTK
 		set_intr_gate(i, &early_idt_handlers[i]);
Index: linux-2.6/arch/x86/kernel/head_64.S
===================================================================
--- linux-2.6.orig/arch/x86/kernel/head_64.S
+++ linux-2.6/arch/x86/kernel/head_64.S
@@ -141,8 +141,6 @@ ident_complete:
 	addq	%rbp, trampoline_level4_pgt + (511*8)(%rip)
 #endif
 
-	movq    $KERNEL_IMAGE_SIZE / PAGE_SIZE, max_pfn_mapped(%rip)
-
 	/* Due to ENTRY(), sometimes the empty space gets filled with
 	 * zeros. Better take a jmp than relying on empty space being
 	 * filled with 0x90 (nop)

[tip:core/memblock] x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

[tip-bot for Jeremy Fitzhardinge]

Commit-ID:  67e87f0a1c5cbc750f81ebf6a128e8ff6f4376cc
Gitweb:     http://git.kernel.org/tip/67e87f0a1c5cbc750f81ebf6a128e8ff6f4376cc
Author:     Jeremy Fitzhardinge <jeremy@goop.org>
AuthorDate: Wed, 13 Oct 2010 16:34:15 -0700
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Thu, 14 Oct 2010 09:06:49 +0200

x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S

head_64.S maps up to 512 MiB, but that is not necessarity true for
other entry paths, such as Xen.

Thus, co-locate the setting of max_pfn_mapped with the code to
actually set up the page tables in head_64.S.  The 32-bit code is
already so co-located.  (The Xen code already sets max_pfn_mapped
correctly for its own use case.)

-v2:

 Yinghai fixed the following bug in this patch:

 |
 | max_pfn_mapped is in .bss section, so we need to set that
 | after bss get cleared. Without that we crash on bootup.
 |
 | That is safe because Xen does not call x86_64_start_kernel().
 |

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Fixed-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
LKML-Reference: <4CB6AB24.9020504@kernel.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 arch/x86/kernel/head64.c |    2 ++
 arch/x86/kernel/setup.c  |    1 -
 2 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
index 97adf98..2d2673c 100644
--- a/arch/x86/kernel/head64.c
+++ b/arch/x86/kernel/head64.c
@@ -80,6 +80,8 @@ void __init x86_64_start_kernel(char * real_mode_data)
 	/* Cleanup the over mapped high alias */
 	cleanup_highmap();
 
+	max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
+
 	for (i = 0; i < NUM_EXCEPTION_VECTORS; i++) {
 #ifdef CONFIG_EARLY_PRINTK
 		set_intr_gate(i, &early_idt_handlers[i]);
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index b11a238..c3cebfe 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -932,7 +932,6 @@ void __init setup_arch(char **cmdline_p)
 		max_low_pfn = max_pfn;
 
 	high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1;
-	max_pfn_mapped = KERNEL_IMAGE_SIZE >> PAGE_SHIFT;
 #endif
 
 	/*

Debug patches for memblock

[Yinghai Lu]

Jeremy,

Please check some debug patches for memblock in

git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock

hope you can use them to find out causes for some xen failure.

Thanks

	Yinghai

Re: Debug patches for memblock

[Jeremy Fitzhardinge]

On 11/03/2010 01:05 AM, Yinghai Lu wrote:
> Jeremy,
>
> Please check some debug patches for memblock in
>
> git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-2.6-yinghai.git memblock
>
> hope you can use them to find out causes for some xen failure.

Thanks, I'll take a look.

    J