Memory size unaligned to section boundary

Memory size unaligned to section boundary

[Stefan Agner]

Hi,

It seems to me that I hit an issue in low memory mapping (map_lowmem).
I'm using a custom memory size, which leads to an freeze on Linux 4.0
and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid
and NVIDIA Tegra 3):

With mem=259744K
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
(ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
[    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
instruction cache
[    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
Board
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] cma: Reserved 16 MiB at 0x8e400000
[    0.000000] Memory policy: Data cache writeback
<freeze>

I dug a bit more into that, and it unveiled that when creating the
mapping for the non-kernel_x part (if (kernel_x_end < end) in
map_lowmem), the unaligned section at the end leads to the freeze. In
alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte
gets called which allocates a PTE outside of the initialized region (in
early_alloc_aligned). The system freezes@the call of memset in
early_alloc_aligned function.

With some debug print, this can be better illustrated:
[    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
[    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
                         => actual end of memory ^^^^^^^^
[    0.000000] alloc_init_pte
[    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
[    0.000000] early_pte_alloc
[    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align
00001000
     => PTE allocated outside of initialized area ^^^^^^^^

It seems that memory gets allocation in the last section. When the last
section was in the previous PMD, the allocation works, however if the
last section is within the same PMD, the allocation ends up in the
non-initialized area. So:

In other words, sizes which end in a upper part of the 2MB sized PMD
fail, while sizes in the lower part of a PMD work.
0xFF80000 => fails (mem=261632K)
0xFE80000 => works (mem=260608K)
0xFD80000 => fails (mem=261632K)
...

While I understand the reason for the freeze, I don't know to properly
fix it. It looks to me that in alloc_init_pmd, we should use
__map_init_section first to map the last aligned section, before calling
alloc_init_pte on the non aligned section.

Background: I tried to reuse the boot loader part of the simplefb
implementation for sunxi. It decreases memory size by the size of the
framebuffer. Hence the actually memory size can be unaligned, depending
on the display size used. In the case at hand, a framebuffer of the size
800x600 worked while 1024x600 did not work... The implementation uses
device tree to report the memory size, but the kernel arguments show the
same behavior.

Maybe a regression of e651eab0af ("ARM: 7677/1: LPAE: Fix mapping in
alloc_init_section for unaligned addresses"). I currently do not have a
platform at hand which works on that Linux version out of the box.

--
Stefan

[REGRESSION?] ARM: 7677/1: LPAE: Fix mapping in alloc_init_section for unaligned addresses (was Re: Memory size unaligned to section boundary)

[Hans de Goede]

Hi all,

On 23-04-15 15:19, Stefan Agner wrote:
> Hi,
>
> It seems to me that I hit an issue in low memory mapping (map_lowmem).
> I'm using a custom memory size, which leads to an freeze on Linux 4.0
> and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid
> and NVIDIA Tegra 3):
>
> With mem=259744K
> [    0.000000] Booting Linux on physical CPU 0x0
> [    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
> (ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
> 4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
> [    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
> cr=10c5387d
> [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
> instruction cache
> [    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
> Board
> [    0.000000] bootconsole [earlycon0] enabled
> [    0.000000] cma: Reserved 16 MiB at 0x8e400000
> [    0.000000] Memory policy: Data cache writeback
> <freeze>
>
> I dug a bit more into that, and it unveiled that when creating the
> mapping for the non-kernel_x part (if (kernel_x_end < end) in
> map_lowmem), the unaligned section at the end leads to the freeze. In
> alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte
> gets called which allocates a PTE outside of the initialized region (in
> early_alloc_aligned). The system freezes at the call of memset in
> early_alloc_aligned function.
>
> With some debug print, this can be better illustrated:
> [    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
> [    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
> [    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
>                           => actual end of memory ^^^^^^^^
> [    0.000000] alloc_init_pte
> [    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
> [    0.000000] early_pte_alloc
> [    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align
> 00001000
>       => PTE allocated outside of initialized area ^^^^^^^^
>
> It seems that memory gets allocation in the last section. When the last
> section was in the previous PMD, the allocation works, however if the
> last section is within the same PMD, the allocation ends up in the
> non-initialized area. So:
>
> In other words, sizes which end in a upper part of the 2MB sized PMD
> fail, while sizes in the lower part of a PMD work.
> 0xFF80000 => fails (mem=261632K)
> 0xFE80000 => works (mem=260608K)
> 0xFD80000 => fails (mem=261632K)
> ...
>
> While I understand the reason for the freeze, I don't know to properly
> fix it. It looks to me that in alloc_init_pmd, we should use
> __map_init_section first to map the last aligned section, before calling
> alloc_init_pte on the non aligned section.
>
> Background: I tried to reuse the boot loader part of the simplefb
> implementation for sunxi. It decreases memory size by the size of the
> framebuffer. Hence the actually memory size can be unaligned, depending
> on the display size used. In the case at hand, a framebuffer of the size
> 800x600 worked while 1024x600 did not work... The implementation uses
> device tree to report the memory size, but the kernel arguments show the
> same behavior.
>
> Maybe a regression of e651eab0af ("ARM: 7677/1: LPAE: Fix mapping in
> alloc_init_section for unaligned addresses"). I currently do not have a
> platform at hand which works on that Linux version out of the box.

I'm seeing this to an Allwinner Cortex A7 based SoCs, specifically
on tablets with a 1024x600 lcd screen it seems that shaving exactly the
amount of memory needed for a 32bpp 1024x600 framebuffer of from the
top of memory triggers this.

Regards,

Hans

[REGRESSION?] ARM: 7677/1: LPAE: Fix mapping in alloc_init_section for unaligned addresses (was Re: Memory size unaligned to section boundary)

[Mark Rutland]

[Adding potentially interested parties, those involved in 7677/1]

On Tue, Apr 28, 2015 at 11:05:37AM +0100, Hans de Goede wrote:
> Hi all,
> 
> On 23-04-15 15:19, Stefan Agner wrote:
> > Hi,
> >
> > It seems to me that I hit an issue in low memory mapping (map_lowmem).
> > I'm using a custom memory size, which leads to an freeze on Linux 4.0
> > and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid
> > and NVIDIA Tegra 3):
> >
> > With mem=259744K
> > [    0.000000] Booting Linux on physical CPU 0x0
> > [    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
> > (ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
> > 4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
> > [    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
> > cr=10c5387d
> > [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
> > instruction cache
> > [    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
> > Board
> > [    0.000000] bootconsole [earlycon0] enabled
> > [    0.000000] cma: Reserved 16 MiB at 0x8e400000
> > [    0.000000] Memory policy: Data cache writeback
> > <freeze>
> >
> > I dug a bit more into that, and it unveiled that when creating the
> > mapping for the non-kernel_x part (if (kernel_x_end < end) in
> > map_lowmem), the unaligned section at the end leads to the freeze. In
> > alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte
> > gets called which allocates a PTE outside of the initialized region (in
> > early_alloc_aligned). The system freezes at the call of memset in
> > early_alloc_aligned function.
> >
> > With some debug print, this can be better illustrated:
> > [    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
> > [    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
> > [    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
> >                           => actual end of memory ^^^^^^^^
> > [    0.000000] alloc_init_pte
> > [    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
> > [    0.000000] early_pte_alloc
> > [    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align
> > 00001000
> >       => PTE allocated outside of initialized area ^^^^^^^^
> >
> > It seems that memory gets allocation in the last section. When the last
> > section was in the previous PMD, the allocation works, however if the
> > last section is within the same PMD, the allocation ends up in the
> > non-initialized area. So:
> >
> > In other words, sizes which end in a upper part of the 2MB sized PMD
> > fail, while sizes in the lower part of a PMD work.
> > 0xFF80000 => fails (mem=261632K)
> > 0xFE80000 => works (mem=260608K)
> > 0xFD80000 => fails (mem=261632K)
> > ...
> >
> > While I understand the reason for the freeze, I don't know to properly
> > fix it. It looks to me that in alloc_init_pmd, we should use
> > __map_init_section first to map the last aligned section, before calling
> > alloc_init_pte on the non aligned section.
> >
> > Background: I tried to reuse the boot loader part of the simplefb
> > implementation for sunxi. It decreases memory size by the size of the
> > framebuffer. Hence the actually memory size can be unaligned, depending
> > on the display size used. In the case at hand, a framebuffer of the size
> > 800x600 worked while 1024x600 did not work... The implementation uses
> > device tree to report the memory size, but the kernel arguments show the
> > same behavior.
> >
> > Maybe a regression of e651eab0af ("ARM: 7677/1: LPAE: Fix mapping in
> > alloc_init_section for unaligned addresses"). I currently do not have a
> > platform at hand which works on that Linux version out of the box.
> 
> I'm seeing this to an Allwinner Cortex A7 based SoCs, specifically
> on tablets with a 1024x600 lcd screen it seems that shaving exactly the
> amount of memory needed for a 32bpp 1024x600 framebuffer of from the
> top of memory triggers this.

I'm able to trigger the issue on TC2 by passing mem=259744K. If I hack
sanity_check_meminfo to round the memblock limit down to PMD_SIZE I
avoid the immediate freeze, but later things blew up seemingly due to an
unmapped DTB (panic below) I'm not entirely sure why that's the case.

I wasn't able to come up with a DTB that would trigger this. Do you have
an example set of memory nodes + memreserves? Where are your kernel and
DTB loaded in memory?

Thanks,
Mark.

Unable to handle kernel paging request at virtual address 9fee6000
pgd = 80004000
[9fee6000] *pgd=00000000
Internal error: Oops: 5 [#1] SMP ARM
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.0-rc1+ #17
Hardware name: ARM-Versatile Express
task: 8065e7a8 ti: 8065a000 task.ti: 8065a000
PC is at fdt_check_header+0x0/0x74
LR is at __unflatten_device_tree+0x1c/0x128
pc : [<80490350>]    lr : [<803a1554>]    psr: a00001d3
sp : 8065bf28  ip : 806a7d77  fp : 80000200
r10: 8056d84c  r9 : 8069fc9c  r8 : 80635b0c
r7 : 80683140  r6 : 9fee6000  r5 : 8063eac4  r4 : 80635b0c
r3 : 8069fcb4  r2 : 80635b0c  r1 : 8069fc9c  r0 : 9fee6000
Flags: NzCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM  Segment kernel
Control: 10c5387d  Table: 8000406a  DAC: 00000015
Process swapper (pid: 0, stack limit = 0x8065a210)
Stack: (0x8065bf28 to 0x8065c000)
bf20:                   ffffffff 00000000 ffffffff 0008fbfd 00000000 00000000
bf40: 00000000 80635b0c 8063eac4 8065f79c 80683140 8068d5e4 806650e0 806366e8
bf60: 8065c3c8 8061b43c ffffffff 10c5387d 80683000 8fbfb340 80008000 8064aa88
bf80: 00000000 00000000 00000000 80058674 8056c3e8 8065bfb4 00000000 00000000
bfa0: 80683000 00000001 8065c3c0 ffffffff 00000000 00000000 00000000 8061895c
bfc0: 00000000 00000000 00000000 00000000 00000000 8064aa88 80683394 8065c440
bfe0: 8064aa84 8065f8bc 8000406a 412fc0f1 00000000 8000807c 00000000 00000000
[<80490350>] (fdt_check_header) from [<803a1554>] (__unflatten_device_tree+0x1c/0x128)
[<803a1554>] (__unflatten_device_tree) from [<806366e8>] (unflatten_device_tree+0x28/0x34)
[<806366e8>] (unflatten_device_tree) from [<8061b43c>] (setup_arch+0x778/0x984)
[<8061b43c>] (setup_arch) from [<8061895c>] (start_kernel+0x9c/0x3ac)
[<8061895c>] (start_kernel) from [<8000807c>] (0x8000807c)
Code: e3e0300d eafd2608 e3e0300d eafd260d (e5903000)
---[ end trace cb88537fdc8fa200 ]---
Kernel panic - not syncing: Attempted to kill the idle task!
---[ end Kernel panic - not syncing: Attempted to kill the idle task!

[REGRESSION?] ARM: 7677/1: LPAE: Fix mapping in alloc_init_section for unaligned addresses (was Re: Memory size unaligned to section boundary)

[Hans de Goede]

Hi Mark,

On 05-05-15 16:22, Mark Rutland wrote:
> [Adding potentially interested parties, those involved in 7677/1]
>
> On Tue, Apr 28, 2015 at 11:05:37AM +0100, Hans de Goede wrote:
>> Hi all,
>>
>> On 23-04-15 15:19, Stefan Agner wrote:
>>> Hi,
>>>
>>> It seems to me that I hit an issue in low memory mapping (map_lowmem).
>>> I'm using a custom memory size, which leads to an freeze on Linux 4.0
>>> and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid
>>> and NVIDIA Tegra 3):
>>>
>>> With mem=259744K
>>> [    0.000000] Booting Linux on physical CPU 0x0
>>> [    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
>>> (ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
>>> 4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
>>> [    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
>>> cr=10c5387d
>>> [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
>>> instruction cache
>>> [    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
>>> Board
>>> [    0.000000] bootconsole [earlycon0] enabled
>>> [    0.000000] cma: Reserved 16 MiB at 0x8e400000
>>> [    0.000000] Memory policy: Data cache writeback
>>> <freeze>
>>>
>>> I dug a bit more into that, and it unveiled that when creating the
>>> mapping for the non-kernel_x part (if (kernel_x_end < end) in
>>> map_lowmem), the unaligned section at the end leads to the freeze. In
>>> alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte
>>> gets called which allocates a PTE outside of the initialized region (in
>>> early_alloc_aligned). The system freezes at the call of memset in
>>> early_alloc_aligned function.
>>>
>>> With some debug print, this can be better illustrated:
>>> [    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
>>> [    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
>>> [    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
>>>                            => actual end of memory ^^^^^^^^
>>> [    0.000000] alloc_init_pte
>>> [    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
>>> [    0.000000] early_pte_alloc
>>> [    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align
>>> 00001000
>>>        => PTE allocated outside of initialized area ^^^^^^^^
>>>
>>> It seems that memory gets allocation in the last section. When the last
>>> section was in the previous PMD, the allocation works, however if the
>>> last section is within the same PMD, the allocation ends up in the
>>> non-initialized area. So:
>>>
>>> In other words, sizes which end in a upper part of the 2MB sized PMD
>>> fail, while sizes in the lower part of a PMD work.
>>> 0xFF80000 => fails (mem=261632K)
>>> 0xFE80000 => works (mem=260608K)
>>> 0xFD80000 => fails (mem=261632K)
>>> ...
>>>
>>> While I understand the reason for the freeze, I don't know to properly
>>> fix it. It looks to me that in alloc_init_pmd, we should use
>>> __map_init_section first to map the last aligned section, before calling
>>> alloc_init_pte on the non aligned section.
>>>
>>> Background: I tried to reuse the boot loader part of the simplefb
>>> implementation for sunxi. It decreases memory size by the size of the
>>> framebuffer. Hence the actually memory size can be unaligned, depending
>>> on the display size used. In the case at hand, a framebuffer of the size
>>> 800x600 worked while 1024x600 did not work... The implementation uses
>>> device tree to report the memory size, but the kernel arguments show the
>>> same behavior.
>>>
>>> Maybe a regression of e651eab0af ("ARM: 7677/1: LPAE: Fix mapping in
>>> alloc_init_section for unaligned addresses"). I currently do not have a
>>> platform at hand which works on that Linux version out of the box.
>>
>> I'm seeing this to an Allwinner Cortex A7 based SoCs, specifically
>> on tablets with a 1024x600 lcd screen it seems that shaving exactly the
>> amount of memory needed for a 32bpp 1024x600 framebuffer of from the
>> top of memory triggers this.
>
> I'm able to trigger the issue on TC2 by passing mem=259744K. If I hack
> sanity_check_meminfo to round the memblock limit down to PMD_SIZE I
> avoid the immediate freeze, but later things blew up seemingly due to an
> unmapped DTB (panic below) I'm not entirely sure why that's the case.

Thank you for looking into this.

> I wasn't able to come up with a DTB that would trigger this. Do you have
> an example set of memory nodes + memreserves? Where are your kernel and
> DTB loaded in memory?

We've a single memory node/bank from 0x40000000 to end of memory, we
carve out a framebuffer at the end, and do not report that to Linux, so
the end becomes 0x40000000 + memory-size - fb-size, and we use no mem
reserves, because if we do use mem-reserves the mmap of the fbmem by
the simplefb driver fails as that maps it non cach-able and if it is
part of the main membank it already is mapped cachable.

We substract exactly the necessary fb-size, one known fb-size which
triggers this is 1024x600 which means we end up substracting
1024x600x4 bytes from the end of memory, so effectively we are
doing the same as passing a mem= argument which is not 2MiB aligned.

As for our kernel location, we have the following in u-boot's
environment:

         "kernel_addr_r=0x42000000\0" \
         "fdt_addr_r=0x43000000\0" \
         "scriptaddr=0x43100000\0" \
         "pxefile_addr_r=0x43200000\0" \
         "ramdisk_addr_r=0x43300000\0"

Regards,

Hans


> Unable to handle kernel paging request at virtual address 9fee6000
> pgd = 80004000
> [9fee6000] *pgd=00000000
> Internal error: Oops: 5 [#1] SMP ARM
> Modules linked in:
> CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.0-rc1+ #17
> Hardware name: ARM-Versatile Express
> task: 8065e7a8 ti: 8065a000 task.ti: 8065a000
> PC is at fdt_check_header+0x0/0x74
> LR is at __unflatten_device_tree+0x1c/0x128
> pc : [<80490350>]    lr : [<803a1554>]    psr: a00001d3
> sp : 8065bf28  ip : 806a7d77  fp : 80000200
> r10: 8056d84c  r9 : 8069fc9c  r8 : 80635b0c
> r7 : 80683140  r6 : 9fee6000  r5 : 8063eac4  r4 : 80635b0c
> r3 : 8069fcb4  r2 : 80635b0c  r1 : 8069fc9c  r0 : 9fee6000
> Flags: NzCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM  Segment kernel
> Control: 10c5387d  Table: 8000406a  DAC: 00000015
> Process swapper (pid: 0, stack limit = 0x8065a210)
> Stack: (0x8065bf28 to 0x8065c000)
> bf20:                   ffffffff 00000000 ffffffff 0008fbfd 00000000 00000000
> bf40: 00000000 80635b0c 8063eac4 8065f79c 80683140 8068d5e4 806650e0 806366e8
> bf60: 8065c3c8 8061b43c ffffffff 10c5387d 80683000 8fbfb340 80008000 8064aa88
> bf80: 00000000 00000000 00000000 80058674 8056c3e8 8065bfb4 00000000 00000000
> bfa0: 80683000 00000001 8065c3c0 ffffffff 00000000 00000000 00000000 8061895c
> bfc0: 00000000 00000000 00000000 00000000 00000000 8064aa88 80683394 8065c440
> bfe0: 8064aa84 8065f8bc 8000406a 412fc0f1 00000000 8000807c 00000000 00000000
> [<80490350>] (fdt_check_header) from [<803a1554>] (__unflatten_device_tree+0x1c/0x128)
> [<803a1554>] (__unflatten_device_tree) from [<806366e8>] (unflatten_device_tree+0x28/0x34)
> [<806366e8>] (unflatten_device_tree) from [<8061b43c>] (setup_arch+0x778/0x984)
> [<8061b43c>] (setup_arch) from [<8061895c>] (start_kernel+0x9c/0x3ac)
> [<8061895c>] (start_kernel) from [<8000807c>] (0x8000807c)
> Code: e3e0300d eafd2608 e3e0300d eafd260d (e5903000)
> ---[ end trace cb88537fdc8fa200 ]---
> Kernel panic - not syncing: Attempted to kill the idle task!
> ---[ end Kernel panic - not syncing: Attempted to kill the idle task!
>

[REGRESSION?] ARM: 7677/1: LPAE: Fix mapping in alloc_init_section for unaligned addresses (was Re: Memory size unaligned to section boundary)

[Mark Rutland]

> > I wasn't able to come up with a DTB that would trigger this. Do you have
> > an example set of memory nodes + memreserves? Where are your kernel and
> > DTB loaded in memory?
> 
> We've a single memory node/bank from 0x40000000 to end of memory, we
> carve out a framebuffer at the end, and do not report that to Linux, so
> the end becomes 0x40000000 + memory-size - fb-size, and we use no mem
> reserves, because if we do use mem-reserves the mmap of the fbmem by
> the simplefb driver fails as that maps it non cach-able and if it is
> part of the main membank it already is mapped cachable.

Sure. The only reason for caring about any memreserves was in case they
inadvertently affected the memblock_limit or anything else generated by
iterating over the memblock array. If you have none to begin with then
they clearly aren't involved.

> We substract exactly the necessary fb-size, one known fb-size which
> triggers this is 1024x600 which means we end up substracting
> 1024x600x4 bytes from the end of memory, so effectively we are
> doing the same as passing a mem= argument which is not 2MiB aligned.

Thanks for the info.

It turns out my bootloader was silently rewriting the memory nodes,
which was why I couldn't reproduce the issue with a DTB alone. With the
memory node reg munged to <0 0x80000000 0 0x3FDA8000> without bootloader
interference, TC2 dies similarly to what you described.

As far as I can see the issue is not a regression; it looks like we'd
previously fail to use a (1M) section unless we had precisely 1M or 2M
of the section to map (as those are the only cases when end would be
section aligned).

The below hack prevents the issue by rounding the memblock_limit down to
a full (2M) pmd boundary, so we don't try to allocate from the first
section in a partial pmd. That does mean that if your memory ends on a
1M boundary you lose that last 1M for early memblock allocations.

Balancing the pmd manipulation turned out to be a lot more painful than
I'd anticipated, so I gave up on trying to map the first section in a
partial pmd. If people are happy with the below diff I can respin as a
patch (with comment updates and so on).

Thanks,
Mark.

---->8----
diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index 4e6ef89..2ea13f0 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
                         * occurs before any free memory is mapped.
                         */
                        if (!memblock_limit) {
-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
                                        memblock_limit = block_start;
-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
                                        memblock_limit = arm_lowmem_limit;
                        }
 
@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
         * last full section, which should be mapped.
         */
        if (memblock_limit)
-               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+               memblock_limit = round_down(memblock_limit, PMD_SIZE);
        if (!memblock_limit)
                memblock_limit = arm_lowmem_limit;

Memory size unaligned to section boundary

[Russell King - ARM Linux]

On Thu, Apr 23, 2015 at 03:19:45PM +0200, Stefan Agner wrote:
> Hi,
> 
> It seems to me that I hit an issue in low memory mapping (map_lowmem).
> I'm using a custom memory size, which leads to an freeze on Linux 4.0
> and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid
> and NVIDIA Tegra 3):
> 
> With mem=259744K
> [    0.000000] Booting Linux on physical CPU 0x0
> [    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
> (ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
> 4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
> [    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
> cr=10c5387d
> [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
> instruction cache
> [    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
> Board
> [    0.000000] bootconsole [earlycon0] enabled
> [    0.000000] cma: Reserved 16 MiB at 0x8e400000
> [    0.000000] Memory policy: Data cache writeback
> <freeze>
> 
> I dug a bit more into that, and it unveiled that when creating the
> mapping for the non-kernel_x part (if (kernel_x_end < end) in
> map_lowmem), the unaligned section at the end leads to the freeze. In
> alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte
> gets called which allocates a PTE outside of the initialized region (in
> early_alloc_aligned). The system freezes at the call of memset in
> early_alloc_aligned function.
> 
> With some debug print, this can be better illustrated:
> [    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
> [    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
> [    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
>                          => actual end of memory ^^^^^^^^
> [    0.000000] alloc_init_pte
> [    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
> [    0.000000] early_pte_alloc
> [    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align
> 00001000
>      => PTE allocated outside of initialized area ^^^^^^^^

Right, and the question is why does that happen - and the answer is
this:

        /*
         * Round the memblock limit down to a section size.  This
         * helps to ensure that we will allocate memory from the
         * last full section, which should be mapped.
         */
        if (memblock_limit)
                memblock_limit = round_down(memblock_limit, SECTION_SIZE);

That should round down by 2x SECTION_SIZE to ensure that we don't start
allocating the L2 page table in a section which isn't mapped.  Please
try this patch:

 arch/arm/mm/mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index 4e6ef896c619..387becac5c86 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
 	 * last full section, which should be mapped.
 	 */
 	if (memblock_limit)
-		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+		memblock_limit = round_down(memblock_limit, 2 * SECTION_SIZE);
 	if (!memblock_limit)
 		memblock_limit = arm_lowmem_limit;
 


-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

Memory size unaligned to section boundary

[Mark Rutland]

Hi Russell,

On Wed, May 06, 2015 at 11:11:05AM +0100, Russell King - ARM Linux wrote:
> On Thu, Apr 23, 2015 at 03:19:45PM +0200, Stefan Agner wrote:
> > Hi,
> > 
> > It seems to me that I hit an issue in low memory mapping (map_lowmem).
> > I'm using a custom memory size, which leads to an freeze on Linux 4.0
> > and also with Linus master on two tested ARMv7-A SoC's (Freescale Vybrid
> > and NVIDIA Tegra 3):
> > 
> > With mem=259744K
> > [    0.000000] Booting Linux on physical CPU 0x0
> > [    0.000000] Linux version 4.0.0-00189-ga4d2a4c3-dirty
> > (ags at trochilidae) (gcc version 4.8.3 20140401 (prerelease) (Linaro GCC
> > 4.8-2014.04) ) #506 Thu Apr 23 14:13:21 CEST 2015
> > [    0.000000] CPU: ARMv7 Processor [410fc051] revision 1 (ARMv7),
> > cr=10c5387d
> > [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing
> > instruction cache
> > [    0.000000] Machine model: Toradex Colibri VF61 on Colibri Evaluation
> > Board
> > [    0.000000] bootconsole [earlycon0] enabled
> > [    0.000000] cma: Reserved 16 MiB at 0x8e400000
> > [    0.000000] Memory policy: Data cache writeback
> > <freeze>
> > 
> > I dug a bit more into that, and it unveiled that when creating the
> > mapping for the non-kernel_x part (if (kernel_x_end < end) in
> > map_lowmem), the unaligned section at the end leads to the freeze. In
> > alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte
> > gets called which allocates a PTE outside of the initialized region (in
> > early_alloc_aligned). The system freezes at the call of memset in
> > early_alloc_aligned function.
> > 
> > With some debug print, this can be better illustrated:
> > [    0.000000] pgd 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
> > [    0.000000] pud 800063f0, addr 8fc00000, end 8fda8000, next 8fda8000
> > [    0.000000] pmd 800063f0, addr 8fc00000, next 8fda8000
> >                          => actual end of memory ^^^^^^^^
> > [    0.000000] alloc_init_pte
> > [    0.000000] set_pte_ext, pte 00000000, addr 8fc00000, end 8fda8000
> > [    0.000000] early_pte_alloc
> > [    0.000000] early_alloc_aligned, 00001000, ptr 8fcff000, align
> > 00001000
> >      => PTE allocated outside of initialized area ^^^^^^^^
> 
> Right, and the question is why does that happen - and the answer is
> this:
> 
>         /*
>          * Round the memblock limit down to a section size.  This
>          * helps to ensure that we will allocate memory from the
>          * last full section, which should be mapped.
>          */
>         if (memblock_limit)
>                 memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> 
> That should round down by 2x SECTION_SIZE to ensure that we don't start
> allocating the L2 page table in a section which isn't mapped.  Please
> try this patch:
> 
>  arch/arm/mm/mmu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index 4e6ef896c619..387becac5c86 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
>  	 * last full section, which should be mapped.
>  	 */
>  	if (memblock_limit)
> -		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> +		memblock_limit = round_down(memblock_limit, 2 * SECTION_SIZE);
>  	if (!memblock_limit)
>  		memblock_limit = arm_lowmem_limit;

For LPAE, where SECTION_SIZE == PMD_SIZE, this will set the memblock
limit a section lower than necessary.

In another subthread thread [1] I suggested roudning down to PMD_SIZE
instead, which should do the right thing regardless of LPAE.

Thanks,
Mark.

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-May/340925.html

Memory size unaligned to section boundary

[Catalin Marinas]

On Wed, May 06, 2015 at 11:11:05AM +0100, Russell King - ARM Linux wrote:
> On Thu, Apr 23, 2015 at 03:19:45PM +0200, Stefan Agner wrote:
> > I dug a bit more into that, and it unveiled that when creating the
> > mapping for the non-kernel_x part (if (kernel_x_end < end) in
> > map_lowmem), the unaligned section at the end leads to the freeze. In
> > alloc_init_pmd, if the memory end is section unaligned, alloc_init_pte
> > gets called which allocates a PTE outside of the initialized region (in
> > early_alloc_aligned). The system freezes at the call of memset in
> > early_alloc_aligned function.
[...]
> Right, and the question is why does that happen - and the answer is
> this:
> 
>         /*
>          * Round the memblock limit down to a section size.  This
>          * helps to ensure that we will allocate memory from the
>          * last full section, which should be mapped.
>          */
>         if (memblock_limit)
>                 memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> 
> That should round down by 2x SECTION_SIZE to ensure that we don't start
> allocating the L2 page table in a section which isn't mapped.  Please
> try this patch:
> 
>  arch/arm/mm/mmu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index 4e6ef896c619..387becac5c86 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
>  	 * last full section, which should be mapped.
>  	 */
>  	if (memblock_limit)
> -		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> +		memblock_limit = round_down(memblock_limit, 2 * SECTION_SIZE);

Why not PMD_SIZE? We don't need 4MB round down with LPAE.

-- 
Catalin

Memory size unaligned to section boundary

[Russell King - ARM Linux]

On Wed, May 06, 2015 at 11:51:03AM +0100, Catalin Marinas wrote:
> On Wed, May 06, 2015 at 11:11:05AM +0100, Russell King - ARM Linux wrote:
> > @@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
> >  	 * last full section, which should be mapped.
> >  	 */
> >  	if (memblock_limit)
> > -		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> > +		memblock_limit = round_down(memblock_limit, 2 * SECTION_SIZE);
> 
> Why not PMD_SIZE? We don't need 4MB round down with LPAE.

No reason - PMD_SIZE would be good there.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

[REGRESSION?] ARM: 7677/1: LPAE: Fix mapping in alloc_init_section for unaligned addresses (was Re: Memory size unaligned to section boundary)

[Laura Abbott]

On 05/05/2015 09:11 AM, Mark Rutland wrote:
>>> I wasn't able to come up with a DTB that would trigger this. Do you have
>>> an example set of memory nodes + memreserves? Where are your kernel and
>>> DTB loaded in memory?
>>
>> We've a single memory node/bank from 0x40000000 to end of memory, we
>> carve out a framebuffer at the end, and do not report that to Linux, so
>> the end becomes 0x40000000 + memory-size - fb-size, and we use no mem
>> reserves, because if we do use mem-reserves the mmap of the fbmem by
>> the simplefb driver fails as that maps it non cach-able and if it is
>> part of the main membank it already is mapped cachable.
>
> Sure. The only reason for caring about any memreserves was in case they
> inadvertently affected the memblock_limit or anything else generated by
> iterating over the memblock array. If you have none to begin with then
> they clearly aren't involved.
>
>> We substract exactly the necessary fb-size, one known fb-size which
>> triggers this is 1024x600 which means we end up substracting
>> 1024x600x4 bytes from the end of memory, so effectively we are
>> doing the same as passing a mem= argument which is not 2MiB aligned.
>
> Thanks for the info.
>
> It turns out my bootloader was silently rewriting the memory nodes,
> which was why I couldn't reproduce the issue with a DTB alone. With the
> memory node reg munged to <0 0x80000000 0 0x3FDA8000> without bootloader
> interference, TC2 dies similarly to what you described.
>
> As far as I can see the issue is not a regression; it looks like we'd
> previously fail to use a (1M) section unless we had precisely 1M or 2M
> of the section to map (as those are the only cases when end would be
> section aligned).
>
> The below hack prevents the issue by rounding the memblock_limit down to
> a full (2M) pmd boundary, so we don't try to allocate from the first
> section in a partial pmd. That does mean that if your memory ends on a
> 1M boundary you lose that last 1M for early memblock allocations.
>
> Balancing the pmd manipulation turned out to be a lot more painful than
> I'd anticipated, so I gave up on trying to map the first section in a
> partial pmd. If people are happy with the below diff I can respin as a
> patch (with comment updates and so on).
>
> Thanks,
> Mark.
>
> ---->8----
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index 4e6ef89..2ea13f0 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
>                           * occurs before any free memory is mapped.
>                           */
>                          if (!memblock_limit) {
> -                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
> +                               if (!IS_ALIGNED(block_start, PMD_SIZE))
>                                          memblock_limit = block_start;
> -                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
> +                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
>                                          memblock_limit = arm_lowmem_limit;
>                          }
>
> @@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
>           * last full section, which should be mapped.
>           */
>          if (memblock_limit)
> -               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> +               memblock_limit = round_down(memblock_limit, PMD_SIZE);
>          if (!memblock_limit)
>                  memblock_limit = arm_lowmem_limit;
>

Trying to map just the first section is super messy and I think this
is the correct fix overall. The intention with this code is to
prevent the exact problem we are running into here.

Looking at this made me realize there may be a slight bug elsewhere
though: we use memblock_limit for determining the bounds for memory
that gets passed to the buddy allocator and with this code we may
necessarily omit memory from ZONE_NORMAL. I'll think about that
some more.

Thanks,
Laura

Memory size unaligned to section boundary

[Hans de Goede]

Hi,

On 06-05-15 13:35, Russell King - ARM Linux wrote:
> On Wed, May 06, 2015 at 11:51:03AM +0100, Catalin Marinas wrote:
>> On Wed, May 06, 2015 at 11:11:05AM +0100, Russell King - ARM Linux wrote:
>>> @@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
>>>   	 * last full section, which should be mapped.
>>>   	 */
>>>   	if (memblock_limit)
>>> -		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
>>> +		memblock_limit = round_down(memblock_limit, 2 * SECTION_SIZE);
>>
>> Why not PMD_SIZE? We don't need 4MB round down with LPAE.
>
> No reason - PMD_SIZE would be good there.

Ok, so does that mean that Mark's original patch:

---->8----
diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index 4e6ef89..2ea13f0 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
                          * occurs before any free memory is mapped.
                          */
                         if (!memblock_limit) {
-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
                                         memblock_limit = block_start;
-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
                                         memblock_limit = arm_lowmem_limit;
                         }

@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
          * last full section, which should be mapped.
          */
         if (memblock_limit)
-               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+               memblock_limit = round_down(memblock_limit, PMD_SIZE);
         if (!memblock_limit)
                 memblock_limit = arm_lowmem_limit;


Is good, or do we only need to have the last chunk of this patch ?

Once I know what the final patch should look like I can give this a try
on one of the systems which sparked this whole discussion.

Regards,

Hans

Memory size unaligned to section boundary

[Russell King - ARM Linux]

On Sat, May 09, 2015 at 03:38:16PM +0200, Hans de Goede wrote:
> Ok, so does that mean that Mark's original patch:
> 
> ---->8----
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index 4e6ef89..2ea13f0 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
>                          * occurs before any free memory is mapped.
>                          */
>                         if (!memblock_limit) {
> -                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
> +                               if (!IS_ALIGNED(block_start, PMD_SIZE))
>                                         memblock_limit = block_start;
> -                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
> +                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
>                                         memblock_limit = arm_lowmem_limit;
>                         }
> 
> @@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
>          * last full section, which should be mapped.
>          */
>         if (memblock_limit)
> -               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> +               memblock_limit = round_down(memblock_limit, PMD_SIZE);
>         if (!memblock_limit)
>                 memblock_limit = arm_lowmem_limit;
> 
> 
> Is good, or do we only need to have the last chunk of this patch ?

That should do it, thanks.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

Memory size unaligned to section boundary

[Hans de Goede]

Hi,

On 09-05-15 15:48, Russell King - ARM Linux wrote:
> On Sat, May 09, 2015 at 03:38:16PM +0200, Hans de Goede wrote:
>> Ok, so does that mean that Mark's original patch:
>>
>> ---->8----
>> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
>> index 4e6ef89..2ea13f0 100644
>> --- a/arch/arm/mm/mmu.c
>> +++ b/arch/arm/mm/mmu.c
>> @@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
>>                           * occurs before any free memory is mapped.
>>                           */
>>                          if (!memblock_limit) {
>> -                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
>> +                               if (!IS_ALIGNED(block_start, PMD_SIZE))
>>                                          memblock_limit = block_start;
>> -                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
>> +                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
>>                                          memblock_limit = arm_lowmem_limit;
>>                          }
>>
>> @@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
>>           * last full section, which should be mapped.
>>           */
>>          if (memblock_limit)
>> -               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
>> +               memblock_limit = round_down(memblock_limit, PMD_SIZE);
>>          if (!memblock_limit)
>>                  memblock_limit = arm_lowmem_limit;
>>
>>
>> Is good, or do we only need to have the last chunk of this patch ?
>
> That should do it, thanks.

"that should do it" means the entire patch or only the last chunk?

Regards,

Hans

Memory size unaligned to section boundary

[Russell King - ARM Linux]

On Sat, May 09, 2015 at 03:54:00PM +0200, Hans de Goede wrote:
> Hi,
> 
> On 09-05-15 15:48, Russell King - ARM Linux wrote:
> >On Sat, May 09, 2015 at 03:38:16PM +0200, Hans de Goede wrote:
> >>Ok, so does that mean that Mark's original patch:
> >>
> >>---->8----
> >>diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> >>index 4e6ef89..2ea13f0 100644
> >>--- a/arch/arm/mm/mmu.c
> >>+++ b/arch/arm/mm/mmu.c
> >>@@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
> >>                          * occurs before any free memory is mapped.
> >>                          */
> >>                         if (!memblock_limit) {
> >>-                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
> >>+                               if (!IS_ALIGNED(block_start, PMD_SIZE))
> >>                                         memblock_limit = block_start;
> >>-                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
> >>+                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
> >>                                         memblock_limit = arm_lowmem_limit;
> >>                         }
> >>
> >>@@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
> >>          * last full section, which should be mapped.
> >>          */
> >>         if (memblock_limit)
> >>-               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> >>+               memblock_limit = round_down(memblock_limit, PMD_SIZE);
> >>         if (!memblock_limit)
> >>                 memblock_limit = arm_lowmem_limit;
> >>
> >>
> >>Is good, or do we only need to have the last chunk of this patch ?
> >
> >That should do it, thanks.
> 
> "that should do it" means the entire patch or only the last chunk?

Entire patch.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

Memory size unaligned to section boundary

[Hans de Goede]

Hi,

On 09-05-15 15:54, Russell King - ARM Linux wrote:
> On Sat, May 09, 2015 at 03:54:00PM +0200, Hans de Goede wrote:
>> Hi,
>>
>> On 09-05-15 15:48, Russell King - ARM Linux wrote:
>>> On Sat, May 09, 2015 at 03:38:16PM +0200, Hans de Goede wrote:
>>>> Ok, so does that mean that Mark's original patch:
>>>>
>>>> ---->8----
>>>> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
>>>> index 4e6ef89..2ea13f0 100644
>>>> --- a/arch/arm/mm/mmu.c
>>>> +++ b/arch/arm/mm/mmu.c
>>>> @@ -1125,9 +1125,9 @@ void __init sanity_check_meminfo(void)
>>>>                           * occurs before any free memory is mapped.
>>>>                           */
>>>>                          if (!memblock_limit) {
>>>> -                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
>>>> +                               if (!IS_ALIGNED(block_start, PMD_SIZE))
>>>>                                          memblock_limit = block_start;
>>>> -                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
>>>> +                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
>>>>                                          memblock_limit = arm_lowmem_limit;
>>>>                          }
>>>>
>>>> @@ -1142,7 +1142,7 @@ void __init sanity_check_meminfo(void)
>>>>           * last full section, which should be mapped.
>>>>           */
>>>>          if (memblock_limit)
>>>> -               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
>>>> +               memblock_limit = round_down(memblock_limit, PMD_SIZE);
>>>>          if (!memblock_limit)
>>>>                  memblock_limit = arm_lowmem_limit;
>>>>
>>>>
>>>> Is good, or do we only need to have the last chunk of this patch ?
>>>
>>> That should do it, thanks.
>>
>> "that should do it" means the entire patch or only the last chunk?
>
> Entire patch.

Ok, I'll try to give this a test run on an affected system soonish.

Regards,

Hans

[PATCH] ARM: mm: handle non-pmd-aligned end of RAM

[Mark Rutland]

At boot time we round the memblock limit down to section size in an
attempt to ensure that we will have mapped this RAM with section
mappings prior to allocating from it. When mapping RAM we iterate over
PMD-sized chunks, creating these section mappings.

Section mappings are only created when the end of a chunk is aligned to
section size. Unfortunately, with classic page tables (where PMD_SIZE is
2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in
size the first 1M will not be mapped despite having been accounted for
in the memblock limit. This has been observed to result in page tables
being allocated from unmapped memory, causing boot-time hangs.

This patch modifies the memblock limit rounding to always round down to
PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we
will round the memblock limit down to a 2M boundary, matching the limits
on section mappings, and preventing allocations from unmapped memory.
For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reported-by: Stefan Agner <stefan@agner.ch>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Hans de Goede <hdegoede@redhat.com>
Cc: Laura Abbott <labbott@redhat.com>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Steve Capper <steve.capper@linaro.org>
---
 arch/arm/mm/mmu.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
index 4e6ef89..7186382 100644
--- a/arch/arm/mm/mmu.c
+++ b/arch/arm/mm/mmu.c
@@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
 			}
 
 			/*
-			 * Find the first non-section-aligned page, and point
+			 * Find the first non-pmd-aligned page, and point
 			 * memblock_limit at it. This relies on rounding the
-			 * limit down to be section-aligned, which happens at
-			 * the end of this function.
+			 * limit down to be pmd-aligned, which happens at the
+			 * end of this function.
 			 *
 			 * With this algorithm, the start or end of almost any
-			 * bank can be non-section-aligned. The only exception
-			 * is that the start of the bank 0 must be section-
+			 * bank can be non-pmd-aligned. The only exception is
+			 * that the start of the bank 0 must be section-
 			 * aligned, since otherwise memory would need to be
 			 * allocated when mapping the start of bank 0, which
 			 * occurs before any free memory is mapped.
 			 */
 			if (!memblock_limit) {
-				if (!IS_ALIGNED(block_start, SECTION_SIZE))
+				if (!IS_ALIGNED(block_start, PMD_SIZE))
 					memblock_limit = block_start;
-				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
+				else if (!IS_ALIGNED(block_end, PMD_SIZE))
 					memblock_limit = arm_lowmem_limit;
 			}
 
@@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
 	high_memory = __va(arm_lowmem_limit - 1) + 1;
 
 	/*
-	 * Round the memblock limit down to a section size.  This
+	 * Round the memblock limit down to a pmd size.  This
 	 * helps to ensure that we will allocate memory from the
-	 * last full section, which should be mapped.
+	 * last full pmd, which should be mapped.
 	 */
 	if (memblock_limit)
-		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
+		memblock_limit = round_down(memblock_limit, PMD_SIZE);
 	if (!memblock_limit)
 		memblock_limit = arm_lowmem_limit;
 
-- 
1.9.1

[PATCH] ARM: mm: handle non-pmd-aligned end of RAM

[Stefan Agner]

On 2015-05-11 12:31, Mark Rutland wrote:
> At boot time we round the memblock limit down to section size in an
> attempt to ensure that we will have mapped this RAM with section
> mappings prior to allocating from it. When mapping RAM we iterate over
> PMD-sized chunks, creating these section mappings.
> 
> Section mappings are only created when the end of a chunk is aligned to
> section size. Unfortunately, with classic page tables (where PMD_SIZE is
> 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in
> size the first 1M will not be mapped despite having been accounted for
> in the memblock limit. This has been observed to result in page tables
> being allocated from unmapped memory, causing boot-time hangs.
> 
> This patch modifies the memblock limit rounding to always round down to
> PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we
> will round the memblock limit down to a 2M boundary, matching the limits
> on section mappings, and preventing allocations from unmapped memory.
> For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.

Thanks Mark, just tested that patch on the hardware I had the issue,
looks good.

Tested-by: Stefan Agner <stefan@agner.ch>

> 
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Reported-by: Stefan Agner <stefan@agner.ch>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: Laura Abbott <labbott@redhat.com>
> Cc: Russell King <rmk+kernel@arm.linux.org.uk>
> Cc: Steve Capper <steve.capper@linaro.org>
> ---
>  arch/arm/mm/mmu.c | 20 ++++++++++----------
>  1 file changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index 4e6ef89..7186382 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
>  			}
>  
>  			/*
> -			 * Find the first non-section-aligned page, and point
> +			 * Find the first non-pmd-aligned page, and point
>  			 * memblock_limit at it. This relies on rounding the
> -			 * limit down to be section-aligned, which happens at
> -			 * the end of this function.
> +			 * limit down to be pmd-aligned, which happens at the
> +			 * end of this function.
>  			 *
>  			 * With this algorithm, the start or end of almost any
> -			 * bank can be non-section-aligned. The only exception
> -			 * is that the start of the bank 0 must be section-
> +			 * bank can be non-pmd-aligned. The only exception is
> +			 * that the start of the bank 0 must be section-
>  			 * aligned, since otherwise memory would need to be
>  			 * allocated when mapping the start of bank 0, which
>  			 * occurs before any free memory is mapped.
>  			 */
>  			if (!memblock_limit) {
> -				if (!IS_ALIGNED(block_start, SECTION_SIZE))
> +				if (!IS_ALIGNED(block_start, PMD_SIZE))
>  					memblock_limit = block_start;
> -				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
> +				else if (!IS_ALIGNED(block_end, PMD_SIZE))
>  					memblock_limit = arm_lowmem_limit;
>  			}
>  
> @@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
>  	high_memory = __va(arm_lowmem_limit - 1) + 1;
>  
>  	/*
> -	 * Round the memblock limit down to a section size.  This
> +	 * Round the memblock limit down to a pmd size.  This
>  	 * helps to ensure that we will allocate memory from the
> -	 * last full section, which should be mapped.
> +	 * last full pmd, which should be mapped.
>  	 */
>  	if (memblock_limit)
> -		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> +		memblock_limit = round_down(memblock_limit, PMD_SIZE);
>  	if (!memblock_limit)
>  		memblock_limit = arm_lowmem_limit;

[PATCH] ARM: mm: handle non-pmd-aligned end of RAM

[Laura Abbott]

On 05/11/2015 03:31 AM, Mark Rutland wrote:
> At boot time we round the memblock limit down to section size in an
> attempt to ensure that we will have mapped this RAM with section
> mappings prior to allocating from it. When mapping RAM we iterate over
> PMD-sized chunks, creating these section mappings.
>
> Section mappings are only created when the end of a chunk is aligned to
> section size. Unfortunately, with classic page tables (where PMD_SIZE is
> 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in
> size the first 1M will not be mapped despite having been accounted for
> in the memblock limit. This has been observed to result in page tables
> being allocated from unmapped memory, causing boot-time hangs.
>
> This patch modifies the memblock limit rounding to always round down to
> PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we
> will round the memblock limit down to a 2M boundary, matching the limits
> on section mappings, and preventing allocations from unmapped memory.
> For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.
>
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Reported-by: Stefan Agner <stefan@agner.ch>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: Laura Abbott <labbott@redhat.com>
> Cc: Russell King <rmk+kernel@arm.linux.org.uk>
> Cc: Steve Capper <steve.capper@linaro.org>

Acked-by: Laura Abbott <labbott@redhat.com>

> ---
>   arch/arm/mm/mmu.c | 20 ++++++++++----------
>   1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index 4e6ef89..7186382 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
>   			}
>
>   			/*
> -			 * Find the first non-section-aligned page, and point
> +			 * Find the first non-pmd-aligned page, and point
>   			 * memblock_limit at it. This relies on rounding the
> -			 * limit down to be section-aligned, which happens at
> -			 * the end of this function.
> +			 * limit down to be pmd-aligned, which happens at the
> +			 * end of this function.
>   			 *
>   			 * With this algorithm, the start or end of almost any
> -			 * bank can be non-section-aligned. The only exception
> -			 * is that the start of the bank 0 must be section-
> +			 * bank can be non-pmd-aligned. The only exception is
> +			 * that the start of the bank 0 must be section-
>   			 * aligned, since otherwise memory would need to be
>   			 * allocated when mapping the start of bank 0, which
>   			 * occurs before any free memory is mapped.
>   			 */
>   			if (!memblock_limit) {
> -				if (!IS_ALIGNED(block_start, SECTION_SIZE))
> +				if (!IS_ALIGNED(block_start, PMD_SIZE))
>   					memblock_limit = block_start;
> -				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
> +				else if (!IS_ALIGNED(block_end, PMD_SIZE))
>   					memblock_limit = arm_lowmem_limit;
>   			}
>
> @@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
>   	high_memory = __va(arm_lowmem_limit - 1) + 1;
>
>   	/*
> -	 * Round the memblock limit down to a section size.  This
> +	 * Round the memblock limit down to a pmd size.  This
>   	 * helps to ensure that we will allocate memory from the
> -	 * last full section, which should be mapped.
> +	 * last full pmd, which should be mapped.
>   	 */
>   	if (memblock_limit)
> -		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> +		memblock_limit = round_down(memblock_limit, PMD_SIZE);
>   	if (!memblock_limit)
>   		memblock_limit = arm_lowmem_limit;
>
>

[PATCH] ARM: mm: handle non-pmd-aligned end of RAM

[Hans de Goede]

Hi,

On 11-05-15 13:43, Stefan Agner wrote:
> On 2015-05-11 12:31, Mark Rutland wrote:
>> At boot time we round the memblock limit down to section size in an
>> attempt to ensure that we will have mapped this RAM with section
>> mappings prior to allocating from it. When mapping RAM we iterate over
>> PMD-sized chunks, creating these section mappings.
>>
>> Section mappings are only created when the end of a chunk is aligned to
>> section size. Unfortunately, with classic page tables (where PMD_SIZE is
>> 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in
>> size the first 1M will not be mapped despite having been accounted for
>> in the memblock limit. This has been observed to result in page tables
>> being allocated from unmapped memory, causing boot-time hangs.
>>
>> This patch modifies the memblock limit rounding to always round down to
>> PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we
>> will round the memblock limit down to a 2M boundary, matching the limits
>> on section mappings, and preventing allocations from unmapped memory.
>> For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.
>
> Thanks Mark, just tested that patch on the hardware I had the issue,
> looks good.
>
> Tested-by: Stefan Agner <stefan@agner.ch>

Same for me, this also fixes the issue I was seeing no an Allwinner A33
tablet with 1024z600 lcd screen.

Tested-by: Hans de Goede <hdegoede@redhat.com>

Can we get this Cc-ed to stable at vger.kernel.org please? At least for 4.0 ?

Regards,

Hans


>
>>
>> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
>> Reported-by: Stefan Agner <stefan@agner.ch>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Hans de Goede <hdegoede@redhat.com>
>> Cc: Laura Abbott <labbott@redhat.com>
>> Cc: Russell King <rmk+kernel@arm.linux.org.uk>
>> Cc: Steve Capper <steve.capper@linaro.org>
>> ---
>>   arch/arm/mm/mmu.c | 20 ++++++++++----------
>>   1 file changed, 10 insertions(+), 10 deletions(-)
>>
>> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
>> index 4e6ef89..7186382 100644
>> --- a/arch/arm/mm/mmu.c
>> +++ b/arch/arm/mm/mmu.c
>> @@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
>>   			}
>>
>>   			/*
>> -			 * Find the first non-section-aligned page, and point
>> +			 * Find the first non-pmd-aligned page, and point
>>   			 * memblock_limit at it. This relies on rounding the
>> -			 * limit down to be section-aligned, which happens at
>> -			 * the end of this function.
>> +			 * limit down to be pmd-aligned, which happens at the
>> +			 * end of this function.
>>   			 *
>>   			 * With this algorithm, the start or end of almost any
>> -			 * bank can be non-section-aligned. The only exception
>> -			 * is that the start of the bank 0 must be section-
>> +			 * bank can be non-pmd-aligned. The only exception is
>> +			 * that the start of the bank 0 must be section-
>>   			 * aligned, since otherwise memory would need to be
>>   			 * allocated when mapping the start of bank 0, which
>>   			 * occurs before any free memory is mapped.
>>   			 */
>>   			if (!memblock_limit) {
>> -				if (!IS_ALIGNED(block_start, SECTION_SIZE))
>> +				if (!IS_ALIGNED(block_start, PMD_SIZE))
>>   					memblock_limit = block_start;
>> -				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
>> +				else if (!IS_ALIGNED(block_end, PMD_SIZE))
>>   					memblock_limit = arm_lowmem_limit;
>>   			}
>>
>> @@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
>>   	high_memory = __va(arm_lowmem_limit - 1) + 1;
>>
>>   	/*
>> -	 * Round the memblock limit down to a section size.  This
>> +	 * Round the memblock limit down to a pmd size.  This
>>   	 * helps to ensure that we will allocate memory from the
>> -	 * last full section, which should be mapped.
>> +	 * last full pmd, which should be mapped.
>>   	 */
>>   	if (memblock_limit)
>> -		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
>> +		memblock_limit = round_down(memblock_limit, PMD_SIZE);
>>   	if (!memblock_limit)
>>   		memblock_limit = arm_lowmem_limit;
>

[PATCH] ARM: mm: handle non-pmd-aligned end of RAM

[Mark Rutland]

On Wed, May 13, 2015 at 02:40:16PM +0100, Hans de Goede wrote:
> Hi,
> 
> On 11-05-15 13:43, Stefan Agner wrote:
> > On 2015-05-11 12:31, Mark Rutland wrote:
> >> At boot time we round the memblock limit down to section size in an
> >> attempt to ensure that we will have mapped this RAM with section
> >> mappings prior to allocating from it. When mapping RAM we iterate over
> >> PMD-sized chunks, creating these section mappings.
> >>
> >> Section mappings are only created when the end of a chunk is aligned to
> >> section size. Unfortunately, with classic page tables (where PMD_SIZE is
> >> 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in
> >> size the first 1M will not be mapped despite having been accounted for
> >> in the memblock limit. This has been observed to result in page tables
> >> being allocated from unmapped memory, causing boot-time hangs.
> >>
> >> This patch modifies the memblock limit rounding to always round down to
> >> PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we
> >> will round the memblock limit down to a 2M boundary, matching the limits
> >> on section mappings, and preventing allocations from unmapped memory.
> >> For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.
> >
> > Thanks Mark, just tested that patch on the hardware I had the issue,
> > looks good.
> >
> > Tested-by: Stefan Agner <stefan@agner.ch>
> 
> Same for me, this also fixes the issue I was seeing no an Allwinner A33
> tablet with 1024z600 lcd screen.
> 
> Tested-by: Hans de Goede <hdegoede@redhat.com>

Great. Thanks for testing!

> Can we get this Cc-ed to stable at vger.kernel.org please? At least for 4.0 ?

Sure, done.

Russell, on the assumption you're ok with the patch I've submitted it to
the patch system as 8356/1.

Thanks,
Mark.

> Regards,
> 
> Hans
> 
> 
> >
> >>
> >> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> >> Reported-by: Stefan Agner <stefan@agner.ch>
> >> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >> Cc: Hans de Goede <hdegoede@redhat.com>
> >> Cc: Laura Abbott <labbott@redhat.com>
> >> Cc: Russell King <rmk+kernel@arm.linux.org.uk>
> >> Cc: Steve Capper <steve.capper@linaro.org>
> >> ---
> >>   arch/arm/mm/mmu.c | 20 ++++++++++----------
> >>   1 file changed, 10 insertions(+), 10 deletions(-)
> >>
> >> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> >> index 4e6ef89..7186382 100644
> >> --- a/arch/arm/mm/mmu.c
> >> +++ b/arch/arm/mm/mmu.c
> >> @@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
> >>   			}
> >>
> >>   			/*
> >> -			 * Find the first non-section-aligned page, and point
> >> +			 * Find the first non-pmd-aligned page, and point
> >>   			 * memblock_limit at it. This relies on rounding the
> >> -			 * limit down to be section-aligned, which happens at
> >> -			 * the end of this function.
> >> +			 * limit down to be pmd-aligned, which happens at the
> >> +			 * end of this function.
> >>   			 *
> >>   			 * With this algorithm, the start or end of almost any
> >> -			 * bank can be non-section-aligned. The only exception
> >> -			 * is that the start of the bank 0 must be section-
> >> +			 * bank can be non-pmd-aligned. The only exception is
> >> +			 * that the start of the bank 0 must be section-
> >>   			 * aligned, since otherwise memory would need to be
> >>   			 * allocated when mapping the start of bank 0, which
> >>   			 * occurs before any free memory is mapped.
> >>   			 */
> >>   			if (!memblock_limit) {
> >> -				if (!IS_ALIGNED(block_start, SECTION_SIZE))
> >> +				if (!IS_ALIGNED(block_start, PMD_SIZE))
> >>   					memblock_limit = block_start;
> >> -				else if (!IS_ALIGNED(block_end, SECTION_SIZE))
> >> +				else if (!IS_ALIGNED(block_end, PMD_SIZE))
> >>   					memblock_limit = arm_lowmem_limit;
> >>   			}
> >>
> >> @@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
> >>   	high_memory = __va(arm_lowmem_limit - 1) + 1;
> >>
> >>   	/*
> >> -	 * Round the memblock limit down to a section size.  This
> >> +	 * Round the memblock limit down to a pmd size.  This
> >>   	 * helps to ensure that we will allocate memory from the
> >> -	 * last full section, which should be mapped.
> >> +	 * last full pmd, which should be mapped.
> >>   	 */
> >>   	if (memblock_limit)
> >> -		memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> >> +		memblock_limit = round_down(memblock_limit, PMD_SIZE);
> >>   	if (!memblock_limit)
> >>   		memblock_limit = arm_lowmem_limit;
> >
> 

[PATCH] ARM: mm: handle non-pmd-aligned end of RAM

[Russell King - ARM Linux]

On Wed, May 13, 2015 at 03:11:52PM +0100, Mark Rutland wrote:
> Russell, on the assumption you're ok with the patch I've submitted it to
> the patch system as 8356/1.

Thanks.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

[PATCH] ARM: mm: handle non-pmd-aligned end of RAM

[Javier Martinez Canillas]

[Adding Krzysztof, Doug, Kukjin and Sjoerd to cc since they are also
familiar with this platform]

Hello Mark,

On Mon, May 11, 2015 at 12:31 PM, Mark Rutland <mark.rutland@arm.com> wrote:
> At boot time we round the memblock limit down to section size in an
> attempt to ensure that we will have mapped this RAM with section
> mappings prior to allocating from it. When mapping RAM we iterate over
> PMD-sized chunks, creating these section mappings.
>
> Section mappings are only created when the end of a chunk is aligned to
> section size. Unfortunately, with classic page tables (where PMD_SIZE is
> 2 * SECTION_SIZE) this means that if a chunk is between 1M and 2M in
> size the first 1M will not be mapped despite having been accounted for
> in the memblock limit. This has been observed to result in page tables
> being allocated from unmapped memory, causing boot-time hangs.
>
> This patch modifies the memblock limit rounding to always round down to
> PMD_SIZE instead of SECTION_SIZE. For classic MMU this means that we
> will round the memblock limit down to a 2M boundary, matching the limits
> on section mappings, and preventing allocations from unmapped memory.
> For LPAE there should be no change as PMD_SIZE == SECTION_SIZE.
>
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Reported-by: Stefan Agner <stefan@agner.ch>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Hans de Goede <hdegoede@redhat.com>
> Cc: Laura Abbott <labbott@redhat.com>
> Cc: Russell King <rmk+kernel@arm.linux.org.uk>
> Cc: Steve Capper <steve.capper@linaro.org>
> ---
>  arch/arm/mm/mmu.c | 20 ++++++++++----------
>  1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c
> index 4e6ef89..7186382 100644
> --- a/arch/arm/mm/mmu.c
> +++ b/arch/arm/mm/mmu.c
> @@ -1112,22 +1112,22 @@ void __init sanity_check_meminfo(void)
>                         }
>
>                         /*
> -                        * Find the first non-section-aligned page, and point
> +                        * Find the first non-pmd-aligned page, and point
>                          * memblock_limit at it. This relies on rounding the
> -                        * limit down to be section-aligned, which happens at
> -                        * the end of this function.
> +                        * limit down to be pmd-aligned, which happens at the
> +                        * end of this function.
>                          *
>                          * With this algorithm, the start or end of almost any
> -                        * bank can be non-section-aligned. The only exception
> -                        * is that the start of the bank 0 must be section-
> +                        * bank can be non-pmd-aligned. The only exception is
> +                        * that the start of the bank 0 must be section-
>                          * aligned, since otherwise memory would need to be
>                          * allocated when mapping the start of bank 0, which
>                          * occurs before any free memory is mapped.
>                          */
>                         if (!memblock_limit) {
> -                               if (!IS_ALIGNED(block_start, SECTION_SIZE))
> +                               if (!IS_ALIGNED(block_start, PMD_SIZE))
>                                         memblock_limit = block_start;
> -                               else if (!IS_ALIGNED(block_end, SECTION_SIZE))
> +                               else if (!IS_ALIGNED(block_end, PMD_SIZE))
>                                         memblock_limit = arm_lowmem_limit;
>                         }
>
> @@ -1137,12 +1137,12 @@ void __init sanity_check_meminfo(void)
>         high_memory = __va(arm_lowmem_limit - 1) + 1;
>
>         /*
> -        * Round the memblock limit down to a section size.  This
> +        * Round the memblock limit down to a pmd size.  This
>          * helps to ensure that we will allocate memory from the
> -        * last full section, which should be mapped.
> +        * last full pmd, which should be mapped.
>          */
>         if (memblock_limit)
> -               memblock_limit = round_down(memblock_limit, SECTION_SIZE);
> +               memblock_limit = round_down(memblock_limit, PMD_SIZE);
>         if (!memblock_limit)
>                 memblock_limit = arm_lowmem_limit;
>
> --

There were reports of 4.1 not booting on the Exynos Chromebooks. I did
a bisect on an Exynos5420 Peach Pit Chromebook and tracked it down to
$subject as the cause. Reverting this commit makes it to boot again.

In case someone is not familiar with the Exynos Chromebooks, they have
a read-only u-boot that can only boot signed images so there are two
ways to boot a kernel:

1) Booting a signed FIT image that contains the zImage + FDT
2) Chain loading a signed u-boot image (i.e: mainline u-boot) that can
in turn boot a non signed kernel image

4.1 is failing to boot only for 1), doing 2) leads to the kernel
booting successfully.

Adding some printouts I found that the memory layout and total amount
of RAM is different when booting a FIT image directly and when chain
loading a mainline u-boot.

when chain loading a mainline u-boot:
------------------------------------------------------

memory block: block_start 0x40000000 size 0x60000000

This 1.5 GiB block is both PMD_SIZE and SECTION_SIZE aligned

when booting using the vendor u-boot and a FIT:
---------------------------------------------------------------------

memory block: block_start 0x20000000 size 0x1e00000
memory block: block_start 0x21f00000 size 0x7e100000

The first 30MiB block is both PMD_SIZE and SECTION_SIZE aligned but
the second (~2GiB) block start is SECTION_SIZE aligned but no PMD_SIZE
aligned.

The Device Tree memory node in arch/arm/boot/dts/exynos5420-peach-pit.dts is:

memory {
      reg = <0x20000000 0x80000000>;
};

So besides wondering which memory layout is the correct one, I wonder
if this patch is not just exposing another latent bug on this
platform.

Since before this patch, memblock_limit was always set to
arm_lowmem_limit in both cases because the memory blocks were always
aligned to SECTION_SIZE regardless if booting a FIT directly or chain
loading mainline u-boot.

But this is not true anymore when rounding memblock_limit down to
PMD_SIZE since when booting using a FIT, the second memory area start
is only SECTION_SIZE aligned but no PMD_SIZE aligned.

Best regards,
Javier

[PATCH] ARM: mm: handle non-pmd-aligned end of RAM

[Javier Martinez Canillas]

On Tue, Jun 30, 2015 at 12:01 PM, Javier Martinez Canillas
<javier@dowhile0.org> wrote:

[...]

>
> There were reports of 4.1 not booting on the Exynos Chromebooks. I did
> a bisect on an Exynos5420 Peach Pit Chromebook and tracked it down to
> $subject as the cause. Reverting this commit makes it to boot again.
>

I found that the following posted patch by Laura Abbot makes the
system to boot again:

[PATCH] arm: Update memblock limit after mapping lowmem [0].

Best regards,
Javier

[0]: https://lkml.org/lkml/2015/6/4/606