* ARM64 kernel Image size limitations?
@ 2017-08-28 19:55 Florian Fainelli
2017-08-28 20:01 ` Ard Biesheuvel
2017-08-29 10:31 ` Mark Rutland
0 siblings, 2 replies; 7+ messages in thread
From: Florian Fainelli @ 2017-08-28 19:55 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
I have been fighting with a reasonably large kernel Image: 46MB and
found unable to boot it. This is on v4.13-rc6, but I could go back as
far as v4.9~ish. It is so big, because it's got some debugging enabled,
and the initramfs compression is disabled, but still that ought to be
possible.
After some experimentation I found that the breaking size is somewhere
around 33MB total for arch/arm64/boot/Image and when that happens, I get
the following backtrace which suggests that there is some memory
corruption of some kind (or so it seems to me), other experiments
indicated that the backtrace may point back to where the bootloader had
set up its exception vector. I am fairly confident that the bootloader
is not responsible for corrupting the Image that is loaded.
Is there such a size limitation or am I possibly tripping over something
else?
(gdb) bt
#0 0xffffff800880721c in arch_spin_lock (lock=<optimized out>)
at ./arch/arm64/include/asm/spinlock.h:89
#1 do_raw_spin_lock_flags (flags=<optimized out>, lock=<optimized out>)
at ./include/linux/spinlock.h:155
#2 __raw_spin_lock_irqsave (lock=<optimized out>)
at ./include/linux/spinlock_api_smp.h:119
#3 _raw_spin_lock_irqsave (lock=0xffffff800a1100f8)
at kernel/locking/spinlock.c:159
#4 0xffffff80080888b4 in die (str=0xffffff800894a490 "Oops",
regs=0xffffff800a013630, err=-1778384892) at
arch/arm64/kernel/traps.c:279
#5 0x80ffffff080952c4 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb) info reg
x0 0xffffff800a1100f8 18446743524122624248
x1 0x1c0 448
x2 0x10000 65536
x3 0x1 1
x4 0x0 0
x5 0xffffff8008857dc8 18446743524096703944
x6 0xffffff800a00494f 18446743524121528655
x7 0x6572646461206c65 7310015527464758373
x8 0x65676e6172207373 7306930285237531507
x9 0x0 0
x10 0xffffff800a0134e0 18446743524121588960
x11 0xffffff800a0134e0 18446743524121588960
x12 0x40d00 265472
x13 0x72422820352e382e 8233187187582973998
x14 0x73206d6f6364616f 8295750838792773999
x15 0x0 0
x16 0x0 0
x17 0x0 0
x18 0x0 0
x19 0xffffff800a110000 18446743524122624000
x20 0x96000004 2516582404
x21 0xffffff800894a490 18446743524097696912
x22 0xffffff800a013630 18446743524121589296
x23 0xffffff800a013630 18446743524121589296
x24 0x25 37
x25 0xffffff800a018000 18446743524121608192
x26 0xffffff8008955622 18446743524097742370
x27 0xffffff8008955622 18446743524097742370
x28 0xffffff800a01dd80 18446743524121632128
x29 0xffffff800a0134d0 18446743524121588944
x30 0xffffff80080888b4 18446743524088514740
sp 0xffffff800a0134d0 0xffffff800a0134d0
pc 0xffffff800880721c 0xffffff800880721c
<_raw_spin_lock_irqsave+44>
CPSR 0x600001c5 1610613189
(gdb)
--
Florian
^ permalink raw reply [flat|nested] 7+ messages in thread
* ARM64 kernel Image size limitations?
2017-08-28 19:55 ARM64 kernel Image size limitations? Florian Fainelli
@ 2017-08-28 20:01 ` Ard Biesheuvel
2017-08-28 21:24 ` Florian Fainelli
2017-08-29 10:31 ` Mark Rutland
1 sibling, 1 reply; 7+ messages in thread
From: Ard Biesheuvel @ 2017-08-28 20:01 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On 28 August 2017 at 20:55, Florian Fainelli <f.fainelli@gmail.com> wrote:
> Hi,
>
> I have been fighting with a reasonably large kernel Image: 46MB and
> found unable to boot it. This is on v4.13-rc6, but I could go back as
> far as v4.9~ish. It is so big, because it's got some debugging enabled,
> and the initramfs compression is disabled, but still that ought to be
> possible.
>
> After some experimentation I found that the breaking size is somewhere
> around 33MB total for arch/arm64/boot/Image and when that happens, I get
> the following backtrace which suggests that there is some memory
> corruption of some kind (or so it seems to me), other experiments
> indicated that the backtrace may point back to where the bootloader had
> set up its exception vector. I am fairly confident that the bootloader
> is not responsible for corrupting the Image that is loaded.
>
> Is there such a size limitation or am I possibly tripping over something
> else?
>
What boot environment are you using? Is it possible the DTB is copied
on top of the kernel by the boot loader?
^ permalink raw reply [flat|nested] 7+ messages in thread
* ARM64 kernel Image size limitations?
2017-08-28 20:01 ` Ard Biesheuvel
@ 2017-08-28 21:24 ` Florian Fainelli
2017-08-29 8:15 ` Ard Biesheuvel
0 siblings, 1 reply; 7+ messages in thread
From: Florian Fainelli @ 2017-08-28 21:24 UTC (permalink / raw)
To: linux-arm-kernel
(fixed Will's address, apologies)
On 08/28/2017 01:01 PM, Ard Biesheuvel wrote:
> Hi,
>
> On 28 August 2017 at 20:55, Florian Fainelli <f.fainelli@gmail.com> wrote:
>> Hi,
>>
>> I have been fighting with a reasonably large kernel Image: 46MB and
>> found unable to boot it. This is on v4.13-rc6, but I could go back as
>> far as v4.9~ish. It is so big, because it's got some debugging enabled,
>> and the initramfs compression is disabled, but still that ought to be
>> possible.
>>
>> After some experimentation I found that the breaking size is somewhere
>> around 33MB total for arch/arm64/boot/Image and when that happens, I get
>> the following backtrace which suggests that there is some memory
>> corruption of some kind (or so it seems to me), other experiments
>> indicated that the backtrace may point back to where the bootloader had
>> set up its exception vector. I am fairly confident that the bootloader
>> is not responsible for corrupting the Image that is loaded.
>>
>> Is there such a size limitation or am I possibly tripping over something
>> else?
>>
>
> What boot environment are you using? Is it possible the DTB is copied
> on top of the kernel by the boot loader?
Bootloader is BOLT (Broadcom's own implementation, has PSCI and is
capable of doing FDT live patching), and yes the DTB is provided by the
bootloader at an address higher than the kernel.
Physical offset of the RAM on this platform is 0, so the load is going
to be at PA 0x80000, 0x80000 + size (35MB) = 0x2280000, DTB is placed at
0x771f000 and is 0x90ef bytes.
--
Florian
^ permalink raw reply [flat|nested] 7+ messages in thread
* ARM64 kernel Image size limitations?
2017-08-28 21:24 ` Florian Fainelli
@ 2017-08-29 8:15 ` Ard Biesheuvel
2017-08-29 16:51 ` Florian Fainelli
0 siblings, 1 reply; 7+ messages in thread
From: Ard Biesheuvel @ 2017-08-29 8:15 UTC (permalink / raw)
To: linux-arm-kernel
On 28 August 2017 at 22:24, Florian Fainelli <f.fainelli@gmail.com> wrote:
> (fixed Will's address, apologies)
>
> On 08/28/2017 01:01 PM, Ard Biesheuvel wrote:
>> Hi,
>>
>> On 28 August 2017 at 20:55, Florian Fainelli <f.fainelli@gmail.com> wrote:
>>> Hi,
>>>
>>> I have been fighting with a reasonably large kernel Image: 46MB and
>>> found unable to boot it. This is on v4.13-rc6, but I could go back as
>>> far as v4.9~ish. It is so big, because it's got some debugging enabled,
>>> and the initramfs compression is disabled, but still that ought to be
>>> possible.
>>>
>>> After some experimentation I found that the breaking size is somewhere
>>> around 33MB total for arch/arm64/boot/Image and when that happens, I get
>>> the following backtrace which suggests that there is some memory
>>> corruption of some kind (or so it seems to me), other experiments
>>> indicated that the backtrace may point back to where the bootloader had
>>> set up its exception vector. I am fairly confident that the bootloader
>>> is not responsible for corrupting the Image that is loaded.
>>>
>>> Is there such a size limitation or am I possibly tripping over something
>>> else?
>>>
>>
>> What boot environment are you using? Is it possible the DTB is copied
>> on top of the kernel by the boot loader?
>
> Bootloader is BOLT (Broadcom's own implementation, has PSCI and is
> capable of doing FDT live patching), and yes the DTB is provided by the
> bootloader at an address higher than the kernel.
>
> Physical offset of the RAM on this platform is 0, so the load is going
> to be at PA 0x80000, 0x80000 + size (35MB) = 0x2280000, DTB is placed at
> 0x771f000 and is 0x90ef bytes.
>
Could you try to dump __log_buf from gdb?
^ permalink raw reply [flat|nested] 7+ messages in thread
* ARM64 kernel Image size limitations?
2017-08-29 8:15 ` Ard Biesheuvel
@ 2017-08-29 16:51 ` Florian Fainelli
2017-08-29 16:58 ` Florian Fainelli
0 siblings, 1 reply; 7+ messages in thread
From: Florian Fainelli @ 2017-08-29 16:51 UTC (permalink / raw)
To: linux-arm-kernel
On 08/29/2017 01:15 AM, Ard Biesheuvel wrote:
> On 28 August 2017 at 22:24, Florian Fainelli <f.fainelli@gmail.com> wrote:
>> (fixed Will's address, apologies)
>>
>> On 08/28/2017 01:01 PM, Ard Biesheuvel wrote:
>>> Hi,
>>>
>>> On 28 August 2017 at 20:55, Florian Fainelli <f.fainelli@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I have been fighting with a reasonably large kernel Image: 46MB and
>>>> found unable to boot it. This is on v4.13-rc6, but I could go back as
>>>> far as v4.9~ish. It is so big, because it's got some debugging enabled,
>>>> and the initramfs compression is disabled, but still that ought to be
>>>> possible.
>>>>
>>>> After some experimentation I found that the breaking size is somewhere
>>>> around 33MB total for arch/arm64/boot/Image and when that happens, I get
>>>> the following backtrace which suggests that there is some memory
>>>> corruption of some kind (or so it seems to me), other experiments
>>>> indicated that the backtrace may point back to where the bootloader had
>>>> set up its exception vector. I am fairly confident that the bootloader
>>>> is not responsible for corrupting the Image that is loaded.
>>>>
>>>> Is there such a size limitation or am I possibly tripping over something
>>>> else?
>>>>
>>>
>>> What boot environment are you using? Is it possible the DTB is copied
>>> on top of the kernel by the boot loader?
>>
>> Bootloader is BOLT (Broadcom's own implementation, has PSCI and is
>> capable of doing FDT live patching), and yes the DTB is provided by the
>> bootloader at an address higher than the kernel.
>>
>> Physical offset of the RAM on this platform is 0, so the load is going
>> to be at PA 0x80000, 0x80000 + size (35MB) = 0x2280000, DTB is placed at
>> 0x771f000 and is 0x90ef bytes.
>>
>
> Could you try to dump __log_buf from gdb?
Silly me, this is a bootloader limitation, unlike what I thought the
load address is not at PA 0, it is at PA 0x06FFC000 and there is only
~34MB worth of space available, mayhem ensues.
Thanks!
--
Florian
^ permalink raw reply [flat|nested] 7+ messages in thread
* ARM64 kernel Image size limitations?
2017-08-29 16:51 ` Florian Fainelli
@ 2017-08-29 16:58 ` Florian Fainelli
0 siblings, 0 replies; 7+ messages in thread
From: Florian Fainelli @ 2017-08-29 16:58 UTC (permalink / raw)
To: linux-arm-kernel
On 08/29/2017 09:51 AM, Florian Fainelli wrote:
> On 08/29/2017 01:15 AM, Ard Biesheuvel wrote:
>> On 28 August 2017 at 22:24, Florian Fainelli <f.fainelli@gmail.com> wrote:
>>> (fixed Will's address, apologies)
>>>
>>> On 08/28/2017 01:01 PM, Ard Biesheuvel wrote:
>>>> Hi,
>>>>
>>>> On 28 August 2017 at 20:55, Florian Fainelli <f.fainelli@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I have been fighting with a reasonably large kernel Image: 46MB and
>>>>> found unable to boot it. This is on v4.13-rc6, but I could go back as
>>>>> far as v4.9~ish. It is so big, because it's got some debugging enabled,
>>>>> and the initramfs compression is disabled, but still that ought to be
>>>>> possible.
>>>>>
>>>>> After some experimentation I found that the breaking size is somewhere
>>>>> around 33MB total for arch/arm64/boot/Image and when that happens, I get
>>>>> the following backtrace which suggests that there is some memory
>>>>> corruption of some kind (or so it seems to me), other experiments
>>>>> indicated that the backtrace may point back to where the bootloader had
>>>>> set up its exception vector. I am fairly confident that the bootloader
>>>>> is not responsible for corrupting the Image that is loaded.
>>>>>
>>>>> Is there such a size limitation or am I possibly tripping over something
>>>>> else?
>>>>>
>>>>
>>>> What boot environment are you using? Is it possible the DTB is copied
>>>> on top of the kernel by the boot loader?
>>>
>>> Bootloader is BOLT (Broadcom's own implementation, has PSCI and is
>>> capable of doing FDT live patching), and yes the DTB is provided by the
>>> bootloader at an address higher than the kernel.
>>>
>>> Physical offset of the RAM on this platform is 0, so the load is going
>>> to be at PA 0x80000, 0x80000 + size (35MB) = 0x2280000, DTB is placed at
>>> 0x771f000 and is 0x90ef bytes.
>>>
>>
>> Could you try to dump __log_buf from gdb?
>
> Silly me, this is a bootloader limitation, unlike what I thought the
> load address is not at PA 0, it is at PA 0x06FFC000 and there is only
> ~34MB worth of space available, mayhem ensues.
Most definitively -ENOCOFFEE it is loaded at PA 0x80000 as I expect it
to, I will try to get you a log after drinking some coffee ;)
--
Florian
^ permalink raw reply [flat|nested] 7+ messages in thread
* ARM64 kernel Image size limitations?
2017-08-28 19:55 ARM64 kernel Image size limitations? Florian Fainelli
2017-08-28 20:01 ` Ard Biesheuvel
@ 2017-08-29 10:31 ` Mark Rutland
1 sibling, 0 replies; 7+ messages in thread
From: Mark Rutland @ 2017-08-29 10:31 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Aug 28, 2017 at 12:55:23PM -0700, Florian Fainelli wrote:
> Hi,
>
> I have been fighting with a reasonably large kernel Image: 46MB and
> found unable to boot it. This is on v4.13-rc6, but I could go back as
> far as v4.9~ish. It is so big, because it's got some debugging enabled,
> and the initramfs compression is disabled, but still that ought to be
> possible.
I regularly boot 90+ MB kernels, so this is certainly possible, in
regular-ish configurations.
> After some experimentation I found that the breaking size is somewhere
> around 33MB total for arch/arm64/boot/Image and when that happens, I get
> the following backtrace which suggests that there is some memory
> corruption of some kind (or so it seems to me), other experiments
> indicated that the backtrace may point back to where the bootloader had
> set up its exception vector. I am fairly confident that the bootloader
> is not responsible for corrupting the Image that is loaded.
>
> Is there such a size limitation or am I possibly tripping over something
> else?
Are you using 16K pages, by any chance?
With 16K pages, the early page tables can only map up to 32M, and thus
cannot map a kernel image larger than this.
When those go wrong, you'd see what looks like memory corruption, or
unmapped memory.
With 4K page, we can map up to 1G, and with 64K we can map up to 512M.
Thanks,
Mark.
>
> (gdb) bt
> #0 0xffffff800880721c in arch_spin_lock (lock=<optimized out>)
> at ./arch/arm64/include/asm/spinlock.h:89
> #1 do_raw_spin_lock_flags (flags=<optimized out>, lock=<optimized out>)
> at ./include/linux/spinlock.h:155
> #2 __raw_spin_lock_irqsave (lock=<optimized out>)
> at ./include/linux/spinlock_api_smp.h:119
> #3 _raw_spin_lock_irqsave (lock=0xffffff800a1100f8)
> at kernel/locking/spinlock.c:159
> #4 0xffffff80080888b4 in die (str=0xffffff800894a490 "Oops",
> regs=0xffffff800a013630, err=-1778384892) at
> arch/arm64/kernel/traps.c:279
> #5 0x80ffffff080952c4 in ?? ()
> Backtrace stopped: previous frame inner to this frame (corrupt stack?)
> (gdb) info reg
> x0 0xffffff800a1100f8 18446743524122624248
> x1 0x1c0 448
> x2 0x10000 65536
> x3 0x1 1
> x4 0x0 0
> x5 0xffffff8008857dc8 18446743524096703944
> x6 0xffffff800a00494f 18446743524121528655
> x7 0x6572646461206c65 7310015527464758373
> x8 0x65676e6172207373 7306930285237531507
> x9 0x0 0
> x10 0xffffff800a0134e0 18446743524121588960
> x11 0xffffff800a0134e0 18446743524121588960
> x12 0x40d00 265472
> x13 0x72422820352e382e 8233187187582973998
> x14 0x73206d6f6364616f 8295750838792773999
> x15 0x0 0
> x16 0x0 0
> x17 0x0 0
> x18 0x0 0
> x19 0xffffff800a110000 18446743524122624000
> x20 0x96000004 2516582404
> x21 0xffffff800894a490 18446743524097696912
> x22 0xffffff800a013630 18446743524121589296
> x23 0xffffff800a013630 18446743524121589296
> x24 0x25 37
> x25 0xffffff800a018000 18446743524121608192
> x26 0xffffff8008955622 18446743524097742370
> x27 0xffffff8008955622 18446743524097742370
> x28 0xffffff800a01dd80 18446743524121632128
> x29 0xffffff800a0134d0 18446743524121588944
> x30 0xffffff80080888b4 18446743524088514740
> sp 0xffffff800a0134d0 0xffffff800a0134d0
> pc 0xffffff800880721c 0xffffff800880721c
> <_raw_spin_lock_irqsave+44>
> CPSR 0x600001c5 1610613189
> (gdb)
>
> --
> Florian
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-08-29 16:58 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-28 19:55 ARM64 kernel Image size limitations? Florian Fainelli
2017-08-28 20:01 ` Ard Biesheuvel
2017-08-28 21:24 ` Florian Fainelli
2017-08-29 8:15 ` Ard Biesheuvel
2017-08-29 16:51 ` Florian Fainelli
2017-08-29 16:58 ` Florian Fainelli
2017-08-29 10:31 ` Mark Rutland
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.