[PATCH 0/2] ARM: decompressor: relax the loading restriction of the decompressed kernel

[PATCH 0/2] ARM: decompressor: relax the loading restriction of the decompressed kernel

[Zhen Lei]

Patch 1 is baseed on Ard Biesheuvel's series:
https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=arm-p2v-v2


Zhen Lei (2):
  ARM: p2v: fix trivial comments
  ARM: decompressor: relax the loading restriction of the decompressed
    kernel

 arch/arm/boot/compressed/head.S | 33 ++++++++++++++-------------------
 arch/arm/kernel/phys2virt.S     |  4 ++--
 2 files changed, 16 insertions(+), 21 deletions(-)

-- 
1.8.3

[PATCH 1/2] ARM: p2v: fix trivial comments

[Zhen Lei]

Change pv_offset to __pv_offset.

Fixes: f52bb722547f ("ARM: mm: Correct virt_to_phys patching for 64 bit physical addresses")
Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
---
 arch/arm/kernel/phys2virt.S | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/kernel/phys2virt.S b/arch/arm/kernel/phys2virt.S
index fb53db78fe7850b..baadf2b8919a986 100644
--- a/arch/arm/kernel/phys2virt.S
+++ b/arch/arm/kernel/phys2virt.S
@@ -53,8 +53,8 @@ ENDPROC(__fixup_pv_table)
 	.text
 __fixup_a_pv_table:
 	adr_l	r6, __pv_offset
-	ldr	r0, [r6, #HIGH_OFFSET]	@ pv_offset high word
-	ldr	r6, [r6, #LOW_OFFSET]	@ pv_offset low word
+	ldr	r0, [r6, #HIGH_OFFSET]	@ __pv_offset high word
+	ldr	r6, [r6, #LOW_OFFSET]	@ __pv_offset low word
 	cmn	r0, #1
 #ifdef CONFIG_THUMB2_KERNEL
 	@
-- 
1.8.3

[PATCH 2/2] ARM: decompressor: relax the loading restriction of the decompressed kernel

[Zhen Lei]

mov	r4, pc
and	r4, r4, #0xf8000000	//truncated to 128MiB boundary
add	r4, r4, #TEXT_OFFSET	//PA(_start)

Currently, the decompressed kernel must be placed at the position: 128MiB
boundary + TEXT_OFFSET. This limitation is just because we masked PC with
0xf80000000. Actually, we can directly obtain PA(_start) by using formula
: VA(_start) + (PHYS_OFFSET - PAGE_OFFSET).

So the "PA(_start) - TEXT_OFFSET" can be 2MiB boundary, 1MiB boundary,
and so on.

Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
---
 arch/arm/boot/compressed/head.S | 33 ++++++++++++++-------------------
 1 file changed, 14 insertions(+), 19 deletions(-)

diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 434a16982e344fe..e5ba2ad2ea4700f 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -255,26 +255,16 @@ not_angel:
 
 #ifdef CONFIG_AUTO_ZRELADDR
 		/*
-		 * Find the start of physical memory.  As we are executing
-		 * without the MMU on, we are in the physical address space.
-		 * We just need to get rid of any offset by aligning the
-		 * address.
-		 *
-		 * This alignment is a balance between the requirements of
-		 * different platforms - we have chosen 128MB to allow
-		 * platforms which align the start of their physical memory
-		 * to 128MB to use this feature, while allowing the zImage
-		 * to be placed within the first 128MB of memory on other
-		 * platforms.  Increasing the alignment means we place
-		 * stricter alignment requirements on the start of physical
-		 * memory, but relaxing it means that we break people who
-		 * are already placing their zImage in (eg) the top 64MB
-		 * of this range.
+		 * Find ZRELADDR (Address where the decompressed kernel was
+		 * placed, usually == PHYS_OFFSET + TEXT_OFFSET). That's the
+		 * start physical address of the text section, PA(_start).
+		 * As we are executing without the MMU on, we are in the
+		 * physical address space.
 		 */
-		mov	r4, pc
-		and	r4, r4, #0xf8000000
-		/* Determine final kernel image address. */
-		add	r4, r4, #TEXT_OFFSET
+		adr	r0, LC2
+		ldmia	r0, {r3-r4}
+		sub	r3, r0, r3		@ PHYS_OFFSET - PAGE_OFFSET
+		add	r4, r4, r3		@ PA(_start)
 #else
 		ldr	r4, =zreladdr
 #endif
@@ -660,6 +650,11 @@ LC1:		.word	.L_user_stack_end - LC1	@ sp
 		.word	_edata - LC1		@ r6
 		.size	LC1, . - LC1
 
+		.align	2
+		.type	LC2, #object
+LC2:		.word	LC2
+		.word	_start			@ start VA of text section
+
 .Lheadroom:
 		.word	_end - restart + 16384 + 1024*1024
 
-- 
1.8.3

Re: [PATCH 2/2] ARM: decompressor: relax the loading restriction of the decompressed kernel

[Ard Biesheuvel]

On Mon, 28 Sep 2020 at 11:27, Zhen Lei <thunder.leizhen@huawei.com> wrote:
>
> mov     r4, pc
> and     r4, r4, #0xf8000000     //truncated to 128MiB boundary
> add     r4, r4, #TEXT_OFFSET    //PA(_start)
>
> Currently, the decompressed kernel must be placed at the position: 128MiB
> boundary + TEXT_OFFSET. This limitation is just because we masked PC with
> 0xf80000000. Actually, we can directly obtain PA(_start) by using formula
> : VA(_start) + (PHYS_OFFSET - PAGE_OFFSET).
>
> So the "PA(_start) - TEXT_OFFSET" can be 2MiB boundary, 1MiB boundary,
> and so on.
>
> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>

No, this won't work.

The whole reason for rounding to a multiple of 128 MB is that we
cannot infer the start of DRAM from the placement of the zImage (which
provides _start).

There are patches on the list by Geert [0] to obtain the start of DRAM
from the device tree directly, to work around this masking. And when
booting via EFI (which is supported by u-boot too these days), the
zImage could be anywhere in [32-bit addressable] memory, and the start
of DRAM is obtained from the EFI memory map.

Note to Geert: this is a followup to an effort by Zhen Lei and myself
[1] to lower the minimum alignment of PHYS_OFFSET from 16 MiB to 2
MiB, and this affects your patch as well.


[0] https://lore.kernel.org/linux-arm-kernel/20200902153606.13652-1-geert+renesas@glider.be/
[1] https://lore.kernel.org/linux-arm-kernel/20200921154117.757-1-ardb@kernel.org/


> ---
>  arch/arm/boot/compressed/head.S | 33 ++++++++++++++-------------------
>  1 file changed, 14 insertions(+), 19 deletions(-)
>
> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> index 434a16982e344fe..e5ba2ad2ea4700f 100644
> --- a/arch/arm/boot/compressed/head.S
> +++ b/arch/arm/boot/compressed/head.S
> @@ -255,26 +255,16 @@ not_angel:
>
>  #ifdef CONFIG_AUTO_ZRELADDR
>                 /*
> -                * Find the start of physical memory.  As we are executing
> -                * without the MMU on, we are in the physical address space.
> -                * We just need to get rid of any offset by aligning the
> -                * address.
> -                *
> -                * This alignment is a balance between the requirements of
> -                * different platforms - we have chosen 128MB to allow
> -                * platforms which align the start of their physical memory
> -                * to 128MB to use this feature, while allowing the zImage
> -                * to be placed within the first 128MB of memory on other
> -                * platforms.  Increasing the alignment means we place
> -                * stricter alignment requirements on the start of physical
> -                * memory, but relaxing it means that we break people who
> -                * are already placing their zImage in (eg) the top 64MB
> -                * of this range.
> +                * Find ZRELADDR (Address where the decompressed kernel was
> +                * placed, usually == PHYS_OFFSET + TEXT_OFFSET). That's the
> +                * start physical address of the text section, PA(_start).
> +                * As we are executing without the MMU on, we are in the
> +                * physical address space.
>                  */
> -               mov     r4, pc
> -               and     r4, r4, #0xf8000000
> -               /* Determine final kernel image address. */
> -               add     r4, r4, #TEXT_OFFSET
> +               adr     r0, LC2
> +               ldmia   r0, {r3-r4}
> +               sub     r3, r0, r3              @ PHYS_OFFSET - PAGE_OFFSET
> +               add     r4, r4, r3              @ PA(_start)
>  #else
>                 ldr     r4, =zreladdr
>  #endif
> @@ -660,6 +650,11 @@ LC1:               .word   .L_user_stack_end - LC1 @ sp
>                 .word   _edata - LC1            @ r6
>                 .size   LC1, . - LC1
>
> +               .align  2
> +               .type   LC2, #object
> +LC2:           .word   LC2
> +               .word   _start                  @ start VA of text section
> +
>  .Lheadroom:
>                 .word   _end - restart + 16384 + 1024*1024
>
> --
> 1.8.3
>
>

Re: [PATCH 2/2] ARM: decompressor: relax the loading restriction of the decompressed kernel

[Leizhen (ThunderTown)]

On 2020/9/28 18:14, Ard Biesheuvel wrote:
> On Mon, 28 Sep 2020 at 11:27, Zhen Lei <thunder.leizhen@huawei.com> wrote:
>>
>> mov     r4, pc
>> and     r4, r4, #0xf8000000     //truncated to 128MiB boundary
>> add     r4, r4, #TEXT_OFFSET    //PA(_start)
>>
>> Currently, the decompressed kernel must be placed at the position: 128MiB
>> boundary + TEXT_OFFSET. This limitation is just because we masked PC with
>> 0xf80000000. Actually, we can directly obtain PA(_start) by using formula
>> : VA(_start) + (PHYS_OFFSET - PAGE_OFFSET).
>>
>> So the "PA(_start) - TEXT_OFFSET" can be 2MiB boundary, 1MiB boundary,
>> and so on.
>>
>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
> 
> No, this won't work.

But it works well on my board.

> 
> The whole reason for rounding to a multiple of 128 MB is that we
> cannot infer the start of DRAM from the placement of the zImage (which
> provides _start).

Maybe this is further guaranteed by the following code:
        /*
	 * Set up a page table only if it won't overwrite ourself.
	 * That means r4 < pc || r4 - 16k page directory > &_end.
	 * Given that r4 > &_end is most unfrequent, we add a rough
	 * additional 1MB of room for a possible appended DTB.
	 */

In addition, the Image has already been loaded when this position is reached.

----------- <--128MiB boundary
|          |
----------- <--TEXT_OFFSET <--
| (1)Image |                 |
------------                 |
|          |                 |
-----------  (2)--copyto-----
| (2)Image |
-----------

I don't think it's the case of (2), but (1). Because no code modification is
required for the case (2).

By the way, I'm not familiar with the arm32 code, so I'm just speculating.

> 
> There are patches on the list by Geert [0] to obtain the start of DRAM
> from the device tree directly, to work around this masking. And when
> booting via EFI (which is supported by u-boot too these days), the
> zImage could be anywhere in [32-bit addressable] memory, and the start
> of DRAM is obtained from the EFI memory map.
> 
> Note to Geert: this is a followup to an effort by Zhen Lei and myself
> [1] to lower the minimum alignment of PHYS_OFFSET from 16 MiB to 2
> MiB, and this affects your patch as well.
> 
> 
> [0] https://lore.kernel.org/linux-arm-kernel/20200902153606.13652-1-geert+renesas@glider.be/
> [1] https://lore.kernel.org/linux-arm-kernel/20200921154117.757-1-ardb@kernel.org/
> 
> 
>> ---
>>  arch/arm/boot/compressed/head.S | 33 ++++++++++++++-------------------
>>  1 file changed, 14 insertions(+), 19 deletions(-)
>>
>> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
>> index 434a16982e344fe..e5ba2ad2ea4700f 100644
>> --- a/arch/arm/boot/compressed/head.S
>> +++ b/arch/arm/boot/compressed/head.S
>> @@ -255,26 +255,16 @@ not_angel:
>>
>>  #ifdef CONFIG_AUTO_ZRELADDR
>>                 /*
>> -                * Find the start of physical memory.  As we are executing
>> -                * without the MMU on, we are in the physical address space.
>> -                * We just need to get rid of any offset by aligning the
>> -                * address.
>> -                *
>> -                * This alignment is a balance between the requirements of
>> -                * different platforms - we have chosen 128MB to allow
>> -                * platforms which align the start of their physical memory
>> -                * to 128MB to use this feature, while allowing the zImage
>> -                * to be placed within the first 128MB of memory on other
>> -                * platforms.  Increasing the alignment means we place
>> -                * stricter alignment requirements on the start of physical
>> -                * memory, but relaxing it means that we break people who
>> -                * are already placing their zImage in (eg) the top 64MB
>> -                * of this range.
>> +                * Find ZRELADDR (Address where the decompressed kernel was
>> +                * placed, usually == PHYS_OFFSET + TEXT_OFFSET). That's the
>> +                * start physical address of the text section, PA(_start).
>> +                * As we are executing without the MMU on, we are in the
>> +                * physical address space.
>>                  */
>> -               mov     r4, pc
>> -               and     r4, r4, #0xf8000000
>> -               /* Determine final kernel image address. */
>> -               add     r4, r4, #TEXT_OFFSET
>> +               adr     r0, LC2
>> +               ldmia   r0, {r3-r4}
>> +               sub     r3, r0, r3              @ PHYS_OFFSET - PAGE_OFFSET
>> +               add     r4, r4, r3              @ PA(_start)
>>  #else
>>                 ldr     r4, =zreladdr
>>  #endif
>> @@ -660,6 +650,11 @@ LC1:               .word   .L_user_stack_end - LC1 @ sp
>>                 .word   _edata - LC1            @ r6
>>                 .size   LC1, . - LC1
>>
>> +               .align  2
>> +               .type   LC2, #object
>> +LC2:           .word   LC2
>> +               .word   _start                  @ start VA of text section
>> +
>>  .Lheadroom:
>>                 .word   _end - restart + 16384 + 1024*1024
>>
>> --
>> 1.8.3
>>
>>
> 
> .
> 

Re: [PATCH 2/2] ARM: decompressor: relax the loading restriction of the decompressed kernel

[Ard Biesheuvel]

On Mon, 28 Sep 2020 at 13:57, Leizhen (ThunderTown)
<thunder.leizhen@huawei.com> wrote:
>
>
>
> On 2020/9/28 18:14, Ard Biesheuvel wrote:
> > On Mon, 28 Sep 2020 at 11:27, Zhen Lei <thunder.leizhen@huawei.com> wrote:
> >>
> >> mov     r4, pc
> >> and     r4, r4, #0xf8000000     //truncated to 128MiB boundary
> >> add     r4, r4, #TEXT_OFFSET    //PA(_start)
> >>
> >> Currently, the decompressed kernel must be placed at the position: 128MiB
> >> boundary + TEXT_OFFSET. This limitation is just because we masked PC with
> >> 0xf80000000. Actually, we can directly obtain PA(_start) by using formula
> >> : VA(_start) + (PHYS_OFFSET - PAGE_OFFSET).
> >>
> >> So the "PA(_start) - TEXT_OFFSET" can be 2MiB boundary, 1MiB boundary,
> >> and so on.
> >>
> >> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
> >
> > No, this won't work.
>
> But it works well on my board.
>

That is because you load zImage at the base of DRAM.

> >
> > The whole reason for rounding to a multiple of 128 MB is that we
> > cannot infer the start of DRAM from the placement of the zImage (which
> > provides _start).
>
> Maybe this is further guaranteed by the following code:
>         /*
>          * Set up a page table only if it won't overwrite ourself.
>          * That means r4 < pc || r4 - 16k page directory > &_end.
>          * Given that r4 > &_end is most unfrequent, we add a rough
>          * additional 1MB of room for a possible appended DTB.
>          */
>
> In addition, the Image has already been loaded when this position is reached.
>
> ----------- <--128MiB boundary
> |          |
> ----------- <--TEXT_OFFSET <--
> | (1)Image |                 |
> ------------                 |
> |          |                 |
> -----------  (2)--copyto-----
> | (2)Image |
> -----------
>
> I don't think it's the case of (2), but (1). Because no code modification is
> required for the case (2).
>
> By the way, I'm not familiar with the arm32 code, so I'm just speculating.
>

The zImage code that runs has not received *any* information from the
platform on where DRAM starts, so the only info it has is the current
placement of zImage.

So when zImage is loaded at the intended base of DRAM, things work fine.

If the zImage is loaded close to the end of a 128 MB region, the
rounding would pick the start of that 128 MB region. However, the
_start symbol you are using will point to an address that is close to
the end of the 128 MB [given that it is inside zImage] so your logic
will pick an address that is much higher in memory.

Re: [PATCH 2/2] ARM: decompressor: relax the loading restriction of the decompressed kernel

[Geert Uytterhoeven]

Hi Zhen,

On Mon, Sep 28, 2020 at 2:15 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> On Mon, 28 Sep 2020 at 13:57, Leizhen (ThunderTown)
> <thunder.leizhen@huawei.com> wrote:
> > On 2020/9/28 18:14, Ard Biesheuvel wrote:
> > > On Mon, 28 Sep 2020 at 11:27, Zhen Lei <thunder.leizhen@huawei.com> wrote:
> > >>
> > >> mov     r4, pc
> > >> and     r4, r4, #0xf8000000     //truncated to 128MiB boundary
> > >> add     r4, r4, #TEXT_OFFSET    //PA(_start)
> > >>
> > >> Currently, the decompressed kernel must be placed at the position: 128MiB
> > >> boundary + TEXT_OFFSET. This limitation is just because we masked PC with
> > >> 0xf80000000. Actually, we can directly obtain PA(_start) by using formula
> > >> : VA(_start) + (PHYS_OFFSET - PAGE_OFFSET).
> > >>
> > >> So the "PA(_start) - TEXT_OFFSET" can be 2MiB boundary, 1MiB boundary,
> > >> and so on.
> > >>
> > >> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
> > >
> > > No, this won't work.
> >
> > But it works well on my board.
> >
>
> That is because you load zImage at the base of DRAM.
>
> > >
> > > The whole reason for rounding to a multiple of 128 MB is that we
> > > cannot infer the start of DRAM from the placement of the zImage (which
> > > provides _start).
> >
> > Maybe this is further guaranteed by the following code:
> >         /*
> >          * Set up a page table only if it won't overwrite ourself.
> >          * That means r4 < pc || r4 - 16k page directory > &_end.
> >          * Given that r4 > &_end is most unfrequent, we add a rough
> >          * additional 1MB of room for a possible appended DTB.
> >          */
> >
> > In addition, the Image has already been loaded when this position is reached.
> >
> > ----------- <--128MiB boundary
> > |          |
> > ----------- <--TEXT_OFFSET <--
> > | (1)Image |                 |
> > ------------                 |
> > |          |                 |
> > -----------  (2)--copyto-----
> > | (2)Image |
> > -----------
> >
> > I don't think it's the case of (2), but (1). Because no code modification is
> > required for the case (2).
> >
> > By the way, I'm not familiar with the arm32 code, so I'm just speculating.
> >
>
> The zImage code that runs has not received *any* information from the
> platform on where DRAM starts, so the only info it has is the current
> placement of zImage.
>
> So when zImage is loaded at the intended base of DRAM, things work fine.
>
> If the zImage is loaded close to the end of a 128 MB region, the
> rounding would pick the start of that 128 MB region. However, the
> _start symbol you are using will point to an address that is close to
> the end of the 128 MB [given that it is inside zImage] so your logic
> will pick an address that is much higher in memory.

https://people.kernel.org/linusw/how-the-arm32-linux-kernel-decompresses
https://people.kernel.org/linusw/how-the-arm32-kernel-starts
are good reads.

Gr{oetje,eeting}s,

                        Geert


--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH 2/2] ARM: decompressor: relax the loading restriction of the decompressed kernel

[Leizhen (ThunderTown)]

On 2020/9/28 20:57, Geert Uytterhoeven wrote:
> Hi Zhen,
> 
> On Mon, Sep 28, 2020 at 2:15 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>> On Mon, 28 Sep 2020 at 13:57, Leizhen (ThunderTown)
>> <thunder.leizhen@huawei.com> wrote:
>>> On 2020/9/28 18:14, Ard Biesheuvel wrote:
>>>> On Mon, 28 Sep 2020 at 11:27, Zhen Lei <thunder.leizhen@huawei.com> wrote:
>>>>>
>>>>> mov     r4, pc
>>>>> and     r4, r4, #0xf8000000     //truncated to 128MiB boundary
>>>>> add     r4, r4, #TEXT_OFFSET    //PA(_start)
>>>>>
>>>>> Currently, the decompressed kernel must be placed at the position: 128MiB
>>>>> boundary + TEXT_OFFSET. This limitation is just because we masked PC with
>>>>> 0xf80000000. Actually, we can directly obtain PA(_start) by using formula
>>>>> : VA(_start) + (PHYS_OFFSET - PAGE_OFFSET).
>>>>>
>>>>> So the "PA(_start) - TEXT_OFFSET" can be 2MiB boundary, 1MiB boundary,
>>>>> and so on.
>>>>>
>>>>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
>>>>
>>>> No, this won't work.
>>>
>>> But it works well on my board.
>>>
>>
>> That is because you load zImage at the base of DRAM.
>>
>>>>
>>>> The whole reason for rounding to a multiple of 128 MB is that we
>>>> cannot infer the start of DRAM from the placement of the zImage (which
>>>> provides _start).
>>>
>>> Maybe this is further guaranteed by the following code:
>>>         /*
>>>          * Set up a page table only if it won't overwrite ourself.
>>>          * That means r4 < pc || r4 - 16k page directory > &_end.
>>>          * Given that r4 > &_end is most unfrequent, we add a rough
>>>          * additional 1MB of room for a possible appended DTB.
>>>          */
>>>
>>> In addition, the Image has already been loaded when this position is reached.
>>>
>>> ----------- <--128MiB boundary
>>> |          |
>>> ----------- <--TEXT_OFFSET <--
>>> | (1)Image |                 |
>>> ------------                 |
>>> |          |                 |
>>> -----------  (2)--copyto-----
>>> | (2)Image |
>>> -----------
>>>
>>> I don't think it's the case of (2), but (1). Because no code modification is
>>> required for the case (2).
>>>
>>> By the way, I'm not familiar with the arm32 code, so I'm just speculating.
>>>
>>
>> The zImage code that runs has not received *any* information from the
>> platform on where DRAM starts, so the only info it has is the current
>> placement of zImage.
>>
>> So when zImage is loaded at the intended base of DRAM, things work fine.
>>
>> If the zImage is loaded close to the end of a 128 MB region, the
>> rounding would pick the start of that 128 MB region. However, the
>> _start symbol you are using will point to an address that is close to
>> the end of the 128 MB [given that it is inside zImage] so your logic
>> will pick an address that is much higher in memory.
> 
> https://people.kernel.org/linusw/how-the-arm32-linux-kernel-decompresses
> https://people.kernel.org/linusw/how-the-arm32-kernel-starts
> are good reads.

Thanks for your information.

> 
> Gr{oetje,eeting}s,
> 
>                         Geert
> 
> 
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
> 
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
>                                 -- Linus Torvalds
> 
> .
>