ARM : Kernel : Setting up of MMU in head.S

ARM : Kernel : Setting up of MMU in head.S

[Prakash K.B.]

Hello.

Please do not hesitate to let me know if this must be posted elsewhere.

I have been trying to understand the code that sets up the MMU. I do have a
fair understanding of the way MMU is meant to be setup, but something in the
kernel code is tripping me.

The code that kickstarts setting up of MMU is __create_page_tables in
/arch/arm/kernel/head.S.

This code is position independent.

It basically
- Reserves 16KB of memory in RAM just before the start of the uncompressed
kernel.
- Clears the 16KB meant to serve as L1 lookup table containing 4096 entries
- Creates a section entry in L1 table for the kernel code to be mapped into
physical memory

It is this creation of section entry code that is puzzling me.

The index used to program this entry is based on physical address of the
kernel base.

The way it ought to work is this.  When the CPU issues a virtual address,
the top bits are used as an index into this L1 table and then through a
couple of table walk throughs, the physical address is arrived at. So the
index used to program the L1 table ought to have been

Now look at this code.

__create_page_tables:
    pgtbl    r4                @ page table address

    /*
     * Clear the 16K level 1 swapper page table
     */
    mov    r0, r4               @r0 = 0x80004000
    mov    r3, #0
    add    r6, r0, #0x4000      @r6 = 0x80008000
1:    str    r3, [r0], #4
    str    r3, [r0], #4
    str    r3, [r0], #4
    str    r3, [r0], #4
    teq    r0, r6
    bne    1b


  /* r10 contains proc_info pointer */
    ldr    r7, [r10, #PROCINFO_MM_MMUFLAGS] @ mm_mmuflags

    /*
     * Create identity mapping to cater for __enable_mmu.
     * This identity mapping will be removed by paging_init().
     */
    adr    r0, __enable_mmu_loc
    ldmia    r0, {r3, r5, r6}

    sub    r0, r0, r3            @ virt->phys offset
    add    r5, r5, r0            @ phys __enable_mmu
    add    r6, r6, r0            @ phys __enable_mmu_end

    mov    r5, r5, lsr #20
    mov    r6, r6, lsr #20

1:  orr    r3, r7, r5, lsl #20        @ flags + kernel base
    str    r3, [r4, r5, lsl #2]        @ identity mapping

    teq    r5, r6
    addne    r5, r5, #1            @ next section
    bne    1b


The 2 lines above
1:  orr    r3, r7, r5, lsl #20        @ flags + kernel base ==> Correct
    str    r3, [r4, r5, lsl #2]        @ identity mapping  ==> ??

create a section entry using index based on physical address.

Am I missing something here?

Regards,
Prakash
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110330/046219e1/attachment.html 

ARM : Kernel : Setting up of MMU in head.S

[sk.syed2]

> The 2 lines above
> 1:? orr??? r3, r7, r5, lsl #20??? ??? @ flags + kernel base ==> Correct
> ??? str??? r3, [r4, r5, lsl #2]??? ??? @ identity mapping? ==> ??
>
> create a section entry using index based on physical address.
Lets say before mmu is turned on PC is at physical address XXX, also
say at XXX there is going to be mmu on instruction. The next
instruction fetch would be from XXX + 4 which would now be VIRTUAL
address(as mmu is turned on), which should still get converted to (via
page tables set up as above) to XXX + 4 (in physical). This is called
identity mapping as specified in the comments. Hope this helps.

-syed

ARM : Kernel : Setting up of MMU in head.S

[Dave Hylands]

Hi Prakash,

On Wed, Mar 30, 2011 at 8:19 AM, Prakash K.B. <prakashk75@gmail.com> wrote:
> Hello.
>
> Please do not hesitate to let me know if this must be posted elsewhere.
>
> I have been trying to understand the code that sets up the MMU. I do have a
> fair understanding of the way MMU is meant to be setup, but something in the
> kernel code is tripping me.
>
> The code that kickstarts setting up of MMU is __create_page_tables in
> /arch/arm/kernel/head.S.
>
> This code is position independent.
>
> It basically
> - Reserves 16KB of memory in RAM just before the start of the uncompressed
> kernel.
> - Clears the 16KB meant to serve as L1 lookup table containing 4096 entries
> - Creates a section entry in L1 table for the kernel code to be mapped into
> physical memory
>
> It is this creation of section entry code that is puzzling me.
>
> The index used to program this entry is based on physical address of the
> kernel base.
>
> The way it ought to work is this.? When the CPU issues a virtual address,
> the top bits are used as an index into this L1 table and then through a
> couple of table walk throughs, the physical address is arrived at. So the
> index used to program the L1 table ought to have been
>
> Now look at this code.

So the initial mapping is done using a single level table. The top 12
bits (3 nibbles) of the virtual address is used as the index into the
table, and each entry in the table maps 1Mb of memory.

At this stage of the boot, only the kernel direct memory is mapped.

So, if your physical memory starts at 0x80000000 and the kernel
virtual space starts at 0xc0000000 then you should see entries like

0x800xxxxx
0x801xxxxx
0x802xxxxx
0x803xxxxx

starting at 0x80007000.

The first level table starts at an offset of 0x4000 into physical
memory (0x80004000 - 0x80007fff physical or 0xc0004000 - 0xc0007fff
virtual).

If you take the top 3 nibbles of 0xc0000000 you get 0xc00 which when
multipled by 4 (each entry is 4 bytes long), gives 0x3000. 0x80004000
+ 0x3000 = 0x80007000.

Later on, when the kernel is up and running it uses a 2 level table
for memory allocated with get_pages. The 2-level table allows for 4k
pages. The kernel direct memory remains mapped with 1 Mb entries.

If you have access to it (you'll need to register and create an
account), https://silver.arm.com/download/ARM_and_AMBA_Architecture/AR570-DC-11001-r0p0-00rel4/DDI0406B_arm_architecture_reference_manual_errata_markup_8_0.pdf
on page B3-8 shows the layout of the entries which can exist in this
top-level table.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

ARM : Kernel : Setting up of MMU in head.S

[Dave Hylands]

Hi Prakash,

On Wed, Mar 30, 2011 at 2:35 PM, Dave Hylands <dhylands@gmail.com> wrote:
> Hi Prakash,
>
> On Wed, Mar 30, 2011 at 8:19 AM, Prakash K.B. <prakashk75@gmail.com> wrote:
>> Hello.
>>
>> Please do not hesitate to let me know if this must be posted elsewhere.
>>
>> I have been trying to understand the code that sets up the MMU. I do have a
>> fair understanding of the way MMU is meant to be setup, but something in the
>> kernel code is tripping me.

Some further explanation is due.

When the kernel starts, the MMU is off, and ther ARM is running with
an implicit identity mapping (i.e. each virtual address maps to the
same physical address).

If your physical memory starts at 0x80000000, then the PC will be 0x800xxxxx.

When the MMU table is turned on, the PC is still at 0x800xxxx, so even
though the kernel has 0xc00xxxxx mapped to 0x800xxxxx it also has to
have 0x800xxxxx mapped to 0x800xxxxx.

So this mapping of 0x800xxxxx to 0x800xxxxx is the "identity" portion
and is needed while switching the MMU on. The 0xc00xxxxx to 0x800xxxxx
mapping is what's used while the kernel is running.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

ARM : Kernel : Setting up of MMU in head.S

[Prakash K.B.]

Merci mate. :-)

On Thu, Mar 31, 2011 at 3:59 AM, Dave Hylands <dhylands@gmail.com> wrote:

> Hi Prakash,
>
> On Wed, Mar 30, 2011 at 2:35 PM, Dave Hylands <dhylands@gmail.com> wrote:
> > Hi Prakash,
> >
> > On Wed, Mar 30, 2011 at 8:19 AM, Prakash K.B. <prakashk75@gmail.com>
> wrote:
> >> Hello.
> >>
> >> Please do not hesitate to let me know if this must be posted elsewhere.
> >>
> >> I have been trying to understand the code that sets up the MMU. I do
> have a
> >> fair understanding of the way MMU is meant to be setup, but something in
> the
> >> kernel code is tripping me.
>
> Some further explanation is due.
>
> When the kernel starts, the MMU is off, and ther ARM is running with
> an implicit identity mapping (i.e. each virtual address maps to the
> same physical address).
>
[Prakash] Aha...So what I ignored as a routine code comment had a deeper
meaning.. :-)

>
> If your physical memory starts at 0x80000000, then the PC will be
> 0x800xxxxx.
>
[Prakash] Agreed.

>
> When the MMU table is turned on, the PC is still at 0x800xxxx, so even
> though the kernel has 0xc00xxxxx mapped to 0x800xxxxx it also has to
> have 0x800xxxxx mapped to 0x800xxxxx.
>
[Prakash] I think you meant to say "So even though the kernel intends to map
0xc00XXXX to 0x800XXX in the future, it has currently mapped 0x800xxx to
0x800xxx.

Now that I know this identity mapping is done on purpose, I hope to make
good progress with the succeeding sequence.

Do you confirm that only one entry is written into this L1 table because
both mmu_enable and enable_mmu_end are on the same section?

>
> So this mapping of 0x800xxxxx to 0x800xxxxx is the "identity" portion
> and is needed while switching the MMU on. The 0xc00xxxxx to 0x800xxxxx
> mapping is what's used while the kernel is running.
>
> --
> Dave Hylands
> Shuswap, BC, Canada
> http://www.davehylands.com
>

-Prakash
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110331/50125820/attachment-0001.html 

ARM : Kernel : Setting up of MMU in head.S

[Prakash K.B.]

Hi Syed.

On Thu, Mar 31, 2011 at 12:55 AM, sk.syed2 <sk.syed2@gmail.com> wrote:

> > The 2 lines above
> > 1:  orr    r3, r7, r5, lsl #20        @ flags + kernel base ==> Correct
> >     str    r3, [r4, r5, lsl #2]        @ identity mapping  ==> ??
> >
> > create a section entry using index based on physical address.
> Lets say before mmu is turned on PC is at physical address XXX, also
> say at XXX there is going to be mmu on instruction. The next
> instruction fetch would be from XXX + 4 which would now be VIRTUAL
> address(as mmu is turned on), which should still get converted to (via
> page tables set up as above) to XXX + 4 (in physical). This is called
> identity mapping as specified in the comments. Hope this helps.
>
> [Prakash]Awesome. This is indeed very insightful. I am now starting to
understand this portion of the kernel better. Thanks a lot. :-)

> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>

-Prakash
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110331/0354d099/attachment.html 

ARM : Kernel : Setting up of MMU in head.S

[Dave Hylands]

Hi Prakash,

On Wed, Mar 30, 2011 at 11:01 PM, Prakash K.B. <prakashk75@gmail.com> wrote:
> Merci mate. :-)
>
> On Thu, Mar 31, 2011 at 3:59 AM, Dave Hylands <dhylands@gmail.com> wrote:
>>
>> Hi Prakash,
>>
>> On Wed, Mar 30, 2011 at 2:35 PM, Dave Hylands <dhylands@gmail.com> wrote:
...snip...
>> When the MMU table is turned on, the PC is still at 0x800xxxx, so even
>> though the kernel has 0xc00xxxxx mapped to 0x800xxxxx it also has to
>> have 0x800xxxxx mapped to 0x800xxxxx.
>
> [Prakash] I think you meant to say "So even though the kernel intends to map
> 0xc00XXXX to 0x800XXX in the future, it has currently mapped 0x800xxx to
> 0x800xxx.

No. the create_page_table function creates a page table which has
0x800xxxxx and 0xc00xxxxx both mapped to 0x800xxxxx. The one store
intrustion saves the identity mapping (for 1 Mb) and the loop sets up
the 0xc00xxxxx to 0x800xxxxx mapping.

The identity portion is used as we discussed, and a few instructions
after turning on the MMU, the CPU then does a jump from the 0x800xxxxx
space to the 0xc00xxxxx space. After that, the identity mapping is no
longer needed. Shortly after this, head.S calls into the start_kernel
function (from init/main.c) and the paging_init function reinitializes
the MMU removing the identity mapping.

>
> Now that I know this identity mapping is done on purpose, I hope to make
> good progress with the succeeding sequence.
>
> Do you confirm that only one entry is written into this L1 table because
> both mmu_enable and enable_mmu_end are on the same section?

Yeah - essentially, that one mapping entry covers 1Mb of code space,
which is sufficient to cover all of head.S, which is always at the
front of the kernel image.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

ARM : Kernel : Setting up of MMU in head.S

[Prakash K.B.]

Hi Dave.

On Thu, Mar 31, 2011 at 7:29 PM, Dave Hylands <dhylands@gmail.com> wrote:

> Hi Prakash,
>
> On Wed, Mar 30, 2011 at 11:01 PM, Prakash K.B. <prakashk75@gmail.com>
> wrote:
> > Merci mate. :-)
> >
> > On Thu, Mar 31, 2011 at 3:59 AM, Dave Hylands <dhylands@gmail.com>
> wrote:
> >>
> >> Hi Prakash,
> >>
> >> On Wed, Mar 30, 2011 at 2:35 PM, Dave Hylands <dhylands@gmail.com>
> wrote:
> ...snip...
> >> When the MMU table is turned on, the PC is still at 0x800xxxx, so even
> >> though the kernel has 0xc00xxxxx mapped to 0x800xxxxx it also has to
> >> have 0x800xxxxx mapped to 0x800xxxxx.
> >
> > [Prakash] I think you meant to say "So even though the kernel intends to
> map
> > 0xc00XXXX to 0x800XXX in the future, it has currently mapped 0x800xxx to
> > 0x800xxx.
>
> No. the create_page_table function creates a page table which has
> 0x800xxxxx and 0xc00xxxxx both mapped to 0x800xxxxx. The one store
> intrustion saves the identity mapping (for 1 Mb) and the loop sets up
> the 0xc00xxxxx to 0x800xxxxx mapping.
>
> The identity portion is used as we discussed, and a few instructions
> after turning on the MMU, the CPU then does a jump from the 0x800xxxxx
> space to the 0xc00xxxxx space. After that, the identity mapping is no
> longer needed. Shortly after this, head.S calls into the start_kernel
> function (from init/main.c) and the paging_init function reinitializes
> the MMU removing the identity mapping.
>

[Prakash] I confirm I now see this exactly as you describe above and below.
Thanks a bunch Dave.

>
> >
> > Now that I know this identity mapping is done on purpose, I hope to make
> > good progress with the succeeding sequence.
> >
> > Do you confirm that only one entry is written into this L1 table because
> > both mmu_enable and enable_mmu_end are on the same section?
>
> Yeah - essentially, that one mapping entry covers 1Mb of code space,
> which is sufficient to cover all of head.S, which is always at the
> front of the kernel image.
>
> --
> Dave Hylands
> Shuswap, BC, Canada
> http://www.davehylands.com
>

-Prakash
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110403/67b4d3cb/attachment.html