[PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB

[PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB

[Christophe Leroy]

Activating CONFIG_PIN_TLB allows access to the 24 first Mbytes of memory at
bootup instead of 8. It is needed for "big" kernels for instance when activating
CONFIG_LOCKDEP_SUPPORT. This needs to be taken into account in init_32 too,
otherwise memory allocation soon fails after startup.

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>

diff -ur linux-3.11.org/arch/powerpc/kernel/head_8xx.S linux-3.11/arch/powerpc/kernel/head_8xx.S
--- linux-3.11.org/arch/powerpc/mm/init_32.c	2013-09-02 22:46:10.000000000 +0200
+++ linux-3.11/arch/powerpc/mm/init_32.c	2013-09-09 11:28:54.000000000 +0200
@@ -213,7 +213,12 @@
 	 */
 	BUG_ON(first_memblock_base != 0);
 
+#ifdef CONFIG_PIN_TLB
+	/* 8xx can only access 24MB at the moment */
+	memblock_set_current_limit(min_t(u64, first_memblock_size, 0x01800000));
+#else
 	/* 8xx can only access 8MB at the moment */
 	memblock_set_current_limit(min_t(u64, first_memblock_size, 0x00800000));
+#endif
 }
 #endif /* CONFIG_8xx */

Re: [PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB

[Joakim Tjernlund]

"Linuxppc-dev" 
<linuxppc-dev-bounces+joakim.tjernlund=transmode.se@lists.ozlabs.org> 
wrote on 2013/10/11 14:56:40:
> 
> Activating CONFIG_PIN_TLB allows access to the 24 first Mbytes of memory 
at
> bootup instead of 8. It is needed for "big" kernels for instance when 
activating
> CONFIG_LOCKDEP_SUPPORT. This needs to be taken into account in init_32 
too,
> otherwise memory allocation soon fails after startup.
> 
> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> 
> diff -ur linux-3.11.org/arch/powerpc/kernel/head_8xx.S 
linux-3.11/arch/powerpc/kernel/head_8xx.S
> --- linux-3.11.org/arch/powerpc/mm/init_32.c   2013-09-02 
22:46:10.000000000 +0200
> +++ linux-3.11/arch/powerpc/mm/init_32.c   2013-09-09 11:28:54.000000000 
+0200
> @@ -213,7 +213,12 @@
>      */
>     BUG_ON(first_memblock_base != 0);
> 
> +#ifdef CONFIG_PIN_TLB
> +   /* 8xx can only access 24MB at the moment */
> +   memblock_set_current_limit(min_t(u64, first_memblock_size, 
0x01800000));
> +#else
>     /* 8xx can only access 8MB at the moment */
>     memblock_set_current_limit(min_t(u64, first_memblock_size, 
0x00800000));
> +#endif
>  }
>  #endif /* CONFIG_8xx */

hmm, I think you should always map 24 MB (or less if RAM < 24 MB) and do 
the same
in head_8xx.S.

Or to keep it simple, just always map at least 16 MB here and in 
head_8xx.S, assuming 
that 16 MB is min RAM for any 8xx system running 3.x kernels.

Much of the need for pinning would go away if you adopted my 8MB pages 
from 2.4 to 3.x 

   Jocke

Re: [PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB

[leroy christophe]

Le 11/10/2013 17:13, Joakim Tjernlund a écrit :
> "Linuxppc-dev"
> <linuxppc-dev-bounces+joakim.tjernlund=transmode.se@lists.ozlabs.org>
> wrote on 2013/10/11 14:56:40:
>> Activating CONFIG_PIN_TLB allows access to the 24 first Mbytes of memory
> at
>> bootup instead of 8. It is needed for "big" kernels for instance when
> activating
>> CONFIG_LOCKDEP_SUPPORT. This needs to be taken into account in init_32
> too,
>> otherwise memory allocation soon fails after startup.
>>
>> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
>>
>> diff -ur linux-3.11.org/arch/powerpc/kernel/head_8xx.S
> linux-3.11/arch/powerpc/kernel/head_8xx.S
>> --- linux-3.11.org/arch/powerpc/mm/init_32.c   2013-09-02
> 22:46:10.000000000 +0200
>> +++ linux-3.11/arch/powerpc/mm/init_32.c   2013-09-09 11:28:54.000000000
> +0200
>> @@ -213,7 +213,12 @@
>>       */
>>      BUG_ON(first_memblock_base != 0);
>>
>> +#ifdef CONFIG_PIN_TLB
>> +   /* 8xx can only access 24MB at the moment */
>> +   memblock_set_current_limit(min_t(u64, first_memblock_size,
> 0x01800000));
>> +#else
>>      /* 8xx can only access 8MB at the moment */
>>      memblock_set_current_limit(min_t(u64, first_memblock_size,
> 0x00800000));
>> +#endif
>>   }
>>   #endif /* CONFIG_8xx */
> hmm, I think you should always map 24 MB (or less if RAM < 24 MB) and do
> the same
> in head_8xx.S.
>
> Or to keep it simple, just always map at least 16 MB here and in
> head_8xx.S, assuming
> that 16 MB is min RAM for any 8xx system running 3.x kernels.
Yes we could do a more elaborated modification in the future. However it 
also has an impact on the boot loader, so I'm not sure we should make it 
the default without thinking twice.

In the meantime, my patch does take into account the existing situation 
where you have 8Mb by default and 24Mb when you activate CONFIG_PIN_TLB.
I see it as a bug fix and I believe we should include it at least in 
order to allow including in the stable releases.

Do you see any issue with this approach ?

>
> Much of the need for pinning would go away if you adopted my 8MB pages
> from 2.4 to 3.x
>
>     Jocke
>
Christophe

Re: [PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB

[Joakim Tjernlund]

leroy christophe <christophe.leroy@c-s.fr> wrote on 2013/10/15 18:27:00:
>=20
>=20
> Le 11/10/2013 17:13, Joakim Tjernlund a =E9crit :
> > "Linuxppc-dev"
> > <linuxppc-dev-bounces+joakim.tjernlund=3Dtransmode.se@lists.ozlabs.org>
> > wrote on 2013/10/11 14:56:40:
> >> Activating CONFIG=5FPIN=5FTLB allows access to the 24 first Mbytes of =

memory
> > at
> >> bootup instead of 8. It is needed for "big" kernels for instance when
> > activating
> >> CONFIG=5FLOCKDEP=5FSUPPORT. This needs to be taken into account in=20
init=5F32
> > too,
> >> otherwise memory allocation soon fails after startup.
> >>
> >> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> >>
> >> diff -ur linux-3.11.org/arch/powerpc/kernel/head=5F8xx.S
> > linux-3.11/arch/powerpc/kernel/head=5F8xx.S
> >> --- linux-3.11.org/arch/powerpc/mm/init=5F32.c   2013-09-02
> > 22:46:10.000000000 +0200
> >> +++ linux-3.11/arch/powerpc/mm/init=5F32.c   2013-09-09=20
11:28:54.000000000
> > +0200
> >> @@ -213,7 +213,12 @@
> >>       */
> >>      BUG=5FON(first=5Fmemblock=5Fbase !=3D 0);
> >>
> >> +#ifdef CONFIG=5FPIN=5FTLB
> >> +   /* 8xx can only access 24MB at the moment */
> >> +   memblock=5Fset=5Fcurrent=5Flimit(min=5Ft(u64, first=5Fmemblock=5Fs=
ize,
> > 0x01800000));
> >> +#else
> >>      /* 8xx can only access 8MB at the moment */
> >>      memblock=5Fset=5Fcurrent=5Flimit(min=5Ft(u64, first=5Fmemblock=5F=
size,
> > 0x00800000));
> >> +#endif
> >>   }
> >>   #endif /* CONFIG=5F8xx */
> > hmm, I think you should always map 24 MB (or less if RAM < 24 MB) and=20
do
> > the same
> > in head=5F8xx.S.
> >
> > Or to keep it simple, just always map at least 16 MB here and in
> > head=5F8xx.S, assuming
> > that 16 MB is min RAM for any 8xx system running 3.x kernels.
> Yes we could do a more elaborated modification in the future. However it =


> also has an impact on the boot loader, so I'm not sure we should make it =


> the default without thinking twice.
>=20
> In the meantime, my patch does take into account the existing situation=20
> where you have 8Mb by default and 24Mb when you activate CONFIG=5FPIN=5FT=
LB.
> I see it as a bug fix and I believe we should include it at least in=20
> order to allow including in the stable releases.
>=20
> Do you see any issue with this approach ?

Not at all, so:
Acked-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>

Re: [PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB

[Scott Wood]

On Tue, 2013-10-15 at 18:27 +0200, leroy christophe wrote:
> Le 11/10/2013 17:13, Joakim Tjernlund a =C3=A9crit :
> > "Linuxppc-dev"
> > <linuxppc-dev-bounces+joakim.tjernlund=3Dtransmode.se@lists.ozlabs.or=
g>
> > wrote on 2013/10/11 14:56:40:
> >> Activating CONFIG_PIN_TLB allows access to the 24 first Mbytes of me=
mory
> > at
> >> bootup instead of 8. It is needed for "big" kernels for instance whe=
n
> > activating
> >> CONFIG_LOCKDEP_SUPPORT. This needs to be taken into account in init_=
32
> > too,
> >> otherwise memory allocation soon fails after startup.
> >>
> >> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> >>
> >> diff -ur linux-3.11.org/arch/powerpc/kernel/head_8xx.S
> > linux-3.11/arch/powerpc/kernel/head_8xx.S
> >> --- linux-3.11.org/arch/powerpc/mm/init_32.c   2013-09-02
> > 22:46:10.000000000 +0200
> >> +++ linux-3.11/arch/powerpc/mm/init_32.c   2013-09-09 11:28:54.00000=
0000
> > +0200
> >> @@ -213,7 +213,12 @@
> >>       */
> >>      BUG_ON(first_memblock_base !=3D 0);
> >>
> >> +#ifdef CONFIG_PIN_TLB
> >> +   /* 8xx can only access 24MB at the moment */
> >> +   memblock_set_current_limit(min_t(u64, first_memblock_size,
> > 0x01800000));
> >> +#else
> >>      /* 8xx can only access 8MB at the moment */
> >>      memblock_set_current_limit(min_t(u64, first_memblock_size,
> > 0x00800000));
> >> +#endif
> >>   }
> >>   #endif /* CONFIG_8xx */
> > hmm, I think you should always map 24 MB (or less if RAM < 24 MB) and=
 do
> > the same
> > in head_8xx.S.
> >
> > Or to keep it simple, just always map at least 16 MB here and in
> > head_8xx.S, assuming
> > that 16 MB is min RAM for any 8xx system running 3.x kernels.
> Yes we could do a more elaborated modification in the future. However i=
t=20
> also has an impact on the boot loader, so I'm not sure we should make i=
t=20
> the default without thinking twice.
>=20
> In the meantime, my patch does take into account the existing situation=
=20
> where you have 8Mb by default and 24Mb when you activate CONFIG_PIN_TLB=
.
> I see it as a bug fix and I believe we should include it at least in=20
> order to allow including in the stable releases.
>=20
> Do you see any issue with this approach ?

The patch is fine, but I don't think it's stable material (BTW, if it
were, you should have marked it as such when submitting).  If I
understand the situation correctly, there's no regression, and nothing
fails to work with CONFIG_PIN_TLB that would have worked without it.
It's just making CONFIG_PIN_TLB more useful.

-Scott

Re: [PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB

[leroy christophe]

[-- Attachment #1: Type: text/plain, Size: 3347 bytes --]


Le 15/10/2013 22:33, Scott Wood a écrit :
> On Tue, 2013-10-15 at 18:27 +0200, leroy christophe wrote:
>> Le 11/10/2013 17:13, Joakim Tjernlund a écrit :
>>> "Linuxppc-dev"
>>> <linuxppc-dev-bounces+joakim.tjernlund=transmode.se@lists.ozlabs.org>
>>> wrote on 2013/10/11 14:56:40:
>>>> Activating CONFIG_PIN_TLB allows access to the 24 first Mbytes of memory
>>> at
>>>> bootup instead of 8. It is needed for "big" kernels for instance when
>>> activating
>>>> CONFIG_LOCKDEP_SUPPORT. This needs to be taken into account in init_32
>>> too,
>>>> otherwise memory allocation soon fails after startup.
>>>>
>>>> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
>>>>
>>>> diff -ur linux-3.11.org/arch/powerpc/kernel/head_8xx.S
>>> linux-3.11/arch/powerpc/kernel/head_8xx.S
>>>> --- linux-3.11.org/arch/powerpc/mm/init_32.c   2013-09-02
>>> 22:46:10.000000000 +0200
>>>> +++ linux-3.11/arch/powerpc/mm/init_32.c   2013-09-09 11:28:54.000000000
>>> +0200
>>>> @@ -213,7 +213,12 @@
>>>>        */
>>>>       BUG_ON(first_memblock_base != 0);
>>>>
>>>> +#ifdef CONFIG_PIN_TLB
>>>> +   /* 8xx can only access 24MB at the moment */
>>>> +   memblock_set_current_limit(min_t(u64, first_memblock_size,
>>> 0x01800000));
>>>> +#else
>>>>       /* 8xx can only access 8MB at the moment */
>>>>       memblock_set_current_limit(min_t(u64, first_memblock_size,
>>> 0x00800000));
>>>> +#endif
>>>>    }
>>>>    #endif /* CONFIG_8xx */
>>> hmm, I think you should always map 24 MB (or less if RAM < 24 MB) and do
>>> the same
>>> in head_8xx.S.
>>>
>>> Or to keep it simple, just always map at least 16 MB here and in
>>> head_8xx.S, assuming
>>> that 16 MB is min RAM for any 8xx system running 3.x kernels.
>> Yes we could do a more elaborated modification in the future. However it
>> also has an impact on the boot loader, so I'm not sure we should make it
>> the default without thinking twice.
>>
>> In the meantime, my patch does take into account the existing situation
>> where you have 8Mb by default and 24Mb when you activate CONFIG_PIN_TLB.
>> I see it as a bug fix and I believe we should include it at least in
>> order to allow including in the stable releases.
>>
>> Do you see any issue with this approach ?
> The patch is fine, but I don't think it's stable material (BTW, if it
> were, you should have marked it as such when submitting).  If I
> understand the situation correctly, there's no regression, and nothing
> fails to work with CONFIG_PIN_TLB that would have worked without it.
> It's just making CONFIG_PIN_TLB more useful.
>
>
Yes the patch is definitly stable. How should I have mark it ?

The situation is that in 2010, I discovered that I was not able to start 
a big Kernel because of the 8Mb limit.
You told me (see attached mail) that in order to get rid of that limit I 
shall use CONFIG_PIN_TLB: it was the first step, it helped pass the 
memory zeroize at init, but it was not enough as I then got problems 
with the Device Tree being erased because being put inside the first 8Mb 
area too. Then I temporarely gave up at that time.

Recently I started again. After fixing my bootloader to get the device 
tree somewhere else, I discovered this 8Mb limit hardcoded in 
mm/init_32.c for the 8xx
With the patch I submitted I can now boot a kernel which is bigger than 8Mb.

So, I'm a bit lost here on what to do.

[-- Attachment #2: Message joint --]
[-- Type: message/rfc822, Size: 6767 bytes --]

From: Scott Wood <scottwood@freescale.com>
To: leroy christophe <christophe.leroy@c-s.fr>
Cc: Joakim Tjernlund <joakim.tjernlund@transmode.se>, <linuxppc-dev@lists.ozlabs.org>, <linux-kernel@vger.kernel.org>, Paul Mackerras <paulus@samba.org>
Subject: Re: [PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB
Date: Tue, 15 Oct 2013 15:33:36 -0500
Message-ID: <1381869216.7979.718.camel@snotra.buserror.net>

On Tue, 2013-10-15 at 18:27 +0200, leroy christophe wrote:
> Le 11/10/2013 17:13, Joakim Tjernlund a écrit :
> > "Linuxppc-dev"
> > <linuxppc-dev-bounces+joakim.tjernlund=transmode.se@lists.ozlabs.org>
> > wrote on 2013/10/11 14:56:40:
> >> Activating CONFIG_PIN_TLB allows access to the 24 first Mbytes of memory
> > at
> >> bootup instead of 8. It is needed for "big" kernels for instance when
> > activating
> >> CONFIG_LOCKDEP_SUPPORT. This needs to be taken into account in init_32
> > too,
> >> otherwise memory allocation soon fails after startup.
> >>
> >> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> >>
> >> diff -ur linux-3.11.org/arch/powerpc/kernel/head_8xx.S
> > linux-3.11/arch/powerpc/kernel/head_8xx.S
> >> --- linux-3.11.org/arch/powerpc/mm/init_32.c   2013-09-02
> > 22:46:10.000000000 +0200
> >> +++ linux-3.11/arch/powerpc/mm/init_32.c   2013-09-09 11:28:54.000000000
> > +0200
> >> @@ -213,7 +213,12 @@
> >>       */
> >>      BUG_ON(first_memblock_base != 0);
> >>
> >> +#ifdef CONFIG_PIN_TLB
> >> +   /* 8xx can only access 24MB at the moment */
> >> +   memblock_set_current_limit(min_t(u64, first_memblock_size,
> > 0x01800000));
> >> +#else
> >>      /* 8xx can only access 8MB at the moment */
> >>      memblock_set_current_limit(min_t(u64, first_memblock_size,
> > 0x00800000));
> >> +#endif
> >>   }
> >>   #endif /* CONFIG_8xx */
> > hmm, I think you should always map 24 MB (or less if RAM < 24 MB) and do
> > the same
> > in head_8xx.S.
> >
> > Or to keep it simple, just always map at least 16 MB here and in
> > head_8xx.S, assuming
> > that 16 MB is min RAM for any 8xx system running 3.x kernels.
> Yes we could do a more elaborated modification in the future. However it 
> also has an impact on the boot loader, so I'm not sure we should make it 
> the default without thinking twice.
> 
> In the meantime, my patch does take into account the existing situation 
> where you have 8Mb by default and 24Mb when you activate CONFIG_PIN_TLB.
> I see it as a bug fix and I believe we should include it at least in 
> order to allow including in the stable releases.
> 
> Do you see any issue with this approach ?

The patch is fine, but I don't think it's stable material (BTW, if it
were, you should have marked it as such when submitting).  If I
understand the situation correctly, there's no regression, and nothing
fails to work with CONFIG_PIN_TLB that would have worked without it.
It's just making CONFIG_PIN_TLB more useful.

-Scott

Re: [PATCH] powerpc 8xx: Fixing memory init issue with CONFIG_PIN_TLB

[Scott Wood]

On Wed, 2013-10-16 at 07:45 +0200, leroy christophe wrote:
> Le 15/10/2013 22:33, Scott Wood a =C3=A9crit :
> > On Tue, 2013-10-15 at 18:27 +0200, leroy christophe wrote:
> >> Le 11/10/2013 17:13, Joakim Tjernlund a =C3=A9crit :
> >>> "Linuxppc-dev"
> >>> <linuxppc-dev-bounces+joakim.tjernlund=3Dtransmode.se@lists.ozlabs.=
org>
> >>> wrote on 2013/10/11 14:56:40:
> >>>> Activating CONFIG_PIN_TLB allows access to the 24 first Mbytes of =
memory
> >>> at
> >>>> bootup instead of 8. It is needed for "big" kernels for instance w=
hen
> >>> activating
> >>>> CONFIG_LOCKDEP_SUPPORT. This needs to be taken into account in ini=
t_32
> >>> too,
> >>>> otherwise memory allocation soon fails after startup.
> >>>>
> >>>> Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
> >>>>
> >>>> diff -ur linux-3.11.org/arch/powerpc/kernel/head_8xx.S
> >>> linux-3.11/arch/powerpc/kernel/head_8xx.S
> >>>> --- linux-3.11.org/arch/powerpc/mm/init_32.c   2013-09-02
> >>> 22:46:10.000000000 +0200
> >>>> +++ linux-3.11/arch/powerpc/mm/init_32.c   2013-09-09 11:28:54.000=
000000
> >>> +0200
> >>>> @@ -213,7 +213,12 @@
> >>>>        */
> >>>>       BUG_ON(first_memblock_base !=3D 0);
> >>>>
> >>>> +#ifdef CONFIG_PIN_TLB
> >>>> +   /* 8xx can only access 24MB at the moment */
> >>>> +   memblock_set_current_limit(min_t(u64, first_memblock_size,
> >>> 0x01800000));
> >>>> +#else
> >>>>       /* 8xx can only access 8MB at the moment */
> >>>>       memblock_set_current_limit(min_t(u64, first_memblock_size,
> >>> 0x00800000));
> >>>> +#endif
> >>>>    }
> >>>>    #endif /* CONFIG_8xx */
> >>> hmm, I think you should always map 24 MB (or less if RAM < 24 MB) a=
nd do
> >>> the same
> >>> in head_8xx.S.
> >>>
> >>> Or to keep it simple, just always map at least 16 MB here and in
> >>> head_8xx.S, assuming
> >>> that 16 MB is min RAM for any 8xx system running 3.x kernels.
> >> Yes we could do a more elaborated modification in the future. Howeve=
r it
> >> also has an impact on the boot loader, so I'm not sure we should mak=
e it
> >> the default without thinking twice.
> >>
> >> In the meantime, my patch does take into account the existing situat=
ion
> >> where you have 8Mb by default and 24Mb when you activate CONFIG_PIN_=
TLB.
> >> I see it as a bug fix and I believe we should include it at least in
> >> order to allow including in the stable releases.
> >>
> >> Do you see any issue with this approach ?
> > The patch is fine, but I don't think it's stable material (BTW, if it
> > were, you should have marked it as such when submitting).  If I
> > understand the situation correctly, there's no regression, and nothin=
g
> > fails to work with CONFIG_PIN_TLB that would have worked without it.
> > It's just making CONFIG_PIN_TLB more useful.
> >
> >
> Yes the patch is definitly stable.

It's not about whether the patch itself is "stable", but whether it is a
critical bugfix that should be applied to the 3.11.x stable tree and the
3.12 release, rather than being queued for 3.13.

>  How should I have mark it ?

https://www.kernel.org/doc/Documentation/stable_kernel_rules.txt

> The situation is that in 2010, I discovered that I was not able to star=
t=20
> a big Kernel because of the 8Mb limit.
> You told me (see attached mail) that in order to get rid of that limit =
I=20
> shall use CONFIG_PIN_TLB: it was the first step, it helped pass the=20
> memory zeroize at init, but it was not enough as I then got problems=20
> with the Device Tree being erased because being put inside the first 8M=
b=20
> area too. Then I temporarely gave up at that time.
>=20
> Recently I started again. After fixing my bootloader to get the device=20
> tree somewhere else, I discovered this 8Mb limit hardcoded in=20
> mm/init_32.c for the 8xx
> With the patch I submitted I can now boot a kernel which is bigger than=
 8Mb.
>=20
> So, I'm a bit lost here on what to do.

There's nothing you need to do -- I'll apply the patch and send it to
Ben for 3.13.

-Scott