Long boot delay on 460EX with 2.6.28-rc8

Long boot delay on 460EX with 2.6.28-rc8

[Felix Radensky]

Hi,

I'm experiencing a long (around 20 sec) delay from the moment
u-boot-1.3.4 passes control to linux and until the first kernel message
appears on console. This happens on custom board, I don't have access
to Canyonlands to test how it behaves. I did not modify anything in kernel 
code to make it boot on custom board, just removed PCIX and PCIE0 nodes
from DTS.

Has anyone experienced similar behavior with 460EX ? Any hints what can
cause this ?

Thanks a lot.

Felix.


-- 
View this message in context: http://www.nabble.com/Long-boot-delay-on-460EX-with-2.6.28-rc8-tp21070179p21070179.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Long boot delay on 460EX with 2.6.28-rc8

[Anton Vorontsov]

On Thu, Dec 18, 2008 at 02:29:52AM -0800, Felix Radensky wrote:
> 
> Hi,
> 
> I'm experiencing a long (around 20 sec) delay from the moment
> u-boot-1.3.4 passes control to linux and until the first kernel message
> appears on console.

I saw somewhat similar behaviour on 83xx target, the cause was
that my .config file contained bogus UDBG_RTAS_CONSOLE=y symbol
(or was it PPC_EARLY_DEBUG_RTAS_PANEL=y? Or HVC_UDBG=y? I don't
quite remember).

Obviously, on 83xx we don't have RTAS or HVC stuff, but kernel
didn't oops or hang completely, it was busy with executing various
delays in those drivers and then just continued to load.

OTOH, it might be that the kernel just calibrates the delay loop
for too long, and you actually don't have any early messages
support enabled so you don't see what the kernel does at the moment.

You could provide dmesg and .config file to make things clearer, so
at least somebody with 460EX boards could try to reproduce the
result.

Thanks,

-- 
Anton Vorontsov
email: cbouatmailru@gmail.com
irc://irc.freenode.net/bd2

Re: Long boot delay on 460EX with 2.6.28-rc8

[Kumar Gala]

On Dec 18, 2008, at 10:02 AM, Anton Vorontsov wrote:

> On Thu, Dec 18, 2008 at 02:29:52AM -0800, Felix Radensky wrote:
>>
>> Hi,
>>
>> I'm experiencing a long (around 20 sec) delay from the moment
>> u-boot-1.3.4 passes control to linux and until the first kernel  
>> message
>> appears on console.
>
> I saw somewhat similar behaviour on 83xx target, the cause was
> that my .config file contained bogus UDBG_RTAS_CONSOLE=y symbol
> (or was it PPC_EARLY_DEBUG_RTAS_PANEL=y? Or HVC_UDBG=y? I don't
> quite remember).
>
> Obviously, on 83xx we don't have RTAS or HVC stuff, but kernel
> didn't oops or hang completely, it was busy with executing various
> delays in those drivers and then just continued to load.

Why are these drivers even doing anything on 83xx?  Seems like they  
need something to bind against or some detection of "HW".

- k

Re: Long boot delay on 460EX with 2.6.28-rc8

[Anton Vorontsov]

On Thu, Dec 18, 2008 at 12:51:29PM -0600, Kumar Gala wrote:
>
> On Dec 18, 2008, at 10:02 AM, Anton Vorontsov wrote:
>
>> On Thu, Dec 18, 2008 at 02:29:52AM -0800, Felix Radensky wrote:
>>>
>>> Hi,
>>>
>>> I'm experiencing a long (around 20 sec) delay from the moment
>>> u-boot-1.3.4 passes control to linux and until the first kernel  
>>> message
>>> appears on console.
>>
>> I saw somewhat similar behaviour on 83xx target, the cause was
>> that my .config file contained bogus UDBG_RTAS_CONSOLE=y symbol
>> (or was it PPC_EARLY_DEBUG_RTAS_PANEL=y? Or HVC_UDBG=y? I don't
>> quite remember).
>>
>> Obviously, on 83xx we don't have RTAS or HVC stuff, but kernel
>> didn't oops or hang completely, it was busy with executing various
>> delays in those drivers and then just continued to load.
>
> Why are these drivers even doing anything on 83xx?  Seems like they need 
> something to bind against or some detection of "HW".

Early console/UDBG is known to cause problems if enabled on
inappropriate hardware. They start working very early, so it's hard
for them to do the hw detection (no device tree yet, I presume).

-- 
Anton Vorontsov
email: cbouatmailru@gmail.com
irc://irc.freenode.net/bd2

Re: Long boot delay on 460EX with 2.6.28-rc8

[Josh Boyer]

On Thu, Dec 18, 2008 at 02:29:52AM -0800, Felix Radensky wrote:
>
>Hi,
>
>I'm experiencing a long (around 20 sec) delay from the moment
>u-boot-1.3.4 passes control to linux and until the first kernel message
>appears on console. This happens on custom board, I don't have access
>to Canyonlands to test how it behaves. I did not modify anything in kernel 
>code to make it boot on custom board, just removed PCIX and PCIE0 nodes
>from DTS.
>
>Has anyone experienced similar behavior with 460EX ? Any hints what can
>cause this ?

I booted on a canyonlands this morning (after having removed the brown
paper bag that preventing me from booting it yesterday).  I saw no
delay.

josh

Re: Long boot delay on 460EX with 2.6.28-rc8

[Wolfgang Denk]

Dear Felix Radensky,

In message <21070179.post@talk.nabble.com> you wrote:
> 
> I'm experiencing a long (around 20 sec) delay from the moment
> u-boot-1.3.4 passes control to linux and until the first kernel message
> appears on console. This happens on custom board, I don't have access
> to Canyonlands to test how it behaves. I did not modify anything in kernel 
> code to make it boot on custom board, just removed PCIX and PCIE0 nodes
> from DTS.

Try using a recent version of U-Boot as a first step (v2009.01-rc1).

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
Der Horizont vieler Menschen ist ein Kreis mit Radius Null --
und das nennen sie ihren Standpunkt.

Re: Long boot delay on 460EX with 2.6.28-rc8

[Michael Ellerman]

[-- Attachment #1: Type: text/plain, Size: 1275 bytes --]

On Thu, 2008-12-18 at 22:01 +0300, Anton Vorontsov wrote:
> On Thu, Dec 18, 2008 at 12:51:29PM -0600, Kumar Gala wrote:

> >> I saw somewhat similar behaviour on 83xx target, the cause was
> >> that my .config file contained bogus UDBG_RTAS_CONSOLE=y symbol
> >> (or was it PPC_EARLY_DEBUG_RTAS_PANEL=y? Or HVC_UDBG=y? I don't
> >> quite remember).
> >>
> >> Obviously, on 83xx we don't have RTAS or HVC stuff, but kernel
> >> didn't oops or hang completely, it was busy with executing various
> >> delays in those drivers and then just continued to load.
> >
> > Why are these drivers even doing anything on 83xx?  Seems like they need 
> > something to bind against or some detection of "HW".
> 
> Early console/UDBG is known to cause problems if enabled on
> inappropriate hardware. They start working very early, so it's hard
> for them to do the hw detection (no device tree yet, I presume).

Yes that's the whole point, it's _EARLY_ debug, there's even a big fat
warning in the Kconfig.

cheers

-- 
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Long boot delay on 460EX with 2.6.28-rc8

[Felix Radensky]

Josh Boyer-3 wrote:
> 
> On Thu, Dec 18, 2008 at 02:29:52AM -0800, Felix Radensky wrote:
>>
> 
> 
> I booted on a canyonlands this morning (after having removed the brown
> paper bag that preventing me from booting it yesterday).  I saw no
> delay.
> 
> josh
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
> 
> 

Thanks a lot to everyone who replied. I've managed to identify the cause
of the delay. The board is equipped with 256 MiB Samsung NAND flash. 
Since NAND support is a must for this platform, I've intergated the ndfc
driver recently posted by Sean McLennan. With this driver enabled the delay
is present, without it messages appear on the console immediately.

Is this an expected behavior - detection of 256 MiB NAND flash
takes around 20 seconds. The ndfc driver works fine after boot.

Felix.
-- 
View this message in context: http://www.nabble.com/Long-boot-delay-on-460EX-with-2.6.28-rc8-tp21070179p21087072.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Long boot delay on 460EX with 2.6.28-rc8

[Stefan Roese]

On Friday 19 December 2008, Felix Radensky wrote:
> Thanks a lot to everyone who replied. I've managed to identify the cause
> of the delay. The board is equipped with 256 MiB Samsung NAND flash.
> Since NAND support is a must for this platform, I've intergated the ndfc
> driver recently posted by Sean McLennan. With this driver enabled the delay
> is present, without it messages appear on the console immediately.
>
> Is this an expected behavior - detection of 256 MiB NAND flash
> takes around 20 seconds. The ndfc driver works fine after boot.

No, 20 seconds is definitely too long. Something must be wrong with the ndfc 
driver or the NAND dts entries.

Best regards,
Stefan

Re: Long boot delay on 460EX with 2.6.28-rc8

[Felix Radensky]

Stefan Roese wrote:
> 
> 
>> Is this an expected behavior - detection of 256 MiB NAND flash
>> takes around 20 seconds. The ndfc driver works fine after boot.
> 
> No, 20 seconds is definitely too long. Something must be wrong with the
> ndfc 
> driver or the NAND dts entries.
> 
> Best regards,
> Stefan
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
> 

Thanks, Stefan

My ndfc DTS entry looks like this

nand@3,0 {
    compatible = "amcc,ndfc";
    reg = <0x00000003 0x00000000 0x00002000>;
    ccr = <0x00001000>;
    bank-settings = <0x80002222>;
    #address-cells = <1>;
    #size-cells = <1>;

    partition@0 {
        label = "data";
        reg = <0x00000000 0x10000000>;
    };
};

I also had to modify u-boot to provide ndfc ebc mappings.
Not sure if it's the best solution, but it worked. Sorry for
posting u-boot stuff here, I'll stop doing that if list thinks
it's irrelevant.

My u-boot code looks like this (modified canyonlands code):

void ft_board_setup(void *blob, bd_t *bd)
{
        u32 val[8];
        int rc;

        ft_cpu_setup(blob, bd);

        /* Fixup NOR mapping */
        val[0] = 0;                             /* chip select number */
        val[1] = 0;                             /* always 0 */
        val[2] = CFG_FLASH_BASE_PHYS_L;         /* we fixed up this address
*/
        val[3] = gd->bd->bi_flashsize;

        /* Set NAND Controller mapping */
        val[4] = CFG_NAND_CS;                   /* chip select number */
        val[5] = 0;                                         /* always 0 */
        val[6] = CFG_NAND_ADDR;              /* NAND address */
        val[7] = 0x2000;                               /* NAND register area
size */

        rc = fdt_find_and_setprop(blob, "/plb/opb/ebc", "ranges",
                                  val, sizeof(val), 1);
        if (rc) {
                printf("Unable to update property NOR mapping, err=%s\n",
                       fdt_strerror(rc));
        }
}

Felix.

-- 
View this message in context: http://www.nabble.com/Long-boot-delay-on-460EX-with-2.6.28-rc8-tp21070179p21087540.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Long boot delay on 460EX with 2.6.28-rc8

[Felix Radensky]

Stefan Roese wrote:
> 
> 
>> Is this an expected behavior - detection of 256 MiB NAND flash
>> takes around 20 seconds. The ndfc driver works fine after boot.
> 
> No, 20 seconds is definitely too long. Something must be wrong with the
> ndfc 
> driver or the NAND dts entries.
> 
> Best regards,
> Stefan
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
> 

Thanks, Stefan

My ndfc DTS entry looks like this

nand@3,0 {
    compatible = "amcc,ndfc";
    reg = <0x00000003 0x00000000 0x00002000>;
    ccr = <0x00001000>;
    bank-settings = <0x80002222>;
    #address-cells = <1>;
    #size-cells = <1>;

    partition@0 {
        label = "data";
        reg = <0x00000000 0x10000000>;
    };
};

I also had to modify u-boot to provide ndfc ebc mappings.
Not sure if it's the best solution, but it worked. Sorry for
posting u-boot stuff here, I'll stop doing that if list thinks
it's irrelevant.

My u-boot code looks like this (modified canyonlands code):

void ft_board_setup(void *blob, bd_t *bd)
{
        u32 val[8];
        int rc;

        ft_cpu_setup(blob, bd);

        /* Fixup NOR mapping */
        val[0] = 0;                             /* chip select number */
        val[1] = 0;                             /* always 0 */
        val[2] = CFG_FLASH_BASE_PHYS_L;         /* we fixed up this address
*/
        val[3] = gd->bd->bi_flashsize;

        /* Set NAND Controller mapping */
        val[4] = CFG_NAND_CS;                   /* chip select number */
        val[5] = 0;                                         /* always 0 */
        val[6] = CFG_NAND_ADDR;              /* NAND address */
        val[7] = 0x2000;                               /* NAND register area
size */

        rc = fdt_find_and_setprop(blob, "/plb/opb/ebc", "ranges",
                                  val, sizeof(val), 1);
        if (rc) {
                printf("Unable to update property NOR mapping, err=%s\n",
                       fdt_strerror(rc));
        }
}

Felix.

-- 
View this message in context: http://www.nabble.com/Long-boot-delay-on-460EX-with-2.6.28-rc8-tp21070179p21087541.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Long boot delay on 460EX with 2.6.28-rc8

[Felix Radensky]

Stefan Roese wrote:
> On Friday 19 December 2008, Felix Radensky wrote:
>   
>> Thanks a lot to everyone who replied. I've managed to identify the cause
>> of the delay. The board is equipped with 256 MiB Samsung NAND flash.
>> Since NAND support is a must for this platform, I've intergated the ndfc
>> driver recently posted by Sean McLennan. With this driver enabled the delay
>> is present, without it messages appear on the console immediately.
>>
>> Is this an expected behavior - detection of 256 MiB NAND flash
>> takes around 20 seconds. The ndfc driver works fine after boot.
>>     
>
> No, 20 seconds is definitely too long. Something must be wrong with the ndfc 
> driver or the NAND dts entries.
>
> Best regards,
> Stefan
>   
Hi, Sean

Do you have any ideas what can cause such delay ?

Thanks.

Felix.

Re: Long boot delay on 460EX with 2.6.28-rc8

[Sean MacLennan]

On Fri, 19 Dec 2008 23:23:58 +0200
"Felix Radensky" <felix@embedded-sol.com> wrote:

> Hi, Sean
> 
> Do you have any ideas what can cause such delay ?

Sorry, I haven't been following this thread :(

We also use a 256M NAND, although a Spansion S29GL. While there is a
small delay in u-boot, there is no noticeable during the ndfc startup.

Can you read and write the NAND properly in u-boot? That is always the
first step. If that works, we can look at what might be different
between the u-boot and the ndfc driver.

Cheers,
   Sean

Re: Long boot delay on 460EX with 2.6.28-rc8

[Felix Radensky]

Sean MacLennan-2 wrote:
> 
> On Fri, 19 Dec 2008 23:23:58 +0200
> "Felix Radensky" <felix@embedded-sol.com> wrote:
> 
>> Hi, Sean
>> 
>> Do you have any ideas what can cause such delay ?
> 
> Sorry, I haven't been following this thread :(
> 
> We also use a 256M NAND, although a Spansion S29GL. While there is a
> small delay in u-boot, there is no noticeable during the ndfc startup.
> 
> Can you read and write the NAND properly in u-boot? That is always the
> first step. If that works, we can look at what might be different
> between the u-boot and the ndfc driver.
> 
> Cheers,
>    Sean
> 
> 

Thanks, Sean

I've found the cause of the delay. It was a stupid error on my part, not
related to ndfc driver, which is fine. Thanks a lot for you work on this.

I apologize for the noise.

Felix.
-- 
View this message in context: http://www.nabble.com/Long-boot-delay-on-460EX-with-2.6.28-rc8-tp21070179p21102440.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.

Re: Long boot delay on 460EX with 2.6.28-rc8

[Leon Woestenberg]

Felix,

On Sat, Dec 20, 2008 at 7:51 AM, Felix Radensky <felix@embedded-sol.com> wrote:
>
> I've found the cause of the delay. It was a stupid error on my part, not
> related to ndfc driver, which is fine. Thanks a lot for you work on this.
>
Could you share the error?* (I'm sure I can easily exceed your level
of stupidity)

It helps people who are curiously following the thread (either now or
in two years time, hi there!) learn if it matches their problem.

> I apologize for the noise.
>
Apologies not required.

Regards,
-- 
Leon

* unless of course, it was really really really stupid in the first
place and will cost you your daytime job. Then you are excused.

Re: Long boot delay on 460EX with 2.6.28-rc8

[Felix Radensky]

Leon Woestenberg-3 wrote:
> 
> Felix,
> 
> On Sat, Dec 20, 2008 at 7:51 AM, Felix Radensky <felix@embedded-sol.com>
> wrote:
>>
>> I've found the cause of the delay. It was a stupid error on my part, not
>> related to ndfc driver, which is fine. Thanks a lot for you work on this.
>>
> Could you share the error?* (I'm sure I can easily exceed your level
> of stupidity)
> 
> It helps people who are curiously following the thread (either now or
> in two years time, hi there!) learn if it matches their problem.
> 
> 
> Regards,
> -- 
> Leon
> 
> * unless of course, it was really really really stupid in the first
> place and will cost you your daytime job. Then you are excused.
> 
> 

Well, I was simply using the wrong kernel for testing :) I did all work
on 2.6.26, and backported ndfc driver to it. At this stage of the project
it was too risky to use 2.6.28. But I did the minimal porting work, to see
how 2.8.28 behaves on this hardware, and I thought that I saw this long
delay on 2.6.28 as well as on 2.6.26. I was wrong. The reason for long
delay on 2.6.26 is nand ecc code, which was since completely rewritten.
After I've copied  new drivers/mtd/nand/nand_ecc,c to 2.6.26,  the delay
disappeared.

Felix.
-- 
View this message in context: http://www.nabble.com/Long-boot-delay-on-460EX-with-2.6.28-rc8-tp21070179p21156958.html
Sent from the linuxppc-dev mailing list archive at Nabble.com.