arm64: pstore: printk causing hang during boot in __memcpy_toio with pstore enabled

arm64: pstore: printk causing hang during boot in __memcpy_toio with pstore enabled

[Ivid Suvarna]

Hi,

When pstore is enabled and a *pr_info(any printk) in
__memcpy_toio(arch/arm64/kernel/io.c)* is added, kernel wont boot and
just hangs.

The path where __memcpy_toio is called is below:

->persistent_ram_update
  -> memcpy_toio
    -> __memcpy_toio

I tried with trace_printk and kernel boots fine. I understand that
printk has overhead, but is this expected when we use some printk
statement in __memcpy_toio?

Or is it some kind of bug? Tested on db410c.

Please let me know your opinion.

Thanks,

Re: arm64: pstore: printk causing hang during boot in __memcpy_toio with pstore enabled

[Guenter Roeck]

On 08/01/2018 05:35 AM, Ivid Suvarna wrote:
> Hi,
> 
> When pstore is enabled and a *pr_info(any printk) in
> __memcpy_toio(arch/arm64/kernel/io.c)* is added, kernel wont boot and
> just hangs.
> 
> The path where __memcpy_toio is called is below:
> 
> ->persistent_ram_update
>    -> memcpy_toio
>      -> __memcpy_toio
> 
> I tried with trace_printk and kernel boots fine. I understand that
> printk has overhead, but is this expected when we use some printk
> statement in __memcpy_toio?
> 

I think the problem may be that the printk() output is copied to pstore.
Since pstore calls memcpy_toio(), you get a nice recursion if you add a
printk() call to it.

Guenter

Re: arm64: pstore: printk causing hang during boot in __memcpy_toio with pstore enabled

[Ivid Suvarna]

On Wed, Aug 1, 2018 at 6:49 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> On 08/01/2018 05:35 AM, Ivid Suvarna wrote:
>>
>> Hi,
>>
>> When pstore is enabled and a *pr_info(any printk) in
>> __memcpy_toio(arch/arm64/kernel/io.c)* is added, kernel wont boot and
>> just hangs.
>>
>> The path where __memcpy_toio is called is below:
>>
>> ->persistent_ram_update
>>    -> memcpy_toio
>>      -> __memcpy_toio
>>
>> I tried with trace_printk and kernel boots fine. I understand that
>> printk has overhead, but is this expected when we use some printk
>> statement in __memcpy_toio?
>>
>
> I think the problem may be that the printk() output is copied to pstore.
> Since pstore calls memcpy_toio(), you get a nice recursion if you add a
> printk() call to it.
>

Is there any solution to this other than not adding printk :p

Re: arm64: pstore: printk causing hang during boot in __memcpy_toio with pstore enabled

[Guenter Roeck]

On Wed, Aug 01, 2018 at 10:46:06PM +0530, Ivid Suvarna wrote:
> On Wed, Aug 1, 2018 at 6:49 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> > On 08/01/2018 05:35 AM, Ivid Suvarna wrote:
> >>
> >> Hi,
> >>
> >> When pstore is enabled and a *pr_info(any printk) in
> >> __memcpy_toio(arch/arm64/kernel/io.c)* is added, kernel wont boot and
> >> just hangs.
> >>
> >> The path where __memcpy_toio is called is below:
> >>
> >> ->persistent_ram_update
> >>    -> memcpy_toio
> >>      -> __memcpy_toio
> >>
> >> I tried with trace_printk and kernel boots fine. I understand that
> >> printk has overhead, but is this expected when we use some printk
> >> statement in __memcpy_toio?
> >>
> >
> > I think the problem may be that the printk() output is copied to pstore.
> > Since pstore calls memcpy_toio(), you get a nice recursion if you add a
> > printk() call to it.
> >
> 
> Is there any solution to this other than not adding printk :p

Well, disabling pstore would help. You could also use a trace function,
as you already found out, or you could use gdb for debugging. Since
this is obviously a debug image, you should be willing to accept some
limitations/restrictions.

Guenter

Re: arm64: pstore: printk causing hang during boot in __memcpy_toio with pstore enabled

[Ivid Suvarna]

On Wed, Aug 1, 2018 at 10:55 PM, Guenter Roeck <linux@roeck-us.net> wrote:
> On Wed, Aug 01, 2018 at 10:46:06PM +0530, Ivid Suvarna wrote:
>> On Wed, Aug 1, 2018 at 6:49 PM, Guenter Roeck <linux@roeck-us.net> wrote:
>> > On 08/01/2018 05:35 AM, Ivid Suvarna wrote:
>> >>
>> >> Hi,
>> >>
>> >> When pstore is enabled and a *pr_info(any printk) in
>> >> __memcpy_toio(arch/arm64/kernel/io.c)* is added, kernel wont boot and
>> >> just hangs.
>> >>
>> >> The path where __memcpy_toio is called is below:
>> >>
>> >> ->persistent_ram_update
>> >>    -> memcpy_toio
>> >>      -> __memcpy_toio
>> >>
>> >> I tried with trace_printk and kernel boots fine. I understand that
>> >> printk has overhead, but is this expected when we use some printk
>> >> statement in __memcpy_toio?
>> >>
>> >
>> > I think the problem may be that the printk() output is copied to pstore.
>> > Since pstore calls memcpy_toio(), you get a nice recursion if you add a
>> > printk() call to it.
>> >
>>
>> Is there any solution to this other than not adding printk :p
>
> Well, disabling pstore would help. You could also use a trace function,
> as you already found out, or you could use gdb for debugging. Since
> this is obviously a debug image, you should be willing to accept some
> limitations/restrictions.
>

Ok. But trace_printk output is available only after boot, so to debug
any boot issues,
it may not be helpful. And gdb for kernel debugging? Maybe u meant kgdb?

Let me check if this issue is only for pstore path or not.