From: luojiaxing <luojiaxing@huawei.com>
To: Petr Mladek <pmladek@suse.com>,
Sergey Senozhatsky <senozhatsky@chromium.org>
Cc: <sergey.senozhatsky@gmail.com>, <rostedt@goodmis.org>,
<john.ogness@linutronix.de>, <linux-kernel@vger.kernel.org>,
<linuxarm@huawei.com>, <bobo.shaobowang@huawei.com>
Subject: Re: [PATCH] printk: stop spining waiter when console resume to flush prb
Date: Mon, 10 May 2021 16:29:11 +0800 [thread overview]
Message-ID: <46df3838-e1a6-ee95-b398-bef0896d2b03@huawei.com> (raw)
In-Reply-To: <YJVnNQ7RGvx9JKxV@alley>
On 2021/5/8 0:13, Petr Mladek wrote:
> On Thu 2021-05-06 23:07:19, Sergey Senozhatsky wrote:
>> Thanks for Cc-ing Petr
>>
>> On (21/05/06 15:39), Petr Mladek wrote:
>>> Many printk messages might get accumulated when consoles were suspended.
>>> They are proceed when console_unlock() is called in resume_console().
>>>
>>> The possibility to pass the console lock owner was added to reduce the risk
>>> of softlockup when too many messages were handled in an atomic context.
>>>
>>> Now, resume_console() is always in a preemptible context that is safe
>>> to handle all accumulated messages. The possibility to pass the console
>>> lock owner actually makes things worse. The new owner might be in an atomic
>>> context and might cause softlockup when processing all messages accumulated
>>> when the console was suspended.
>>>
>>> Create new console_unlock_preemptible() that will not allow to pass
>>> the console lock owner. As a result, all accumulated messages will
>>> be proceed in the safe preemptible process.
>> If we have a lot of pending messages in the logbuf, then there is
>> something chatty - some context (task, irq) or maybe several contexts.
>> And those contexts can continue adding messages, while we print them
>> _exclusively_ from preemptible context only. without ever throttling down
>> printk() callers - something that console_owner spinning and handover
>> does for us. And those printk() callers can even preempt
>> console_unlock_preemptible() and cause delays and lost messages.
> Luo, please, correct me if I am wrong.
Hi, Petr, I reply the test result on your first reply for this patch.
Please check it when you are free.
>
> This patch a about one well defined scenario. The messages are
> accumulated between suspend_console() and resume_console().
> It is "small" part of the system hibernation. And we need
> to get them out now. There might be many if something special
> was debugged.
>
> I am pretty sure that Luo did not see any flood of messages:
>
> + Flood in more contexts would be balanced by switching
> the console_owner.
>
> + Flood in one context would be naturally throttled because
> this context will become the console_owner.
>
> In each case, these messages would be generated after
> console_resume(). Luo's original patch was explicitly talking
> about messages accumulated during the suspend.
>
> Luo, could you please provide some log showing the problem?
Sure, But, it is not easy to find that the printk got such problem in
the user's logs.
So I'm attaching a log of the simulation test. You can clearly see that
the driver thread calling dev_info() is blocked.
My test method is as follows:
Kernel thread A causes the console to enter suspend and then resume it
10 seconds later.
Kernel thread B repeatedly invokes dev_info() for 15 seconds after the
console suspend.
Part of dmesg log I save before:
[ 288.013869] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 288.013870] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 288.013871] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 288.013872] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 288.013873] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 288.013874] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 288.013875] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 288.013876] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 288.013877] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 289.670256] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 299.286325] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 299.291198] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 299.296063] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 299.300924] hisi_sas_v3_hw 0000:b4:02.0: print test
[ 299.305787] hisi_sas_v3_hw 0000:b4:02.0: print test
The log shows that when thread B invokes dev_info, dev_info is blocked
for several seconds.
Thanks
Jiaxing
>
> Best Regards,
> Petr
>
> .
>
next prev parent reply other threads:[~2021-05-10 8:29 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-06 8:00 [PATCH] printk: stop spining waiter when console resume to flush prb Luo Jiaxing
2021-05-06 13:13 ` Steven Rostedt
2021-05-07 8:35 ` luojiaxing
2021-05-06 13:39 ` Petr Mladek
2021-05-06 14:07 ` Sergey Senozhatsky
2021-05-06 14:12 ` Sergey Senozhatsky
2021-05-06 15:14 ` John Ogness
2021-05-07 7:58 ` luojiaxing
2021-05-07 7:33 ` luojiaxing
2021-05-07 7:49 ` Sergey Senozhatsky
2021-05-07 16:36 ` Petr Mladek
2021-05-10 8:26 ` Sergey Senozhatsky
2021-05-10 10:17 ` Petr Mladek
2021-05-10 10:32 ` John Ogness
2021-05-10 11:16 ` Sergey Senozhatsky
2021-05-10 11:43 ` Sergey Senozhatsky
2021-05-07 16:13 ` Petr Mladek
2021-05-10 8:29 ` luojiaxing [this message]
2021-05-10 9:50 ` Petr Mladek
2021-05-10 12:06 ` Sergey Senozhatsky
2021-05-10 7:41 ` luojiaxing
2021-05-10 9:30 ` Petr Mladek
2021-05-11 7:32 ` luojiaxing
2021-05-11 9:08 ` Petr Mladek
2021-05-13 7:55 ` luojiaxing
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=46df3838-e1a6-ee95-b398-bef0896d2b03@huawei.com \
--to=luojiaxing@huawei.com \
--cc=bobo.shaobowang@huawei.com \
--cc=john.ogness@linutronix.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxarm@huawei.com \
--cc=pmladek@suse.com \
--cc=rostedt@goodmis.org \
--cc=senozhatsky@chromium.org \
--cc=sergey.senozhatsky@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).