All of lore.kernel.org
 help / color / mirror / Atom feed
From: Petr Mladek <pmladek@suse.com>
To: Stephen Brennan <stephen.s.brennan@oracle.com>
Cc: Sergey Senozhatsky <senozhatsky@chromium.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	John Ogness <john.ogness@linutronix.de>,
	Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/4] printk: disable optimistic spin during panic
Date: Tue, 25 Jan 2022 13:42:03 +0100	[thread overview]
Message-ID: <Ye/wG5IsIYer7akH@alley> (raw)
In-Reply-To: <20220121190222.572694-3-stephen.s.brennan@oracle.com>

On Fri 2022-01-21 11:02:20, Stephen Brennan wrote:
> A CPU executing with console lock spinning enabled might be halted
> during a panic. Before the panicking CPU calls console_flush_on_panic(),
> it may call console_trylock(), which attempts to optimistically spin,
> deadlocking the panic CPU:
> 
> CPU 0 (panic CPU)             CPU 1
> -----------------             ------
>                               printk() {
>                                 vprintk_func() {
>                                   vprintk_default() {
>                                     vprintk_emit() {
>                                       console_unlock() {
>                                         console_lock_spinning_enable();
>                                         ... printing to console ...
> panic() {
>   crash_smp_send_stop() {
>     NMI  -------------------> HALT
>   }
>   atomic_notifier_call_chain() {
>     printk() {
>       ...
>       console_trylock_spinnning() {
>         // optimistic spin infinitely
> 
> This hang during panic can be induced when a kdump kernel is loaded, and
> crash_kexec_post_notifiers=1 is present on the kernel command line. The
> following script which concurrently writes to /dev/kmsg, and triggers a
> panic, can result in this hang:
> 
>     #!/bin/bash
>     date
>     # 991 chars (based on log buffer size):
>     chars="$(printf 'a%.0s' {1..991})"
>     while :; do
>         echo $chars > /dev/kmsg
>     done &
>     echo c > /proc/sysrq-trigger &
>     date
>     exit
> 
> To avoid this deadlock, ensure that console_trylock_spinning() does not
> allow spinning once a panic has begun.
> 
> Fixes: dbdda842fe96 ("printk: Add console owner and waiter logic to load balance console writes")
> 
> Suggested-by: Petr Mladek <pmladek@suse.com>
> Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>

Looks good to me:

Reviewed-by: Petr Mladek <pmladek@suse.com>

Best Regards,
Petr

  reply	other threads:[~2022-01-25 12:45 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-21 19:02 [PATCH 0/4] printk: reduce deadlocks during panic Stephen Brennan
2022-01-21 19:02 ` [PATCH 1/4] panic: Add panic_in_progress helper Stephen Brennan
2022-01-25 11:48   ` Petr Mladek
2022-01-26 17:37     ` Stephen Brennan
2022-01-21 19:02 ` [PATCH 2/4] printk: disable optimistic spin during panic Stephen Brennan
2022-01-25 12:42   ` Petr Mladek [this message]
2022-01-26  9:18   ` Sergey Senozhatsky
2022-01-26  9:45     ` John Ogness
2022-01-26 10:06       ` Sergey Senozhatsky
2022-01-26 18:15         ` Stephen Brennan
2022-01-27  7:11           ` Sergey Senozhatsky
2022-01-27  9:09             ` John Ogness
2022-01-27 11:38             ` Petr Mladek
2022-01-27 12:43               ` John Ogness
2022-01-27 14:25                 ` Petr Mladek
2022-01-21 19:02 ` [PATCH 3/4] printk: Avoid livelock with heavy printk " Stephen Brennan
2022-01-25 14:25   ` Petr Mladek
2022-01-21 19:02 ` [PATCH 4/4] printk: Drop console_sem " Stephen Brennan
2022-01-24 16:12   ` John Ogness
2022-01-24 16:26     ` John Ogness
2022-01-25 15:04     ` Petr Mladek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Ye/wG5IsIYer7akH@alley \
    --to=pmladek@suse.com \
    --cc=john.ogness@linutronix.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=senozhatsky@chromium.org \
    --cc=sergey.senozhatsky@gmail.com \
    --cc=stephen.s.brennan@oracle.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.