linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Petr Mladek <pmladek@suse.com>
To: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Andrew Morton <akpm@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>,
	John Ogness <john.ogness@linutronix.de>,
	Petr Tesarik <ptesarik@suse.cz>,
	Konstantin Khlebnikov <koct9i@gmail.com>,
	x86@kernel.org, linux-kernel@vger.kernel.org,
	Petr Mladek <pmladek@suse.com>
Subject: [PATCH 0/2] panic/printk/x86: Prevent some more printk-related deadlocks in panic()
Date: Tue, 16 Jul 2019 09:28:03 +0200	[thread overview]
Message-ID: <20190716072805.22445-1-pmladek@suse.com> (raw)

Hi,

I have found some spare duct tape and wrapped some more printk-related
deadlocks in panic().

More seriously, someone reported a deadlock in panic(). Some non-trivial
debugging pointed out a problem with the following combination:

     + x86_64 architecture
     + panic()
     + pstore configured as message dumper (kmsg_dump())
     + crash kernel configured
     + crash_kexec_post_notifiers

In this case, CPUs are stopped by crash_smp_send_stop(). It uses
NMI but it does not modify cpu_online_mask. Therefore logbuf_lock
might stay locked, see 2nd patch for more details.

The above is a real corner case. But similar problem seems to be
even in the common situations on architectures that do not use
NMI in smp_send_stop() as a fallback, see 1st patch.

Back to the duct tape. I hope that we will get rid of these problems
with the lockless printk ringbuffer rather sooner than later.
But it still might take some time. And the two fixes might be
useful also for stable kernels.


Petr Mladek (2):
  printk/panic: Access the main printk log in panic() only when safe
  printk/panic/x86: Allow to access printk log buffer after
    crash_smp_send_stop()

 arch/x86/kernel/crash.c     |  6 +++++-
 include/linux/printk.h      |  6 ++++++
 kernel/panic.c              | 49 +++++++++++++++++++++++++++------------------
 kernel/printk/printk_safe.c | 37 ++++++++++++++++++++++------------
 4 files changed, 65 insertions(+), 33 deletions(-)

-- 
2.16.4


             reply	other threads:[~2019-07-16  7:28 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-16  7:28 Petr Mladek [this message]
2019-07-16  7:28 ` [PATCH 1/2] printk/panic: Access the main printk log in panic() only when safe Petr Mladek
2019-07-17  9:56   ` Sergey Senozhatsky
2019-07-18  8:36     ` Petr Mladek
2019-07-18  9:49       ` Sergey Senozhatsky
2019-07-19 12:57         ` Petr Mladek
2019-07-23  3:13           ` Sergey Senozhatsky
2019-07-24 12:27             ` Petr Mladek
2019-07-31  6:08               ` Sergey Senozhatsky
2019-07-31  6:59                 ` Sergey Senozhatsky
2019-07-18 10:11   ` Sergey Senozhatsky
2019-07-16  7:28 ` [PATCH 2/2] printk/panic/x86: Allow to access printk log buffer after crash_smp_send_stop() Petr Mladek
2019-07-18 10:47   ` Sergey Senozhatsky
2019-07-18 11:07     ` Konstantin Khlebnikov
2019-07-18 11:29       ` Sergey Senozhatsky
2019-07-19 12:19         ` Petr Mladek
2019-07-18  9:59 ` [PATCH 0/2] panic/printk/x86: Prevent some more printk-related deadlocks in panic() Konstantin Khlebnikov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190716072805.22445-1-pmladek@suse.com \
    --to=pmladek@suse.com \
    --cc=akpm@linux-foundation.org \
    --cc=john.ogness@linutronix.de \
    --cc=koct9i@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=ptesarik@suse.cz \
    --cc=rostedt@goodmis.org \
    --cc=sergey.senozhatsky@gmail.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).