linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Ben Hutchings <ben@decadent.org.uk>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: akpm@linux-foundation.org, Denis Kirjanov <kda@linux-powerpc.org>,
	"Sergey Senozhatsky" <sergey.senozhatsky@gmail.com>,
	"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
	"Daniel Wang" <wonderfly@google.com>,
	"Jiri Slaby" <jslaby@suse.com>, "Petr Mladek" <pmladek@suse.com>,
	"Sergey Senozhatsky" <sergey.senozhatsky.work@gmail.com>,
	"Peter Feiner" <pfeiner@google.com>,
	"Alan Cox" <gnomes@lxorguk.ukuu.org.uk>,
	linux-serial@vger.kernel.org,
	"Linus Torvalds" <torvalds@linux-foundation.org>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Steven Rostedt" <rostedt@goodmis.org>
Subject: [PATCH 3.16 14/99] panic: avoid deadlocks in re-entrant console drivers
Date: Tue, 02 Apr 2019 14:38:27 +0100	[thread overview]
Message-ID: <lsq.1554212307.529644269@decadent.org.uk> (raw)
In-Reply-To: <lsq.1554212307.17110877@decadent.org.uk>

3.16.65-rc1 review patch.  If anyone has any objections, please let me know.

------------------

From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>

commit c7c3f05e341a9a2bd1a92993d4f996cfd6e7348e upstream.

=46romprintk()/serial console point of view panic() is special, because
it may force CPU to re-enter printk() or/and serial console driver.
Therefore, some of serial consoles drivers are re-entrant. E.g. 8250:

serial8250_console_write()
{
	if (port->sysrq)
		locked = 0;
	else if (oops_in_progress)
		locked = spin_trylock_irqsave(&port->lock, flags);
	else
		spin_lock_irqsave(&port->lock, flags);
	...
}

panic() does set oops_in_progress via bust_spinlocks(1), so in theory
we should be able to re-enter serial console driver from panic():

	CPU0
	<NMI>
	uart_console_write()
	serial8250_console_write()		// if (oops_in_progress)
						//    spin_trylock_irqsave()
	call_console_drivers()
	console_unlock()
	console_flush_on_panic()
	bust_spinlocks(1)			// oops_in_progress++
	panic()
	<NMI/>
	spin_lock_irqsave(&port->lock, flags)   // spin_lock_irqsave()
	serial8250_console_write()
	call_console_drivers()
	console_unlock()
	printk()
	...

However, this does not happen and we deadlock in serial console on
port->lock spinlock. And the problem is that console_flush_on_panic()
called after bust_spinlocks(0):

void panic(const char *fmt, ...)
{
	bust_spinlocks(1);
	...
	bust_spinlocks(0);
	console_flush_on_panic();
	...
}

bust_spinlocks(0) decrements oops_in_progress, so oops_in_progress
can go back to zero. Thus even re-entrant console drivers will simply
spin on port->lock spinlock. Given that port->lock may already be
locked either by a stopped CPU, or by the very same CPU we execute
panic() on (for instance, NMI panic() on printing CPU) the system
deadlocks and does not reboot.

Fix this by removing bust_spinlocks(0), so oops_in_progress is always
set in panic() now and, thus, re-entrant console drivers will trylock
the port->lock instead of spinning on it forever, when we call them
from console_flush_on_panic().

Link: http://lkml.kernel.org/r/20181025101036.6823-1-sergey.senozhatsky@gmail.com
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Daniel Wang <wonderfly@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Alan Cox <gnomes@lxorguk.ukuu.org.uk>
Cc: Jiri Slaby <jslaby@suse.com>
Cc: Peter Feiner <pfeiner@google.com>
Cc: linux-serial@vger.kernel.org
Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
---
 kernel/panic.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -13,6 +13,7 @@
 #include <linux/kmsg_dump.h>
 #include <linux/kallsyms.h>
 #include <linux/notifier.h>
+#include <linux/vt_kern.h>
 #include <linux/module.h>
 #include <linux/random.h>
 #include <linux/ftrace.h>
@@ -178,7 +179,10 @@ void panic(const char *fmt, ...)
 	 */
 	crash_kexec(NULL);
 
-	bust_spinlocks(0);
+#ifdef CONFIG_VT
+	unblank_screen();
+#endif
+	console_unblank();
 
 	/*
 	 * We may have ended up stopping the CPU holding the lock (in


  parent reply	other threads:[~2019-04-02 13:40 UTC|newest]

Thread overview: 106+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-02 13:38 [PATCH 3.16 00/99] 3.16.65-rc1 review Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 58/99] genwqe: Fix size check Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 24/99] MIPS: Expand MIPS32 ASIDs to 64 bits Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 62/99] ext4: avoid declaring fs inconsistent due to invalid file handles Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 33/99] gpiolib: Fix return value of gpio_to_desc() stub if !GPIOLIB Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 42/99] perf parse-events: Fix unchecked usage of strncpy() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 66/99] ath6kl: Only use match sets when firmware supports it Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 69/99] fbdev: fbmem: behave better with small rotated displays and many CPUs Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 65/99] scsi: megaraid_sas: Use 63-bit DMA addressing Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 40/99] perf svghelper: Fix unchecked usage of strncpy() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 22/99] sunrpc: fix cache_head leak due to queued request Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 44/99] USB: serial: pl2303: add ids for Hewlett-Packard HP POS pole displays Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 46/99] pinctrl: sh-pfc: r8a7740: Add missing LCD0 marks to lcd0_data24_1 group Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 55/99] KVM: arm/arm64: Fix VMID alloc race by reverting to lock-less Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 09/99] MIPS: SiByte: Enable ZONE_DMA32 for LittleSur Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 29/99] tools/lib/lockdep: Rename "trywlock" into "trywrlock" Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 10/99] power: supply: olpc_battery: correct the temperature units Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 25/99] misc: vexpress: Off by one in vexpress_syscfg_exec() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 45/99] pinctrl: sh-pfc: r8a7740: Add missing REF125CK pin to gether_gmii group Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 13/99] perf pmu: Suppress potential format-truncation warning Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 38/99] Btrfs: fix fsync of files with multiple hard links in new directories Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 35/99] btrfs: dev-replace: go back to suspended state if target device is missing Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 17/99] lib/string.c: remove duplicated function Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 31/99] ALSA: pcm: Fix potential Spectre v1 vulnerability Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 67/99] powerpc/configs: Don't enable PPC_EARLY_DEBUG in defconfigs Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 36/99] Btrfs: fill ->last_trans for delayed inode in btrfs_fill_inode Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 04/99] pcrypt: use format specifier in kobject_add Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 16/99] f2fs: read page index before freeing Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 20/99] b43: Fix error in cordic routine Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 26/99] mips: bpf: fix encoding bug for mm_srlv32_op Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 30/99] ALSA: emux: Fix potential Spectre v1 vulnerabilities Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 11/99] MIPS: Ensure pmd_present() returns false after pmd_mknotpresent() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 05/99] dlm: fixed memory leaks after failed ls_remove_names allocation Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 64/99] ext4: check for shutdown and r/o file system in ext4_write_inode() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 07/99] dlm: lost put_lkb on error path in receive_convert() and receive_unlock() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 48/99] pinctrl: sh-pfc: r8a7791: Remove bogus marks from vin1_b_data18 group Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 37/99] Btrfs: fix stale dir entries after unlink, inode eviction and fsync Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 32/99] crypto: user - support incremental algorithm dumps Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 18/99] altera-stapl: check for a null key before strcasecmp'ing it Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 02/99] drm/i915/ringbuffer: Delay after EMIT_INVALIDATE for gen4/gen5 Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 21/99] ext4: missing unlock/put_page() in ext4_try_to_write_inline_data() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 68/99] cdc-acm: fix abnormal DATA RX issue for Mediatek Preloader Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 52/99] pinctrl: sh-pfc: sh7269: Add missing PCIOR0 field Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 06/99] dlm: possible memory leak on error path in create_lkb() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 41/99] perf ui helpline: Use strlcpy() as a shorter form of strncpy() + explicit set nul Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 47/99] pinctrl: sh-pfc: r8a7791: Remove bogus ctrl marks from qspi_data4_b group Ben Hutchings
2019-04-02 13:38 ` Ben Hutchings [this message]
2019-04-02 13:38 ` [PATCH 3.16 34/99] kvm: vmx: Set IA32_TSC_AUX for legacy mode guests Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 63/99] ext4: force inode writes when nfsd calls commit_metadata() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 19/99] serial: imx: fix error handling in console_setup Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 43/99] net/mlx5: Continue driver initialization despite debugfs failure Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 50/99] pinctrl: sh-pfc: sh7734: Add missing IPSR11 field Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 53/99] pinctrl: sh-pfc: sh7734: Remove bogus IPSR10 value Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 27/99] scsi: zfcp: fix posting too many status read buffers leading to adapter shutdown Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 59/99] ALSA: rme9652: Fix potential Spectre v1 vulnerability Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 49/99] pinctrl: sh-pfc: sh73a0: Add missing TO pin to tpu4_to3 group Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 61/99] ext4: include terminating u32 in size of xattr entries when expanding inodes Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 54/99] pinctrl: sh-pfc: sh7734: Fix shifted values in IPSR10 Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 15/99] drm: rcar-du: Fix vblank initialization Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 60/99] ALSA: emu10k1: Fix potential Spectre v1 vulnerabilities Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 23/99] tty/ldsem: Wake up readers after timed out down_write() Ben Hutchings
2019-04-02 14:22   ` Dmitry Safonov
2019-04-02 14:32     ` Ben Hutchings
2019-04-02 14:39       ` Dmitry Safonov
2019-05-07 18:38         ` Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 51/99] pinctrl: sh-pfc: sh7264: Fix PFCR3 and PFCR0 register configuration Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 03/99] x86/PCI: Fix Broadcom CNB20LE unintended sign extension (redux) Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 01/99] wireless: airo: potential buffer overflow in sprintf() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 56/99] IB/qib: Fix an error code in qib_sdma_verbs_send() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 28/99] sata_rcar: fix deferred probing Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 57/99] usb: r8a66597: Fix a possible concurrency use-after-free bug in r8a66597_endpoint_disable() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 12/99] MIPS: Align kernel load address to 64KB Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 08/99] dlm: memory leaks on error path in dlm_user_request() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 39/99] perf help: Remove needless use of strncpy() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 94/99] ext4: fix special inode number checks in __ext4_iget() Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 88/99] mm: migration: fix migration of huge PMD shared pages Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 74/99] Input: nomadik-ske-keypad - fix a loop timeout test Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 97/99] Driver: Vmxnet3: Fix regression caused by 5738a09 Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 98/99] Revert "cifs: empty TargetInfo leads to crash on recovery" Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 90/99] hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 82/99] 9p/net: put a lower bound on msize Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 89/99] mm, memory_hotplug: do not clear numa_node association after hot_remove Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 73/99] powerpc/tm: Unset MSR[TS] if not recheckpointing Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 80/99] ext4: ext4_inline_data_fiemap should respect callers argument Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 75/99] kvm: Disallow wraparound in kvm_gfn_to_hva_cache_init Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 96/99] net/hamradio/6pack: use mod_timer() to rearm timers Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 93/99] ibmveth: fix DMA unmap error in ibmveth_xmit_start error path Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 72/99] powerpc/tm: Set MSR[TS] just prior to recheckpoint Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 71/99] igb: Fix an issue that PME is not enabled during runtime suspend Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 86/99] mm Documentation: undoc non-linear vmas Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 84/99] ceph: don't update importing cap's mseq when handing cap export Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 95/99] ALSA: hda/tegra: clear pending irq handlers Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 87/99] mm: rmap use pte lock not mmap_sem to set PageMlocked Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 76/99] KVM: x86: Use jmp to invoke kvm_spurious_fault() from .fixup Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 85/99] sunrpc: use SVC_NET() in svcauth_gss_* functions Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 81/99] ext4: fix a potential fiemap/page fault deadlock w/ inline_data Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 70/99] fbdev: fbcon: Fix unregister crash when more than one framebuffer Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 92/99] block/swim3: Fix -EBUSY error when re-opening device after unmount Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 79/99] ext4: make sure enough credits are reserved for dioread_nolock writes Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 77/99] MIPS: BCM63XX: fix switch core reset on BCM6368 Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 99/99] CIFS: Enable encryption during session setup phase Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 91/99] ext4: avoid kernel warning when writing the superblock to a dead device Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 83/99] 9p/net: fix memory leak in p9_client_create Ben Hutchings
2019-04-02 13:38 ` [PATCH 3.16 78/99] CIFS: Fix error mapping for SMB2_LOCK command which caused OFD lock problem Ben Hutchings
2019-04-03  2:22 ` [PATCH 3.16 00/99] 3.16.65-rc1 review Guenter Roeck
2019-04-03 14:12   ` Ben Hutchings

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=lsq.1554212307.529644269@decadent.org.uk \
    --to=ben@decadent.org.uk \
    --cc=akpm@linux-foundation.org \
    --cc=gnomes@lxorguk.ukuu.org.uk \
    --cc=gregkh@linuxfoundation.org \
    --cc=jslaby@suse.com \
    --cc=kda@linux-powerpc.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-serial@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=pfeiner@google.com \
    --cc=pmladek@suse.com \
    --cc=rostedt@goodmis.org \
    --cc=sergey.senozhatsky.work@gmail.com \
    --cc=sergey.senozhatsky@gmail.com \
    --cc=stable@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=wonderfly@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).