* + kdump-fix-crash_kexec-smp_send_stop-race-in-panic.patch added to -mm tree
@ 2011-10-28 23:11 akpm
0 siblings, 0 replies; 2+ messages in thread
From: akpm @ 2011-10-28 23:11 UTC (permalink / raw)
To: mm-commits; +Cc: holzheu, ebiederm, heiko.carstens, schwidefsky, vgoyal
The patch titled
Subject: kdump: fix crash_kexec()/smp_send_stop() race in panic
has been added to the -mm tree. Its filename is
kdump-fix-crash_kexec-smp_send_stop-race-in-panic.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/SubmitChecklist when testing your code ***
See http://userweb.kernel.org/~akpm/stuff/added-to-mm.txt to find
out what to do about this
The current -mm tree may be found at http://userweb.kernel.org/~akpm/mmotm/
------------------------------------------------------
From: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Subject: kdump: fix crash_kexec()/smp_send_stop() race in panic
When two CPUs call panic at the same time there is a possible race
condition that can stop kdump. The first CPU calls crash_kexec() and the
second CPU calls smp_send_stop() in panic() before crash_kexec() finished
on the first CPU. So the second CPU stops the first CPU and therefore
kdump fails:
1st CPU:
panic()->crash_kexec()->mutex_trylock(&kexec_mutex)-> do kdump
2nd CPU:
panic()->crash_kexec()->kexec_mutex already held by 1st CPU
->smp_send_stop()-> stop 1st CPU (stop kdump)
This patch fixes the problem by introducing a spinlock in panic that
allows only one CPU to process crash_kexec() and the subsequent panic
code.
Signed-off-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
kernel/panic.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff -puN kernel/panic.c~kdump-fix-crash_kexec-smp_send_stop-race-in-panic kernel/panic.c
--- a/kernel/panic.c~kdump-fix-crash_kexec-smp_send_stop-race-in-panic
+++ a/kernel/panic.c
@@ -59,6 +59,7 @@ EXPORT_SYMBOL(panic_blink);
*/
NORET_TYPE void panic(const char * fmt, ...)
{
+ static DEFINE_SPINLOCK(panic_lock);
static char buf[1024];
va_list args;
long i, i_next = 0;
@@ -82,6 +83,13 @@ NORET_TYPE void panic(const char * fmt,
#endif
/*
+ * Only one CPU is allowed to execute the panic code from here. For
+ * multiple parallel invocations of panic all other CPUs will wait on
+ * the panic_lock. They are stopped afterwards by smp_send_stop().
+ */
+ spin_lock(&panic_lock);
+
+ /*
* If we have crashed and we have a crash kernel loaded let it handle
* everything else.
* Do we want to call this before we try to display a message?
_
Subject: Subject: kdump: fix crash_kexec()/smp_send_stop() race in panic
Patches currently in -mm which might be from holzheu@linux.vnet.ibm.com are
linux-next.patch
lib-stringc-fix-strim-semantics-for-strings-that-have-only-blanks.patch
kdump-fix-crash_kexec-smp_send_stop-race-in-panic.patch
kdump-add-udev-events-for-memory-online-offline.patch
^ permalink raw reply [flat|nested] 2+ messages in thread
* + kdump-fix-crash_kexec-smp_send_stop-race-in-panic.patch added to -mm tree
@ 2011-11-30 22:55 akpm
0 siblings, 0 replies; 2+ messages in thread
From: akpm @ 2011-11-30 22:55 UTC (permalink / raw)
To: mm-commits; +Cc: holzheu, cmetcalf
The patch titled
Subject: kdump: fix crash_kexec()/smp_send_stop() race in panic()
has been added to the -mm tree. Its filename is
kdump-fix-crash_kexec-smp_send_stop-race-in-panic.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/SubmitChecklist when testing your code ***
See http://userweb.kernel.org/~akpm/stuff/added-to-mm.txt to find
out what to do about this
The current -mm tree may be found at http://userweb.kernel.org/~akpm/mmotm/
------------------------------------------------------
From: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Subject: kdump: fix crash_kexec()/smp_send_stop() race in panic()
When two CPUs call panic at the same time there is a possible race
condition that can stop kdump. The first CPU calls crash_kexec() and the
second CPU calls smp_send_stop() in panic() before crash_kexec() finished
on the first CPU. So the second CPU stops the first CPU and therefore
kdump fails:
1st CPU:
panic()->crash_kexec()->mutex_trylock(&kexec_mutex)-> do kdump
2nd CPU:
panic()->crash_kexec()->kexec_mutex already held by 1st CPU
->smp_send_stop()-> stop 1st CPU (stop kdump)
This patch fixes the problem by introducing a spinlock in panic that
allows only one CPU to process crash_kexec() and the subsequent panic
code.
All other CPUs call the weak function panic_smp_self_stop() that stops the
CPU itself. This function can be overloaded by architecture code. For
example "tile" can use their lower-power "nap" instruction for that.
Signed-off-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Acked-by: Chris Metcalf <cmetcalf@tilera.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
kernel/panic.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff -puN kernel/panic.c~kdump-fix-crash_kexec-smp_send_stop-race-in-panic kernel/panic.c
--- a/kernel/panic.c~kdump-fix-crash_kexec-smp_send_stop-race-in-panic
+++ a/kernel/panic.c
@@ -49,6 +49,15 @@ static long no_blink(int state)
long (*panic_blink)(int state);
EXPORT_SYMBOL(panic_blink);
+/*
+ * Stop ourself in panic -- architecture code may override this
+ */
+void __weak panic_smp_self_stop(void)
+{
+ while (1)
+ cpu_relax();
+}
+
/**
* panic - halt the system
* @fmt: The text string to print
@@ -59,6 +68,7 @@ EXPORT_SYMBOL(panic_blink);
*/
void panic(const char *fmt, ...)
{
+ static DEFINE_SPINLOCK(panic_lock);
static char buf[1024];
va_list args;
long i, i_next = 0;
@@ -68,8 +78,14 @@ void panic(const char *fmt, ...)
* It's possible to come here directly from a panic-assertion and
* not have preempt disabled. Some functions called from here want
* preempt to be disabled. No point enabling it later though...
+ *
+ * Only one CPU is allowed to execute the panic code from here. For
+ * multiple parallel invocations of panic, all other CPUs either
+ * stop themself or will wait until they are stopped by the 1st CPU
+ * with smp_send_stop().
*/
- preempt_disable();
+ if (!spin_trylock(&panic_lock))
+ panic_smp_self_stop();
console_verbose();
bust_spinlocks(1);
_
Subject: Subject: kdump: fix crash_kexec()/smp_send_stop() race in panic()
Patches currently in -mm which might be from holzheu@linux.vnet.ibm.com are
linux-next.patch
kdump-add-missing-ram-resource-in-crash_shrink_memory.patch
kdump-add-udev-events-for-memory-online-offline.patch
kdump-crashk_res-init-check-for-sys-kernel-kexec_crash_size.patch
kdump-fix-crash_kexec-smp_send_stop-race-in-panic.patch
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2011-11-30 22:55 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-10-28 23:11 + kdump-fix-crash_kexec-smp_send_stop-race-in-panic.patch added to -mm tree akpm
2011-11-30 22:55 akpm
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.