[PATCH 0/2] Dump stack after certain machine checks

* [PATCH 0/2] Dump stack after certain machine checks
@ 2022-09-22 19:51 Tony Luck
  2022-09-22 19:51 ` [PATCH 1/2] x86/mce: Use severity table to handle uncorrected errors in kernel Tony Luck
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Tony Luck @ 2022-09-22 19:51 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Yazen Ghannam, Smita Koralahalli, Carlos Bilbao, x86, linux-edac,
	linux-kernel, Tony Luck

In general it isn't very useful to dump the kernel stack in the panic
from a fatal machine check. The problem is almost always hardware
related, so knowing how the kernel got to the routine that triggered the
machine check isn't useful.

But Linux now has the capability to recover from most user mode and a
few kernel mode memory related machine checks. Validation folks are
testing that out and occasionally bring a kernel log like this to me:

[69608.047771] mce: [Hardware Error]: Machine check: Data load in unrecoverable area of kernel
[69608.021729] mce: [Hardware Error]: TSC 7874eb580177 ADDR 43bb84bd00 MISC 86 PPIN 9f061818e1a92082 
[69608.047773] Kernel panic - not syncing: Fatal local machine check
[69608.021720] mce: [Hardware Error]: RIP 10:<ffffffff8b767517> {copy_page+0x7/0x10}

All I can tell them is that Linux was copying a page and hit poison in
the source of the copy. But there are lots of reasons why Linux may be
copying a page. A stack trace would help figure out if:
1) the test was bad and just injected an error into the wrong location
2) an injected error sat around in memory and was later consumed

Case 2 will help identify places where Linux might use a "safe" copy
function that returns an error to the caller which may attempt some sort
of recovery.

Patch 1 cleans up the Intel severity calculation by using a new severity
table entry instead of some, now dubious, code to adjust the severity
for errors in kernel context.

Patch 2 adds a new severity level that triggers printing a stack trace.

I've only updated the Intel severity calculation to use this new
severity level. I'm not sure if AMD also has situations where this would
be useful. If so, then mce_severity_amd() would need to be updated too
to return different severity for IN_KERNEL and IN_KERNEL_RECOV cases.

I've tested this out on systems that do both broadcast and local machine
checks.

Tony Luck (2):
  x86/mce: Use severity table to handle uncorrected errors in kernel
  x86/mce: Dump the stack for recoverable machine checks in kernel
    context

 arch/x86/kernel/cpu/mce/internal.h |  1 +
 arch/x86/kernel/cpu/mce/core.c     | 11 +++++++++--
 arch/x86/kernel/cpu/mce/severity.c | 10 ++++++----
 3 files changed, 16 insertions(+), 6 deletions(-)

base-commit: 521a547ced6477c54b4b0cc206000406c221b4d6
-- 
2.37.3

^ permalink raw reply	[flat|nested] 9+ messages in thread