linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Luck, Tony" <tony.luck@intel.com>
To: "paulmck@kernel.org" <paulmck@kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"x86@kernel.org" <x86@kernel.org>,
	"linux-edac@vger.kernel.org" <linux-edac@vger.kernel.org>
Cc: "bp@alien8.de" <bp@alien8.de>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"hpa@zytor.com" <hpa@zytor.com>,
	"kernel-team@fb.com" <kernel-team@fb.com>
Subject: RE: [PATCH RFC x86/mce] Make mce_timed_out() identify holdout CPUs
Date: Wed, 6 Jan 2021 18:39:30 +0000	[thread overview]
Message-ID: <3513b04e2bb543d2871ca8c152dcf5ae@intel.com> (raw)
In-Reply-To: <20210106174102.GA23874@paulmck-ThinkPad-P72>

> The "Timeout: Not all CPUs entered broadcast exception handler" message
> will appear from time to time given enough systems, but this message does
> not identify which CPUs failed to enter the broadcast exception handler.
> This information would be valuable if available, for example, in order to
> correlated with other hardware-oriented error messages.  This commit
> therefore maintains a cpumask_t of CPUs that have entered this handler,
> and prints out which ones failed to enter in the event of a timeout.

I tried doing this a while back, but found that in my test case where I forced
an error that would cause both threads from one core to be "missing", the
output was highly unpredictable. Some random number of extra CPUs were
reported as missing. After I added some extra breadcrumbs it became clear
that pretty much all the CPUs (except the missing pair) entered do_machine_check(),
but some got hung up at various points beyond the entry point. My only theory
was that they were trying to snoop caches from the dead core (or access some
other resource held by the dead core) and so they hung too.

Your code is much neater than mine ... and perhaps works in other cases, but
maybe the message needs to allow for the fact that some of the cores that
are reported missing may just be collateral damage from the initial problem.

If I get time in the next day or two, I'll run my old test against your code to
see what happens.

-Tony

  parent reply	other threads:[~2021-01-06 18:40 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-06 17:41 [PATCH RFC x86/mce] Make mce_timed_out() identify holdout CPUs Paul E. McKenney
2021-01-06 18:32 ` Borislav Petkov
2021-01-06 19:13   ` Paul E. McKenney
2021-01-07  7:07     ` Borislav Petkov
2021-01-07 17:08       ` Paul E. McKenney
2021-01-08 12:31         ` Borislav Petkov
2021-01-08 14:55           ` Paul E. McKenney
2021-01-08 16:57             ` Borislav Petkov
2021-01-06 18:39 ` Luck, Tony [this message]
2021-01-06 19:17   ` Paul E. McKenney
2021-01-06 22:49     ` Luck, Tony
2021-01-06 23:23       ` Paul E. McKenney
2021-01-07  0:26         ` Luck, Tony
2021-01-07  0:41           ` Paul E. McKenney
2021-01-08 17:09 ` [tip: ras/core] x86/mce: " tip-bot2 for Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3513b04e2bb543d2871ca8c152dcf5ae@intel.com \
    --to=tony.luck@intel.com \
    --cc=bp@alien8.de \
    --cc=hpa@zytor.com \
    --cc=kernel-team@fb.com \
    --cc=linux-edac@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=paulmck@kernel.org \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).