Re: Uhhuh. NMI received for unknown reason 2c on CPU 0.

From: Borislav Petkov <bp@alien8.de>
To: "Rafael J. Wysocki" <rjw@sisk.pl>,
	Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
	Jiri Slaby <jslaby@suse.cz>, Bjorn Helgaas <bhelgaas@google.com>,
	Konstantin Khlebnikov <khlebnikov@openvz.org>,
	x86@kernel.org, lkml <linux-kernel@vger.kernel.org>,
	e1000-devel@lists.sourceforge.net,
	Bruce Allan <bruce.w.allan@intel.com>
Subject: Re: Uhhuh. NMI received for unknown reason 2c on CPU 0.
Date: Fri, 8 Mar 2013 17:47:49 +0100	[thread overview]
Message-ID: <20130308164749.GA14495@pd.tnic> (raw)
In-Reply-To: <20130306001932.GB30189@pd.tnic>

On Wed, Mar 06, 2013 at 01:19:32AM +0100, Borislav Petkov wrote:
> On Wed, Mar 06, 2013 at 01:13:23AM +0100, Rafael J. Wysocki wrote:
> > I suspected that during resume from hibernation the boot kernel (the
> > one that loaded the image) did something to hardware and the restored
> > kernel didn't handle that change properly. It is hard do say what
> > piece of hardware that was, however (it might or might not be the NIC,
> > it may be pure coincidence that the NMI messages appear in the log at
> > this point).
> 
> Agreed with the second part. About the first part, who communicates what
> to whom, come to think of it, it might not be related to any devices at
> all.
> 
> Here's why I think so:
> 
> So one of the things I did to trigger this is boot the machine, run
> powertop and set all the knobs in the "Tunables" tab to "Good". One of
> the tunables is turn-off-nmi-watchdog something which turns off the
> watchdog which is using the perf infrastructure which generates NMIs
> when the counter overflows.
> 
> Now, imagine I do that in the "normal" kernel, then suspend,
> ...<something happens or does not happen>, then resume back into the
> normal kernel and it somehow "forgets" the fact that we disabled the NMI
> watchdog before the suspend cycle. And boom, it gets a single spurious
> NMI.
> 
> Does it make sense? I dunno - I'm just connecting the dots here between
> the observation points which are most likely.
> 
> Anyway, it's getting late, good night. :)

Exactly as I thought: so I'm running the machine with NMI watchdog
enabled, i.e. powertop says:

PowerTOP v2.0     Overview   Idle stats   Frequency stats   Device stats	Tunables

>> Bad           NMI watchdog should be turned off
   Good          VM writeback timeout
....

and no more spurious NMIs.

I'd say the plot thickens: disabling NMIs and suspending to disk right
afterwards doesn't seem to really disable the watchdog. Or the disable
gets delayed leading to one last spurious NMI when resuming... I
probably need to go stare at the code though...

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--