[RFC] Fix NMI watchdog documentation

[RFC] Fix NMI watchdog documentation

[Carl-Daniel Hailfinger]

Hello,

Documentation/nmi_watchdog.txt does not say which CONFIG_XYZ option has
to be enabled to use the NMI watchdog, but it mentions that IO-APIC is
somewhat related.

Documentation/Configure.help is equally unclear. The NMI watchdog is
mentioned as related to CONFIG_SMP, and the help text for
CONFIG_X86_UP_APIC says: "The local APIC supports [..] the NMI watchdog"
That does not necessarily mean that NMI is compiled in once local APIC
support is selected.

Can someone please shed some light on this issue? I'm willing to create
a patch to fix the docs once I know if the NMI watchdog is compiled in
alsways or on what it depends.


Regards,
Carl-Daniel
-- 
http://www.hailfinger.org/

Re: [RFC] Fix NMI watchdog documentation

[Zwane Mwaikambo]

On Sat, 24 May 2003, Carl-Daniel Hailfinger wrote:

> Documentation/nmi_watchdog.txt does not say which CONFIG_XYZ option has
> to be enabled to use the NMI watchdog, but it mentions that IO-APIC is
> somewhat related.
> 
> Documentation/Configure.help is equally unclear. The NMI watchdog is
> mentioned as related to CONFIG_SMP, and the help text for
> CONFIG_X86_UP_APIC says: "The local APIC supports [..] the NMI watchdog"
> That does not necessarily mean that NMI is compiled in once local APIC
> support is selected.
> 
> Can someone please shed some light on this issue? I'm willing to create
> a patch to fix the docs once I know if the NMI watchdog is compiled in
> alsways or on what it depends.

For nmi_watchdog=1 the NMI watchdog uses the programmable interval timer
to trigger interrupt events via the IOAPIC. with nmi_watchdog=2 we use the 
performance counters on the processor to trigger these events. IOAPICs 
are generally found on SMP motherboards (but there are UP boards with 
them, it's chipset dependent). Generally i686+ (save some Athlons and a 
few other processors) will work with nmi_watchdog=2.

	Zwane
-- 
function.linuxpower.ca

Re: [RFC] Fix NMI watchdog documentation

[Zwane Mwaikambo]

On Sat, 24 May 2003, Zwane Mwaikambo wrote:

> For nmi_watchdog=1 the NMI watchdog uses the programmable interval timer
> to trigger interrupt events via the IOAPIC. with nmi_watchdog=2 we use the 
> performance counters on the processor to trigger these events. IOAPICs 
> are generally found on SMP motherboards (but there are UP boards with 
> them, it's chipset dependent). Generally i686+ (save some Athlons and a 
> few other processors) will work with nmi_watchdog=2.

Forgot this bit;

w/ CONFIG_X86_LOCAL_APIC=y you only have nmi_watchdog=2

w/ CONFIG_X86_IO_APIC=y you have nmi_watchdog=1 and 2

w/ CONFIG_SMP you have nmi_watchdog=1 and 2 as it depends on 
CONFIG_X86_IO_APIC

	Zwane

-- 
function.linuxpower.ca

Re: [RFC] Fix NMI watchdog documentation

[Marc-Christian Petersen]

On Saturday 24 May 2003 21:07, Carl-Daniel Hailfinger wrote:

Hi Carl-Daniel,

> Documentation/nmi_watchdog.txt does not say which CONFIG_XYZ option has
> to be enabled to use the NMI watchdog, but it mentions that IO-APIC is
> somewhat related.
>
> Documentation/Configure.help is equally unclear. The NMI watchdog is
> mentioned as related to CONFIG_SMP, and the help text for
> CONFIG_X86_UP_APIC says: "The local APIC supports [..] the NMI watchdog"
> That does not necessarily mean that NMI is compiled in once local APIC
> support is selected.
>
> Can someone please shed some light on this issue? I'm willing to create
> a patch to fix the docs once I know if the NMI watchdog is compiled in
> alsways or on what it depends.
the nmi_watchdog is always compiled in if you select "CONFIG_X86_LOCAL_APIC"

see "arch/i386/kernel/Makefile"

ciao, Marc

Re: [RFC] Fix NMI watchdog documentation

[Carl-Daniel Hailfinger]

Zwane Mwaikambo wrote:
> On Sat, 24 May 2003, Zwane Mwaikambo wrote:
> 
> 
>>For nmi_watchdog=1 the NMI watchdog uses the programmable interval timer
>>to trigger interrupt events via the IOAPIC. with nmi_watchdog=2 we use the 
>>performance counters on the processor to trigger these events. IOAPICs 
>>are generally found on SMP motherboards (but there are UP boards with 
>>them, it's chipset dependent). Generally i686+ (save some Athlons and a 
>>few other processors) will work with nmi_watchdog=2.
> 
> 
> Forgot this bit;
> 
> w/ CONFIG_X86_LOCAL_APIC=y you only have nmi_watchdog=2
> 
> w/ CONFIG_X86_IO_APIC=y you have nmi_watchdog=1 and 2
> 
> w/ CONFIG_SMP you have nmi_watchdog=1 and 2 as it depends on 
> CONFIG_X86_IO_APIC

Marc-Christian Petersen wrote:
> the nmi_watchdog is always compiled in if you select "CONFIG_X86_LOCAL_APIC"
> 
> see "arch/i386/kernel/Makefile"

Thanks to all who responded. I will prepare a patch for
Documentation/nmi_watchdog.txt to clarify it and send the patch to Linus
and Marcelo.


Regards,
Carl-Daniel
-- 
http://www.hailfinger.org/

Re: [RFC] Fix NMI watchdog documentation

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 664 bytes --]

On Sat, 24 May 2003 15:11:06 EDT, Zwane Mwaikambo said:

> Forgot this bit;
> 
> w/ CONFIG_X86_LOCAL_APIC=y you only have nmi_watchdog=2
> 
> w/ CONFIG_X86_IO_APIC=y you have nmi_watchdog=1 and 2
> 
> w/ CONFIG_SMP you have nmi_watchdog=1 and 2 as it depends on 
> CONFIG_X86_IO_APIC

% grep APIC /usr/src/linux-2.5.69/.config
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_UP_APIC=y
CONFIG_X86_UP_IOAPIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y

but 'dmesg' on my Dell Latitude C840 laptop tells me:

Dell Latitude with broken BIOS detected. Refusing to enable the local APIC.

Is this nmi_watchdog="forget about it dave" time, or is there some way to
get this to work?

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [RFC] Fix NMI watchdog documentation

[Zwane Mwaikambo]

On Sat, 24 May 2003 Valdis.Kletnieks@vt.edu wrote:

> but 'dmesg' on my Dell Latitude C840 laptop tells me:
> 
> Dell Latitude with broken BIOS detected. Refusing to enable the local APIC.
> 
> Is this nmi_watchdog="forget about it dave" time, or is there some way to
> get this to work?

It's known broken with that configuration and hence blacklisted.

	Zwane
-- 
function.linuxpower.ca

Re: [RFC] Fix NMI watchdog documentation

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 422 bytes --]

On Sat, 24 May 2003 23:36:26 EDT, Zwane Mwaikambo said:

> > Dell Latitude with broken BIOS detected. Refusing to enable the local APIC.

> It's known broken with that configuration and hence blacklisted.

Yes, I know it's blacklisted.  The question I intended to ask was "Is the
entire concept of IOAPIC irretrievably scrozzled on this machine, or is there
sufficient minimum functionality to get nmi_watchdog working?"


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [RFC] Fix NMI watchdog documentation

[Zwane Mwaikambo]

On Sun, 25 May 2003 Valdis.Kletnieks@vt.edu wrote:

> On Sat, 24 May 2003 23:36:26 EDT, Zwane Mwaikambo said:
> 
> > > Dell Latitude with broken BIOS detected. Refusing to enable the local APIC.
> 
> > It's known broken with that configuration and hence blacklisted.
> 
> Yes, I know it's blacklisted.  The question I intended to ask was "Is the
> entire concept of IOAPIC irretrievably scrozzled on this machine, or is there
> sufficient minimum functionality to get nmi_watchdog working?"

You don't have an IOAPIC at all, but the Local APIC has been known to 
cause problems. So forget about nmi_watchdog.

	Zwane
-- 
function.linuxpower.ca

Re: [RFC] Fix NMI watchdog documentation

[Disconnect]

On Sun, 2003-05-25 at 01:03, Zwane Mwaikambo wrote: 
> On Sun, 25 May 2003 Valdis.Kletnieks@vt.edu wrote:
> 
> > On Sat, 24 May 2003 23:36:26 EDT, Zwane Mwaikambo said:
> > 
> > > > Dell Latitude with broken BIOS detected. Refusing to enable the local APIC.
> > 
> > > It's known broken with that configuration and hence blacklisted.
> > 
> > Yes, I know it's blacklisted.  The question I intended to ask was "Is the
> > entire concept of IOAPIC irretrievably scrozzled on this machine, or is there
> > sufficient minimum functionality to get nmi_watchdog working?"
> 
> You don't have an IOAPIC at all, but the Local APIC has been known to 
> cause problems. So forget about nmi_watchdog.
> 
> 	Zwane

I was reading that code the other day (just out of curiosity, believe it
or not) and I'm wondering how recently that has been tested - most of
the blacklist/oddness workarounds listed in dmi_scan.c are
model-specific, but the APIC entry is any Dell Inspiron or Latitude.  

I'm going to remove the test tomorrow sometime and see what happens -
lots has changed since the Inspiron 8000, including a migration to
p4-mobile, so its worth seeing if the newer Dells are fixed.  If so,
I'll submit a patch to make that more model-specific (probably I'll just
add a whitelist function - no_local_apic_kills_bios or some such; seems
better than listing every dell inspiron individually..)

I'm encouraged by the complete lack of APM or any of the 'enter bios
while running' options present on the older laptops; according to the
comments, even if the APIC kills the bios on entry/exit, it won't matter
since you can't trigger it to begin with..

-- 

Disconnect <lkml@sigkill.net>

Re: [RFC] Fix NMI watchdog documentation

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 2542 bytes --]

On Mon, 26 May 2003 11:21:38 +0200, mikpe@csd.uu.se said:

> --- linux-2.5.69/arch/i386/kernel/apic.c.~1~	2003-04-20 13:08:15.000000000 +
0200
> +++ linux-2.5.69/arch/i386/kernel/apic.c	2003-05-26 11:11:19.000000000 +
0200
> @@ -617,7 +617,7 @@
>  		goto no_apic;
>  	case X86_VENDOR_INTEL:
>  		if (boot_cpu_data.x86 == 6 ||
> -		    (boot_cpu_data.x86 == 15 && cpu_has_apic) ||
> +		    (boot_cpu_data.x86 == 15) ||

OK, there's good news and bad news.  After doing that and lopping out the
dmi_scan.c blacklist entry, we get these msgs on boot (diffing against
a dmesg from before the above:

1c1
< Linux version 2.5.69 (valdis@turing-police.cc.vt.edu) (gcc version 3.2.3 20030422 (Red Hat Linux 3.2.3-4)) #4 Sun May 25 16:22:44 EDT 2003
---
> Linux version 2.5.69 (valdis@turing-police.cc.vt.edu) (gcc version 3.2.3 20030422 (Red Hat Linux 3.2.3-4)) #5 Mon May 26 20:03:44 EDT 2003
24c24,25
< No local APIC present or hardware disabled
---
> Local APIC disabled by BIOS -- reenabling.
> Found and enabled local APIC!
27c28
< Detected 1595.314 MHz processor.
---
> Detected 1595.413 MHz processor.
30c31
< Memory: 254416k/262024k available (2520k kernel code, 6888k reserved, 969k data, 144k init, 0k highmem)
---
> Memory: 254416k/262024k available (2520k kernel code, 6880k reserved, 969k data, 144k init, 0k highmem)
40c41
< CPU:     After generic, caps: 3febf9ff 00000000 00000000 00000080
---
> CPU:     After generic, caps: 3febfbff 00000000 00000000 00000080
48a50,56
> enabled ExtINT on CPU#0
> ESR value before enabling vector: 00000000
> ESR value after enabling vector: 00000000
> Using local APIC timer interrupts.
> calibrating APIC timer ...
> ..... CPU clock speed is 1595.0017 MHz.
> ..... host bus clock speed is 99.0688 MHz.

So yes Virginia, there is a local APIC on the C840.

Now the bad news - out of 7 or 8 tries, the above was the only boot that lived
long enough for me to get a single-user prompt and do a dmesg into a file.
That try hung about 10 seconds later.  It never hung at the same place twice,
and always hung hard enough to require the "power button for 5 seconds to
poweroff" sledgehammer.  The common factor seemed to be hangs while talking
to the IDE drive - while mounting /, while checking the partition table,
etc.

One question - in apic.c, I see where clear_local_APIC() is called in
init_bsp_APIC(), connect_bsp_APIC(), and disable_local_APIC().  I however
don't see where/how it's called in the init_local_APIC() codepath, nor can
I convince myself that it obviously *shouldn't* be called...


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [RFC] Fix NMI watchdog documentation

[mikpe]

On 26 May 2003 01:31:41 -0400, Disconnect <lkml@sigkill.net> wrote:
>> OK, I put together a kernel that had the Latitude blacklist commented out,
>> and it comes up with:
>> 
>> No local APIC present or hardware disabled
>> Initializing CPU#0
>> 
>> So add the Latitude C840 to the "known b0rken" list.
>
>Ditto the Inspiron 8500 - no apic at all (which is different from
>known-broken, since nothing bad happened.)  
...
>Perhaps just a comment above those entries:
>/* Latitude C840 and Inspiron 8500 have no APIC support in hardware */

If these machines are P4-based, then I bet they do have local APICs.
However, if the BIOS boots the kernel with the local APIC disabled
on a P4, we (apic.c) don't try to enable it. The logic behind that
is that "modern" BIOSen _should_ boot with it enabled, unless they're
horribly broken.

So apply the patch below and try the "can we get the machine to hang"
checklist again.

/Mikael

--- linux-2.5.69/arch/i386/kernel/apic.c.~1~	2003-04-20 13:08:15.000000000 +0200
+++ linux-2.5.69/arch/i386/kernel/apic.c	2003-05-26 11:11:19.000000000 +0200
@@ -617,7 +617,7 @@
 		goto no_apic;
 	case X86_VENDOR_INTEL:
 		if (boot_cpu_data.x86 == 6 ||
-		    (boot_cpu_data.x86 == 15 && cpu_has_apic) ||
+		    (boot_cpu_data.x86 == 15) ||
 		    (boot_cpu_data.x86 == 5 && cpu_has_apic))
 			break;
 		goto no_apic;

Re: [RFC] Fix NMI watchdog documentation

[Disconnect]

On Sun, 2003-05-25 at 22:36, Valdis.Kletnieks@vt.edu wrote:
> On Sun, 25 May 2003 12:50:28 +0200, mikpe@csd.uu.se said:
> 
> > The blacklist rule is a catch-all since we don't have detailed DMI
> > data on all Inspiron/Latitude models, and at the time, _all_ of them
> > were broken. Looking through my records, Inspiron 8000 and 8100, and
> > Latitude C600, C610, C640, C800, and C810 are known to be broken. Note
> > that this includes at least one P4-based machine (C640), so it's not
> > restricted to "old" mobile P3s.
> 
> OK, I put together a kernel that had the Latitude blacklist commented out,
> and it comes up with:
> 
> No local APIC present or hardware disabled
> Initializing CPU#0
> 
> So add the Latitude C840 to the "known b0rken" list.

Ditto the Inspiron 8500 - no apic at all (which is different from
known-broken, since nothing bad happened.)  

For cleanliness sake it would be nice to have specific model info in the
blacklist ("your hardware doesn't have a frobber" is nicer than "the
frobber on your hardware causes your pets to catch fire"), but since the
hardware doesn't exist anyway the end result is the same. (And the code
is a lot more readable.)

Perhaps just a comment above those entries:
/* Latitude C840 and Inspiron 8500 have no APIC support in hardware */

Eventually it might even turn into a proper whitelist/blacklist
collection.

-- 
Disconnect <lkml@sigkill.net>

Re: [RFC] Fix NMI watchdog documentation

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

On Sun, 25 May 2003 12:50:28 +0200, mikpe@csd.uu.se said:

> The blacklist rule is a catch-all since we don't have detailed DMI
> data on all Inspiron/Latitude models, and at the time, _all_ of them
> were broken. Looking through my records, Inspiron 8000 and 8100, and
> Latitude C600, C610, C640, C800, and C810 are known to be broken. Note
> that this includes at least one P4-based machine (C640), so it's not
> restricted to "old" mobile P3s.

OK, I put together a kernel that had the Latitude blacklist commented out,
and it comes up with:

No local APIC present or hardware disabled
Initializing CPU#0

So add the Latitude C840 to the "known b0rken" list.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [RFC] Fix NMI watchdog documentation

[mikpe]

On 25 May 2003 01:52:16 -0400, Disconnect <lkml@sigkill.net> wrote:
>I was reading that code the other day (just out of curiosity, believe it
>or not) and I'm wondering how recently that has been tested - most of
>the blacklist/oddness workarounds listed in dmi_scan.c are
>model-specific, but the APIC entry is any Dell Inspiron or Latitude.  
>
>I'm going to remove the test tomorrow sometime and see what happens -
>lots has changed since the Inspiron 8000, including a migration to
>p4-mobile, so its worth seeing if the newer Dells are fixed.  If so,
>I'll submit a patch to make that more model-specific (probably I'll just
>add a whitelist function - no_local_apic_kills_bios or some such; seems
>better than listing every dell inspiron individually..)
>
>I'm encouraged by the complete lack of APM or any of the 'enter bios
>while running' options present on the older laptops; according to the
>comments, even if the APIC kills the bios on entry/exit, it won't matter
>since you can't trigger it to begin with..

You'll also enter BIOS (System Management Mode actually) when inserting
or removing the power cord, or when thermal events occur, and possibly
also when suspending.

The blacklist rule is a catch-all since we don't have detailed DMI
data on all Inspiron/Latitude models, and at the time, _all_ of them
were broken. Looking through my records, Inspiron 8000 and 8100, and
Latitude C600, C610, C640, C800, and C810 are known to be broken. Note
that this includes at least one P4-based machine (C640), so it's not
restricted to "old" mobile P3s.

The only non-broken Dell laptop that I know of is the Latitude D600
(Centrino), and that was reported only two months ago.

I'm all for adding an explicit white-list to dmi_scan.c. For starters,
the D600 should be included. Partial dmidecode data for it is included
below.

(There is also a generic problem in that apm can enter the BIOS
asynchronously if you configure DISPLAY_BLANK or CPU_IDLE, or build
apm as a module and unload it. Both DISPLAY_BLANK and unloading the
module are known lock-up triggers if the local APIC is enabled.)

/Mikael

Handle 0x0100
	DMI type 1, 25 bytes.
	System Information
		Manufacturer: Dell Computer Corporation
		Product Name: Latitude D600                   
Handle 0x0200
	DMI type 2, 9 bytes.
	Base Board Information
		Manufacturer: Dell Computer Corporation
		Product Name: 03U652

Re: [RFC] Fix NMI watchdog documentation

[mikpe]

On Sat, 24 May 2003 23:29:15 -0400, Valdis.Kletnieks@vt.edu wrote:
>CONFIG_X86_UP_APIC=y
>CONFIG_X86_UP_IOAPIC=y
>CONFIG_X86_LOCAL_APIC=y
>CONFIG_X86_IO_APIC=y
>
>but 'dmesg' on my Dell Latitude C840 laptop tells me:
>
>Dell Latitude with broken BIOS detected. Refusing to enable the local APIC.
>
>Is this nmi_watchdog="forget about it dave" time, or is there some way to
>get this to work?

The blacklist rule treats all Latitudes the same, since most of
them are broken. Your C840 may or may not actually work. Simply
remove the blacklist entry in dmi_scan.c and run some tests:
- does it hang at boot?
- does the kernel fail to enable the local APIC? (it's HW absent
  in many mobile CPUs)
- does it hang when the power cord is attached or detached?
- does it hang when the BIOS setup hotkey is pressed?
  (Fn-F1 on my ancient Latitude, may be different now)
- does it hang under long periods of heavy load?
- does it hang when idle for a long time?
- does it hang at suspend or resume?

If it survives these tests, with nmi_watchdog=0 and =2, then
please tell us about it so we can white-list this particular model.

/Mikael