powernow-k8 and stuck change-pending bit

powernow-k8 and stuck change-pending bit

[John Belmonte]

[-- Attachment #1: Type: text/plain, Size: 949 bytes --]

I'm having a problem with powernow-k8 intermittent failures and would
appreciate some help.

system:

  * Athlon 64 2800+, stepping 10
  * Chaintech K8M800 socket 754 (no Cool'n'Quiet support)
  * linux-2.6.12-rc5 (+ powernow-k8 1.40.2 patch)


symptoms:

    After working fine for a while (several minutes to several hours),
cpufreq seems to get into a bad state where:

     * change pending bit set / stuck kernel messages appear

     * cat of /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq
hangs for a few seconds and finally returns "<unknown>".  However a cat
of /proc/cpuinfo works and seems to show the correct cpu MHz.

     * reloading powernow-k8 module fails, with "change pending bit stuck"

     * the problem can sometimes be provoked by stopping and starting
       powernowd

Attached is some kernel output with "cpufreq.debug=3".  The debug output
doesn't seem to provide much information at the point of failure.

--John

[-- Attachment #2: powernow-k8_stuck_bit.log --]
[-- Type: text/x-log, Size: 4208 bytes --]

powernow-k8: Found 1 AMD Athlon 64 / Opteron processors (version 1.40.2)
cpufreq-core: trying to register driver powernow-k8
cpufreq-core: adding CPU 0
powernow-k8:    0 : fid 0xa, vid 0x2
powernow-k8:    1 : fid 0x2, vid 0x12
powernow-k8:    2 : fid 0x3f, vid 0x1f
powernow-k8: invalid freq 7100000 kHz, ignoring
powernow-k8:    3 : fid 0x3f, vid 0x1f
powernow-k8: invalid freq 7100000 kHz, ignoring
powernow-k8:    4 : fid 0x3f, vid 0x1f
powernow-k8: invalid freq 7100000 kHz, ignoring
powernow-k8:    0 : fid 0xa (1800 MHz), vid 0x2 (1500 mV)
powernow-k8:    1 : fid 0x2 (1000 MHz), vid 0x12 (1100 mV)
powernow-k8: cpu0, init lo 0x20a, hi 0x1
powernow-k8: policy current frequency 1800000 kHz
freq-table: table entry 0: 1800000 kHz, 522 index
freq-table: table entry 1: 1000000 kHz, 4610 index
freq-table: table entry 2 is invalid, skipping
freq-table: table entry 3 is invalid, skipping
freq-table: table entry 4 is invalid, skipping
freq-table: setting show_table for cpu 0 to ffff81002b7cef40
cpu_init done, current fid 0xa, vid 0x2
cpufreq-core: setting new policy for CPU 0: 1000000 - 1800000 kHz
freq-table: request for verification of policy (1000000 - 1800000 kHz) for cpu 0
freq-table: verification lead to (1000000 - 1800000 kHz) for cpu 0
freq-table: request for verification of policy (1000000 - 1800000 kHz) for cpu 0
freq-table: verification lead to (1000000 - 1800000 kHz) for cpu 0
cpufreq-core: new min and max freqs are 1000000 - 1800000 kHz
cpufreq-core: governor switch
cpufreq-core: __cpufreq_governor for CPU 0, event 1
cpufreq-core: target for CPU 0: 1800000 kHz, relation 1
powernow-k8: targ: cpu 0, 1800000 kHz, min 1000000, max 1800000, relation 1
powernow-k8: targ: curr fid 0xa, vid 0x2
freq-table: request for target 1800000 kHz (relation: 1) for cpu 0
freq-table: target is 0 (1800000 kHz, 522)
powernow-k8: cpu 0 transition to index 0
powernow-k8: table matched fid 0xa, giving vid 0x2
powernow-k8: target matches current values (fid 0xa, vid 0x2)
cpufreq-core: governor: change or update limits
cpufreq-core: __cpufreq_governor for CPU 0, event 3
cpufreq-core: target for CPU 0: 1800000 kHz, relation 1
powernow-k8: targ: cpu 0, 1800000 kHz, min 1000000, max 1800000, relation 1
powernow-k8: targ: curr fid 0xa, vid 0x2
freq-table: request for target 1800000 kHz (relation: 1) for cpu 0
freq-table: target is 0 (1800000 kHz, 522)
powernow-k8: cpu 0 transition to index 0
powernow-k8: table matched fid 0xa, giving vid 0x2
powernow-k8: target matches current values (fid 0xa, vid 0x2)
cpufreq-core: initialization complete
cpufreq-core: driver powernow-k8 up and running
cpufreq-core: updating policy for CPU 0
cpufreq-core: setting new policy for CPU 0: 1000000 - 1800000 kHz
freq-table: request for verification of policy (1000000 - 1800000 kHz) for cpu 0
freq-table: verification lead to (1000000 - 1800000 kHz) for cpu 0
freq-table: request for verification of policy (1000000 - 1800000 kHz) for cpu 0
freq-table: verification lead to (1000000 - 1800000 kHz) for cpu 0
cpufreq-core: new min and max freqs are 1000000 - 1800000 kHz
cpufreq-core: governor: change or update limits
cpufreq-core: __cpufreq_governor for CPU 0, event 3
cpufreq-core: target for CPU 0: 1800000 kHz, relation 1
powernow-k8: targ: cpu 0, 1800000 kHz, min 1000000, max 1800000, relation 1
powernow-k8: targ: curr fid 0xa, vid 0x2
freq-table: request for target 1800000 kHz (relation: 1) for cpu 0
freq-table: target is 0 (1800000 kHz, 522)
powernow-k8: cpu 0 transition to index 0
powernow-k8: table matched fid 0xa, giving vid 0x2
powernow-k8: target matches current values (fid 0xa, vid 0x2)
.
.
.
powernow-k8: targ: cpu 0, 1800000 kHz, min 1000000, max 1800000, relation 0
powernow-k8: targ: curr fid 0x2, vid 0x12
freq-table: request for target 1800000 kHz (relation: 0) for cpu 0
freq-table: target is 0 (1800000 kHz, 522)
powernow-k8: cpu 0 transition to index 0
powernow-k8: table matched fid 0xa, giving vid 0x2
powernow-k8: cpu 0, changing to fid 0xa, vid 0x2
cpufreq-core: notification 0 of frequency transition to 1800000 kHz
cpufreq-core: scaling loops_per_jiffy to 1765376 for frequency 1800000 kHz
powernow-k8: detected change pending stuck
powernow-k8: transition frequency failed

[-- Attachment #3: Type: text/plain, Size: 147 bytes --]

_______________________________________________
Cpufreq mailing list
Cpufreq@lists.linux.org.uk
http://lists.linux.org.uk/mailman/listinfo/cpufreq

speedstep_smi module loads randomly.

[Fredrik Eriksson]

this is not a real issue since its working.. but i wanted to know why it does 
like this:

root@sajkisen:/home/sajko> modprobe speedstep_smi
FATAL: Error inserting speedstep_smi 
(/lib/modules/2.6.11-sr5/kernel/arch/i386/kernel/cpu/cpufreq/speedstep-smi.ko): 
No such device
root@sajkisen:/home/sajko> modprobe speedstep_smi
FATAL: Error inserting speedstep_smi 
(/lib/modules/2.6.11-sr5/kernel/arch/i386/kernel/cpu/cpufreq/speedstep-smi.ko): 
No such device
root@sajkisen:/home/sajko> modprobe speedstep_smi
FATAL: Error inserting speedstep_smi 
(/lib/modules/2.6.11-sr5/kernel/arch/i386/kernel/cpu/cpufreq/speedstep-smi.ko): 
No such device
root@sajkisen:/home/sajko> modprobe speedstep_smi
FATAL: Error inserting speedstep_smi 
(/lib/modules/2.6.11-sr5/kernel/arch/i386/kernel/cpu/cpufreq/speedstep-smi.ko): 
No such device
root@sajkisen:/home/sajko> modprobe speedstep_smi
root@sajkisen:/home/sajko> 


fyi i havent done anything in another terminal or anything while ive tried to 
load it... ive just spammed 4-5 times the up arrow and pressed enter :P

its wierd since it also takes from 1st attempt to 5 attempts to make it insert 
the module. just thought someone seen it before and know what to do to fix it 
(since when im doing it in modules.conf it returns the same error thus not 
loading the module when i boot my laptop :P)

Re: speedstep_smi module loads randomly.

[Dominik Brodowski]

Hi,

On Sun, Jun 05, 2005 at 12:50:32AM +0200, Fredrik Eriksson wrote:
> this is not a real issue since its working.. but i wanted to know why it does 
> like this:

Could you enable CPUFREQ_DEBUG in the kernel config, please, and add
"cpufreq.debug=2" to the kernel command line, and send me the dmesg output
of (failed) attempts to load speedstep_smi? I'm curious as to _where_ it
fails to load. And, of course, I'm not able to reproduce it locally...

Thanks,
	Dominik

Re: powernow-k8 and stuck change-pending bit

[John Belmonte]

John Belmonte wrote:
> Devriendt, Paul wrote:
> 
>>>symptoms:
>>>
>>>   After working fine for a while (several minutes to several hours),
>>>cpufreq seems to get into a bad state where:
>>>
>>>    * change pending bit set / stuck kernel messages appear
>>>
>>>    * reloading powernow-k8 module fails, with "change 
>>>pending bit stuck"
>>
> [...]
> 
>>The most likely scenario here is the VRM was unable to change
>>voltage, and that means flaky hardware as it has previously been
>>able to successfully do so. The reason I pick on the VRM and not
>>some sort of issue with the HyperTransport fabric is that it is
>>unlikely anything would still be alive if there was some sort of 
>>problem coming back from the LDT_STOP.
>>
>>Assuming the VRM continues to supply the current voltage within 
>>spec, the system will stay up indefinitely, at the current
>>voltage and frequency.
>>
>>The best path to follow would be to investigate warrantee replacement
>>of the board.
> 
> 
> Thanks for the info Paul.  I've had the motherboard replaced with an
> identical model.  This new board seems to have the same problem, except
> that when a transition finally fails, Linux hangs bad enough to require
> a hard reset of the system.
> 
> With two boards having this problem, it suggests either a bad
> motherboard design or some bad interaction between Linux and the
> motherboard.  (It seems strange that the manufacturer wouldn't have seen
> this problem under Windows.)  Much less likely is that I have a bad CPU.

An update to this story:  I've replaced my motherboard with a different
brand and model (DFI K8M800-MLVF) and am encountering the same power
throttling issue.  It does share the same Via chipset as my previous
motherboard.  I guess the next stop on this arduous trek is to try a
different chipset.

--John

Re: powernow-k8 and stuck change-pending bit

[John Belmonte]

Devriendt, Paul wrote:
>>symptoms:
>>
>>    After working fine for a while (several minutes to several hours),
>>cpufreq seems to get into a bad state where:
>>
>>     * change pending bit set / stuck kernel messages appear
>>
>>     * reloading powernow-k8 module fails, with "change 
>>pending bit stuck"
> 
[...]
> The most likely scenario here is the VRM was unable to change
> voltage, and that means flaky hardware as it has previously been
> able to successfully do so. The reason I pick on the VRM and not
> some sort of issue with the HyperTransport fabric is that it is
> unlikely anything would still be alive if there was some sort of 
> problem coming back from the LDT_STOP.
> 
> Assuming the VRM continues to supply the current voltage within 
> spec, the system will stay up indefinitely, at the current
> voltage and frequency.
> 
> The best path to follow would be to investigate warrantee replacement
> of the board.

Thanks for the info Paul.  I've had the motherboard replaced with an
identical model.  This new board seems to have the same problem, except
that when a transition finally fails, Linux hangs bad enough to require
a hard reset of the system.

With two boards having this problem, it suggests either a bad
motherboard design or some bad interaction between Linux and the
motherboard.  (It seems strange that the manufacturer wouldn't have seen
this problem under Windows.)  Much less likely is that I have a bad CPU.

--John

RE: powernow-k8 and stuck change-pending bit

[Devriendt, Paul]

> symptoms:
> 
>     After working fine for a while (several minutes to several hours),
> cpufreq seems to get into a bad state where:
> 
>      * change pending bit set / stuck kernel messages appear
> 
>      * reloading powernow-k8 module fails, with "change 
> pending bit stuck"

I have seen a very small number of reports of this over the past
couple of years. Perhaps 3 or 4. I can tell you what is happening, but
it may not be much help.

The processor has to communicate externally during a frequency or
a voltage change. On a frequency change, all devices on the 
HyperTransport fabric will see a LDT_STOP. On a voltage change, the 
new vid (voltage code) has to be driven out to the external VRM 
(voltage regulator module) which actually supplies the correct 
voltage. Obviously, this external activity takes a little while.

The software interface to this is via a register, and there is a bit
to warn the software that a change is in progress. This is the pending 
bit. A change should never fail to complete. It can fail if the change
request was invalid, but it should not fail to complete.

The driver is written to timeout (i.e. not hang the system) if the
pending bit should ever fail to clear.

It is also written to never attempt to make a change if the pending
bit is set.

This is what you are seeing. A transition was attempted, it never
completed, and the driver is not happy about it.

The most likely scenario here is the VRM was unable to change
voltage, and that means flaky hardware as it has previously been
able to successfully do so. The reason I pick on the VRM and not
some sort of issue with the HyperTransport fabric is that it is
unlikely anything would still be alive if there was some sort of 
problem coming back from the LDT_STOP.

Assuming the VRM continues to supply the current voltage within 
spec, the system will stay up indefinitely, at the current
voltage and frequency.

The best path to follow would be to investigate warrantee replacement
of the board.

Paul.