CpuFreq Laptop Scaling broken?

CpuFreq Laptop Scaling broken?

[Harry van Haaren]

Hi -rt users, and CC's,

I'm attempting to squeeze the lowest audio-latency out of a laptop as possible,
and in doing so would like to set the CPU to performance.

CPU in laptop: Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
ondemand performance

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver
acpi-cpufreq

# Attempt to set performance governer
$ echo "performance" >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
bash: echo: write error: Invalid argument

# Attempt to set min freq of 2GHz (max) of this core2duo
$ echo 2000000 > /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq
bash: /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq: Permission denied

As far as I can tell, it is currently impossible on the -rt kernel to
change CPU governer?
This is necessary for reliable low-latency audio for laptop musicians.

What is necessary to fix cpufreq on -rt? Cheers, -Harry

PS: I've CC'd maintainers of cpufreq, I hope that's OK, otherwise
please inform me of
normal practices on linux-rt-users ML

Re: CpuFreq Laptop Scaling broken?

[Viresh Kumar]

On 25 August 2014 17:58, Harry van Haaren <harryhaaren@gmail.com> wrote:
> I'm attempting to squeeze the lowest audio-latency out of a laptop as possible,
> and in doing so would like to set the CPU to performance.
>
> CPU in laptop: Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz
>
> $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
> ondemand performance
>
> $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver
> acpi-cpufreq
>
> # Attempt to set performance governer
> $ echo "performance" >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
> bash: echo: write error: Invalid argument
>
> # Attempt to set min freq of 2GHz (max) of this core2duo
> $ echo 2000000 > /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq
> bash: /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq: Permission denied

You tried both these as root? Even 'sudo' may not work..

I haven't seen the rt code since sometime, what tag should I look at ?

I don't think there should be any such issues there..

> As far as I can tell, it is currently impossible on the -rt kernel to
> change CPU governer?
> This is necessary for reliable low-latency audio for laptop musicians.
>
> What is necessary to fix cpufreq on -rt? Cheers, -Harry
>
> PS: I've CC'd maintainers of cpufreq, I hope that's OK, otherwise
> please inform me of
> normal practices on linux-rt-users ML

That's fine I suppose.

Re: CpuFreq Laptop Scaling broken?

[Viresh Kumar]

Fixing Rafael's id..

On 25 August 2014 19:25, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> On 25 August 2014 17:58, Harry van Haaren <harryhaaren@gmail.com> wrote:
>> I'm attempting to squeeze the lowest audio-latency out of a laptop as possible,
>> and in doing so would like to set the CPU to performance.
>>
>> CPU in laptop: Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz
>>
>> $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
>> ondemand performance
>>
>> $ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver
>> acpi-cpufreq
>>
>> # Attempt to set performance governer
>> $ echo "performance" >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
>> bash: echo: write error: Invalid argument
>>
>> # Attempt to set min freq of 2GHz (max) of this core2duo
>> $ echo 2000000 > /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq
>> bash: /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq: Permission denied
>
> You tried both these as root? Even 'sudo' may not work..
>
> I haven't seen the rt code since sometime, what tag should I look at ?
>
> I don't think there should be any such issues there..
>
>> As far as I can tell, it is currently impossible on the -rt kernel to
>> change CPU governer?
>> This is necessary for reliable low-latency audio for laptop musicians.
>>
>> What is necessary to fix cpufreq on -rt? Cheers, -Harry
>>
>> PS: I've CC'd maintainers of cpufreq, I hope that's OK, otherwise
>> please inform me of
>> normal practices on linux-rt-users ML
>
> That's fine I suppose.

Re: CpuFreq Laptop Scaling broken?

[Joakim Hernberg]

On Mon, 25 Aug 2014 19:26:03 +0530
Viresh Kumar <viresh.kumar@linaro.org> wrote:

> > On 25 August 2014 17:58, Harry van Haaren <harryhaaren@gmail.com>
> > wrote:
> > You tried both these as root? Even 'sudo' may not work..
> >
> > I haven't seen the rt code since sometime, what tag should I look
> > at ?

Dont' know how this is supposed to work on a system using the pstate
driver, but i have a similar issue:

With the realtime patch, but built with CONFIG_PREEMPT__LL=y
# uname -r
3.14.12-rt9
# uname -v
#1 SMP PREEMPT Mon Aug 25 17:27:37 CEST 2014

# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver
intel_pstate
# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
performance powersave
# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
powersave
# echo "performance" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
performance

But using i7z, I see no change in behaviour on this cpu, while using the
interface in /sys/devices/system/cpu/intel_pstate does seem to work
rather well.

With the realtime patch enabled CONFIG_PREEMPT_RT_FULL=y
# uname -r
3.14.12-rt9-1-rt
# uname -v
#1 SMP PREEMPT RT Sat Aug 23 04:41:50 CEST 2014

# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver
intel_pstate
#  cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
performance powersave
# echo "performance" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
bash: echo: write error: Invalid argument
# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
powersave

I don't if this is normal for the pstate driver, but if one can't
change governor, i suppose it shouldn't show one either?

--

   Joakim

Re: CpuFreq Laptop Scaling broken?

[Harry van Haaren]

On Mon, Aug 25, 2014 at 2:55 PM, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> You tried both these as root? Even 'sudo' may not work..
Yes, the original output was all as root (not sudo, genuine root).

> I haven't seen the rt code since sometime, what tag should I look at ?
I'm running 3.14.12-rt9-1-rt  (as packaged for Arch here:
https://aur.archlinux.org/packages/linux-rt )

Cheers, -Harry

Re: CpuFreq Laptop Scaling broken?

[Fernando Lopez-Lezcano]

On 08/25/2014 07:02 AM, Harry van Haaren wrote:
> On Mon, Aug 25, 2014 at 2:55 PM, Viresh Kumar <viresh.kumar@linaro.org> wrote:
>> You tried both these as root? Even 'sudo' may not work..
> Yes, the original output was all as root (not sudo, genuine root).
>
>> I haven't seen the rt code since sometime, what tag should I look at ?
> I'm running 3.14.12-rt9-1-rt  (as packaged for Arch here:
> https://aur.archlinux.org/packages/linux-rt )

Hi Harry,

Check to see if you see something in /sys/devices/system/cpu/intel_pstate/

If so then the intel pstate driver is what is controlling cpu speed for 
you (that changed since 3.9 or so). You can set min_perf_pct to "100" to 
set your minimum cpu speed to the maximum possible. Even then you will 
see some variation but in my usage case jackd is again usable after this 
change[*].

See:
https://wiki.archlinux.org/index.php/CPU_frequency_scaling

-- Fernando

[*] I think the intel pstate drive is not yet aware of what the 
scheduler does with tasks (or visceversa). In the case of jackd I think 
that the scheduler can move a task to a processor that is throttled down 
and the change in speed after detecting high load does not happen fast 
enough. Jackd runs out of cpu cycles and you get an xrun. By the next 
jackd cycle the cpu is running up to speed and everything is fine but 
the task might be moved again to a different core. Just speculation on 
my part.

Re: CpuFreq Laptop Scaling broken?

[Harry van Haaren]

On Mon, Aug 25, 2014 at 6:18 PM, Fernando Lopez-Lezcano
<nando@ccrma.stanford.edu> wrote:
> Hi Harry,
Hey!

> Check to see if you see something in /sys/devices/system/cpu/intel_pstate/
My core2duo doesn't have intel_pstate, its too old!

> https://wiki.archlinux.org/index.php/CPU_frequency_scaling
These options (cpupower frequency-set -g performance) don't work, see
output as root user:

[root@audio ~]# cpupower frequency-set -g performance
Setting cpu: 0
Error setting new values. Common errors:
- Do you have proper administration rights? (super-user?)
- Is the governor you requested available and modprobed?
- Trying to set an invalid policy?
- Trying to set a specific frequency, but userspace governor is not available,
   for example because of hardware which cannot be set to a specific frequency
   or because the userspace governor isn't loaded?

Cheers, -Harry

Re: CpuFreq Laptop Scaling broken?

[Viresh Kumar]

On 25 August 2014 19:32, Harry van Haaren <harryhaaren@gmail.com> wrote:
> On Mon, Aug 25, 2014 at 2:55 PM, Viresh Kumar <viresh.kumar@linaro.org> wrote:
>> You tried both these as root? Even 'sudo' may not work..
> Yes, the original output was all as root (not sudo, genuine root).
>
>> I haven't seen the rt code since sometime, what tag should I look at ?
> I'm running 3.14.12-rt9-1-rt  (as packaged for Arch here:
> https://aur.archlinux.org/packages/linux-rt )

I don't have a clue of what's going wrong here..

But my guess would be that something is not working with locking, as that's
one of the main difference between rt and non-rt kernels.

Can you try to debug it with some prints?

The functions  of interest would be, drivers/cpufreq/cpufreq.c:
- static ssize_t show(struct kobject *kobj, struct attribute *attr, char *buf)
- static ssize_t store_scaling_governor(struct cpufreq_policy *policy,
const char *buf, size_t count)

Try to add some prints there while returning errors and then try to change
governor from userspace and things might get somewhat better then..

--
viresh

Re: CpuFreq Laptop Scaling broken?

[Harry van Haaren]

On Tue, Aug 26, 2014 at 5:34 AM, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> Can you try to debug it with some prints?
>
> The functions  of interest would be, drivers/cpufreq/cpufreq.c:
> - static ssize_t show(struct kobject *kobj, struct attribute *attr, char *buf)
> - static ssize_t store_scaling_governor(struct cpufreq_policy *policy,
> const char *buf, size_t count)
>
> Try to add some prints there while returning errors and then try to change
> governor from userspace and things might get somewhat better then..

I will: unfortunatly I won't have time due to buisness trip until next week.
I've scheduled some time for rebuilding with prints then.

Thanks for the suggestion, I'll update once I have more info, -Harry

Re: CpuFreq Laptop Scaling broken?

[Harry van Haaren]

On Tue, Aug 26, 2014 at 11:44 PM, Harry van Haaren
<harryhaaren@gmail.com> wrote:
> On Tue, Aug 26, 2014 at 5:34 AM, Viresh Kumar <viresh.kumar@linaro.org> wrote:
>> The functions  of interest would be, drivers/cpufreq/cpufreq.c:
>> - static ssize_t show(struct kobject *kobj, struct attribute *attr, char *buf)
>> - static ssize_t store_scaling_governor(struct cpufreq_policy *policy,
>> const char *buf, size_t count)
>>
>> Try to add some prints there while returning errors and then try to change
>> governor from userspace and things might get somewhat better then..
>
> Thanks for the suggestion, I'll update once I have more info

Found a little time to check this: the cpufreq_get_policy() call
returns non-zero:

static ssize_t store_scaling_governor(struct cpufreq_policy *policy,
                    const char *buf, size_t count)
{
    int ret;
    char    str_governor[16];
    struct cpufreq_policy new_policy;

  printk( "RT debug: store_scaling_governor() with %s\n", buf );

    ret = cpufreq_get_policy(&new_policy, policy->cpu);
    if (ret)
  {
    printk( "RT debug: ERROR: get policy() returned \n" );
        return ret;
  }


dmesg:
[   43.186804] RT debug: store_scaling_governor() with performance
[   43.186811] RT debug: ERROR: get policy() returned


This output is from a fresh boot: so it seems that the getting the
current policy after
a boot isn't functioning right here.

Re: CpuFreq Laptop Scaling broken?

[Harry van Haaren]

On Wed, Aug 27, 2014 at 4:07 PM, Viresh Kumar
> Some more prints inside  cpufreq_get_policy() might take us closer..

I've time to continue this bug hunt again:

With more prints:

struct cpufreq_policy *cpufreq_cpu_get(unsigned int cpu)
{
    struct cpufreq_policy *policy = NULL;
    unsigned long flags;

    if ( cpufreq_disabled() )
  {
    printk("RT test errror: cpufreq_disabled(), returning NULL\n");
    return NULL;
  }
  if (cpu >= nr_cpu_ids)
  {
    printk("cpu > nr_cpu_ids\n");
        return NULL;
  }

    if (!down_read_trylock(&cpufreq_rwsem))
  {
    printk("RT debug: down_read_trylock( &cpufreq_rwsem ) returned NULL\n");
        return NULL;
  }

   <snip>

The down_read_trylock() fails, the other checks are fine.

If i'm understanding correctly, the cpufreq_rwsem locks the module
from being unloaded during critical sections while writing to CPU
speed etc?

Would that mean that an un-matched pair of down_read() up_write()
could cause the failure of changing CPU speed?

-Harry

www.openavproductions.com

Re: CpuFreq Laptop Scaling broken?

[Viresh Kumar]

On 5 September 2014 17:36, Harry van Haaren <harryhaaren@gmail.com> wrote:
> The down_read_trylock() fails, the other checks are fine.

So my initial guess was correct :)

> If i'm understanding correctly, the cpufreq_rwsem locks the module
> from being unloaded during critical sections while writing to CPU
> speed etc?

Its not about changing speed, but its more of a lock for critical sections
only.. So, yes its required..

> Would that mean that an un-matched pair of down_read() up_write()
> could cause the failure of changing CPU speed?

Probably yes, but why doesn't that happen without rt?

@Steven: Any guesses ?

Re: CpuFreq Laptop Scaling broken?

[Thomas Gleixner]

On Fri, 5 Sep 2014, Viresh Kumar wrote:

> On 5 September 2014 17:36, Harry van Haaren <harryhaaren@gmail.com> wrote:
> > The down_read_trylock() fails, the other checks are fine.
> 
> So my initial guess was correct :)
> 
> > If i'm understanding correctly, the cpufreq_rwsem locks the module
> > from being unloaded during critical sections while writing to CPU
> > speed etc?
> 
> Its not about changing speed, but its more of a lock for critical sections
> only.. So, yes its required..
> 
> > Would that mean that an un-matched pair of down_read() up_write()
> > could cause the failure of changing CPU speed?
> 
> Probably yes, but why doesn't that happen without rt?

I think Carsten had looked into this.

Re: CpuFreq Laptop Scaling broken?

[Harry van Haaren]

Hi,

Small update: there's an mail-thread on Linux-Audio-User of somebody
who seems to have a similar issue as this. The thread in question:
http://linux-audio.4202.n7.nabble.com/Problems-with-real-time-kernel-thermal-throttling-and-cpu-freq-governors-td93209.html

Note that the CPU the OP of that thread has is an i7, and they're
using an older -rt kernel (3.14-2-rt-amd64). On a vanilla 3.14-2-amd64
all is reported to be fine (see forum post mentioned in first mail by
OP, link: http://forums.debian.net/viewtopic.php?f=5&t=117613&p=554270#p554270
)

The OP of that thread mentions that the machine hangs, forcing a
hard-reboot. I've been experiencing that too, but had not concidered
that the issues might be one-and-the-same.

What is the next step I can take to help this issue along?
Cheers, -Harry


On Fri, Sep 5, 2014 at 11:47 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> On Fri, 5 Sep 2014, Viresh Kumar wrote:
>
>> On 5 September 2014 17:36, Harry van Haaren <harryhaaren@gmail.com> wrote:
>> > The down_read_trylock() fails, the other checks are fine.
>>
>> So my initial guess was correct :)
>>
>> > If i'm understanding correctly, the cpufreq_rwsem locks the module
>> > from being unloaded during critical sections while writing to CPU
>> > speed etc?
>>
>> Its not about changing speed, but its more of a lock for critical sections
>> only.. So, yes its required..
>>
>> > Would that mean that an un-matched pair of down_read() up_write()
>> > could cause the failure of changing CPU speed?
>>
>> Probably yes, but why doesn't that happen without rt?
>
> I think Carsten had looked into this.
>



-- 

www.openavproductions.com

Re: CpuFreq Laptop Scaling broken?

[Steven Rostedt]

On Mon, 25 Aug 2014 19:25:28 +0530
Viresh Kumar <viresh.kumar@linaro.org> wrote:


> > # Attempt to set min freq of 2GHz (max) of this core2duo
> > $ echo 2000000 > /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq
> > bash: /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq: Permission denied
> 
> You tried both these as root? Even 'sudo' may not work..
> 
> I haven't seen the rt code since sometime, what tag should I look at ?
> 

Can you also try the matching vanilla kernel to see if that works? That
is, if you are using 3.14.12-rt9, build and boot 3.14.12 with the same
config (except change CONFIG_PREEMPT_RT_FULL to just CONFIG_PREEMPT)
and see if that doesn't work either.

-- Steve

Re: CpuFreq Laptop Scaling broken?

[Harry van Haaren]

On Mon, Aug 25, 2014 at 3:57 PM, Pavel Vasilyev <pavel@pavlinux.ru> wrote:
>>>> # Attempt to set min freq of 2GHz (max) of this core2duo
>>>> $ echo 2000000 > /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq
>>>> bash: /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq: Permission
>>>> denied
>
> cpuinfo_min_freq - Read-only file, use scaling_min_freq

$ echo 2000000 > /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq
bash: echo: write error: Invalid argument

No success there.