Fwd: Nice processes prevent frequency increases - possible scheduler regression (known good in 2.6.35)

Fwd: Nice processes prevent frequency increases - possible scheduler regression (known good in 2.6.35)

[Michal Zatloukal]

Hello.
Note 1: I'm not subscribed, please Cc me into replies.
Note 2: I haven't kept up-to-date with the latest kernels in quite
some time so I've only noticed this when I switched my machine from
Kubuntu 10.10 (kernel 2.6.35) to 12.10 (kernel 3.5) last week (both
amd64 arch). I've done some trials to determine what's happening and
think I've got it. It may be a corner case, I'm not sure.
So here's the situation:
- dual-core laptop running BOINC projects (nice-19, single-thread
process on each core, very little IO) without restriction (the
processes are not terminated until they finish themselves or the boinc
daemon is stopped).
- to make the fan noise levels bearable with such a load, I set the
on-demand governor's ignore_nice_load parameter to YES. This makes the
CPU run at 800 MHz when it's only running the BOINC projects and
nothing demanding for the user (nice=0).
On the old kernel, playing an HD video with BOINC running in the
background is not a problem - the CPU immediately kicks into 2.5 GHz
and the video plays smoothly. User tasks which take all the CPU they
can get (compressing files, starting Chromium with ~70 tabs), take all
the CPU they can get, making ondemad governor switch the CPU to full
speed and BOINC projects are starved.
On the new kernel, the nice processes are never starved - even when
starting a tab-laden chromium session, the processes for BOINC keep
about 20% CPU each (that is normalized to all CPUs, ie 40% nice load
on each core). The problem is, the governor now seems to consider the
non-nice task unable to saturate the CPU, and the cores' frequencies
are hovering between 1.0 and 1.8 GHz. The scheduler keeps scheduling
the nice tasks, and the non-nice tasks are progressing much slower,
caused by the lower CPU speed as well as less processing time
allocated to them. HD video stutters often, and Chromium takes at
least 2-3 times longer to fully load.

Questions:
Q1: Is the usage of ignore_nice_load=Y obscure? Ubuntu defaults to
"N", not sure about other distros.
Q2. Is this considered a bug? If so, I'll try to devise a test case
that doesn't require a 30-minute setup and an account on BOINC
projects. Any hints on what I should look out for when I want to
emulate the load of BOINC projects with readily-available tools,
regarding the scheduler? Cgroup, tty, session, thread count?

MZ

Re: Fwd: Nice processes prevent frequency increases - possible scheduler regression (known good in 2.6.35)

[Mike Galbraith]

On Fri, 2012-11-02 at 21:09 +0100, Michal Zatloukal wrote:

> On the new kernel, the nice processes are never starved - even when
> starting a tab-laden chromium session, the processes for BOINC keep
> about 20% CPU each (that is normalized to all CPUs, ie 40% nice load
> on each core). The problem is, the governor now seems to consider the
> non-nice task unable to saturate the CPU, and the cores' frequencies
> are hovering between 1.0 and 1.8 GHz. The scheduler keeps scheduling
> the nice tasks, and the non-nice tasks are progressing much slower,
> caused by the lower CPU speed as well as less processing time
> allocated to them. HD video stutters often, and Chromium takes at
> least 2-3 times longer to fully load.

Your nice 19 tasks receiving 'too much' CPU when there are other
runnable tasks around sounds like you have SCHED_AUTOGROUP enabled.

With this enabled (or if tasks are placed in cgroups by another means),
group A and group B will each receive equal CPU if group shares/weight
are equal, regardless of group content.  Task nice level will affect CPU
distribution within a group, but group A containing a gaggle of nice 19
tasks will receive the same amount of CPU as group B containing your
nice 0 browser with zillion tabs.  Add nice 0 hogs to group A, the nice
19 tasks will receive much less CPU, but the total for group A will
remain unchanged relative to group B, unless group shares/weights are
twiddled.

(no idea what's going on with ondemand)

-Mike

Re: Fwd: Nice processes prevent frequency increases - possible scheduler regression (known good in 2.6.35)

[Mike Galbraith]

On Sat, 2012-11-03 at 04:33 -0700, Mike Galbraith wrote: 
> On Fri, 2012-11-02 at 21:09 +0100, Michal Zatloukal wrote:
> 
> > On the new kernel, the nice processes are never starved - even when
> > starting a tab-laden chromium session, the processes for BOINC keep
> > about 20% CPU each (that is normalized to all CPUs, ie 40% nice load
> > on each core). The problem is, the governor now seems to consider the
> > non-nice task unable to saturate the CPU, and the cores' frequencies
> > are hovering between 1.0 and 1.8 GHz. The scheduler keeps scheduling
> > the nice tasks, and the non-nice tasks are progressing much slower,
> > caused by the lower CPU speed as well as less processing time
> > allocated to them. HD video stutters often, and Chromium takes at
> > least 2-3 times longer to fully load.
> 
> Your nice 19 tasks receiving 'too much' CPU when there are other
> runnable tasks around sounds like you have SCHED_AUTOGROUP enabled.

(forgot to mention: if that's the case, you can add noautogroup to your
kernel command line to turn it off if distro turned it on in .config)

Re: Fwd: Nice processes prevent frequency increases - possible scheduler regression (known good in 2.6.35)

[Michal Zatloukal]

On Sat, Nov 3, 2012 at 12:48 PM, Mike Galbraith <efault@gmx.de> wrote:
> On Sat, 2012-11-03 at 04:33 -0700, Mike Galbraith wrote:
>> On Fri, 2012-11-02 at 21:09 +0100, Michal Zatloukal wrote:
>>
>> > On the new kernel, the nice processes are never starved - even when
>> > starting a tab-laden chromium session, the processes for BOINC keep
>> > about 20% CPU each (that is normalized to all CPUs, ie 40% nice load
>> > on each core). The problem is, the governor now seems to consider the
>> > non-nice task unable to saturate the CPU, and the cores' frequencies
>> > are hovering between 1.0 and 1.8 GHz. The scheduler keeps scheduling
>> > the nice tasks, and the non-nice tasks are progressing much slower,
>> > caused by the lower CPU speed as well as less processing time
>> > allocated to them. HD video stutters often, and Chromium takes at
>> > least 2-3 times longer to fully load.
>>
>> Your nice 19 tasks receiving 'too much' CPU when there are other
>> runnable tasks around sounds like you have SCHED_AUTOGROUP enabled.
>
> (forgot to mention: if that's the case, you can add noautogroup to your
> kernel command line to turn it off if distro turned it on in .config)
>

Thanks, Ubuntu's kernel is indeed configured with that option enabled,
and passing "noautogroup" at grub restores the previous behaviour. I'm
back to happy days again :) BTW, isn't this the "magic 200-line patch"
I was reading about ~2 years ago?

MZ

Re: Fwd: Nice processes prevent frequency increases - possible scheduler regression (known good in 2.6.35)

[Chris Friesen]

On 11/03/2012 09:40 AM, Michal Zatloukal wrote:
> On Sat, Nov 3, 2012 at 12:48 PM, Mike Galbraith<efault@gmx.de>  wrote:
>> On Sat, 2012-11-03 at 04:33 -0700, Mike Galbraith wrote:
>>> On Fri, 2012-11-02 at 21:09 +0100, Michal Zatloukal wrote:

>>> Your nice 19 tasks receiving 'too much' CPU when there are other
>>> runnable tasks around sounds like you have SCHED_AUTOGROUP enabled.
>>
>> (forgot to mention: if that's the case, you can add noautogroup to your
>> kernel command line to turn it off if distro turned it on in .config)
>>
>
> Thanks, Ubuntu's kernel is indeed configured with that option enabled,
> and passing "noautogroup" at grub restores the previous behaviour. I'm
> back to happy days again :) BTW, isn't this the "magic 200-line patch"
> I was reading about ~2 years ago?

Yes.

Chris