High Resolution Timer DOS

High Resolution Timer DOS

[matthieu castet]

Hi,

some programs need to do some short of busyloop. It was often 
implemented as :

while (1) {
	if (can_do_stuff) {
		do_stuff();
	}
	else
		//sleep a very short of time
		usleep(1);
}

usleep(1) or equivalent where used instead of sched_yield, because of 
some priority issue. IIRC doing sched_yield, make the process appears 
like an interactive process, so it has better priority and get call more 
often.

But now if high res timer are enabled, these programs while cause 
something like a DOS : the context switch per second will be bigger than 
500 000 and the cpu usage will be very high.


I don't know if such problem are already know, but I believe a warning 
about such issues should be in the Kconfig description.


Matthieu

Re: High Resolution Timer DOS

[Thomas Gleixner]

On Sat, 2007-04-28 at 23:53 +0200, matthieu castet wrote:
> Hi,
> 
> some programs need to do some short of busyloop. It was often 
> implemented as :
> 
> while (1) {
> 	if (can_do_stuff) {
> 		do_stuff();
> 	}
> 	else
> 		//sleep a very short of time
> 		usleep(1);
> }
> 
> usleep(1) or equivalent where used instead of sched_yield, because of 
> some priority issue. IIRC doing sched_yield, make the process appears 
> like an interactive process, so it has better priority and get call more 
> often.
> 
> But now if high res timer are enabled, these programs while cause 
> something like a DOS : the context switch per second will be bigger than 
> 500 000 and the cpu usage will be very high.

Well, it is not really a DoS. The rescheduling of the process is limited
by the scheduler and the available CPU time (depending on the number of
runnable tasks in the system).

>From the spec:

Implementations may place limitations on the granularity of timer
values. For each interval timer, if the requested timer value requires a
finer granularity than the implementation supports, the actual timer
value shall be rounded up to the next supported value.

The !HIGHRES enabled kernel rounds this up to the HZ interval, the
HIGHRES enabled kernel grants the request for this short sleep.

The program gets what it asked for: a stupid sleep value.

	tglx

Re: High Resolution Timer DOS

[Lee Revell]

On 4/28/07, Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Well, it is not really a DoS. The rescheduling of the process is limited
> by the scheduler and the available CPU time (depending on the number of
> runnable tasks in the system).

Shouldn't an unprivileged process be rate limited somehow to avoid
flooding the machine with interrupts?  We restrict nonroot users from
setting the RTC interrupt rate higher than 64Hz for a similar reason
(granted, this limit dates back to the 486 days and should probably be
increased to 1024 Hz).

Root and SCHED_FIFO tasks could be exempt from rate limiting, to avoid
the need to introduce a new rlimit which would take years for
userspace to catch up to.

Lee

Re: High Resolution Timer DOS

[William Heimbigner]

On Sat, 28 Apr 2007, Lee Revell wrote:

> On 4/28/07, Thomas Gleixner <tglx@linutronix.de> wrote:
>>
>>  Well, it is not really a DoS. The rescheduling of the process is limited
>>  by the scheduler and the available CPU time (depending on the number of
>>  runnable tasks in the system).
>
> Shouldn't an unprivileged process be rate limited somehow to avoid
> flooding the machine with interrupts?  We restrict nonroot users from
> setting the RTC interrupt rate higher than 64Hz for a similar reason
> (granted, this limit dates back to the 486 days and should probably be
> increased to 1024 Hz).

Isn't that what /etc/security/limits.conf is for?
Just limit the CPU usage.

> Root and SCHED_FIFO tasks could be exempt from rate limiting, to avoid
> the need to introduce a new rlimit which would take years for
> userspace to catch up to.
>
> Lee

William Heimbigner
icxcnika@mar.tar.cc

Re: High Resolution Timer DOS

[Ingo Molnar]

* Lee Revell <rlrevell@joe-job.com> wrote:

> > Well, it is not really a DoS. The rescheduling of the process is 
> > limited by the scheduler and the available CPU time (depending on 
> > the number of runnable tasks in the system).
> 
> Shouldn't an unprivileged process be rate limited somehow to avoid 
> flooding the machine with interrupts?  We restrict nonroot users from 
> setting the RTC interrupt rate higher than 64Hz for a similar reason 
> (granted, this limit dates back to the 486 days and should probably be 
> increased to 1024 Hz).

No. An interrupt in this case is really just 'CPU time used up', and an 
unprivileged process can take up as much CPU time as the scheduler 
allows. So it's _not_ a DoS, and neither is any other unprivileged 
infinit loop (or high-rate context-switching task) a DoS.

	Ingo

Re: High Resolution Timer DOS

[matthieu castet]

[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]

Ingo Molnar wrote:
> * Lee Revell <rlrevell@joe-job.com> wrote:
> 
>>> Well, it is not really a DoS. The rescheduling of the process is 
>>> limited by the scheduler and the available CPU time (depending on 
>>> the number of runnable tasks in the system).
>> Shouldn't an unprivileged process be rate limited somehow to avoid 
>> flooding the machine with interrupts?  We restrict nonroot users from 
>> setting the RTC interrupt rate higher than 64Hz for a similar reason 
>> (granted, this limit dates back to the 486 days and should probably be 
>> increased to 1024 Hz).
> 
> No. An interrupt in this case is really just 'CPU time used up', and an 
> unprivileged process can take up as much CPU time as the scheduler 
> allows. So it's _not_ a DoS, and neither is any other unprivileged 
> infinit loop (or high-rate context-switching task) a DoS.
Ok, may be DOS was not the correct term, but with the 2.6.21 hrt there 
is a great difference between an infinite loop and the high-rate 
context-switching task (you can try attached programs).
With the first I the system is still responsive, with the latter it 
isn't (new process take lot's of time to get created, other process are 
very slow).
If it is "just 'CPU time used up'", why I see a such difference between 
the 2 cases ?

Maybe the current scheduler failed to handle correctly this case ?


Matthieu

[-- Attachment #2: infinite_loop.c --]
[-- Type: text/x-csrc, Size: 134 bytes --]

#include <string.h>
#include <sys/time.h>
#include <time.h>

int main()
{
	struct timeval tv;
	while (1)
		gettimeofday(&tv, NULL);
}

[-- Attachment #3: small_sleep.c --]
[-- Type: text/x-csrc, Size: 39 bytes --]

int main()
{
	while (1)
		usleep(1);
}

Re: High Resolution Timer DOS

[Ingo Molnar]

* matthieu castet <castet.matthieu@free.fr> wrote:

> Ok, may be DOS was not the correct term, [...]

ok, good that have that issue put aside ;-)

> [...] but with the 2.6.21 hrt there is a great difference between an 
> infinite loop and the high-rate context-switching task (you can try 
> attached programs). With the first I the system is still responsive, 
> with the latter it isn't (new process take lot's of time to get 
> created, other process are very slow). If it is "just 'CPU time used 
> up'", why I see a such difference between the 2 cases ?

this is a pure scheduler thing: the scheduler treats sleepers 
differently than CPU hogs. Try the same test for example under the 
(ob'plug) CFS scheduler:

   http://redhat.com/~mingo/cfs-scheduler/

and you'll see small_sleep.c being handled the same way as 
infinite_loop.c. This is a CFS box with 20 small_sleep's running:

 top - 20:41:02 up 1 min,  2 users,  load average: 4.92, 1.27, 0.43
 Tasks:  89 total,  22 running,  67 sleeping,   0 stopped,   0 zombie
 Cpu(s):  5.2%us, 46.5%sy,  1.7%ni, 17.7%id, 28.5%wa,  0.3%hi,  0.1%si,  0.0%st
 Mem:   2053204k total,   103300k used,  1949904k free,    12096k buffers
 Swap:  4096564k total,        0k used,  4096564k free,    43040k cached

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  2208 mingo     20   0  1576  256  208 R  4.5  0.0   0:01.08 small_sleep
  2252 mingo     20   0  1580  260  208 R  4.5  0.0   0:00.71 small_sleep
  2254 mingo     20   0  1576  256  208 R  4.5  0.0   0:00.61 small_sleep

and the system is still completely usable.

This isnt really about timers - you can achieve similar effects without 
using any timers.

	Ingo