Scheduling of low-priority background processes

Scheduling of low-priority background processes

[Thomas Koeller]

Dear kernel wizards,

first of all, I wish to apologize in case I am asking a questions that has already been discussed much on this list. I am not a subscriber
(so please CC. me if you respond to this mail) and by browsing the list archive I did not immediatly find any discussion about my topic.
So here is my question:

Some operating systems I have been working with had a scheduling policy different from what I find in Linux.

Re: Scheduling of low-priority background processes

[Rik van Riel]

On Mon, 5 Nov 2001, Thomas Koeller wrote:

> So here is my question:
>
> Some operating systems I have been working with had a scheduling
> policy different from what I find in Linux.

I'm not sure what you want to ask, though I guess you're not
too happy about the fact that niced processes still get a lot
of CPU time in Linux ;)

If this was what you wanted to say, this is something I've been
planning to fix for a while and, now that my VM has been removed
from the kernel, I'll have some time for too...

cheers,

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)

http://www.surriel.com/		http://distro.conectiva.com/

Re: Scheduling of low-priority background processes

[Thomas Koeller]

Sorry, that mail was incomplete, I did not mean to send it.

Thomas

On Monday,  5. November 2001 22:08, Rik van Riel wrote:
> On Mon, 5 Nov 2001, Thomas Koeller wrote:
> > So here is my question:
> >
> > Some operating systems I have been working with had a scheduling
> > policy different from what I find in Linux.
>
> I'm not sure what you want to ask, though I guess you're not
> too happy about the fact that niced processes still get a lot
> of CPU time in Linux ;)
>
> If this was what you wanted to say, this is something I've been
> planning to fix for a while and, now that my VM has been removed
>
> >from the kernel, I'll have some time for too...
>
> cheers,
>
> Rik

-- 
Thomas Koeller
tkoeller@gmx.net

Re: Scheduling of low-priority background processes

[Rik van Riel]

On Mon, 5 Nov 2001, Thomas Koeller wrote:

> On those systems, you could assign a scheduling priority lower than
> the one nomally used by interactive processes to CPU-hogging,
> numbercrunching tasks. These tasks would then use up any CPU time left
> over by interactive processes without otherwise interfering with them.
> I always found this feature very useful (think of SETI@home!).

> But that idea is so obvious, and since nobody did it so far, I am
> probably missing something. What is it?

Priority inversion.  I did a patch which does exactly
what you describe, around the 2.1 timeframe. It worked
fine most of the time, but occasionally the following
happened:

1) a SCHED_IDLE process got hold of some kernel lock
2) a normal, low-priority process started eating CPU
   for a number of seconds
3) a high-priority normal process wanted the lock the
   SCHED_IDLE task had, but had to wait several seconds,
   at times up to a minute, before the SCHED_IDLE task
   got a chance to run and release the lock

This wasn't too much of a problem on my own system, but
of course this is an easily exploitable vulnerability for
attackers.

For me, this just means we should improve the scheduler so
nice levels are stronger ... say that a nice +20 process
only gets 1% of the CPU of a normal priority process ;)

regards,

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/

http://www.surriel.com/		http://distro.conectiva.com/

Re: Scheduling of low-priority background processes

[Kevin Easton]

> > On those systems, you could assign a scheduling priority lower than 
> > the one nomally used by interactive processes to CPU-hogging, 
> > numbercrunching tasks. These tasks would then use up any CPU time left 
> > over by interactive processes without otherwise interfering with them. 
> > I always found this feature very useful (think of SETI@home!). 
>
>
> > But that idea is so obvious, and since nobody did it so far, I am 
> > probably missing something. What is it? 
>
>
> Priority inversion. I did a patch which does exactly 
> what you describe, around the 2.1 timeframe. It worked 
> fine most of the time, but occasionally the following 
> happened: 
>
>
> 1) a SCHED_IDLE process got hold of some kernel lock 
> 2) a normal, low-priority process started eating CPU 
>    for a number of seconds 
> 3) a high-priority normal process wanted the lock the 
>    SCHED_IDLE task had, but had to wait several seconds, 
>    at times up to a minute, before the SCHED_IDLE task 
>    got a chance to run and release the lock 
>
>
> This wasn't too much of a problem on my own system, but 
> of course this is an easily exploitable vulnerability for 
> attackers. 
>
>
> For me, this just means we should improve the scheduler so 
> nice levels are stronger ... say that a nice +20 process 
> only gets 1% of the CPU of a normal priority process ;) 
>

What if the SCHED_IDLE behaviour only applies when the process
is in userspace?  Couldn't scheduler compare the process's
instruction pointer against the kernel/user break point, and
if the process is in the kernel, then just treat it like a
normal process?

>
> regards, 
>
>
> Rik 

Re: Scheduling of low-priority background processes

[Kevin Easton]

I foolishly muttered:

> What if the SCHED_IDLE behaviour only applies when the process 
> is in userspace? Couldn't scheduler compare the process's 
> instruction pointer against the kernel/user break point, and 
> if the process is in the kernel, then just treat it like a 
> normal process? 

...eek.   I clearly wasn't thinking straight with that one.  There
isn't a (non-disgusting) way of determining in the scheduler if a
process is executing a syscall apart from sys_sched_yield, is there.

Carry on...

   - Kevin.

Re: Scheduling of low-priority background processes

[Pavel Machek]

Hi!
> I foolishly muttered:
> 
> > What if the SCHED_IDLE behaviour only applies when the process 
> > is in userspace? Couldn't scheduler compare the process's 
> > instruction pointer against the kernel/user break point, and 
> > if the process is in the kernel, then just treat it like a 
> > normal process? 
> 
> ...eek.   I clearly wasn't thinking straight with that one.  There
> isn't a (non-disgusting) way of determining in the scheduler if a
> process is executing a syscall apart from sys_sched_yield, is there.

Actually, something similar was implemented. New process flag was added,
and when process did syscall, it lost SCHED_IDLE flag, and it was returned
to it when it went back to userland.

-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: Scheduling of low-priority background processes

[Richard Gooch]

Pavel Machek writes:
> Hi!
> > I foolishly muttered:
> > 
> > > What if the SCHED_IDLE behaviour only applies when the process 
> > > is in userspace? Couldn't scheduler compare the process's 
> > > instruction pointer against the kernel/user break point, and 
> > > if the process is in the kernel, then just treat it like a 
> > > normal process? 
> > 
> > ...eek.   I clearly wasn't thinking straight with that one.  There
> > isn't a (non-disgusting) way of determining in the scheduler if a
> > process is executing a syscall apart from sys_sched_yield, is there.
> 
> Actually, something similar was implemented. New process flag was
> added, and when process did syscall, it lost SCHED_IDLE flag, and it
> was returned to it when it went back to userland.

That's the sensible approach. Is anyone maintaining that patch and
sending it to Linus for inclusion?

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: Scheduling of low-priority background processes

[Thomas Koeller]

On Monday,  5. November 2001 23:24, Mark Hahn wrote:
>
> please read the scheduler; it's not that bad, especially if you
> ignore the SMP case.  normal procs are only considered if there
> are no runnable RT procs.

I know the scheduler works this way. But does it have to? What I meant to do 
was suggesting an improvement.

Thomas


-- 
Thomas Koeller
tkoeller@gmx.net