linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Feature proposal (scheduling related)
@ 2003-07-23 10:57 jimis
  2003-07-23 11:24 ` David M. Wilson
                   ` (2 more replies)
  0 siblings, 3 replies; 17+ messages in thread
From: jimis @ 2003-07-23 10:57 UTC (permalink / raw)
  To: linux-kernel

With the current scheduler we can prioritize the CPU usage for each process. 
What I think would be extremely useful (as I have needed it many times) is the 
scheduling of disk I/O and net I/O traffic. 2 examples showing the importance 
(the numbers are estimations just to explain whati I mean):

1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits of 
bandwidth available. What I want to do is listen to icecast radio via xmms (at 
22 kbits), download the kernel sources with wget, and browse the web at the same 
time. Currently I think that this is *impossible* (correct me if I'm wrong) as 
the radio will be full of pauses and the browsing experience painfully slow. 
What I would like to be able to do (let's suppose nice has the --net option to 
set net I/O priority):
$ nice --net -1 xmms
$ nice --net 1 wget ftp://.../KernelSources.tar.bz2
$ mozilla
This way, xmms which has top priority whould always get the 22kbits it needs. 
What remains should go to the browser when I ask for a web page, and when the 
browser doesn't request anything (let's say I'm reading a big doc in tldp) what 
remains should go to wget. Wget has lower priority and won't irritate the 
browsing experience, though the file will be downloaded when there is free 
bandwidth.

2) Normally mozilla starts in 5 seconds after intense disk I/O to load all 
needed libraries. If I run in the background a long disk intense process
(like find / -name 'whatever' -xdev) loading mozilla could need 20 boring 
seconds, or doing other simple tasks might be irritating slow. What I would like 
to be able to do is (once again let's suppose nice has the --disk option to set 
disk I/O priority):
$ nice --disk 1 find / -name 'whatever' -xdev
$ mozilla
and load mozilla ,which has the default disk priority 0, fast. The scheduler 
should give to mozilla most disk troughput when it needs it.

Notes:
1) PLEASE CC REPLIES BACK TO ME since I 'm not subscribed to the list (I can't 
stand the traffic). However I 'll be checking periodically the list via NNTP.
2) As I have no idea of kernel programming I hope what I propose is aplicable 
and relevant to the kernel, as I believe. Sorry if not.
3) I hope what I propose is implementable using the existing scheduler. It would 
be nice to have one scheduler to handle them all.
4) I believe that these features don't need a lot of CPU power because the disk 
and net I/O troughput are relatively slow.
5) If you think that UNIX tradition forbits what I propose I must say that these 
features could be invisible to programs, setting the new  priorities to the 
default 0 or maybe to the same number as CPU priority.

Thank you very much for your time,
Dimitris



^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 10:57 Feature proposal (scheduling related) jimis
@ 2003-07-23 11:24 ` David M. Wilson
  2003-07-23 11:43 ` Pavel Machek
  2003-07-23 14:17 ` Valdis.Kletnieks
  2 siblings, 0 replies; 17+ messages in thread
From: David M. Wilson @ 2003-07-23 11:24 UTC (permalink / raw)
  To: jimis; +Cc: linux-kernel

On Wed, Jul 23, 2003 at 01:57:41PM +0300, jimis@gmx.net wrote:

> 1)I 'm connected to the internet via dial-up, therefore I only have 40 
> kbits of bandwidth available. What I want to do is listen to icecast radio 
> via xmms (at 22 kbits), download the kernel sources with wget, and browse 
> the web at the same time. Currently I think that this is *impossible* 
> (correct me if I'm wrong) as the radio will be full of pauses and the 
> browsing experience painfully slow. What I would like to be able to do 
> (let's suppose nice has the --net option to set net I/O priority):
> $ nice --net -1 xmms
> $ nice --net 1 wget ftp://.../KernelSources.tar.bz2
> $ mozilla

I think what you need to look at is network-level QoS, this is far too
intrusive for what you are trying to do. Try tldp.org.

David.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 10:57 Feature proposal (scheduling related) jimis
  2003-07-23 11:24 ` David M. Wilson
@ 2003-07-23 11:43 ` Pavel Machek
  2003-07-25 20:24   ` Marcelo Tosatti
  2003-07-23 14:17 ` Valdis.Kletnieks
  2 siblings, 1 reply; 17+ messages in thread
From: Pavel Machek @ 2003-07-23 11:43 UTC (permalink / raw)
  To: jimis; +Cc: linux-kernel

Hi!

> With the current scheduler we can prioritize the CPU usage for each 
> process. What I think would be extremely useful (as I have needed it 
> many times) is the scheduling of disk I/O and net I/O traffic. 2 
> examples showing the importance (the numbers are estimations just to 
> explain whati I mean):

Yes that would be nice, and in 2.5 timeframe
there was patch doing that. Port it to 2.6 an test it!
-- 
				Pavel
Written on sharp zaurus, because my Velo1 broke. If you have Velo you don't need...


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 10:57 Feature proposal (scheduling related) jimis
  2003-07-23 11:24 ` David M. Wilson
  2003-07-23 11:43 ` Pavel Machek
@ 2003-07-23 14:17 ` Valdis.Kletnieks
  2003-07-23 14:23   ` Alan Cox
                     ` (3 more replies)
  2 siblings, 4 replies; 17+ messages in thread
From: Valdis.Kletnieks @ 2003-07-23 14:17 UTC (permalink / raw)
  To: jimis; +Cc: linux-kernel

[-- Attachment #1: Type: text/plain, Size: 4619 bytes --]

On Wed, 23 Jul 2003 13:57:41 +0300, jimis@gmx.net  said:
> 1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits of 
> bandwidth available. What I want to do is listen to icecast radio via xmms (at
> 22 kbits), download the kernel sources with wget, and browse the web at the same
> time. Currently I think that this is *impossible* (correct me if I'm wrong) as
> the radio will be full of pauses and the browsing experience painfully slow.

Basically, you're stuck.  The biggest part of the problem is that although you
can certainly control the outbound packets, you have no real control over when
inbound packets arrive at the other end of your dial-up.  One person suggested
using QoS to help things along - but that needs to be implemented at the OTHER
end of the dial-up - which means unless your provider does QoS on the terminal
server, you're basically stuck.  Packets will probably just get queued up in
order of arrival.

And even if the packets do get prioritized so your xmms traffic "goes first",
that's still not going to be perfect, due to a phenomenon canned 'jitter',
which is a measure of how "bursty" the traffic is, and how much *variability*
there is.

Let's say you have xmms going, and a very slow wget.  The last 5 packets of
xmms traffic arrived when the modem was idle, and immediately start coming
across, so the jitter is near zero.  Now a 6th xmms packet arrives - but the
modem is 3 bytes into transmitting a 1500 byte wget packet.  Wham! that packet
gets to wait 300ms for its turn, and you just got 300ms of 'jitter'.  If the
xmms didn't have 300ms of sound already buffered, you're screwed and about to
have a drop-out.  QoS doesn't help here - the best it can do is guarantee that
you only have to wait for the currently transmitting packet and not for any
already-queued packets. (As an aside - TCP windowing will probably guarantee
that any given connection will only have 1 or 2 packets queued up - but if
you're going to run 5 concurrent data-transfer connections, that's still 5-10
packets. If you don't have QoS, any given packet *could* end up with several
seconds of latency here, and the exact order of arrival could make your jitter
anywhere from 0 to several seconds...

You *might* be able to improve the jitter characteristics by dropping the MTU
on the dialup connection from the 1500 byte range down to 576, but this has its
own problems. First, you're going to be sending 3x as many packets, so the
overhead of packet headers is going to triple, cutting into your effective
bandwidth.  Second, there are often issues with windowing, fragmented packets,
and PMTU Discovery when you do this.

> 2) Normally mozilla starts in 5 seconds after intense disk I/O to load all 
> needed libraries. If I run in the background a long disk intense process
> (like find / -name 'whatever' -xdev) loading mozilla could need 20 boring 
> seconds, or doing other simple tasks might be irritating slow. What I would like 
> to be able to do is (once again let's suppose nice has the --disk option to set 
> disk I/O priority):
> $ nice --disk 1 find / -name 'whatever' -xdev
> $ mozilla
> and load mozilla ,which has the default disk priority 0, fast. The scheduler 
> should give to mozilla most disk troughput when it needs it.

Again, here there be serious and nasty dragons.  The problem is that you have
to balance several competing issues:

1) Trying to maintain good overall disk throughput - if you keep pushing
Mozilla's disk requests *right now*, regardless of what the disk is doing, the
disk ends up spending a LOT of time just moving the heads back and forth.  Even
at 5ms a shot, seeks are *expensive*, so you often want to re-order the
requests to minimize the distance travelled.  Think of it as the difference
between planning your trip to the grocery store, the gas station, the post
office, and the bank to make it shortest, as opposed to going back home after
each one because there's something at home that needs doing Right Now...

2) There's a phenomenon called "starvation".  See that 'find' command in your
example?  If mozilla is disk-hungry enough, bad I/O scheduling can mean that
'find' command will sit there *forever*, tying up resources the whole time.
This can cause issues.  For instance - if you've flagged 'mozilla' as the
process that gets first shot at the disk, what do you do if you start paging to
the swap area, and some OTHER process has to read a page in?  What if that
"other process" is the X server or your window manager?  Think REALLY hard here
- just saying "I'll renice them too" is NOT the right answer.. .;)


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 14:17 ` Valdis.Kletnieks
@ 2003-07-23 14:23   ` Alan Cox
  2003-07-23 15:10     ` Richard B. Johnson
  2003-07-23 14:47   ` Greg Stark
                     ` (2 subsequent siblings)
  3 siblings, 1 reply; 17+ messages in thread
From: Alan Cox @ 2003-07-23 14:23 UTC (permalink / raw)
  To: Valdis.Kletnieks; +Cc: jimis, Linux Kernel Mailing List

On Mer, 2003-07-23 at 15:17, Valdis.Kletnieks@vt.edu wrote:
> Basically, you're stuck.  The biggest part of the problem is that although you
> can certainly control the outbound packets, you have no real control over when
> inbound packets arrive at the other end of your dial-up.  One person suggested
> using QoS to help things along - but that needs to be implemented at the OTHER
> end of the dial-up - which means unless your provider does QoS on the terminal
> server, you're basically stuck.  Packets will probably just get queued up in
> order of arrival.

There are a few things that help in the general real world but not
mathematical sense. Use an ftp client like gnome-ftp which can set the
rate it accepts data and window sizes. It'll still jam the modem a
little when it starts a transfer but then it'll generally be ok if you
have a bit of buffering for your icecast stream.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 14:17 ` Valdis.Kletnieks
  2003-07-23 14:23   ` Alan Cox
@ 2003-07-23 14:47   ` Greg Stark
  2003-07-23 22:17   ` jimis
  2003-07-24  4:04   ` Andre Tomt
  3 siblings, 0 replies; 17+ messages in thread
From: Greg Stark @ 2003-07-23 14:47 UTC (permalink / raw)
  To: Valdis.Kletnieks; +Cc: jimis, linux-kernel


Valdis.Kletnieks@vt.edu writes:

> 2) There's a phenomenon called "starvation".  See that 'find' command in your
> example?  If mozilla is disk-hungry enough, bad I/O scheduling can mean that
> 'find' command will sit there *forever*, tying up resources the whole time.
> This can cause issues.  For instance - if you've flagged 'mozilla' as the
> process that gets first shot at the disk, what do you do if you start paging to
> the swap area, and some OTHER process has to read a page in?  What if that
> "other process" is the X server or your window manager?  Think REALLY hard here
> - just saying "I'll renice them too" is NOT the right answer.. .;)

I'm sure it's a serious issue, and yet my network has QoS set up and the low
priority flows still eventually get through just fine even though it has much
lower bandwidth available than my disk controller.

I think it would be really cool to be able to control disk i/o with the same
level of flexibility as network i/o. I could see setting up cbq trees that can
key off things like whether it's paging, a blocking/nonblocking i/o, or a
nonblocking i/o. They could also see what user owns the process, and what
inode the process's executable image is.

I would just wonder about the overhead.

-- 
greg


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 14:23   ` Alan Cox
@ 2003-07-23 15:10     ` Richard B. Johnson
  2003-07-23 15:13       ` Antonio Vargas
  0 siblings, 1 reply; 17+ messages in thread
From: Richard B. Johnson @ 2003-07-23 15:10 UTC (permalink / raw)
  To: Alan Cox; +Cc: Valdis.Kletnieks, jimis, Linux Kernel Mailing List

On Wed, 23 Jul 2003, Alan Cox wrote:

> On Mer, 2003-07-23 at 15:17, Valdis.Kletnieks@vt.edu wrote:
> > Basically, you're stuck.  The biggest part of the problem is that although you
> > can certainly control the outbound packets, you have no real control over when
> > inbound packets arrive at the other end of your dial-up.  One person suggested
> > using QoS to help things along - but that needs to be implemented at the OTHER
> > end of the dial-up - which means unless your provider does QoS on the terminal
> > server, you're basically stuck.  Packets will probably just get queued up in
> > order of arrival.
>
> There are a few things that help in the general real world but not
> mathematical sense. Use an ftp client like gnome-ftp which can set the
> rate it accepts data and window sizes. It'll still jam the modem a
> little when it starts a transfer but then it'll generally be ok if you
> have a bit of buffering for your icecast stream.
>

More info. I use a 56k dialup link and PPD to essentially become
my own ISP for my network at home. The machines at home are nodes
on the company's LAN. I have two dedicated machines, one at
work and one at home who's only purpose is to forward IP packets.
These machines are, otherwise, idle.

When I am using ftp to download some work to one of my other
home computers, it is functional impossible to do any useful
work on any other connection. The connections persist, but
no data will get through when a FTP transfer is in progress
except for the FTP data. Somehow the FTP data stream is able
to hog the entire transmission bandwidth. Attempts to "fix" it
be tuning Nagle off, Van Jacobson, etc., all off doesn't do
anything useful.

I have from time-to-time reported as far back as Linux 2.0, the
fact that there are TCP stalls when using PPP. These stalls
still persist with 2.4.20 and I've just had to accept the so-
called "fact" that 1 to 10 second stalls when using PPP are
"normal".

At one time I thought it was just that the modems had lost
sync so I set up a wire-to-wire PPP link here at work. The
stalls still exist when using the RS-232C link with a null-
modem cable. Evidence is that packets are lost, even with
a direct connection. It takes <too much> time for Linux to
discover the missing datagram and have it re-sent. When a
stream of data (FTP) is being sent, nothing seems to get
lost except the ACKs from the receiving end. It seems as
though RS-232C is not full duplex. If something is being
sent, nothing will be received and vice versa.

But... If you write to RS-232C with a test program, with
pins 2,3 shorted (loop-back), you can always read what
you wrote with no errors IFF hardware CRTCTS is turned OFF.

If I enable hardware handshaking, necessary for a modem,
with RTS jumpered to CTS, all bets are off. There are
many missing bytes, even at 9600 baud. However, when
connected through modems, this affect goes away. So,
it seems as though the PPP problem is related to RS-232C
problems, but I'll be damned if I can find it.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
            Note 96.31% of all statistics are fiction.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 15:10     ` Richard B. Johnson
@ 2003-07-23 15:13       ` Antonio Vargas
  2003-07-23 16:55         ` Disconnect
  0 siblings, 1 reply; 17+ messages in thread
From: Antonio Vargas @ 2003-07-23 15:13 UTC (permalink / raw)
  To: Richard B. Johnson
  Cc: Alan Cox, Valdis.Kletnieks, jimis, Linux Kernel Mailing List

On Wed, Jul 23, 2003 at 11:10:08AM -0400, Richard B. Johnson wrote:
> On Wed, 23 Jul 2003, Alan Cox wrote:
> 
> > On Mer, 2003-07-23 at 15:17, Valdis.Kletnieks@vt.edu wrote:
> > > Basically, you're stuck.  The biggest part of the problem is that although you
> > > can certainly control the outbound packets, you have no real control over when
> > > inbound packets arrive at the other end of your dial-up.  One person suggested
> > > using QoS to help things along - but that needs to be implemented at the OTHER
> > > end of the dial-up - which means unless your provider does QoS on the terminal
> > > server, you're basically stuck.  Packets will probably just get queued up in
> > > order of arrival.
> >
> > There are a few things that help in the general real world but not
> > mathematical sense. Use an ftp client like gnome-ftp which can set the
> > rate it accepts data and window sizes. It'll still jam the modem a
> > little when it starts a transfer but then it'll generally be ok if you
> > have a bit of buffering for your icecast stream.
> >
> 
> More info. I use a 56k dialup link and PPD to essentially become
> my own ISP for my network at home. The machines at home are nodes
> on the company's LAN. I have two dedicated machines, one at
> work and one at home who's only purpose is to forward IP packets.
> These machines are, otherwise, idle.
> 
> When I am using ftp to download some work to one of my other
> home computers, it is functional impossible to do any useful
> work on any other connection. The connections persist, but
> no data will get through when a FTP transfer is in progress
> except for the FTP data. Somehow the FTP data stream is able
> to hog the entire transmission bandwidth. Attempts to "fix" it
> be tuning Nagle off, Van Jacobson, etc., all off doesn't do
> anything useful.

You need QoS at the router level to resolve this. Since you are
running your own routers to connect your ethernet segments, QoS
should be done at both ends of the connection. If it's available
on your distro, try wondershaper, it's a nice script which you tell
your upstream and downstream rates and then it adjusts QoS parameters
to provide great response. The most important thing is that it prioritises
ACK packets above everything else. This helps a lot when there is heavy
traffic (FTP for example) in both directions at the same time.
 
> I have from time-to-time reported as far back as Linux 2.0, the
> fact that there are TCP stalls when using PPP. These stalls
> still persist with 2.4.20 and I've just had to accept the so-
> called "fact" that 1 to 10 second stalls when using PPP are
> "normal".
> 
> At one time I thought it was just that the modems had lost
> sync so I set up a wire-to-wire PPP link here at work. The
> stalls still exist when using the RS-232C link with a null-
> modem cable. Evidence is that packets are lost, even with
> a direct connection. It takes <too much> time for Linux to
> discover the missing datagram and have it re-sent. When a
> stream of data (FTP) is being sent, nothing seems to get
> lost except the ACKs from the receiving end. It seems as
> though RS-232C is not full duplex. If something is being
> sent, nothing will be received and vice versa.
> 
> But... If you write to RS-232C with a test program, with
> pins 2,3 shorted (loop-back), you can always read what
> you wrote with no errors IFF hardware CRTCTS is turned OFF.
> 
> If I enable hardware handshaking, necessary for a modem,
> with RTS jumpered to CTS, all bets are off. There are
> many missing bytes, even at 9600 baud. However, when
> connected through modems, this affect goes away. So,
> it seems as though the PPP problem is related to RS-232C
> problems, but I'll be damned if I can find it.
> 

Greets, Antonio.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 15:13       ` Antonio Vargas
@ 2003-07-23 16:55         ` Disconnect
  2003-07-23 17:22           ` David S. Miller
  0 siblings, 1 reply; 17+ messages in thread
From: Disconnect @ 2003-07-23 16:55 UTC (permalink / raw)
  To: Linux Kernel Mailing List

Alternately, openbsd can do a similar thing with great results (although
I haven't tried it over serial/pppd.)  And it works reasonably well even
when done on only one end.

http://www.benzedrine.cx/ackpri.html

On Wed, 2003-07-23 at 11:13, Antonio Vargas wrote:

> You need QoS at the router level to resolve this. Since you are
> running your own routers to connect your ethernet segments, QoS
> should be done at both ends of the connection. If it's available
> on your distro, try wondershaper, it's a nice script which you tell
> your upstream and downstream rates and then it adjusts QoS parameters
> to provide great response. The most important thing is that it prioritises
> ACK packets above everything else. This helps a lot when there is heavy
> traffic (FTP for example) in both directions at the same time.

-- 
Disconnect <lkml@sigkill.net>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 16:55         ` Disconnect
@ 2003-07-23 17:22           ` David S. Miller
  0 siblings, 0 replies; 17+ messages in thread
From: David S. Miller @ 2003-07-23 17:22 UTC (permalink / raw)
  To: Disconnect; +Cc: linux-kernel

On 23 Jul 2003 12:55:35 -0400
Disconnect <lkml@sigkill.net> wrote:

> Alternately, openbsd can do a similar thing with great results (although
> I haven't tried it over serial/pppd.)  And it works reasonably well even
> when done on only one end.

Wondershaper usually does too.

It's all the same stuff.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 14:17 ` Valdis.Kletnieks
  2003-07-23 14:23   ` Alan Cox
  2003-07-23 14:47   ` Greg Stark
@ 2003-07-23 22:17   ` jimis
  2003-07-24  0:58     ` Mike Fedyk
  2003-07-24  4:04   ` Andre Tomt
  3 siblings, 1 reply; 17+ messages in thread
From: jimis @ 2003-07-23 22:17 UTC (permalink / raw)
  To: linux-kernel

Valdis.Kletnieks@vt.edu wrote:
> On Wed, 23 Jul 2003 13:57:41 +0300, jimis@gmx.net  said:
> 
>>1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits of 
>>bandwidth available. What I want to do is listen to icecast radio via xmms (at
>>22 kbits), download the kernel sources with wget, and browse the web at the same
>>time. Currently I think that this is *impossible* (correct me if I'm wrong) as
>>the radio will be full of pauses and the browsing experience painfully slow.
> 
> 
> Basically, you're stuck.  The biggest part of the problem is that although you
> can certainly control the outbound packets, you have no real control over when
> inbound packets arrive at the other end of your dial-up.  One person suggested

But the incoming packets are those that I request to be sent to me (most times). 
When I know the packet size I will accept I may not request all packets now, but 
a bit later, to leave bandwidth for other requested packets of higher priority.

> using QoS to help things along - but that needs to be implemented at the OTHER
> end of the dial-up - which means unless your provider does QoS on the terminal
> server, you're basically stuck.  Packets will probably just get queued up in
> order of arrival.

This is what I don't like, but I bet there must be a way to slow down the other 
end, even if it has not QoS.

What might help, and needs only be implemented on my side of the connection, is 
requesting the packets of higher priorities first. And if I know my total 
bandwidth, perhaps I can request as many packets needed to fill it (and not 
flood it like it happens now).

Thank you all for your attention,
Dimitris




^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 22:17   ` jimis
@ 2003-07-24  0:58     ` Mike Fedyk
  0 siblings, 0 replies; 17+ messages in thread
From: Mike Fedyk @ 2003-07-24  0:58 UTC (permalink / raw)
  To: jimis; +Cc: linux-kernel

On Thu, Jul 24, 2003 at 01:17:57AM +0300, jimis@gmx.net wrote:
> What might help, and needs only be implemented on my side of the 
> connection, is requesting the packets of higher priorities first. And if I 
> know my total bandwidth, perhaps I can request as many packets needed to 
> fill it (and not flood it like it happens now).

Yes, this is exactly what wondershaper does now.

Please give it a try.

I use it on a dsl line, and it works wonders.  (pardon the pun).

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 14:17 ` Valdis.Kletnieks
                     ` (2 preceding siblings ...)
  2003-07-23 22:17   ` jimis
@ 2003-07-24  4:04   ` Andre Tomt
  3 siblings, 0 replies; 17+ messages in thread
From: Andre Tomt @ 2003-07-24  4:04 UTC (permalink / raw)
  To: Valdis.Kletnieks; +Cc: jimis, linux-kernel

On ons, 2003-07-23 at 16:17, Valdis.Kletnieks@vt.edu wrote:
> On Wed, 23 Jul 2003 13:57:41 +0300, jimis@gmx.net  said:
> > 1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits of 
> > bandwidth available. What I want to do is listen to icecast radio via xmms (at
> > 22 kbits), download the kernel sources with wget, and browse the web at the same
> > time. Currently I think that this is *impossible* (correct me if I'm wrong) as
> > the radio will be full of pauses and the browsing experience painfully slow.
> 
> Basically, you're stuck.  The biggest part of the problem is that although you
> can certainly control the outbound packets, you have no real control over when
> inbound packets arrive at the other end of your dial-up.

Take a look at http://trash.net/~kaber/imq/
Does wonders here, both local and forwarded traffic.

-- 
Cheers,
André Tomt
andre@tomt.net


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-23 11:43 ` Pavel Machek
@ 2003-07-25 20:24   ` Marcelo Tosatti
  2003-07-28  9:29     ` Pavel Machek
  0 siblings, 1 reply; 17+ messages in thread
From: Marcelo Tosatti @ 2003-07-25 20:24 UTC (permalink / raw)
  To: Pavel Machek; +Cc: jimis, linux-kernel



On Wed, 23 Jul 2003, Pavel Machek wrote:

> Hi!
>
> > With the current scheduler we can prioritize the CPU usage for each
> > process. What I think would be extremely useful (as I have needed it
> > many times) is the scheduling of disk I/O and net I/O traffic. 2
> > examples showing the importance (the numbers are estimations just to
> > explain whati I mean):
>
> Yes that would be nice, and in 2.5 timeframe
> there was patch doing that. Port it to 2.6 an test it!

Pavel,

Do you remember who wrote those or where one can find it?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
  2003-07-25 20:24   ` Marcelo Tosatti
@ 2003-07-28  9:29     ` Pavel Machek
  0 siblings, 0 replies; 17+ messages in thread
From: Pavel Machek @ 2003-07-28  9:29 UTC (permalink / raw)
  To: Marcelo Tosatti, cb-lkml; +Cc: Pavel Machek, jimis, linux-kernel

Hi!

> > > With the current scheduler we can prioritize the CPU usage for each
> > > process. What I think would be extremely useful (as I have needed it
> > > many times) is the scheduling of disk I/O and net I/O traffic. 2
> > > examples showing the importance (the numbers are estimations just to
> > > explain whati I mean):
> >
> > Yes that would be nice, and in 2.5 timeframe
> > there was patch doing that. Port it to 2.6 an test it!
>
> Do you remember who wrote those or where one can find it?

After a bit of searching I found this. I'm not 100% sure this is the
same one I am remembering, but it looks like that.

http://marc.theaimsgroup.com/?l=linux-kernel&m=103962160319984&w=2

[No, I'm probably remembering some other patch, this looks way too
simple, but may be good point to start...]
							Pavel

-- 
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* RE: Feature proposal (scheduling related)
@ 2003-07-23 12:42 Frederick, Fabian
  0 siblings, 0 replies; 17+ messages in thread
From: Frederick, Fabian @ 2003-07-23 12:42 UTC (permalink / raw)
  To: jimis, linux-kernel

Jimis,

	I have been thinking about some /etc/nice file where kernel could
pick up process nice
when doing the sys_execve stuff.Maybe it could help ...

Regards,
Fabian

-----Message d'origine-----
De : jimis@gmx.net [mailto:jimis@gmx.net]
Envoyé : mercredi 23 juillet 2003 12:58
À : linux-kernel@vger.kernel.org
Objet : Feature proposal (scheduling related)


With the current scheduler we can prioritize the CPU usage for each process.

What I think would be extremely useful (as I have needed it many times) is
the 
scheduling of disk I/O and net I/O traffic. 2 examples showing the
importance 
(the numbers are estimations just to explain whati I mean):

1)I 'm connected to the internet via dial-up, therefore I only have 40 kbits
of 
bandwidth available. What I want to do is listen to icecast radio via xmms
(at 
22 kbits), download the kernel sources with wget, and browse the web at the
same 
time. Currently I think that this is *impossible* (correct me if I'm wrong)
as 
the radio will be full of pauses and the browsing experience painfully slow.

What I would like to be able to do (let's suppose nice has the --net option
to 
set net I/O priority):
$ nice --net -1 xmms
$ nice --net 1 wget ftp://.../KernelSources.tar.bz2
$ mozilla
This way, xmms which has top priority whould always get the 22kbits it
needs. 
What remains should go to the browser when I ask for a web page, and when
the 
browser doesn't request anything (let's say I'm reading a big doc in tldp)
what 
remains should go to wget. Wget has lower priority and won't irritate the 
browsing experience, though the file will be downloaded when there is free 
bandwidth.

2) Normally mozilla starts in 5 seconds after intense disk I/O to load all 
needed libraries. If I run in the background a long disk intense process
(like find / -name 'whatever' -xdev) loading mozilla could need 20 boring 
seconds, or doing other simple tasks might be irritating slow. What I would
like 
to be able to do is (once again let's suppose nice has the --disk option to
set 
disk I/O priority):
$ nice --disk 1 find / -name 'whatever' -xdev
$ mozilla
and load mozilla ,which has the default disk priority 0, fast. The scheduler

should give to mozilla most disk troughput when it needs it.

Notes:
1) PLEASE CC REPLIES BACK TO ME since I 'm not subscribed to the list (I
can't 
stand the traffic). However I 'll be checking periodically the list via
NNTP.
2) As I have no idea of kernel programming I hope what I propose is
aplicable 
and relevant to the kernel, as I believe. Sorry if not.
3) I hope what I propose is implementable using the existing scheduler. It
would 
be nice to have one scheduler to handle them all.
4) I believe that these features don't need a lot of CPU power because the
disk 
and net I/O troughput are relatively slow.
5) If you think that UNIX tradition forbits what I propose I must say that
these 
features could be invisible to programs, setting the new  priorities to the 
default 0 or maybe to the same number as CPU priority.

Thank you very much for your time,
Dimitris


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Feature proposal (scheduling related)
       [not found] <cpvY.4hH.25@gated-at.bofh.it>
@ 2003-07-23 11:54 ` Ihar "Philips" Filipau
  0 siblings, 0 replies; 17+ messages in thread
From: Ihar "Philips" Filipau @ 2003-07-23 11:54 UTC (permalink / raw)
  To: jimis; +Cc: linux-kernel

jimis@gmx.net wrote:
> With the current scheduler we can prioritize the CPU usage for each 
> process. What I think would be extremely useful (as I have needed it 
> many times) is the scheduling of disk I/O and net I/O traffic. 2 
> examples showing the importance (the numbers are estimations just to 
> explain whati I mean):
> 

    can address only your first problem.

> 1)I 'm connected to the internet via dial-up, therefore I only have 40 
> kbits of bandwidth available. What I want to do is listen to icecast 
> radio via xmms (at 22 kbits), download the kernel sources with wget, and 
> browse the web at the same time. Currently I think that this is 
> *impossible* (correct me if I'm wrong) as the radio will be full of 
> pauses and the browsing experience painfully slow. What I would like to 
> be able to do (let's suppose nice has the --net option to set net I/O 
> priority):
> $ nice --net -1 xmms
> $ nice --net 1 wget ftp://.../KernelSources.tar.bz2
> $ mozilla
> This way, xmms which has top priority whould always get the 22kbits it 
> needs. What remains should go to the browser when I ask for a web page, 
> and when the browser doesn't request anything (let's say I'm reading a 
> big doc in tldp) what remains should go to wget. Wget has lower priority 
> and won't irritate the browsing experience, though the file will be 
> downloaded when there is free bandwidth.
> 

    this is already in kernel and called Quality of Service (QoS) or 
traffic shaping.
    command is called /sbin/tc. (located in iproute.rpm in RH)
    instead of man page (because there is no man page) I can recommend 
to look in Internet.
    e.g. http://users.belgacom.net/staf/ -> QoS intro
    also you can try to "rpm -ql $(which tc)" - probably your distor has 
some docs.


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2003-07-28  9:14 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2003-07-23 10:57 Feature proposal (scheduling related) jimis
2003-07-23 11:24 ` David M. Wilson
2003-07-23 11:43 ` Pavel Machek
2003-07-25 20:24   ` Marcelo Tosatti
2003-07-28  9:29     ` Pavel Machek
2003-07-23 14:17 ` Valdis.Kletnieks
2003-07-23 14:23   ` Alan Cox
2003-07-23 15:10     ` Richard B. Johnson
2003-07-23 15:13       ` Antonio Vargas
2003-07-23 16:55         ` Disconnect
2003-07-23 17:22           ` David S. Miller
2003-07-23 14:47   ` Greg Stark
2003-07-23 22:17   ` jimis
2003-07-24  0:58     ` Mike Fedyk
2003-07-24  4:04   ` Andre Tomt
     [not found] <cpvY.4hH.25@gated-at.bofh.it>
2003-07-23 11:54 ` Ihar "Philips" Filipau
2003-07-23 12:42 Frederick, Fabian

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).