[LARTC] Need big buffer!

[LARTC] Need big buffer!

[Bob Puff@NLE]

Hi gang,

I have an application that is sending streaming media to a server.  The encoder computer sends an
average of 200kbit stream; but for short, 1 second bursts, can hit 400-500kbits.  After it does one
of these bursts, it reduces its output for another second or two, so that it maintains its average
of 200kbits.

The problem is that it is sending UDP packets into a DSL link that has a fixed upstream cap of
300kbit.  All is fine until it decides to burst, then I get dropped packets.

Is there a way I can insert a linux box after the encoding machine that I can use some traffic
shaping to:

1. Make sure it never sends more than 300k up (I do know how to do this now)
2. Make a big buffer so that I can still get all the data through the pipe, albeit with some delay,
when these bursts happen.  I would need this buffer to handle at least 2 seconds worth, maybe more.

I need something like this:

Time    IN       OUT
(secs)  (kbit)   (kbit)
----------------------
0.0	200	200
0.5	200	200
1.0	350	300
1.5	400	300
2.0	25	175
2.5	50	50
3.0	175     175
3.5	200     200
	

...etc...

Ideas?

Bob

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Need big buffer!

[Larry Brigman]

On 2/8/07, Bob Puff@NLE <bob@nleaudio.com> wrote:
> Hi gang,
>
> I have an application that is sending streaming media to a server.  The encoder computer sends an
> average of 200kbit stream; but for short, 1 second bursts, can hit 400-500kbits.  After it does one
> of these bursts, it reduces its output for another second or two, so that it maintains its average
> of 200kbits.
>
> The problem is that it is sending UDP packets into a DSL link that has a fixed upstream cap of
> 300kbit.  All is fine until it decides to burst, then I get dropped packets.
>
> Is there a way I can insert a linux box after the encoding machine that I can use some traffic
> shaping to:
>
> 1. Make sure it never sends more than 300k up (I do know how to do this now)
> 2. Make a big buffer so that I can still get all the data through the pipe, albeit with some delay,
> when these bursts happen.  I would need this buffer to handle at least 2 seconds worth, maybe more.
>
If you know how to do the 300k limit then using that same method add a
queue depth to handle the difference on that class.
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Need big buffer!

[Bob Puff]

> If you know how to do the 300k limit then using that same method add 
> a queue depth to handle the difference on that class.

How do I do that?  (sorry, newbie to tc)

Bob

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Need big buffer!

[Bob Puff@NLE]

Bob Puff wrote:

> 
>>If you know how to do the 300k limit then using that same method add 
>>a queue depth to handle the difference on that class.
> 
> 
> How do I do that?  (sorry, newbie to tc)
> 
> Bob

I've been trying to read up, and still not coming up with concrete info on queue sizes.
Right now, my code for limiting to 300k is:

tc qdisc add dev eth0 root handle 1: htb default 21
tc class add dev eth0 parent 1: classid 1:1 htb rate 300kbit
tc class add dev eth0 parent 1:1 classid 1:20 htb prio 0 rate 100kbit
tc class add dev eth0 parent 1:1 classid 1:21 htb prio 1 rate 100kbit ceil 300k

..with some matches for prioritizing other traffic into class 1:20.

I assume there is something I need to add to the first line, but everything I've read about never
mentions htb.

Bob

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Need big buffer!

[Martin A. Brown]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings Bob,

 : I've been trying to read up, and still not coming up with 
 : concrete info on queue sizes. Right now, my code for limiting to 
 : 300k is:
 : 
 : tc qdisc add dev eth0 root handle 1: htb default 21
 : tc class add dev eth0 parent 1: classid 1:1 htb rate 300kbit
 : tc class add dev eth0 parent 1:1 classid 1:20 htb prio 0 rate 100kbit
 : tc class add dev eth0 parent 1:1 classid 1:21 htb prio 1 rate 100kbit ceil 300k
 : 
 : ..with some matches for prioritizing other traffic into class 
 : 1:20.
 : 
 : I assume there is something I need to add to the first line, but 
 : everything I've read about never mentions htb.

Here are pointers to HTB documentation that I have written [0], and, 
of course, the author's own documentation [1].  Stef Coene, who used 
to be extraordinarily active on this list has some useful (if not 
currently maintained) documentation at [2].  See also Leonardo 
Balliache's thorough examination of traffic control under Linux [3].

As I see it, your problem is not that you haven't found the right 
HTB setting, but rather that you haven't embedded a (b)FIFO in the 
right place.

Try adding a fifo of the appropriate size (in bytes) to the class 
which is handling your bursty traffic.  This FIFO will hold your 
queued packets while HTB dequeues them at the ceil rate.  Trial and 
error will probably help you determine the optimal depth of the FIFO 
for your purposes.

   tc qdisc add dev eth0 parent 1:21 bfifo limit 256000

Best of luck,

- -Martin

 [0] 
 [1] http://luxik.cdi.cz/~devik/qos/htb/
 [2] http://www.docum.org/docum.org/
 [3] http://www.opalsoft.net/qos/DS-28.htm

- -- 
Martin A. Brown
http://linux-ip.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)

iD8DBQFFzJqeHEoZD1iZ+YcRAvFyAJ9ARFRk02h1tY0COJnHvEvHs1HkwgCgpQwF
9AWk9kTyPSGmdxPtFYpFpGg=Y0gF
-----END PGP SIGNATURE-----
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc