* [LARTC] HTB: Problem with excess bandwidth distribution
@ 2004-10-28 16:55 Leslie Patrick Polzer
2004-10-28 17:20 ` Saad S. B. Faruque
` (11 more replies)
0 siblings, 12 replies; 13+ messages in thread
From: Leslie Patrick Polzer @ 2004-10-28 16:55 UTC (permalink / raw)
To: lartc
Hello,
I have a serious problem with HTB which I wasn't able to solve myself.
I run a masquerading router with ppp0 as interface to the Internet.
Three clients need to share a downstream of 1 MBit, which I want
to divide with tc.
When I see a packet being forwarded to one of these clients, I give
it the appropriate unique mark:
iptables -t mangle -A FORWARD -d 192.168.34.141 -j MARK --set-mark 1
iptables -t mangle -A FORWARD -d 192.168.34.140 -j MARK --set-mark 2
iptables -t mangle -A FORWARD -d 192.168.1.2 -j MARK --set-mark 3
Because it might be of interest: 192.168.34.0/24 is on network A
with 10 MBit, 192.168.1.0/24 is on network B with 100 MBit.
I then attach an IMQ device imq0 to the FORWARD table:
# delegate all incoming on ppp+ to imq0
iptables -t mangle -A FORWARD -i ppp+ -j IMQ --todev 0
After all this I create the actual tc setup:
# --- snip ---
# clear root qdisc
tc qdisc del dev imq0 root
# add root qdisc (htb)
tc qdisc add dev imq0 root handle 1: htb default 40
# add root class (needed for bandwidth borrowing)
tc class add dev imq0 parent 1: classid 1:1 htb rate 1mbit ceil 1mbit
# set classes for users
tc class add dev imq0 parent 1:1 classid 1:10 htb rate 333kbit ceil 1mbit \
burst 15k
tc class add dev imq0 parent 1:1 classid 1:20 htb rate 333kbit ceil 1mbit \
burst 15k
tc class add dev imq0 parent 1:1 classid 1:30 htb rate 333kbit ceil 1mbit \
burst 15k
tc class add dev imq0 parent 1:1 classid 1:40 htb rate 5kbps
# set filters to direct ips to their classes
tc filter add dev imq0 protocol ip parent 1: prio 1 handle 1 fw flowid 1:10
tc filter add dev imq0 protocol ip parent 1: prio 1 handle 2 fw flowid 1:20
tc filter add dev imq0 protocol ip parent 1: prio 1 handle 3 fw flowid 1:30
# --- snap ---
1:40 is just for testing.
The 'rate'-argument gets applied correctly if I don't use ceil - but I
do, of
course, want to let the classes borrow free bandwidth, so I use a ceiling
of 1 MBit. And herein lies the problem:
If 1:10 and 1:30 both download a file with full speed, 1:10 gets about
20kb/s (which is under its guaranteed bandwidth!) and 1:30 gets
90 kb/s. What is going wrong here? The shortened output of tc:
class htb 1:1 root rate 1Mbit ceil 1Mbit burst 2909b/8 mpu 0b cburst
2909b/8 mpu 0b level 7
class htb 1:10 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
burst 15Kb/8 mpu 0b cburst
class htb 1:20 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
burst 15Kb/8 mpu 0b cburst
class htb 1:30 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
burst 15Kb/8 mpu 0b cburst
class htb 1:40 parent 1:1 prio 0 quantum 1000 rate 40Kbit ceil 40Kbit
burst 1650b/8 mpu 0b cburst
...shows that each class is configured equal.
Any clues? I'd be very, very grateful if anyone could point out errors.
If more output is needed, just tell me.
Kind regards,
Leslie
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
@ 2004-10-28 17:20 ` Saad S. B. Faruque
2004-10-28 17:45 ` Zviad O. Giorgadze
` (10 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Saad S. B. Faruque @ 2004-10-28 17:20 UTC (permalink / raw)
To: lartc
did u try it with sfq ?
On Thu, 28 Oct 2004 18:55:00 +0200, Leslie Patrick Polzer
<leslie.polzer@gmx.net> wrote:
> Hello,
>
> I have a serious problem with HTB which I wasn't able to solve myself.
>
> I run a masquerading router with ppp0 as interface to the Internet.
> Three clients need to share a downstream of 1 MBit, which I want
> to divide with tc.
> When I see a packet being forwarded to one of these clients, I give
> it the appropriate unique mark:
>
> iptables -t mangle -A FORWARD -d 192.168.34.141 -j MARK --set-mark 1
> iptables -t mangle -A FORWARD -d 192.168.34.140 -j MARK --set-mark 2
> iptables -t mangle -A FORWARD -d 192.168.1.2 -j MARK --set-mark 3
>
> Because it might be of interest: 192.168.34.0/24 is on network A
> with 10 MBit, 192.168.1.0/24 is on network B with 100 MBit.
>
> I then attach an IMQ device imq0 to the FORWARD table:
>
> # delegate all incoming on ppp+ to imq0
> iptables -t mangle -A FORWARD -i ppp+ -j IMQ --todev 0
>
> After all this I create the actual tc setup:
>
> # --- snip ---
> # clear root qdisc
> tc qdisc del dev imq0 root
>
> # add root qdisc (htb)
> tc qdisc add dev imq0 root handle 1: htb default 40
>
> # add root class (needed for bandwidth borrowing)
> tc class add dev imq0 parent 1: classid 1:1 htb rate 1mbit ceil 1mbit
>
> # set classes for users
> tc class add dev imq0 parent 1:1 classid 1:10 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:20 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:30 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:40 htb rate 5kbps
>
> # set filters to direct ips to their classes
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 1 fw flowid 1:10
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 2 fw flowid 1:20
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 3 fw flowid 1:30
>
> # --- snap ---
>
> 1:40 is just for testing.
>
> The 'rate'-argument gets applied correctly if I don't use ceil - but I
> do, of
> course, want to let the classes borrow free bandwidth, so I use a ceiling
> of 1 MBit. And herein lies the problem:
>
> If 1:10 and 1:30 both download a file with full speed, 1:10 gets about
> 20kb/s (which is under its guaranteed bandwidth!) and 1:30 gets
> 90 kb/s. What is going wrong here? The shortened output of tc:
>
> class htb 1:1 root rate 1Mbit ceil 1Mbit burst 2909b/8 mpu 0b cburst
> 2909b/8 mpu 0b level 7
> class htb 1:10 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
> burst 15Kb/8 mpu 0b cburst
> class htb 1:20 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
> burst 15Kb/8 mpu 0b cburst
> class htb 1:30 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
> burst 15Kb/8 mpu 0b cburst
> class htb 1:40 parent 1:1 prio 0 quantum 1000 rate 40Kbit ceil 40Kbit
> burst 1650b/8 mpu 0b cburst
>
> ...shows that each class is configured equal.
>
> Any clues? I'd be very, very grateful if anyone could point out errors.
> If more output is needed, just tell me.
>
> Kind regards,
>
> Leslie
>
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
--
Saad S. B. Faruque
MCSE, RHCT, CCNA
Head of NOC
MTL BD Ltd.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
2004-10-28 17:20 ` Saad S. B. Faruque
@ 2004-10-28 17:45 ` Zviad O. Giorgadze
2004-10-28 22:04 ` Andy Furniss
` (9 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Zviad O. Giorgadze @ 2004-10-28 17:45 UTC (permalink / raw)
To: lartc
Hello Leslie,
I had the same problem for kernel 2.4.27 and it was related to bug in HTB.
Use kernel >=2.6.8.1 or apply patch from Devik's site
http://luxik.cdi.cz/~devik/qos/htb/v3/htbfair.diff.
Best regards,
Zviad
>Hello,
>
>I have a serious problem with HTB which I wasn't able to solve myself.
>
>I run a masquerading router with ppp0 as interface to the Internet.
>Three clients need to share a downstream of 1 MBit, which I want
>to divide with tc.
>When I see a packet being forwarded to one of these clients, I give
>it the appropriate unique mark:
>
>iptables -t mangle -A FORWARD -d 192.168.34.141 -j MARK --set-mark 1
>iptables -t mangle -A FORWARD -d 192.168.34.140 -j MARK --set-mark 2
>iptables -t mangle -A FORWARD -d 192.168.1.2 -j MARK --set-mark 3
>
>Because it might be of interest: 192.168.34.0/24 is on network A
>with 10 MBit, 192.168.1.0/24 is on network B with 100 MBit.
>
>I then attach an IMQ device imq0 to the FORWARD table:
>
># delegate all incoming on ppp+ to imq0
>iptables -t mangle -A FORWARD -i ppp+ -j IMQ --todev 0
>
>After all this I create the actual tc setup:
>
># --- snip ---
># clear root qdisc
> tc qdisc del dev imq0 root
>
># add root qdisc (htb)
> tc qdisc add dev imq0 root handle 1: htb default 40
>
># add root class (needed for bandwidth borrowing)
> tc class add dev imq0 parent 1: classid 1:1 htb rate 1mbit ceil 1mbit
>
># set classes for users
> tc class add dev imq0 parent 1:1 classid 1:10 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:20 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:30 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:40 htb rate 5kbps
>
># set filters to direct ips to their classes
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 1 fw flowid 1:10
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 2 fw flowid 1:20
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 3 fw flowid 1:30
>
># --- snap ---
>
>1:40 is just for testing.
>
>The 'rate'-argument gets applied correctly if I don't use ceil - but I
>do, of
>course, want to let the classes borrow free bandwidth, so I use a ceiling
>of 1 MBit. And herein lies the problem:
>
>If 1:10 and 1:30 both download a file with full speed, 1:10 gets about
>20kb/s (which is under its guaranteed bandwidth!) and 1:30 gets
>90 kb/s. What is going wrong here? The shortened output of tc:
>
>class htb 1:1 root rate 1Mbit ceil 1Mbit burst 2909b/8 mpu 0b cburst
>2909b/8 mpu 0b level 7
>class htb 1:10 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
>burst 15Kb/8 mpu 0b cburst
>class htb 1:20 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
>burst 15Kb/8 mpu 0b cburst
>class htb 1:30 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
>burst 15Kb/8 mpu 0b cburst
>class htb 1:40 parent 1:1 prio 0 quantum 1000 rate 40Kbit ceil 40Kbit
>burst 1650b/8 mpu 0b cburst
>
>...shows that each class is configured equal.
>
>Any clues? I'd be very, very grateful if anyone could point out errors.
>If more output is needed, just tell me.
>
>
>Kind regards,
>
>Leslie
>
>_______________________________________________
>LARTC mailing list / LARTC@mailman.ds9a.nl
>http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
2004-10-28 17:20 ` Saad S. B. Faruque
2004-10-28 17:45 ` Zviad O. Giorgadze
@ 2004-10-28 22:04 ` Andy Furniss
2004-10-29 6:11 ` Leslie Patrick Polzer
` (8 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Andy Furniss @ 2004-10-28 22:04 UTC (permalink / raw)
To: lartc
Leslie Patrick Polzer wrote:
> Hello,
>
> I have a serious problem with HTB which I wasn't able to solve myself.
>
> I run a masquerading router with ppp0 as interface to the Internet.
> Three clients need to share a downstream of 1 MBit, which I want
> to divide with tc.
> When I see a packet being forwarded to one of these clients, I give
> it the appropriate unique mark:
>
> iptables -t mangle -A FORWARD -d 192.168.34.141 -j MARK --set-mark 1
> iptables -t mangle -A FORWARD -d 192.168.34.140 -j MARK --set-mark 2
> iptables -t mangle -A FORWARD -d 192.168.1.2 -j MARK --set-mark 3
>
> Because it might be of interest: 192.168.34.0/24 is on network A
> with 10 MBit, 192.168.1.0/24 is on network B with 100 MBit.
>
> I then attach an IMQ device imq0 to the FORWARD table:
You can't use IMQ in forward AFAIK, see
http://www.docum.org/docum.org/kptd/
You can use it in prerouting, but because you are doing NAT you will
need to select for after NAT in the new IMQ from www.linuximq.net or
patch for NAT if you want to use an older IMQ. You can't mark on de
natted IPs in prerouting so you need to use u32.
Shaping from the narrow end of the bottleneck is a bit of a kludge, you
have to set your rates/ceils lower than link speed or you won't have a
queue to shape with.
If you don't want to have a more complicated script to mark interactive
packets/use prio etc. I would add 30K bfifos to each class - or if you
don't mind patching/tweaking use esfq/sfq with a queue length of about
20, not that these figures are set in stone - but the defaults for htb
with no queue added or untweaked sfq are alot longer.
Andy.
>
> # delegate all incoming on ppp+ to imq0
> iptables -t mangle -A FORWARD -i ppp+ -j IMQ --todev 0
>
> After all this I create the actual tc setup:
>
> # --- snip ---
> # clear root qdisc
> tc qdisc del dev imq0 root
>
> # add root qdisc (htb)
> tc qdisc add dev imq0 root handle 1: htb default 40
>
> # add root class (needed for bandwidth borrowing)
> tc class add dev imq0 parent 1: classid 1:1 htb rate 1mbit ceil 1mbit
>
> # set classes for users
> tc class add dev imq0 parent 1:1 classid 1:10 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:20 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:30 htb rate 333kbit ceil 1mbit \
> burst 15k
> tc class add dev imq0 parent 1:1 classid 1:40 htb rate 5kbps
>
> # set filters to direct ips to their classes
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 1 fw flowid 1:10
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 2 fw flowid 1:20
> tc filter add dev imq0 protocol ip parent 1: prio 1 handle 3 fw flowid 1:30
>
> # --- snap ---
>
> 1:40 is just for testing.
>
> The 'rate'-argument gets applied correctly if I don't use ceil - but I
> do, of
> course, want to let the classes borrow free bandwidth, so I use a ceiling
> of 1 MBit. And herein lies the problem:
>
> If 1:10 and 1:30 both download a file with full speed, 1:10 gets about
> 20kb/s (which is under its guaranteed bandwidth!) and 1:30 gets
> 90 kb/s. What is going wrong here? The shortened output of tc:
>
> class htb 1:1 root rate 1Mbit ceil 1Mbit burst 2909b/8 mpu 0b cburst
> 2909b/8 mpu 0b level 7
> class htb 1:10 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
> burst 15Kb/8 mpu 0b cburst
> class htb 1:20 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
> burst 15Kb/8 mpu 0b cburst
> class htb 1:30 parent 1:1 prio 0 quantum 4262 rate 333Kbit ceil 1Mbit
> burst 15Kb/8 mpu 0b cburst
> class htb 1:40 parent 1:1 prio 0 quantum 1000 rate 40Kbit ceil 40Kbit
> burst 1650b/8 mpu 0b cburst
>
> ...shows that each class is configured equal.
>
> Any clues? I'd be very, very grateful if anyone could point out errors.
> If more output is needed, just tell me.
>
>
> Kind regards,
>
> Leslie
>
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (2 preceding siblings ...)
2004-10-28 22:04 ` Andy Furniss
@ 2004-10-29 6:11 ` Leslie Patrick Polzer
2004-10-29 11:17 ` Leslie Patrick Polzer
` (7 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Leslie Patrick Polzer @ 2004-10-29 6:11 UTC (permalink / raw)
To: lartc
Andy Furniss wrote:
> Leslie Patrick Polzer wrote:
>
>> Hello,
>>
>> I have a serious problem with HTB which I wasn't able to solve myself.
>>
>> I run a masquerading router with ppp0 as interface to the Internet.
>> Three clients need to share a downstream of 1 MBit, which I want
>> to divide with tc.
>> When I see a packet being forwarded to one of these clients, I give
>> it the appropriate unique mark:
>>
>> iptables -t mangle -A FORWARD -d 192.168.34.141 -j MARK --set-mark 1
>> iptables -t mangle -A FORWARD -d 192.168.34.140 -j MARK --set-mark 2
>> iptables -t mangle -A FORWARD -d 192.168.1.2 -j MARK --set-mark 3
>>
>> Because it might be of interest: 192.168.34.0/24 is on network A
>> with 10 MBit, 192.168.1.0/24 is on network B with 100 MBit.
>>
>> I then attach an IMQ device imq0 to the FORWARD table:
>
>
> You can't use IMQ in forward AFAIK, see
>
> http://www.docum.org/docum.org/kptd/
Hmmm, really?
I mean, all intended packets are going through it, no errors
whatsoever. They are being marked correctly by iptables
and tc filter classifies according to mark. The only problem
seems to be the excess bandwidth distribution, which
leaves me to the question:
How could the hooks of IMQ and the excess bandwidth
distribution of HTB relate in this setup?
I hope you are understanding that I do not question your
knowledge. I'm just not fully persuaded of this yet, so I'd
like to discuss it a bit more.
And thanks a lot for the additional information you gave me!
Kind regards,
Leslie
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (3 preceding siblings ...)
2004-10-29 6:11 ` Leslie Patrick Polzer
@ 2004-10-29 11:17 ` Leslie Patrick Polzer
2004-10-29 15:29 ` Andy Furniss
` (6 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Leslie Patrick Polzer @ 2004-10-29 11:17 UTC (permalink / raw)
To: lartc
Andy Furniss wrote:
> Shaping from the narrow end of the bottleneck is a bit of a kludge,
> you have to set your rates/ceils lower than link speed or you won't
> have a queue to shape with.
>
Could you also elaborate this a bit further?
Many thanks so far!
Leslie
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (4 preceding siblings ...)
2004-10-29 11:17 ` Leslie Patrick Polzer
@ 2004-10-29 15:29 ` Andy Furniss
2004-10-29 15:36 ` Leslie Patrick Polzer
` (5 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Andy Furniss @ 2004-10-29 15:29 UTC (permalink / raw)
To: lartc
Leslie Patrick Polzer wrote:
> Andy Furniss wrote:
>
>> Leslie Patrick Polzer wrote:
>>
>>> Hello,
>>>
>>> I have a serious problem with HTB which I wasn't able to solve myself.
>>>
>>> I run a masquerading router with ppp0 as interface to the Internet.
>>> Three clients need to share a downstream of 1 MBit, which I want
>>> to divide with tc.
>>> When I see a packet being forwarded to one of these clients, I give
>>> it the appropriate unique mark:
>>>
>>> iptables -t mangle -A FORWARD -d 192.168.34.141 -j MARK --set-mark 1
>>> iptables -t mangle -A FORWARD -d 192.168.34.140 -j MARK --set-mark 2
>>> iptables -t mangle -A FORWARD -d 192.168.1.2 -j MARK --set-mark 3
>>>
>>> Because it might be of interest: 192.168.34.0/24 is on network A
>>> with 10 MBit, 192.168.1.0/24 is on network B with 100 MBit.
>>>
>>> I then attach an IMQ device imq0 to the FORWARD table:
>>
>>
>>
>> You can't use IMQ in forward AFAIK, see
>>
>> http://www.docum.org/docum.org/kptd/
>
>
> Hmmm, really?
> I mean, all intended packets are going through it, no errors
> whatsoever. They are being marked correctly by iptables
> and tc filter classifies according to mark. The only problem
> seems to be the excess bandwidth distribution, which
> leaves me to the question:
>
> How could the hooks of IMQ and the excess bandwidth
> distribution of HTB relate in this setup?
>
> I hope you are understanding that I do not question your
> knowledge. I'm just not fully persuaded of this yet, so I'd
> like to discuss it a bit more.
>
You are right to question me :-) - I was thinking a bit too much about
my setup (At least I know that works). I use IMQ on ppp so I can shape
traffic headed for local processes as well as forwarded. If you don't
need to do this then you don't need to do it in prerouting anyway.
I am guessing that calling IMQ from forward uses postrouting which is OK
for your needs. I know from a test I did in prerouting that IMQ doesn't
respect where in a table it gets called from. You could test by seeing
if you can shape locally generated traffic marked in output I suppose.
Wherever it hooks you need to set a rate less than link speed and if you
use an old kernel, patch HTB. I said shaping from the wrong end of the
bottleneck is a kludge because if I shape from the fat end then I
control exactly what happens - I can arrange for my latency never to be
increased by more than the time it takes for a packet my MTU long to be
sent at my bitrate. As long as I tweak for link overheads I can use
nearly 100% bandwidth.
Incoming traffic from my ISP has already been through a 600ms fifo -
it's never going to arrive at more than my link speed, so I need to set
the ceils/rate totals to less than link speed - how much less will
determine how fast the queue fills. The behavior of various types of
queues is probably not the same as if they were at the other end of the
bottleneck.
There are also factors out of my control - TCP can get bursty when acks
get buffered elsewhere. There may be packets in long buffers (mainly
P2P) headed for me which are unstoppable, and my queue may not have any
packets from active connections at any given time. The queue also reacts
too late when the bandwidth changes - A new connection will be in TCP
slowstart, which quite quickly will increase rate causing a temporary
filling of ISP buffer - which hurts latency. It doesn't fill enough to
cause drops, though, so as far as bandwidth allocation goes it's OK.
My queues also drop a bit too much when this happens - causing TCP to
resync which can be bursty.
Andy.
> And thanks a lot for the additional information you gave me!
>
>
> Kind regards,
>
> Leslie
>
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (5 preceding siblings ...)
2004-10-29 15:29 ` Andy Furniss
@ 2004-10-29 15:36 ` Leslie Patrick Polzer
2004-10-29 16:19 ` Jason Boxman
` (4 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Leslie Patrick Polzer @ 2004-10-29 15:36 UTC (permalink / raw)
To: lartc
Still problems :(
I upgraded to kernel 2.6.9 now, configured IMQ to hook itself up after
NAT, called it
from prerouting, used u32 (matching works), set the root class to a rate
of 800kBit
(which is 200 less than my link speed) - and the behavior gets even worse :(
Unfortunately, I cannot shape on the outgoing interfaces either, because
there are two.
I really don't know what to do now... I haven't dug deep into CBQ yet -
should I try it?
Leslie
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (6 preceding siblings ...)
2004-10-29 15:36 ` Leslie Patrick Polzer
@ 2004-10-29 16:19 ` Jason Boxman
2004-10-29 17:40 ` Francisco Pereira
` (3 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Jason Boxman @ 2004-10-29 16:19 UTC (permalink / raw)
To: lartc
On Friday 29 October 2004 11:36, Leslie Patrick Polzer wrote:
> Still problems :(
>
> I upgraded to kernel 2.6.9 now, configured IMQ to hook itself up after
> NAT, called it
> from prerouting, used u32 (matching works), set the root class to a rate
> of 800kBit
> (which is 200 less than my link speed) - and the behavior gets even worse
> :(
Define worse? What metric are you using to measure the behavior?
> Unfortunately, I cannot shape on the outgoing interfaces either, because
> there are two.
Wouldn't IMQ work for this too?
> I really don't know what to do now... I haven't dug deep into CBQ yet -
> should I try it?
CBQ won't magically work over multiple interfaces without something like IMQ,
just like HTB.
--
Jason Boxman
Perl Programmer / *NIX Systems Administrator
Shimberg Center for Affordable Housing | University of Florida
http://edseek.com/ - Linux and FOSS stuff
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (7 preceding siblings ...)
2004-10-29 16:19 ` Jason Boxman
@ 2004-10-29 17:40 ` Francisco Pereira
2004-10-29 22:03 ` Andy Furniss
` (2 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: Francisco Pereira @ 2004-10-29 17:40 UTC (permalink / raw)
To: lartc
Quoting Leslie Patrick Polzer <leslie.polzer@gmx.net>:
> Still problems :(
>
> I upgraded to kernel 2.6.9 now, configured IMQ to hook itself up after
> NAT, called it
> from prerouting, used u32 (matching works), set the root class to a rate
> of 800kBit
> (which is 200 less than my link speed) - and the behavior gets even worse :(
>
> Unfortunately, I cannot shape on the outgoing interfaces either, because
> there are two.
Have you tried putting another machine as a bridge? (You dont need the IMQ in
this case)
-------------------------------------------------------------
Elecciones Nacionales 2004
Consulte en el Portal donde votar
http://www.montevideo.com.uy/elecciones2004
-------------------------------------------------------------
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (8 preceding siblings ...)
2004-10-29 17:40 ` Francisco Pereira
@ 2004-10-29 22:03 ` Andy Furniss
2004-10-29 23:24 ` Andy Furniss
2004-10-31 12:03 ` Andy Furniss
11 siblings, 0 replies; 13+ messages in thread
From: Andy Furniss @ 2004-10-29 22:03 UTC (permalink / raw)
To: lartc
Leslie Patrick Polzer wrote:
> Still problems :(
>
> I upgraded to kernel 2.6.9 now, configured IMQ to hook itself up after
> NAT, called it
> from prerouting, used u32 (matching works), set the root class to a rate
> of 800kBit
> (which is 200 less than my link speed) - and the behavior gets even
> worse :(
>
> Unfortunately, I cannot shape on the outgoing interfaces either, because
> there are two.
>
> I really don't know what to do now... I haven't dug deep into CBQ yet -
> should I try it?
Hmm - this should work. I just cobbled together a test - It's not very
elegant because it's based on a slightly different setup, but it works
for me. I use default as my local traffic has a dynamic IP - you don't
need to . Note the U32 filters are attached to 1:0 if I attached them to
1:1 than I would need a rule to send traffic to 1:1.
I wouldn't trust the output of apps for bandwidth tests - their
averaging can be confusing - also if it weren't just a test I would add
queues to the classes. Saying that I did notice that HTB was dropping -
maybe the default queue length is shorter now? It does seem a bit
strange though, I see drops where I expect the queue to be long enough
for my rwin and a class with two tcps on the go had less drops than one
with one - strange.
It did work though use tc -s class ls dev imq0 to see rates (which for
me using the new TC seem to be shown in the wrong units).
You may need to unwrap the lines if you copy n paste this:
set -x
IPTABLES=/usr/local/sbin/iptables
MODPROBE=/sbin/modprobe
IP=/sbin/ip
TC=/sbin/tc
$IPTABLES -t mangle -D PREROUTING -i ppp0 -j IMQ --todev 0 &> /dev/null
$IP link set imq0 down &> /dev/null
$MODPROBE -r imq &> /dev/null
if [ "$1" = "stop" ]
then
echo "stopping"
exit
fi
$MODPROBE imq numdevs=1
$IPTABLES -t mangle -I PREROUTING -i ppp0 -j IMQ --todev 0
$IP link set imq0 up
$TC qdisc add dev imq0 root handle 1:0 htb default 34
$TC class add dev imq0 parent 1:0 classid 1:1 htb rate 400kbit ceil
400kbit burst 6k
#### 1 ####
$TC class add dev imq0 parent 1:1 classid 1:32 htb rate 133kbit ceil
400kbit prio 1
$TC filter add dev imq0 protocol ip parent 1:0 prio 1 u32 match ip dst
192.168.0.2 flowid 1:32
#### 2 ####
$TC class add dev imq0 parent 1:1 classid 1:33 htb rate 133kbit ceil 400kbit
$TC filter add dev imq0 protocol ip parent 1:0 prio 1 u32 match ip dst
192.168.0.3 flowid 1:33
#### Default = traffic for local process ####
$TC class add dev imq0 parent 1:1 classid 1:34 htb rate 133kbit ceil 400kbit
Andy.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (9 preceding siblings ...)
2004-10-29 22:03 ` Andy Furniss
@ 2004-10-29 23:24 ` Andy Furniss
2004-10-31 12:03 ` Andy Furniss
11 siblings, 0 replies; 13+ messages in thread
From: Andy Furniss @ 2004-10-29 23:24 UTC (permalink / raw)
To: lartc
Andy Furniss wrote:
>
> #### 1 ####
> $TC class add dev imq0 parent 1:1 classid 1:32 htb rate 133kbit ceil
> 400kbit prio 1
I meant to delete the prio 1 - I don't know if it matters - I tested
with the other two.
Andy.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [LARTC] HTB: Problem with excess bandwidth distribution
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
` (10 preceding siblings ...)
2004-10-29 23:24 ` Andy Furniss
@ 2004-10-31 12:03 ` Andy Furniss
11 siblings, 0 replies; 13+ messages in thread
From: Andy Furniss @ 2004-10-31 12:03 UTC (permalink / raw)
To: lartc
Andy Furniss wrote:
> Saying that I did notice that HTB was dropping -
> maybe the default queue length is shorter now? It does seem a bit
> strange though, I see drops where I expect the queue to be long enough
> for my rwin and a class with two tcps on the go had less drops than one
> with one - strange.
I took another look at this and it's because the default queue length of
the default class is shorter than the default for a normal class.
Andy.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2004-10-31 12:03 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-10-28 16:55 [LARTC] HTB: Problem with excess bandwidth distribution Leslie Patrick Polzer
2004-10-28 17:20 ` Saad S. B. Faruque
2004-10-28 17:45 ` Zviad O. Giorgadze
2004-10-28 22:04 ` Andy Furniss
2004-10-29 6:11 ` Leslie Patrick Polzer
2004-10-29 11:17 ` Leslie Patrick Polzer
2004-10-29 15:29 ` Andy Furniss
2004-10-29 15:36 ` Leslie Patrick Polzer
2004-10-29 16:19 ` Jason Boxman
2004-10-29 17:40 ` Francisco Pereira
2004-10-29 22:03 ` Andy Furniss
2004-10-29 23:24 ` Andy Furniss
2004-10-31 12:03 ` Andy Furniss
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.