Not hitting rated speed on HTB downstream

Not hitting rated speed on HTB downstream

[Horace]

Hi,

Internet <------> (eth1) Router (bond0.10) <-----> switch <------------------> direct connected clients
                                                          <-- 10mbit link ---> clients (2ms latency)


I've got a very strange problem on tc htb downstream (bond0.10), which I cannot attain rated speed for a htb class 1:2. However, I can attain the rated speed for class 1:3 for the same setting (direct connected clients). I'm almost suspecting it is a hardware related issue, but when I changed the 1:2 class ceil value to 30mbit, I can attain the full 10mbit (granted it is a 10mbit down link). 

I've also tried the settings with upstream on eth1, no problem at all on both 1:2 and 1:3. Seems the problem only appears on downstream. I don't have any policy on the ingress queue right now. And also nothing on the iptables.

Tried turn off tso/gso, enlarge bond0.10 txqueue, set clocksource to hpet, set burst to 20-200k, not much difference.

Any suggestion how to solve this problem?

Here's tc-htb config
==========
tc qdisc del dev bond0.10 root
tc qdisc add dev bond0.10 root handle 1: htb default 4
tc class add dev bond0.10 parent 1: classid 1:1 htb rate 1000Mbit prio 1
tc class add dev bond0.10 parent 1:1 classid 1:2 htb rate 10mbit ceil 10mbit prio 1 
tc class add dev bond0.10 parent 1:1 classid 1:3 htb rate 10mbit ceil 10mbit prio 1
tc class add dev bond0.10 parent 1:1 classid 1:4 htb rate 1000mbit prio 2

tc qdisc add dev bond0.10 parent 1:2 pfifo
tc qdisc add dev bond0.10 parent 1:3 pfifo
tc qdisc add dev bond0.10 parent 1:4 pfifo

tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
match ip dst 1.1.1.8/29 flowid 1:2
tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
match ip dst 1.1.1.81 flowid 1:3

Here's the stats
========
class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 7 
 Sent 71611952 bytes 127507 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8919Kbit 1633pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 16 ctokens: 16

class htb 1:2 parent 1:1 leaf 8828: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0 
 Sent 18463384 bytes 20911 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 4270Kbit 398pps backlog 0b 1p requeues 0 
 lended: 16930 borrowed: 0 giants: 0
 tokens: -18459 ctokens: -18459             <======== is it not supposed to be negative when the rate is not more than 10mbit?

class htb 1:3 parent 1:1 leaf 8829: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0 
 Sent 9921 bytes 61 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 672bit 0pps backlog 0b 0p requeues 0 
 lended: 61 borrowed: 0 giants: 0
 tokens: 16610 ctokens: 16610

class htb 1:4 parent 1:1 leaf 882a: prio 2 quantum 200000 rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 0 
 Sent 53138757 bytes 106536 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 4647Kbit 1235pps backlog 0b 0p requeues 0 
 lended: 97706 borrowed: 0 giants: 0
 tokens: 187 ctokens: 187



Horace Ng

Re: Not hitting rated speed on HTB downstream

[Remy Mudingay]

Hi,

Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4. 
Secondly, I recommend lowering/setting the ceil value to 10mbit (or slightly lower).




Sent from my iPhone 

On Aug 13, 2013, at 6:35, Horace <horace@zmail.hkisl.net> wrote:

> Hi,
> 
> Internet <------> (eth1) Router (bond0.10) <-----> switch <------------------> direct connected clients
>                                                          <-- 10mbit link ---> clients (2ms latency)
> 
> 
> I've got a very strange problem on tc htb downstream (bond0.10), which I cannot attain rated speed for a htb class 1:2. However, I can attain the rated speed for class 1:3 for the same setting (direct connected clients). I'm almost suspecting it is a hardware related issue, but when I changed the 1:2 class ceil value to 30mbit, I can attain the full 10mbit (granted it is a 10mbit down link). 
> 
> I've also tried the settings with upstream on eth1, no problem at all on both 1:2 and 1:3. Seems the problem only appears on downstream. I don't have any policy on the ingress queue right now. And also nothing on the iptables.
> 
> Tried turn off tso/gso, enlarge bond0.10 txqueue, set clocksource to hpet, set burst to 20-200k, not much difference.
> 
> Any suggestion how to solve this problem?
> 
> Here's tc-htb config
> ==========
> tc qdisc del dev bond0.10 root
> tc qdisc add dev bond0.10 root handle 1: htb default 4
> tc class add dev bond0.10 parent 1: classid 1:1 htb rate 1000Mbit prio 1
> tc class add dev bond0.10 parent 1:1 classid 1:2 htb rate 10mbit ceil 10mbit prio 1 
> tc class add dev bond0.10 parent 1:1 classid 1:3 htb rate 10mbit ceil 10mbit prio 1
> tc class add dev bond0.10 parent 1:1 classid 1:4 htb rate 1000mbit prio 2
> 
> tc qdisc add dev bond0.10 parent 1:2 pfifo
> tc qdisc add dev bond0.10 parent 1:3 pfifo
> tc qdisc add dev bond0.10 parent 1:4 pfifo
> 
> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
> match ip dst 1.1.1.8/29 flowid 1:2
> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
> match ip dst 1.1.1.81 flowid 1:3
> 
> Here's the stats
> ========
> class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 7 
> Sent 71611952 bytes 127507 pkt (dropped 0, overlimits 0 requeues 0) 
> rate 8919Kbit 1633pps backlog 0b 0p requeues 0 
> lended: 0 borrowed: 0 giants: 0
> tokens: 16 ctokens: 16
> 
> class htb 1:2 parent 1:1 leaf 8828: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0 
> Sent 18463384 bytes 20911 pkt (dropped 0, overlimits 0 requeues 0) 
> rate 4270Kbit 398pps backlog 0b 1p requeues 0 
> lended: 16930 borrowed: 0 giants: 0
> tokens: -18459 ctokens: -18459             <======== is it not supposed to be negative when the rate is not more than 10mbit?
> 
> class htb 1:3 parent 1:1 leaf 8829: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0 
> Sent 9921 bytes 61 pkt (dropped 0, overlimits 0 requeues 0) 
> rate 672bit 0pps backlog 0b 0p requeues 0 
> lended: 61 borrowed: 0 giants: 0
> tokens: 16610 ctokens: 16610
> 
> class htb 1:4 parent 1:1 leaf 882a: prio 2 quantum 200000 rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 0 
> Sent 53138757 bytes 106536 pkt (dropped 0, overlimits 0 requeues 0) 
> rate 4647Kbit 1235pps backlog 0b 0p requeues 0 
> lended: 97706 borrowed: 0 giants: 0
> tokens: 187 ctokens: 187
> 
> 
> 
> Horace Ng
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Not hitting rated speed on HTB downstream

[Wolfgang Hennerbichler]

On Aug 13, 2013, at 17:57 , Remy Mudingay <remy.mudingay@gmail.com> wrote:

> Hi,
> 
> Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4. 

This is a question I had on my mind for a long time: what if I want to oversubscribe (a lot of) clients? Will this cause any troubles? Why do you recommend doing this? 

Wolfgang

Re: Not hitting rated speed on HTB downstream

[Mike Schmidt]

On Tue, Aug 13, 2013 at 3:04 PM, Wolfgang Hennerbichler <wogri@wogri.com> wrote:
> On Aug 13, 2013, at 17:57 , Remy Mudingay <remy.mudingay@gmail.com> wrote:
>
>> Hi,
>>
>> Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4.
>
> This is a question I had on my mind for a long time: what if I want to oversubscribe (a lot of) clients? Will this cause any troubles? Why do you recommend doing this?
>
> Wolfgang--

Generally, if you oversubscribe the children (i.e. sum(rates of
children)  > rate of parent class) in htb, you basically kill the
bandwidth management of the parent, since it can no longer manage the
children's throughput. I personally set all the children to a low
value for rate, but give them a high ceiling, this way the parent has
a lot of flexibility. Don't forget that the 'rate' in htb is
essentially a guaranteed throughput. However, I find HFSC more
flexible, although somewhat harder to understand.

-- 
Mike SCHMIDT
CTO
Intello Technologies Inc.
mike.schmidt@intello.com
Canada: 1-888-404-6261 x320
USA: 1-888-404-6268 x320
Mobile: 514-409-6898
www.intello.com

Re: Not hitting rated speed on HTB downstream

[Wolfgang Hennerbichler]

-- 
http://www.wogri.at

On Aug 13, 2013, at 22:25 , Mike Schmidt <mike.schmidt@intello.com> wrote:

> On Tue, Aug 13, 2013 at 3:04 PM, Wolfgang Hennerbichler <wogri@wogri.com> wrote:
>> On Aug 13, 2013, at 17:57 , Remy Mudingay <remy.mudingay@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4.
>> 
>> This is a question I had on my mind for a long time: what if I want to oversubscribe (a lot of) clients? Will this cause any troubles? Why do you recommend doing this?
>> 
>> Wolfgang--
> 
> Generally, if you oversubscribe the children (i.e. sum(rates of
> children)  > rate of parent class) in htb, you basically kill the
> bandwidth management of the parent, since it can no longer manage the
> children's throughput. I personally set all the children to a low
> value for rate, but give them a high ceiling, this way the parent has
> a lot of flexibility. Don't forget that the 'rate' in htb is
> essentially a guaranteed throughput. However, I find HFSC more
> flexible, although somewhat harder to understand.

finding proper HFSC doc is a problem for me…
thanks for clarifying.

> -- 
> Mike SCHMIDT
> CTO
> Intello Technologies Inc.
> mike.schmidt@intello.com
> Canada: 1-888-404-6261 x320
> USA: 1-888-404-6268 x320
> Mobile: 514-409-6898
> www.intello.com

Re: Not hitting rated speed on HTB downstream

[Horace]

I've tried to modify classid 1:4 to 500mbit, seems no difference. What do you mean by lower the ceil value?

Horace Ng
-
Internet Solutions Limited
E-mail: horace@hkisl.net
Tel: +852 27109880
Fax: +852 27704631

----- Original Message -----
From: "Remy Mudingay" <remy.mudingay@gmail.com>
To: "Horace Ng" <horace@hkisl.net>
Cc: lartc@vger.kernel.org
Sent: Tuesday, August 13, 2013 11:57:29 PM
Subject: Re: Not hitting rated speed on HTB downstream

Hi,

Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4. 
Secondly, I recommend lowering/setting the ceil value to 10mbit (or slightly lower).




Sent from my iPhone 

On Aug 13, 2013, at 6:35, Horace <horace@zmail.hkisl.net> wrote:

> Hi,
> 
> Internet <------> (eth1) Router (bond0.10) <-----> switch <------------------> direct connected clients
>                                                          <-- 10mbit link ---> clients (2ms latency)
> 
> 
> I've got a very strange problem on tc htb downstream (bond0.10), which I cannot attain rated speed for a htb class 1:2. However, I can attain the rated speed for class 1:3 for the same setting (direct connected clients). I'm almost suspecting it is a hardware related issue, but when I changed the 1:2 class ceil value to 30mbit, I can attain the full 10mbit (granted it is a 10mbit down link). 
> 
> I've also tried the settings with upstream on eth1, no problem at all on both 1:2 and 1:3. Seems the problem only appears on downstream. I don't have any policy on the ingress queue right now. And also nothing on the iptables.
> 
> Tried turn off tso/gso, enlarge bond0.10 txqueue, set clocksource to hpet, set burst to 20-200k, not much difference.
> 
> Any suggestion how to solve this problem?
> 
> Here's tc-htb config
> ==========
> tc qdisc del dev bond0.10 root
> tc qdisc add dev bond0.10 root handle 1: htb default 4
> tc class add dev bond0.10 parent 1: classid 1:1 htb rate 1000Mbit prio 1
> tc class add dev bond0.10 parent 1:1 classid 1:2 htb rate 10mbit ceil 10mbit prio 1 
> tc class add dev bond0.10 parent 1:1 classid 1:3 htb rate 10mbit ceil 10mbit prio 1
> tc class add dev bond0.10 parent 1:1 classid 1:4 htb rate 1000mbit prio 2
> 
> tc qdisc add dev bond0.10 parent 1:2 pfifo
> tc qdisc add dev bond0.10 parent 1:3 pfifo
> tc qdisc add dev bond0.10 parent 1:4 pfifo
> 
> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
> match ip dst 1.1.1.8/29 flowid 1:2
> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
> match ip dst 1.1.1.81 flowid 1:3
> 
> Here's the stats
> ========
> class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 7 
> Sent 71611952 bytes 127507 pkt (dropped 0, overlimits 0 requeues 0) 
> rate 8919Kbit 1633pps backlog 0b 0p requeues 0 
> lended: 0 borrowed: 0 giants: 0
> tokens: 16 ctokens: 16
> 
> class htb 1:2 parent 1:1 leaf 8828: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0 
> Sent 18463384 bytes 20911 pkt (dropped 0, overlimits 0 requeues 0) 
> rate 4270Kbit 398pps backlog 0b 1p requeues 0 
> lended: 16930 borrowed: 0 giants: 0
> tokens: -18459 ctokens: -18459             <======== is it not supposed to be negative when the rate is not more than 10mbit?
> 
> class htb 1:3 parent 1:1 leaf 8829: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0 
> Sent 9921 bytes 61 pkt (dropped 0, overlimits 0 requeues 0) 
> rate 672bit 0pps backlog 0b 0p requeues 0 
> lended: 61 borrowed: 0 giants: 0
> tokens: 16610 ctokens: 16610
> 
> class htb 1:4 parent 1:1 leaf 882a: prio 2 quantum 200000 rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 0 
> Sent 53138757 bytes 106536 pkt (dropped 0, overlimits 0 requeues 0) 
> rate 4647Kbit 1235pps backlog 0b 0p requeues 0 
> lended: 97706 borrowed: 0 giants: 0
> tokens: 187 ctokens: 187
> 
> 
> 
> Horace Ng
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Not hitting rated speed on HTB downstream

[Horace]

For HFSC, I think this is a good read. I'll also try to implement it to test the throughput on my current set up.

http://linux-ip.net/articles/hfsc.en/

Horace Ng
-
Internet Solutions Limited
E-mail: horace@hkisl.net
Tel: +852 27109880
Fax: +852 27704631

----- Original Message -----
From: "Wolfgang Hennerbichler" <wogri@wogri.com>
To: "Mike Schmidt" <mike.schmidt@intello.com>
Cc: lartc@vger.kernel.org
Sent: Wednesday, August 14, 2013 4:33:24 AM
Subject: Re: Not hitting rated speed on HTB downstream


-- 
http://www.wogri.at

On Aug 13, 2013, at 22:25 , Mike Schmidt <mike.schmidt@intello.com> wrote:

> On Tue, Aug 13, 2013 at 3:04 PM, Wolfgang Hennerbichler <wogri@wogri.com> wrote:
>> On Aug 13, 2013, at 17:57 , Remy Mudingay <remy.mudingay@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4.
>> 
>> This is a question I had on my mind for a long time: what if I want to oversubscribe (a lot of) clients? Will this cause any troubles? Why do you recommend doing this?
>> 
>> Wolfgang--
> 
> Generally, if you oversubscribe the children (i.e. sum(rates of
> children)  > rate of parent class) in htb, you basically kill the
> bandwidth management of the parent, since it can no longer manage the
> children's throughput. I personally set all the children to a low
> value for rate, but give them a high ceiling, this way the parent has
> a lot of flexibility. Don't forget that the 'rate' in htb is
> essentially a guaranteed throughput. However, I find HFSC more
> flexible, although somewhat harder to understand.

finding proper HFSC doc is a problem for me…
thanks for clarifying.

> -- 
> Mike SCHMIDT
> CTO
> Intello Technologies Inc.
> mike.schmidt@intello.com
> Canada: 1-888-404-6261 x320
> USA: 1-888-404-6268 x320
> Mobile: 514-409-6898
> www.intello.com

--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Not hitting rated speed on HTB downstream

[Remy Mudingay]

HI Horace,

Correct me if I miss understood your network topology - Your maximum
download rate should be set to 10mbit because that is your limit.
Therefore your shaping rules for your default class should also
reflect this by setting the ceil to 10mbit also. Your class statistics
show that 4270Kbit (4.3mbits) are being shaped by classid 1:2, classid
1:3 has 672bit and the default classid (1:4) has 4647Kbit(4.6mbit).
So roughly half of your network traffic if going into your default
class (1:4) and since your default class has a guaranteed rate of
1Gbit...

If you choose to go the HFSC route, it is important to note that HFSC
is more complicated to setup than HTB and it will for example drop any
packet which has not been explicitly classified, whereas HTB will
either bypass (direct_packet_stat) or shape these packets in the
default class.

I highly recommend that you identify the following :

 a) what is going through your default class (classid 1:4)?
#This can be achieved by using the mirred action filter. firstly,
modify this line tc qdisc add dev bond0.10 parent 1:4 pfifo to the
following
tc qdisc add dev bond0.10 parent 1:4 handly 4:1 pfifo

#Next add a dummy interface
modprobe dummy
#Then add a classification rule
tc filter add dev bon0.10 prio 1 protocol all parent 4:1 action mirred
egress mirror dev dummy0
#run tcpdump
tcpdump -i dummy0

Now you have identified the network that is using up half your of your
total bandwidth.

b) The rate sum of classid 1:2, 1:3 and 1:4 should not be greater than 10mbit.

I hope this helps.

Remy

On 15 August 2013 03:59, Horace <horace@zmail.hkisl.net> wrote:
> I've tried to modify classid 1:4 to 500mbit, seems no difference. What do you mean by lower the ceil value?
>
> Horace Ng
> -
> Internet Solutions Limited
> E-mail: horace@hkisl.net
> Tel: +852 27109880
> Fax: +852 27704631
>
> ----- Original Message -----
> From: "Remy Mudingay" <remy.mudingay@gmail.com>
> To: "Horace Ng" <horace@hkisl.net>
> Cc: lartc@vger.kernel.org
> Sent: Tuesday, August 13, 2013 11:57:29 PM
> Subject: Re: Not hitting rated speed on HTB downstream
>
> Hi,
>
> Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4.
> Secondly, I recommend lowering/setting the ceil value to 10mbit (or slightly lower).
>
>
>
>
> Sent from my iPhone
>
> On Aug 13, 2013, at 6:35, Horace <horace@zmail.hkisl.net> wrote:
>
>> Hi,
>>
>> Internet <------> (eth1) Router (bond0.10) <-----> switch <------------------> direct connected clients
>>                                                          <-- 10mbit link ---> clients (2ms latency)
>>
>>
>> I've got a very strange problem on tc htb downstream (bond0.10), which I cannot attain rated speed for a htb class 1:2. However, I can attain the rated speed for class 1:3 for the same setting (direct connected clients). I'm almost suspecting it is a hardware related issue, but when I changed the 1:2 class ceil value to 30mbit, I can attain the full 10mbit (granted it is a 10mbit down link).
>>
>> I've also tried the settings with upstream on eth1, no problem at all on both 1:2 and 1:3. Seems the problem only appears on downstream. I don't have any policy on the ingress queue right now. And also nothing on the iptables.
>>
>> Tried turn off tso/gso, enlarge bond0.10 txqueue, set clocksource to hpet, set burst to 20-200k, not much difference.
>>
>> Any suggestion how to solve this problem?
>>
>> Here's tc-htb config
>> ==========
>> tc qdisc del dev bond0.10 root
>> tc qdisc add dev bond0.10 root handle 1: htb default 4
>> tc class add dev bond0.10 parent 1: classid 1:1 htb rate 1000Mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:2 htb rate 10mbit ceil 10mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:3 htb rate 10mbit ceil 10mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:4 htb rate 1000mbit prio 2
>>
>> tc qdisc add dev bond0.10 parent 1:2 pfifo
>> tc qdisc add dev bond0.10 parent 1:3 pfifo
>> tc qdisc add dev bond0.10 parent 1:4 pfifo
>>
>> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
>> match ip dst 1.1.1.8/29 flowid 1:2
>> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
>> match ip dst 1.1.1.81 flowid 1:3
>>
>> Here's the stats
>> ========
>> class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 7
>> Sent 71611952 bytes 127507 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 8919Kbit 1633pps backlog 0b 0p requeues 0
>> lended: 0 borrowed: 0 giants: 0
>> tokens: 16 ctokens: 16
>>
>> class htb 1:2 parent 1:1 leaf 8828: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0
>> Sent 18463384 bytes 20911 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 4270Kbit 398pps backlog 0b 1p requeues 0
>> lended: 16930 borrowed: 0 giants: 0
>> tokens: -18459 ctokens: -18459             <======== is it not supposed to be negative when the rate is not more than 10mbit?
>>
>> class htb 1:3 parent 1:1 leaf 8829: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0
>> Sent 9921 bytes 61 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 672bit 0pps backlog 0b 0p requeues 0
>> lended: 61 borrowed: 0 giants: 0
>> tokens: 16610 ctokens: 16610
>>
>> class htb 1:4 parent 1:1 leaf 882a: prio 2 quantum 200000 rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 0
>> Sent 53138757 bytes 106536 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 4647Kbit 1235pps backlog 0b 0p requeues 0
>> lended: 97706 borrowed: 0 giants: 0
>> tokens: 187 ctokens: 187
>>
>>
>>
>> Horace Ng
>> --
>> To unsubscribe from this list: send the line "unsubscribe lartc" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Not hitting rated speed on HTB downstream

[Horace]

Hi Mudingay,

My actual maximum throughput is 1Gbps, I just merely try to chunk out 10mbps for one of my client. As I mentioned before, if I only modify the ceil value to 30mbit for 1:2 class, it can actually attain full 10mbit for that link, nothing else need to be modified.
One thing that I noticed is that the 1:2 class sometimes got a negative token sum when transferring at 4mbps average, which from my understanding, it is either bursting too much (not really, the physical line speed is only 10mbps), or somehow the token is just not adding fast enough for 10mbit!! (clocksource problem? tried hpet already)

Horace Ng
-
Internet Solutions Limited
E-mail: horace@hkisl.net
Tel: +852 27109880
Fax: +852 27704631

----- Original Message -----
From: "Remy Mudingay" <remy.mudingay@gmail.com>
To: "Horace Ng" <horace@hkisl.net>
Cc: lartc@vger.kernel.org
Sent: Thursday, August 15, 2013 2:23:12 PM
Subject: Re: Not hitting rated speed on HTB downstream

HI Horace,

Correct me if I miss understood your network topology - Your maximum
download rate should be set to 10mbit because that is your limit.
Therefore your shaping rules for your default class should also
reflect this by setting the ceil to 10mbit also. Your class statistics
show that 4270Kbit (4.3mbits) are being shaped by classid 1:2, classid
1:3 has 672bit and the default classid (1:4) has 4647Kbit(4.6mbit).
So roughly half of your network traffic if going into your default
class (1:4) and since your default class has a guaranteed rate of
1Gbit...

If you choose to go the HFSC route, it is important to note that HFSC
is more complicated to setup than HTB and it will for example drop any
packet which has not been explicitly classified, whereas HTB will
either bypass (direct_packet_stat) or shape these packets in the
default class.

I highly recommend that you identify the following :

 a) what is going through your default class (classid 1:4)?
#This can be achieved by using the mirred action filter. firstly,
modify this line tc qdisc add dev bond0.10 parent 1:4 pfifo to the
following
tc qdisc add dev bond0.10 parent 1:4 handly 4:1 pfifo

#Next add a dummy interface
modprobe dummy
#Then add a classification rule
tc filter add dev bon0.10 prio 1 protocol all parent 4:1 action mirred
egress mirror dev dummy0
#run tcpdump
tcpdump -i dummy0

Now you have identified the network that is using up half your of your
total bandwidth.

b) The rate sum of classid 1:2, 1:3 and 1:4 should not be greater than 10mbit.

I hope this helps.

Remy

On 15 August 2013 03:59, Horace <horace@zmail.hkisl.net> wrote:
> I've tried to modify classid 1:4 to 500mbit, seems no difference. What do you mean by lower the ceil value?
>
> Horace Ng
> -
> Internet Solutions Limited
> E-mail: horace@hkisl.net
> Tel: +852 27109880
> Fax: +852 27704631
>
> ----- Original Message -----
> From: "Remy Mudingay" <remy.mudingay@gmail.com>
> To: "Horace Ng" <horace@hkisl.net>
> Cc: lartc@vger.kernel.org
> Sent: Tuesday, August 13, 2013 11:57:29 PM
> Subject: Re: Not hitting rated speed on HTB downstream
>
> Hi,
>
> Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4.
> Secondly, I recommend lowering/setting the ceil value to 10mbit (or slightly lower).
>
>
>
>
> Sent from my iPhone
>
> On Aug 13, 2013, at 6:35, Horace <horace@zmail.hkisl.net> wrote:
>
>> Hi,
>>
>> Internet <------> (eth1) Router (bond0.10) <-----> switch <------------------> direct connected clients
>>                                                          <-- 10mbit link ---> clients (2ms latency)
>>
>>
>> I've got a very strange problem on tc htb downstream (bond0.10), which I cannot attain rated speed for a htb class 1:2. However, I can attain the rated speed for class 1:3 for the same setting (direct connected clients). I'm almost suspecting it is a hardware related issue, but when I changed the 1:2 class ceil value to 30mbit, I can attain the full 10mbit (granted it is a 10mbit down link).
>>
>> I've also tried the settings with upstream on eth1, no problem at all on both 1:2 and 1:3. Seems the problem only appears on downstream. I don't have any policy on the ingress queue right now. And also nothing on the iptables.
>>
>> Tried turn off tso/gso, enlarge bond0.10 txqueue, set clocksource to hpet, set burst to 20-200k, not much difference.
>>
>> Any suggestion how to solve this problem?
>>
>> Here's tc-htb config
>> ==========
>> tc qdisc del dev bond0.10 root
>> tc qdisc add dev bond0.10 root handle 1: htb default 4
>> tc class add dev bond0.10 parent 1: classid 1:1 htb rate 1000Mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:2 htb rate 10mbit ceil 10mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:3 htb rate 10mbit ceil 10mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:4 htb rate 1000mbit prio 2
>>
>> tc qdisc add dev bond0.10 parent 1:2 pfifo
>> tc qdisc add dev bond0.10 parent 1:3 pfifo
>> tc qdisc add dev bond0.10 parent 1:4 pfifo
>>
>> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
>> match ip dst 1.1.1.8/29 flowid 1:2
>> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
>> match ip dst 1.1.1.81 flowid 1:3
>>
>> Here's the stats
>> ========
>> class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 7
>> Sent 71611952 bytes 127507 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 8919Kbit 1633pps backlog 0b 0p requeues 0
>> lended: 0 borrowed: 0 giants: 0
>> tokens: 16 ctokens: 16
>>
>> class htb 1:2 parent 1:1 leaf 8828: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0
>> Sent 18463384 bytes 20911 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 4270Kbit 398pps backlog 0b 1p requeues 0
>> lended: 16930 borrowed: 0 giants: 0
>> tokens: -18459 ctokens: -18459             <======== is it not supposed to be negative when the rate is not more than 10mbit?
>>
>> class htb 1:3 parent 1:1 leaf 8829: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0
>> Sent 9921 bytes 61 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 672bit 0pps backlog 0b 0p requeues 0
>> lended: 61 borrowed: 0 giants: 0
>> tokens: 16610 ctokens: 16610
>>
>> class htb 1:4 parent 1:1 leaf 882a: prio 2 quantum 200000 rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 0
>> Sent 53138757 bytes 106536 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 4647Kbit 1235pps backlog 0b 0p requeues 0
>> lended: 97706 borrowed: 0 giants: 0
>> tokens: 187 ctokens: 187
>>
>>
>>
>> Horace Ng
>> --
>> To unsubscribe from this list: send the line "unsubscribe lartc" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Not hitting rated speed on HTB downstream

[Wolfgang Hennerbichler]

I read through it, and will also give it a try. Thanks for sharing the link. 

Wolfgang
-- 
http://www.wogri.at

On Aug 15, 2013, at 04:05 , Horace <horace@zmail.hkisl.net> wrote:

> For HFSC, I think this is a good read. I'll also try to implement it to test the throughput on my current set up.
> 
> http://linux-ip.net/articles/hfsc.en/
> 
> Horace Ng
> -
> Internet Solutions Limited
> E-mail: horace@hkisl.net
> Tel: +852 27109880
> Fax: +852 27704631
> 
> ----- Original Message -----
> From: "Wolfgang Hennerbichler" <wogri@wogri.com>
> To: "Mike Schmidt" <mike.schmidt@intello.com>
> Cc: lartc@vger.kernel.org
> Sent: Wednesday, August 14, 2013 4:33:24 AM
> Subject: Re: Not hitting rated speed on HTB downstream
> 
> 
> -- 
> http://www.wogri.at
> 
> On Aug 13, 2013, at 22:25 , Mike Schmidt <mike.schmidt@intello.com> wrote:
> 
>> On Tue, Aug 13, 2013 at 3:04 PM, Wolfgang Hennerbichler <wogri@wogri.com> wrote:
>>> On Aug 13, 2013, at 17:57 , Remy Mudingay <remy.mudingay@gmail.com> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4.
>>> 
>>> This is a question I had on my mind for a long time: what if I want to oversubscribe (a lot of) clients? Will this cause any troubles? Why do you recommend doing this?
>>> 
>>> Wolfgang--
>> 
>> Generally, if you oversubscribe the children (i.e. sum(rates of
>> children)  > rate of parent class) in htb, you basically kill the
>> bandwidth management of the parent, since it can no longer manage the
>> children's throughput. I personally set all the children to a low
>> value for rate, but give them a high ceiling, this way the parent has
>> a lot of flexibility. Don't forget that the 'rate' in htb is
>> essentially a guaranteed throughput. However, I find HFSC more
>> flexible, although somewhat harder to understand.
> 
> finding proper HFSC doc is a problem for me…
> thanks for clarifying.
> 
>> -- 
>> Mike SCHMIDT
>> CTO
>> Intello Technologies Inc.
>> mike.schmidt@intello.com
>> Canada: 1-888-404-6261 x320
>> USA: 1-888-404-6268 x320
>> Mobile: 514-409-6898
>> www.intello.com
> 
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Not hitting rated speed on HTB downstream

[Horace]

I think I will migrate the ingress traffic to IFB  as it mentioned in other thread and drop the egress shaping at the internal interface. Didn't know that IMQ was a thing of past and IFB is a successor to it. There's not many documentations about IMQ and IFB on the internet.

Horace Ng
-
Internet Solutions Limited
E-mail: horace@hkisl.net
Tel: +852 27109880
Fax: +852 27704631

----- Original Message -----
From: "Horace" <horace@zmail.hkisl.net>
To: "Remy Mudingay" <remy.mudingay@gmail.com>
Cc: lartc@vger.kernel.org
Sent: Thursday, August 15, 2013 3:39:22 PM
Subject: Re: Not hitting rated speed on HTB downstream

Hi Mudingay,

My actual maximum throughput is 1Gbps, I just merely try to chunk out 10mbps for one of my client. As I mentioned before, if I only modify the ceil value to 30mbit for 1:2 class, it can actually attain full 10mbit for that link, nothing else need to be modified.
One thing that I noticed is that the 1:2 class sometimes got a negative token sum when transferring at 4mbps average, which from my understanding, it is either bursting too much (not really, the physical line speed is only 10mbps), or somehow the token is just not adding fast enough for 10mbit!! (clocksource problem? tried hpet already)

Horace Ng
-
Internet Solutions Limited
E-mail: horace@hkisl.net
Tel: +852 27109880
Fax: +852 27704631

----- Original Message -----
From: "Remy Mudingay" <remy.mudingay@gmail.com>
To: "Horace Ng" <horace@hkisl.net>
Cc: lartc@vger.kernel.org
Sent: Thursday, August 15, 2013 2:23:12 PM
Subject: Re: Not hitting rated speed on HTB downstream

HI Horace,

Correct me if I miss understood your network topology - Your maximum
download rate should be set to 10mbit because that is your limit.
Therefore your shaping rules for your default class should also
reflect this by setting the ceil to 10mbit also. Your class statistics
show that 4270Kbit (4.3mbits) are being shaped by classid 1:2, classid
1:3 has 672bit and the default classid (1:4) has 4647Kbit(4.6mbit).
So roughly half of your network traffic if going into your default
class (1:4) and since your default class has a guaranteed rate of
1Gbit...

If you choose to go the HFSC route, it is important to note that HFSC
is more complicated to setup than HTB and it will for example drop any
packet which has not been explicitly classified, whereas HTB will
either bypass (direct_packet_stat) or shape these packets in the
default class.

I highly recommend that you identify the following :

 a) what is going through your default class (classid 1:4)?
#This can be achieved by using the mirred action filter. firstly,
modify this line tc qdisc add dev bond0.10 parent 1:4 pfifo to the
following
tc qdisc add dev bond0.10 parent 1:4 handly 4:1 pfifo

#Next add a dummy interface
modprobe dummy
#Then add a classification rule
tc filter add dev bon0.10 prio 1 protocol all parent 4:1 action mirred
egress mirror dev dummy0
#run tcpdump
tcpdump -i dummy0

Now you have identified the network that is using up half your of your
total bandwidth.

b) The rate sum of classid 1:2, 1:3 and 1:4 should not be greater than 10mbit.

I hope this helps.

Remy

On 15 August 2013 03:59, Horace <horace@zmail.hkisl.net> wrote:
> I've tried to modify classid 1:4 to 500mbit, seems no difference. What do you mean by lower the ceil value?
>
> Horace Ng
> -
> Internet Solutions Limited
> E-mail: horace@hkisl.net
> Tel: +852 27109880
> Fax: +852 27704631
>
> ----- Original Message -----
> From: "Remy Mudingay" <remy.mudingay@gmail.com>
> To: "Horace Ng" <horace@hkisl.net>
> Cc: lartc@vger.kernel.org
> Sent: Tuesday, August 13, 2013 11:57:29 PM
> Subject: Re: Not hitting rated speed on HTB downstream
>
> Hi,
>
> Firstly I'd change classid's rate so that the sum rate of all 1:1 child classes are not greater than its rate value. This is applicable to your default class 1:4.
> Secondly, I recommend lowering/setting the ceil value to 10mbit (or slightly lower).
>
>
>
>
> Sent from my iPhone
>
> On Aug 13, 2013, at 6:35, Horace <horace@zmail.hkisl.net> wrote:
>
>> Hi,
>>
>> Internet <------> (eth1) Router (bond0.10) <-----> switch <------------------> direct connected clients
>>                                                          <-- 10mbit link ---> clients (2ms latency)
>>
>>
>> I've got a very strange problem on tc htb downstream (bond0.10), which I cannot attain rated speed for a htb class 1:2. However, I can attain the rated speed for class 1:3 for the same setting (direct connected clients). I'm almost suspecting it is a hardware related issue, but when I changed the 1:2 class ceil value to 30mbit, I can attain the full 10mbit (granted it is a 10mbit down link).
>>
>> I've also tried the settings with upstream on eth1, no problem at all on both 1:2 and 1:3. Seems the problem only appears on downstream. I don't have any policy on the ingress queue right now. And also nothing on the iptables.
>>
>> Tried turn off tso/gso, enlarge bond0.10 txqueue, set clocksource to hpet, set burst to 20-200k, not much difference.
>>
>> Any suggestion how to solve this problem?
>>
>> Here's tc-htb config
>> ==========
>> tc qdisc del dev bond0.10 root
>> tc qdisc add dev bond0.10 root handle 1: htb default 4
>> tc class add dev bond0.10 parent 1: classid 1:1 htb rate 1000Mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:2 htb rate 10mbit ceil 10mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:3 htb rate 10mbit ceil 10mbit prio 1
>> tc class add dev bond0.10 parent 1:1 classid 1:4 htb rate 1000mbit prio 2
>>
>> tc qdisc add dev bond0.10 parent 1:2 pfifo
>> tc qdisc add dev bond0.10 parent 1:3 pfifo
>> tc qdisc add dev bond0.10 parent 1:4 pfifo
>>
>> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
>> match ip dst 1.1.1.8/29 flowid 1:2
>> tc filter add dev bond0.10 protocol ip parent 1: prio 1 u32 \
>> match ip dst 1.1.1.81 flowid 1:3
>>
>> Here's the stats
>> ========
>> class htb 1:1 root rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 7
>> Sent 71611952 bytes 127507 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 8919Kbit 1633pps backlog 0b 0p requeues 0
>> lended: 0 borrowed: 0 giants: 0
>> tokens: 16 ctokens: 16
>>
>> class htb 1:2 parent 1:1 leaf 8828: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0
>> Sent 18463384 bytes 20911 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 4270Kbit 398pps backlog 0b 1p requeues 0
>> lended: 16930 borrowed: 0 giants: 0
>> tokens: -18459 ctokens: -18459             <======== is it not supposed to be negative when the rate is not more than 10mbit?
>>
>> class htb 1:3 parent 1:1 leaf 8829: prio 1 quantum 125000 rate 10000Kbit ceil 10000Kbit burst 1600b/8 mpu 0b overhead 0b cburst 1600b/8 mpu 0b overhead 0b level 0
>> Sent 9921 bytes 61 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 672bit 0pps backlog 0b 0p requeues 0
>> lended: 61 borrowed: 0 giants: 0
>> tokens: 16610 ctokens: 16610
>>
>> class htb 1:4 parent 1:1 leaf 882a: prio 2 quantum 200000 rate 1000Mbit ceil 1000Mbit burst 1375b/8 mpu 0b overhead 0b cburst 1375b/8 mpu 0b overhead 0b level 0
>> Sent 53138757 bytes 106536 pkt (dropped 0, overlimits 0 requeues 0)
>> rate 4647Kbit 1235pps backlog 0b 0p requeues 0
>> lended: 97706 borrowed: 0 giants: 0
>> tokens: 187 ctokens: 187
>>
>>
>>
>> Horace Ng
>> --
>> To unsubscribe from this list: send the line "unsubscribe lartc" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Not hitting rated speed on HTB downstream

[Andrew Beverley]

On Wed, 2013-08-21 at 09:46 +0800, Horace wrote:
> Didn't know that IMQ was a thing of past and IFB is a successor to it.
> There's not many documentations about IMQ and IFB on the internet.

IMQ is still current. It provides the ability to hook into traffic
passing through the netfilter stack. It's not in the vanilla kernel and
from what I understand is unlikely to ever be. Disadvantage: requires
kernel patching.

IFB sits before netfilter, which provides less flexibility and means
that you can't mark packets to classify your traffic for shaping. It is,
however, in the vanilla kernel. Disadvantage: less flexibility.

So, depends on your use case, but if you can you can manage with the
reduced features then your preference should be IFB as no kernel
patching is required.

Re: Not hitting rated speed on HTB downstream

[Horace]

Thanks. What exactly need to be marked with iptables if you can simply use tc u32 filter instead? Just for stateful marking, i.e. NEW/ESTABLISHED? I suspect if there's any performance impact using u32 vs iptables.

Horace Ng
-
Internet Solutions Limited
E-mail: horace@hkisl.net
Tel: +852 27109880
Fax: +852 27704631

----- Original Message -----
From: "Andrew Beverley" <andy@andybev.com>
To: "Horace Ng" <horace@hkisl.net>
Cc: "Remy Mudingay" <remy.mudingay@gmail.com>, lartc@vger.kernel.org
Sent: Wednesday, August 21, 2013 2:29:11 PM
Subject: Re: Not hitting rated speed on HTB downstream

On Wed, 2013-08-21 at 09:46 +0800, Horace wrote:
> Didn't know that IMQ was a thing of past and IFB is a successor to it.
> There's not many documentations about IMQ and IFB on the internet.

IMQ is still current. It provides the ability to hook into traffic
passing through the netfilter stack. It's not in the vanilla kernel and
from what I understand is unlikely to ever be. Disadvantage: requires
kernel patching.

IFB sits before netfilter, which provides less flexibility and means
that you can't mark packets to classify your traffic for shaping. It is,
however, in the vanilla kernel. Disadvantage: less flexibility.

So, depends on your use case, but if you can you can manage with the
reduced features then your preference should be IFB as no kernel
patching is required.


--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Not hitting rated speed on HTB downstream

[Andrew Beverley]

On Fri, 2013-08-23 at 17:30 +0800, Horace wrote:
> IFB sits before netfilter, which provides less flexibility and means
> that you can't mark packets to classify your traffic for shaping. It is,
> however, in the vanilla kernel. Disadvantage: less flexibility.
>
> Thanks. What exactly need to be marked with iptables if you can simply
> use tc u32 filter instead? Just for stateful marking, i.e.
> NEW/ESTABLISHED?

Stateful marking is one advantage, but there are many more reasons, as
you have access to the whole range of iptables matching rules.

> BTW, I've tested with IFB ingress shaping, the problem still persists.
> I can see the tokens going negative even when the rate does not reach
> the limit.

We'll need some more information in order to help. Please post your
exact current set up and rules, and if possible narrow it down to a
particular instance that's showing the problem.

Andy