RE: Qdisc HTB root overlimits and requeues

["Dzmitry Shykuts" <dshykuts@brest.beltelecom.by>]

Now have such a tc stat:

#tc -s qdisc show dev eth0
qdisc htb 1: root refcnt 6 r2q 10 default 12 direct_packets_stat 95 direct_qlen 1000
 Sent 1391798728 bytes 1862214 pkt (dropped 37906, overlimits 2006475 requeues 122)
 backlog 0b 0p requeues 122
qdisc sfq 11: parent 1:11 limit 127p quantum 1514b depth 127 divisor 2048 perturb 10sec
 Sent 905146766 bytes 842124 pkt (dropped 37656, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc sfq 12: parent 1:12 limit 127p quantum 1514b depth 127 divisor 2048 perturb 10sec
 Sent 486520788 bytes 1019985 pkt (dropped 223, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc ingress ffff: parent ffff:fff1 ----------------
 Sent 1735656945 bytes 2071052 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

#tc -s class show dev eth0
class htb 1:11 parent 1:1 leaf 11: prio 1 rate 30000Kbit ceil 60000Kbit burst 1593b cburst 1590b
 Sent 923690013 bytes 879301 pkt (dropped 25090, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 570597 borrowed: 284812 giants: 0
 tokens: 6164 ctokens: 3082

class htb 1:1 root rate 60000Kbit ceil 60000Kbit burst 1590b cburst 1590b
 Sent 1473354142 bytes 2046438 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 284812 borrowed: 0 giants: 0
 tokens: 3082 ctokens: 3082

class htb 1:12 parent 1:1 leaf 12: prio 2 rate 30000Kbit ceil 30000Kbit burst 1593b cburst 1593b
 Sent 549664129 bytes 1167137 pkt (dropped 248, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 1007042 borrowed: 0 giants: 0
 tokens: 6397 ctokens: 6397

Traffic shaping as needed (equally between clients).

Is my stat normal? Nothing wrong with it?

P.S. Thanks' for your help and attention.

-----Original Message-----
From: Dave Taht [mailto:dave.taht@gmail.com] 
Sent: Monday, January 26, 2015 9:04 AM
To: Dzmitry Shykuts
Subject: Re: Qdisc HTB root overlimits and requeues

fq_codel has been available as a module since 3.6. If it isn't compiled in in your distro, I would be sad.

The rest of the script looked fine, but I like my sqm-scripts.

On Sun, Jan 25, 2015 at 9:59 PM, Dzmitry Shykuts <dshykuts@brest.beltelecom.by> wrote:
> Have a 3.13.0-44-generic kernel only, so no fq_codel as module.
>
> And what about script structure (handlers, classids)? Is it correct?
>
> -----Original Message-----
> From: lartc-owner@vger.kernel.org [mailto:lartc-owner@vger.kernel.org] 
> On Behalf Of Dave Taht
> Sent: Monday, January 26, 2015 12:48 AM
> To: Дмитрий Шикуть
> Cc: lartc@vger.kernel.org
> Subject: Re: Qdisc HTB root overlimits and requeues
>
> give fq_codel a shot instead of sfq.
>
> see also the sqm-scripts.
>
> requeues means that you hit a limit and had to wait for the next scheduled round.
>
>
> On Sun, Jan 25, 2015 at 12:27 PM, Дмитрий Шикуть <dshykuts@brest.beltelecom.by> wrote:
>> Hello!
>>
>> I created script for traffic shaping. After run, there are many 
>> overlimits and requeues in tc stat for htb root discipline. Is this 
>> normal or something going wrong?
>>
>> Here is script:
>>
>> #!/bin/sh
>> ### BEGIN INIT INFO
>> # Provides: trafficshaper
>> # Required-Start: $network $syslog
>> # Required-Stop: $network $syslog
>> # Should-Start:
>> # Should-Stop:
>> # Default-Start: 2 3 4 5
>> # Default-Stop: 0 1 6
>> # Short-Description: Start and stop trafficshaper # Description:
>> trafficshaper ### END INIT INFO
>>
>> PATH=/sbin:/bin:/usr/bin
>>
>> . /lib/init/vars.sh
>> . /lib/lsb/init-functions
>>
>> # Reads config file (will override defaults above) [ -r 
>> /etc/default/trafficshaper ] && . /etc/default/trafficshaper
>>
>> # Devices calculation
>> devDL=$dev
>> devUL=ifb0
>>
>> # Flush discuiplines for DL and UL devices delete_qdisc () {
>>     tc qdisc del dev $devDL ingress 2>/dev/null
>>     tc qdisc del dev $devUL root 2>/dev/null
>>     tc qdisc del dev $devDL root 2>/dev/null }
>>
>> case "$1" in
>>     start|"")
>>         log_daemon_msg "Starting trafficshaper" "ok"
>>
>>         # Global constants
>>         divisor 48
>>         quantum\x1514
>>         perturb\x10
>>
>>         reNum="\([0-9]*\)[a-zA-Z]*"
>>         reStr="[0-9]*\([a-zA-Z]*\)"
>>
>>         # Bandwidth calculation
>>         bwPrivateDL=`expr \( $bwDL : $reNum \) / 2``expr $bwDL : $reStr`
>>         bwPublicDL=$bwPrivateDL
>>
>>         bwPrivateUL=`expr \( $bwUL : $reNum \) / 2``expr $bwUL : $reStr`
>>         bwPublicUL=$bwPrivateUL
>>
>>         # Delete discuiplines for DL and UL devices
>>         delete_qdisc
>>
>>         ifconfig $devUL txqueuelen 1000 2>/dev/null
>>         ifconfig $devUL up 2>/dev/null
>>
>>         # Add discuiplines for DL device
>>         tc qdisc add dev $devDL root handle 1: htb default 12
>>
>>         # Add classes for DL device
>>         tc class add dev $devDL parent 1: classid 1:1 htb rate $bwDL 
>> ceil $bwDL quantum $quantum
>>         tc class add dev $devDL parent 1:1 classid 1:11 htb prio 1 
>> rate $bwPrivateDL ceil $bwDL quantum $quantum
>>         tc class add dev $devDL parent 1:1 classid 1:12 htb prio 2 
>> rate $bwPublicDL ceil $bwPublicDL quantum $quantum
>>
>>         tc qdisc add dev $devDL parent 1:11 handle 11: sfq divisor 
>> $divisor perturb $perturb
>>         tc qdisc add dev $devDL parent 1:12 handle 12: sfq divisor 
>> $divisor perturb $perturb
>>
>>         tc filter add dev $devDL protocol all prio 1 parent 11: 
>> handle
>> 11 flow hash keys dst divisor $divisor baseclass 1:11
>>         tc filter add dev $devDL protocol all prio 2 parent 12: 
>> handle
>> 12 flow hash keys dst divisor $divisor baseclass 1:12
>>
>>         # Add filters for DL device
>>         if [ -s $filePrivate ]; then
>>             for ip in `grep -v "#" $filePrivate`; do
>>                 if [ `expr index $ip :` = 0 ]; then
>>                     tc filter add dev $devDL protocol all prio 1 parent 1:
>> u32 match ip dst $ip flowid 1:11
>>                 else
>>                     echo $ip
>>                     tc filter add dev $devDL protocol all prio 1 parent 1:
>> u32 match ip6 dst $ip flowid 1:11
>>                 fi
>>             done
>>         fi
>>
>>         tc filter add dev $devDL protocol all prio 2 parent 1: u32 
>> match ip dst 0/0 flowid 1:12
>>         tc filter add dev $devDL protocol all prio 2 parent 1: u32 
>> match ip6 dst ::/0 flowid 1:12
>>
>>         # Add discuiplines for UL device
>>         tc qdisc add dev $devUL root handle 2: htb default 22
>>
>>         # Add classes for UL device
>>         tc class add dev $devUL parent 2: classid 2:2 htb rate $bwUL 
>> ceil $bwUL quantum $quantum
>>         tc class add dev $devUL parent 2:2 classid 2:21 htb prio 1 
>> rate $bwPrivateUL ceil $bwUL quantum $quantum
>>         tc class add dev $devUL parent 2:2 classid 2:22 htb prio 2 
>> rate $bwPublicUL ceil $bwPublicUL quantum $quantum
>>
>>         tc qdisc add dev $devUL parent 2:21 handle 21: sfq divisor 
>> $divisor perturb $perturb
>>         tc qdisc add dev $devUL parent 2:22 handle 22: sfq divisor 
>> $divisor perturb $perturb
>>
>>         tc filter add dev $devUL protocol all prio 1 parent 21: 
>> handle
>> 21 flow hash keys src divisor $divisor baseclass 2:21
>>         tc filter add dev $devUL protocol all prio 2 parent 22: 
>> handle
>> 22 flow hash keys src divisor $divisor baseclass 2:22
>>
>>         # Add filters for UL device
>>         if [ -s $filePrivate ]; then
>>             for ip in `grep -v "#" $filePrivate`; do
>>                 if [ `expr index $ip :` = 0 ]; then
>>                     tc filter add dev $devUL protocol all prio 1 parent 2:
>> u32 match ip src $ip flowid 2:21
>>                 else
>>                     tc filter add dev $devUL protocol all prio 1 parent 2:
>> u32 match ip6 src $ip flowid 2:21
>>                 fi
>>             done
>>         fi
>>
>>         tc filter add dev $devUL protocol all prio 2 parent 2: u32 
>> match ip src 0/0 flowid 2:22
>>         tc filter add dev $devUL protocol all prio 2 parent 2: u32 
>> match ip6 src ::/0 flowid 2:22
>>
>>         # Add mirred redirect discuiplines for UL device
>>         tc qdisc add dev $devDL ingress
>>         tc filter add dev $devDL parent ffff: protocol all u32 match
>> u32 0 0 action mirred egress redirect dev $devUL
>>
>>         log_end_msg 0
>>         ;;
>>
>>     stop)
>>         log_daemon_msg "Stopping trafficshaper" "ok"
>>
>>         # Delete discuiplines for DL and UL devices
>>         delete_qdisc
>>
>>         log_end_msg 0
>>         ;;
>>
>>     restart|force-reload)
>>         $0 stop
>>         $0 start
>>         ;;
>>
>>     *)
>>         echo "Usage: trafficshaper [start|stop|restart|force-reload]" >&2
>>         exit 3
>>         ;;
>> esac
>>
>> :
>>
>> And there is tc show output:
>>
>> # tc -s qdisc show dev eth0
>>
>> qdisc htb 1: root refcnt 6 r2q 10 default 12 direct_packets_stat 5 
>> direct_qlen 1000  Sent 147698795 bytes 958679 pkt (dropped 0, 
>> overlimits 43066 requeues 148)  backlog 0b 0p requeues 148 qdisc sfq
>> 11: parent 1:11 limit 127p quantum 1514b depth 127 divisor 2048 
>> perturb 10sec  Sent 81156975 bytes 527521 pkt (dropped 0, overlimits 
>> 0 requeues 0)  backlog 0b 0p requeues 0 qdisc sfq 12: parent 1:12 
>> limit 127p quantum 1514b depth 127 divisor 2048 perturb 10sec  Sent 
>> 66541452 bytes 431152 pkt (dropped 0, overlimits 0 requeues 0)  
>> backlog 0b 0p requeues 0 qdisc ingress ffff: parent ffff:fff1 
>> ----------------  Sent
>> 2333918666 bytes 1999165 pkt (dropped 421, overlimits 0 requeues 0) 
>> backlog 0b 0p requeues 0
>>
>> # tc -s class show dev eth0
>>
>> class htb 1:11 parent 1:1 leaf 11: prio 1 rate 30000Kbit ceil 
>> 60000Kbit burst 1593b cburst 1590b  Sent 82530637 bytes 547960 pkt 
>> (dropped 0, overlimits 0 requeues 0)  rate 0bit 0pps backlog 0b 0p 
>> requeues 0
>>  lended: 521119 borrowed: 5867 giants: 0
>>  tokens: 6097 ctokens: 3048
>>
>> class htb 1:1 root rate 60000Kbit ceil 60000Kbit burst 1590b cburst 
>> 1590b  Sent 150027823 bytes 987451 pkt (dropped 0, overlimits 0 
>> requeues 0)  rate 0bit 0pps backlog 0b 0p requeues 0
>>  lended: 5867 borrowed: 0 giants: 0
>>  tokens: 3215 ctokens: 3215
>>
>> class htb 1:12 parent 1:1 leaf 12: prio 2 rate 30000Kbit ceil 
>> 30000Kbit burst 1593b cburst 1593b  Sent 67497186 bytes 439491 pkt 
>> (dropped 0, overlimits 0 requeues 0)  rate 0bit 0pps backlog 0b 0p 
>> requeues 0
>>  lended: 437399 borrowed: 0 giants: 0
>>  tokens: 6431 ctokens: 6431
>>
>> Is my script correct or I'm something missing?
>>
>> Thank's.
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe lartc" in 
>> the body of a message to majordomo@vger.kernel.org More majordomo 
>> info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Dave Täht
>
> thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in 
> the body of a message to majordomo@vger.kernel.org More majordomo info 
> at  http://vger.kernel.org/majordomo-info.html
>



--
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks