How to classify a port range?

How to classify a port range?

[Yassen Damyanov]

Hello LARTC guys,

I am working on an OSS Python wrapper library intended to help with 
expressing a traffic control structure as a tree of Python objects. This 
structure should later be able to represent itself as a series of tc 
commands. (Your suggestions for getting this thing useful would be 
invaluable.)

I have questions, inevitably. Currently heaviest part seems to be the 
issue of classifying a set of tcp or udp ports to get shaped under a 
common rate limit. (I need to later simulate packet loss for flows on 
these ports, but first things first.)

Can you help me get on the right direction here? Using u32 seems 
daunting for this particular case. Is there another way to do the match?

I've read the relevant parts of the LARTC HowTo and couple more 
documents but still cannot get it right.

Any help would be much appreciated!
Thanks in advance,
Yassen D.

-- 

Yassen Damyanov
M: +359-888-665-235
E: <yd@itlabs.bg>

Re: How to classify a port range?

[Andy Furniss]

Yassen Damyanov wrote:
> Hello LARTC guys,
>
> I am working on an OSS Python wrapper library intended to help with
> expressing a traffic control structure as a tree of Python objects. This
> structure should later be able to represent itself as a series of tc
> commands. (Your suggestions for getting this thing useful would be
> invaluable.)
>
> I have questions, inevitably. Currently heaviest part seems to be the
> issue of classifying a set of tcp or udp ports to get shaped under a
> common rate limit. (I need to later simulate packet loss for flows on
> these ports, but first things first.)
>
> Can you help me get on the right direction here? Using u32 seems
> daunting for this particular case. Is there another way to do the match?
>
> I've read the relevant parts of the LARTC HowTo and couple more
> documents but still cannot get it right.
>
> Any help would be much appreciated!
> Thanks in advance,
> Yassen D.
>

I've never used ematch so don't know if this is correct or not, but -

http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter

Re: How to classify a port range?

[Yassen Damyanov]

On 11/25/2016 1:29 AM, Andy Furniss wrote:
> I've never used ematch so don't know if this is correct or not, but -
>
> http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter

Thanks much, Andy. Would be great if this solves the problem, but it 
doesn't seem to work, unfortunately:

# tc qdisc add dev $DEV root handle 1:0 htb
# tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
# tc filter add dev $DEV parent 1:0 protocol ip prio 1 basic match 
"cmp(u16 at 0 layer transport gt 4000) and cmp(u16 at 0 layer transport 
lt 6000)" flowid 1:1

After running an iperf client against another machine in the local net, 
there's no shaping happening, and the 1:1 class is not visited:

class htb 1:1 root prio 0 quantum 25000 rate 2000Kbit ceil 2000Kbit 
linklayer ethernet burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu 
0b overhead 0b level 0
  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
  rate 0bit 0pps backlog 0b 0p requeues 0
  lended: 0 borrowed: 0 giants: 0
  tokens: 100000 ctokens: 100000

If I use a single port match:
# tc qdisc add dev $DEV root handle 1:0 htb
# tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
# tc filter add dev $DEV parent 1:0 protocol ip prio 1 u32 match ip 
dport 5001 0xffff flowid 1:1

then the traffic is indeed limited to 1.9 Mbits/sec and the class stats 
look different:

class htb 1:1 root prio 0 quantum 25000 rate 2000Kbit ceil 2000Kbit 
linklayer ethernet burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu 
0b overhead 0b level 0
  Sent 1507824 bytes 1000 pkt (dropped 0, overlimits 0 requeues 0)
  rate 0bit 0pps backlog 0b 0p requeues 0
  lended: 484 borrowed: 0 giants: 0
  tokens: -3139 ctokens: -3139

Does anyone know what might be wrong with that ematch use?

-Y.


On 11/25/2016 1:29 AM, Andy Furniss wrote:
> Yassen Damyanov wrote:
>> Hello LARTC guys,
>>
>> I am working on an OSS Python wrapper library intended to help with
>> expressing a traffic control structure as a tree of Python objects. This
>> structure should later be able to represent itself as a series of tc
>> commands. (Your suggestions for getting this thing useful would be
>> invaluable.)
>>
>> I have questions, inevitably. Currently heaviest part seems to be the
>> issue of classifying a set of tcp or udp ports to get shaped under a
>> common rate limit. (I need to later simulate packet loss for flows on
>> these ports, but first things first.)
>>
>> Can you help me get on the right direction here? Using u32 seems
>> daunting for this particular case. Is there another way to do the match?
>>
>> I've read the relevant parts of the LARTC HowTo and couple more
>> documents but still cannot get it right.
>>
>> Any help would be much appreciated!
>> Thanks in advance,
>> Yassen D.
>>
>
> I've never used ematch so don't know if this is correct or not, but -
>
> http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter


-- 

Yassen Damyanov
M: +359-888-665-235
E: <yd@itlabs.bg>

Re: How to classify a port range?

[Andy Furniss]

Yassen Damyanov wrote:
> On 11/25/2016 1:29 AM, Andy Furniss wrote:
>> I've never used ematch so don't know if this is correct or not, but -
>>
>> http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter
>>
>
> Thanks much, Andy. Would be great if this solves the problem, but it
> doesn't seem to work, unfortunately:
>
> # tc qdisc add dev $DEV root handle 1:0 htb
> # tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
> # tc filter add dev $DEV parent 1:0 protocol ip prio 1 basic match
> "cmp(u16 at 0 layer transport gt 4000) and cmp(u16 at 0 layer transport
> lt 6000)" flowid 1:1
>
> After running an iperf client against another machine in the local net,
> there's no shaping happening, and the 1:1 class is not visited:
>
> class htb 1:1 root prio 0 quantum 25000 rate 2000Kbit ceil 2000Kbit
> linklayer ethernet burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu
> 0b overhead 0b level 0
>   Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>   rate 0bit 0pps backlog 0b 0p requeues 0
>   lended: 0 borrowed: 0 giants: 0
>   tokens: 100000 ctokens: 100000
>
> If I use a single port match:
> # tc qdisc add dev $DEV root handle 1:0 htb
> # tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
> # tc filter add dev $DEV parent 1:0 protocol ip prio 1 u32 match ip
> dport 5001 0xffff flowid 1:1

dport would be u16 at 2

>
> then the traffic is indeed limited to 1.9 Mbits/sec and the class stats
> look different:
>
> class htb 1:1 root prio 0 quantum 25000 rate 2000Kbit ceil 2000Kbit
> linklayer ethernet burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu
> 0b overhead 0b level 0
>   Sent 1507824 bytes 1000 pkt (dropped 0, overlimits 0 requeues 0)
>   rate 0bit 0pps backlog 0b 0p requeues 0
>   lended: 484 borrowed: 0 giants: 0
>   tokens: -3139 ctokens: -3139
>
> Does anyone know what might be wrong with that ematch use?
>
> -Y.
>
>
> On 11/25/2016 1:29 AM, Andy Furniss wrote:
>> Yassen Damyanov wrote:
>>> Hello LARTC guys,
>>>
>>> I am working on an OSS Python wrapper library intended to help with
>>> expressing a traffic control structure as a tree of Python objects. This
>>> structure should later be able to represent itself as a series of tc
>>> commands. (Your suggestions for getting this thing useful would be
>>> invaluable.)
>>>
>>> I have questions, inevitably. Currently heaviest part seems to be the
>>> issue of classifying a set of tcp or udp ports to get shaped under a
>>> common rate limit. (I need to later simulate packet loss for flows on
>>> these ports, but first things first.)
>>>
>>> Can you help me get on the right direction here? Using u32 seems
>>> daunting for this particular case. Is there another way to do the match?
>>>
>>> I've read the relevant parts of the LARTC HowTo and couple more
>>> documents but still cannot get it right.
>>>
>>> Any help would be much appreciated!
>>> Thanks in advance,
>>> Yassen D.
>>>
>>
>> I've never used ematch so don't know if this is correct or not, but -
>>
>> http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter
>>
>
>

Re: How to classify a port range?

[Yassen Damyanov]

On 11/25/2016 7:19 PM, Andy Furniss wrote:
> Yassen Damyanov wrote:
>> On 11/25/2016 1:29 AM, Andy Furniss wrote:
>>> I've never used ematch so don't know if this is correct or not, but -
>>>
>>> http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter
>>>
>>>
>>
>> Thanks much, Andy. Would be great if this solves the problem, but it
>> doesn't seem to work, unfortunately:
>>
>> # tc qdisc add dev $DEV root handle 1:0 htb
>> # tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
>> # tc filter add dev $DEV parent 1:0 protocol ip prio 1 basic match
>> "cmp(u16 at 0 layer transport gt 4000) and cmp(u16 at 0 layer transport
>> lt 6000)" flowid 1:1
>>
>
> dport would be u16 at 2

Thanks so much, Andy (and stupid me). Yep, that was it, works like a 
charm! (Rodney, no need to look that up, problem solved, thanks buddy.)

For anyone else who might be stumbling on this: here's the correct 
sequence for my case (where I tried to shape tcp traffic with a dport 
range 5000-6000, excl.):

# tc qdisc add dev $DEV root handle 1:0 htb
# tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
# tc filter add dev $DEV parent 1:0 protocol ip prio 1 basic match 
"cmp(u16 at 2 layer transport gt 5000) and cmp(u16 at 2 layer transport 
lt 6000)" flowid 1:1

$DEV is the network device name (e.g. eth0) and the root qdisc is left 
w/o a default so that we do not shape unclassified traffic.

(thumbs up!)


>> After running an iperf client against another machine in the local net,
>> there's no shaping happening, and the 1:1 class is not visited:
>>
>> class htb 1:1 root prio 0 quantum 25000 rate 2000Kbit ceil 2000Kbit
>> linklayer ethernet burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu
>> 0b overhead 0b level 0
>>   Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>   rate 0bit 0pps backlog 0b 0p requeues 0
>>   lended: 0 borrowed: 0 giants: 0
>>   tokens: 100000 ctokens: 100000
>>
>> If I use a single port match:
>> # tc qdisc add dev $DEV root handle 1:0 htb
>> # tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
>> # tc filter add dev $DEV parent 1:0 protocol ip prio 1 u32 match ip
>> dport 5001 0xffff flowid 1:1
>>
>> then the traffic is indeed limited to 1.9 Mbits/sec and the class stats
>> look different:
>>
>> class htb 1:1 root prio 0 quantum 25000 rate 2000Kbit ceil 2000Kbit
>> linklayer ethernet burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu
>> 0b overhead 0b level 0
>>   Sent 1507824 bytes 1000 pkt (dropped 0, overlimits 0 requeues 0)
>>   rate 0bit 0pps backlog 0b 0p requeues 0
>>   lended: 484 borrowed: 0 giants: 0
>>   tokens: -3139 ctokens: -3139
>>
>> Does anyone know what might be wrong with that ematch use?
>>
>> -Y.
>>
>>
>> On 11/25/2016 1:29 AM, Andy Furniss wrote:
>>> Yassen Damyanov wrote:
>>>> Hello LARTC guys,
>>>>
>>>> I am working on an OSS Python wrapper library intended to help with
>>>> expressing a traffic control structure as a tree of Python objects.
>>>> This
>>>> structure should later be able to represent itself as a series of tc
>>>> commands. (Your suggestions for getting this thing useful would be
>>>> invaluable.)
>>>>
>>>> I have questions, inevitably. Currently heaviest part seems to be the
>>>> issue of classifying a set of tcp or udp ports to get shaped under a
>>>> common rate limit. (I need to later simulate packet loss for flows on
>>>> these ports, but first things first.)
>>>>
>>>> Can you help me get on the right direction here? Using u32 seems
>>>> daunting for this particular case. Is there another way to do the
>>>> match?
>>>>
>>>> I've read the relevant parts of the LARTC HowTo and couple more
>>>> documents but still cannot get it right.
>>>>
>>>> Any help would be much appreciated!
>>>> Thanks in advance,
>>>> Yassen D.
>>>>
>>>
>>> I've never used ematch so don't know if this is correct or not, but -
>>>
>>> http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter
>>>
>>>
>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe lartc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 

Yassen Damyanov
M: +359-888-665-235
E: <yd@itlabs.bg>

Re: How to classify a port range?

[Yassen Damyanov]

Hi again LARTC list,

Andy helped recently to spot a bug in an ematch filter definition that 
classifies a range of ports (thanks again!)

Now I again have a problem with that same classifier, this time on an 
ifb device which is "connected" to an eth device to allow for richer 
ingress control:

# ip link set dev ifb0 up
# tc qdisc add dev eth0 handle ffff: ingress
# tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 
action mirred egress redirect dev ifb0

# tc qdisc add dev ifb0 root handle 1:0 htb
# tc class add dev ifb0 parent 1:0 classid 1:1 htb rate 30gbit
# tc filter add dev ifb0 parent 1:0 protocol ip prio 1 u32 match ip 
protocol 6 0xff flowid 1:1
# tc qdisc add dev ifb0 parent 1:1 handle 2:0 htb
# tc class add dev ifb0 parent 2:0 classid 2:1 htb rate 2mbit
# tc filter add dev ifb0 parent 2:0 protocol ip prio 1 basic match 
"cmp(u16 at 2 layer transport gt 8079) and cmp(u16 at 2 layer transport 
lt 8089)" flowid 2:1

When doing measurements, the speed within the shaped range is way below 
what would be if no shaping is done, but substantially higher than the 
htb rate specified (2mbit). It also differs significantly from test to test.

If I just put a u32 filter like this:

# tc filter add dev ifb0 parent 2:0 protocol ip prio 1 u32 match ip 
dport 8080 0xffff flowid 2:1

-- this works as expected.

I see this in the ematch man page and I guess it might be related, but I 
am not sure how:

"""
When using the ipset ematch with the "ifb" device, the outgoing device
will be the ifb device itself, e.g. "ifb0".  The original interface
(i.e. the device the packet arrived on) is treated as the incoming
interface.
"""

Does anyone see what am I missing and how to get that ematch working on 
the ifb? Thanks!

-Y.


On 11/25/2016 8:34 PM, Yassen Damyanov wrote:
> On 11/25/2016 7:19 PM, Andy Furniss wrote:
>> Yassen Damyanov wrote:
>>> On 11/25/2016 1:29 AM, Andy Furniss wrote:
>>>> I've never used ematch so don't know if this is correct or not, but -
>>>>
>>>> http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter
>>>>
>>>>
>>>>
>>>
>>> Thanks much, Andy. Would be great if this solves the problem, but it
>>> doesn't seem to work, unfortunately:
>>>
>>> # tc qdisc add dev $DEV root handle 1:0 htb
>>> # tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
>>> # tc filter add dev $DEV parent 1:0 protocol ip prio 1 basic match
>>> "cmp(u16 at 0 layer transport gt 4000) and cmp(u16 at 0 layer transport
>>> lt 6000)" flowid 1:1
>>>
>>
>> dport would be u16 at 2
>
> Thanks so much, Andy (and stupid me). Yep, that was it, works like a
> charm! (Rodney, no need to look that up, problem solved, thanks buddy.)
>
> For anyone else who might be stumbling on this: here's the correct
> sequence for my case (where I tried to shape tcp traffic with a dport
> range 5000-6000, excl.):
>
> # tc qdisc add dev $DEV root handle 1:0 htb
> # tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
> # tc filter add dev $DEV parent 1:0 protocol ip prio 1 basic match
> "cmp(u16 at 2 layer transport gt 5000) and cmp(u16 at 2 layer transport
> lt 6000)" flowid 1:1
>
> $DEV is the network device name (e.g. eth0) and the root qdisc is left
> w/o a default so that we do not shape unclassified traffic.
>
> (thumbs up!)
>
>
>>> After running an iperf client against another machine in the local net,
>>> there's no shaping happening, and the 1:1 class is not visited:
>>>
>>> class htb 1:1 root prio 0 quantum 25000 rate 2000Kbit ceil 2000Kbit
>>> linklayer ethernet burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu
>>> 0b overhead 0b level 0
>>>   Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>   rate 0bit 0pps backlog 0b 0p requeues 0
>>>   lended: 0 borrowed: 0 giants: 0
>>>   tokens: 100000 ctokens: 100000
>>>
>>> If I use a single port match:
>>> # tc qdisc add dev $DEV root handle 1:0 htb
>>> # tc class add dev $DEV parent 1:0 classid 1:1 htb rate 2mbit
>>> # tc filter add dev $DEV parent 1:0 protocol ip prio 1 u32 match ip
>>> dport 5001 0xffff flowid 1:1
>>>
>>> then the traffic is indeed limited to 1.9 Mbits/sec and the class stats
>>> look different:
>>>
>>> class htb 1:1 root prio 0 quantum 25000 rate 2000Kbit ceil 2000Kbit
>>> linklayer ethernet burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu
>>> 0b overhead 0b level 0
>>>   Sent 1507824 bytes 1000 pkt (dropped 0, overlimits 0 requeues 0)
>>>   rate 0bit 0pps backlog 0b 0p requeues 0
>>>   lended: 484 borrowed: 0 giants: 0
>>>   tokens: -3139 ctokens: -3139
>>>
>>> Does anyone know what might be wrong with that ematch use?
>>>
>>> -Y.
>>>
>>>
>>> On 11/25/2016 1:29 AM, Andy Furniss wrote:
>>>> Yassen Damyanov wrote:
>>>>> Hello LARTC guys,
>>>>>
>>>>> I am working on an OSS Python wrapper library intended to help with
>>>>> expressing a traffic control structure as a tree of Python objects.
>>>>> This
>>>>> structure should later be able to represent itself as a series of tc
>>>>> commands. (Your suggestions for getting this thing useful would be
>>>>> invaluable.)
>>>>>
>>>>> I have questions, inevitably. Currently heaviest part seems to be the
>>>>> issue of classifying a set of tcp or udp ports to get shaped under a
>>>>> common rate limit. (I need to later simulate packet loss for flows on
>>>>> these ports, but first things first.)
>>>>>
>>>>> Can you help me get on the right direction here? Using u32 seems
>>>>> daunting for this particular case. Is there another way to do the
>>>>> match?
>>>>>
>>>>> I've read the relevant parts of the LARTC HowTo and couple more
>>>>> documents but still cannot get it right.
>>>>>
>>>>> Any help would be much appreciated!
>>>>> Thanks in advance,
>>>>> Yassen D.
>>>>>
>>>>
>>>> I've never used ematch so don't know if this is correct or not, but -
>>>>
>>>> http://serverfault.com/questions/231880/how-to-match-port-range-using-u32-filter
>>>>
>>>>
>>>>
>>>
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe lartc" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

-- 

Yassen Damyanov
E: <yd-at-itlabs-dot-bg>

Re: How to classify a port range?

[Andy Furniss]

Yassen Damyanov wrote:
> Hi again LARTC list,
>
> Andy helped recently to spot a bug in an ematch filter definition that
> classifies a range of ports (thanks again!)
>
> Now I again have a problem with that same classifier, this time on an
> ifb device which is "connected" to an eth device to allow for richer
> ingress control:
>
> # ip link set dev ifb0 up
> # tc qdisc add dev eth0 handle ffff: ingress
> # tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0
> action mirred egress redirect dev ifb0
>
> # tc qdisc add dev ifb0 root handle 1:0 htb
> # tc class add dev ifb0 parent 1:0 classid 1:1 htb rate 30gbit
> # tc filter add dev ifb0 parent 1:0 protocol ip prio 1 u32 match ip
> protocol 6 0xff flowid 1:1
> # tc qdisc add dev ifb0 parent 1:1 handle 2:0 htb
> # tc class add dev ifb0 parent 2:0 classid 2:1 htb rate 2mbit
> # tc filter add dev ifb0 parent 2:0 protocol ip prio 1 basic match
> "cmp(u16 at 2 layer transport gt 8079) and cmp(u16 at 2 layer transport
> lt 8089)" flowid 2:1
>
> When doing measurements, the speed within the shaped range is way below
> what would be if no shaping is done, but substantially higher than the
> htb rate specified (2mbit). It also differs significantly from test to
> test.
>
> If I just put a u32 filter like this:
>
> # tc filter add dev ifb0 parent 2:0 protocol ip prio 1 u32 match ip
> dport 8080 0xffff flowid 2:1
>
> -- this works as expected.
>
> I see this in the ematch man page and I guess it might be related, but I
> am not sure how:
>
> """
> When using the ipset ematch with the "ifb" device, the outgoing device
> will be the ifb device itself, e.g. "ifb0".  The original interface
> (i.e. the device the packet arrived on) is treated as the incoming
> interface.
> """
>
> Does anyone see what am I missing and how to get that ematch working on
> the ifb? Thanks!

Seems to work for me with a quick test on lan.

Random thoughts -

Putting htb qdisc under htb is allowed, but not normal and
gets in dmesg  "htb: htb qdisc 2: is non-work-conserving?"

30 gig is really high and I see htb sets burst and cburst to 0.

Your qlens will be picked up from ifb, default 32 which may be too
low for the high bandwidth/htb direct traffic.

Nics may be doing hardware/driver software offloads, causing
giant packets to hit the qdisc - you can disable with ethtool.