Limiting bandwidth on router

Limiting bandwidth on router

[Elison Niven]

Hi list,
I have an ARM based board with 2 ethernet interfaces eth0 and eth1.
eth0 - LAN interface. 192.168.2.149
eth1 - WAN interface. 192.168.1.149
I have setup a simple router using iptables that allows PCs on the LAN
to browse the internet via the WAN.
I want to limit incoming bandwidth on the LAN interface to about
8mbit/s as when there are massive data transfers initiating from the
PCs on the LAN, the system is under heavy load and is not able to
execute any applications.

I tried to limit the bandwidth using tc's tbf and ingress but it isn't
really working well.
For example from a PC on the LAN (ip 192.168.2.13, default gw 192.168.2.149)
# ping -f -s 1000 192.168.1.3
This is correctly limited and ping results show the loss.

# ping -f -s 1 192.168.1.3
Here the system is under heavy load and is not able to execute any
applications. No loss in ping.

Also when I create a udp socket from this PC to an address on the WAN,
the rate is not limited.
#./udpclient 20000 192.168.1.3 20000 1000
This utility sends 20000 packets of 1000 bytes each to 192.168.1.3.
Here also the rate is not limited.

Do I need to use some other tools than tc? Or use hashlimit in iptables?
Can anyone help me the tc configuration?

I am using linux-2.6.30 and iptables-1.4.4.

Thanks and Regards,
Elison

Re: Limiting bandwidth on router

[Richard Horton]

On 23 July 2010 11:37, Elison Niven <elison.niven@gmail.com> wrote:

> Do I need to use some other tools than tc? Or use hashlimit in iptables?
> Can anyone help me the tc configuration?
>
> I am using linux-2.6.30 and iptables-1.4.4.

Without seeing your relevant iptables and tc rules/commands its hard /
impossible to say.

-- 
Richard Horton
Users are like a virus: Each causing a thousand tiny crises until the
host finally dies.
http://www.pbase.com/arimus - My online photogallery
http://uk.linkedin.com/in/richardhorton1972 - My linkedin profile
http://www.solstans.co.uk/richard - Online CV

Re: Limiting bandwidth on router

[Pascal Hambourg]

Hello,

Elison Niven a écrit :
> I have an ARM based board with 2 ethernet interfaces eth0 and eth1.
> eth0 - LAN interface. 192.168.2.149
> eth1 - WAN interface. 192.168.1.149
> I have setup a simple router using iptables that allows PCs on the LAN
> to browse the internet via the WAN.
> I want to limit incoming bandwidth on the LAN interface to about
> 8mbit/s as when there are massive data transfers initiating from the
> PCs on the LAN, the system is under heavy load and is not able to
> execute any applications.
> 
> I tried to limit the bandwidth using tc's tbf and ingress but it isn't
> really working well.
> For example from a PC on the LAN (ip 192.168.2.13, default gw 192.168.2.149)
> # ping -f -s 1000 192.168.1.3
> This is correctly limited and ping results show the loss.
>
> # ping -f -s 1 192.168.1.3
> Here the system is under heavy load and is not able to execute any
> applications. No loss in ping.

This result may indicate that the router's load depends more on the 
packet rate than the bandwidth, which makes sense. So it may be more 
efficient to limit the packet rate instead of the bandwidth.

> Also when I create a udp socket from this PC to an address on the WAN,
> the rate is not limited.
> #./udpclient 20000 192.168.1.3 20000 1000
> This utility sends 20000 packets of 1000 bytes each to 192.168.1.3.
> Here also the rate is not limited.

I did not understand well whether you want to limit incoming (WAN to 
LAN) or outgoing (LAN to WAN) traffic. The ping test above does not tell 
whether incoming (echo reply) or outgoing (echo request) traffic is limited.

Also, remember that by definition received traffic rate cannot be 
directly controlled. All you can do is drop incoming packets. Due to TCP 
acknowledgement and congestion control mechanism, the sender will 
interpret the packet loss as congestion and reduce the sending rate. But 
UDP has no such mechanism. Anyway, does this really matter ? If the 
heavy data transfers use TCP, it should work. Did you test it ?

Re: Limiting bandwidth on router

[Elison Niven]

On Fri, Jul 23, 2010 at 6:53 PM, Pascal Hambourg
<pascal.mail@plouf.fr.eu.org> wrote:
> Hello,
>
> Elison Niven a écrit :
>>
>> I have an ARM based board with 2 ethernet interfaces eth0 and eth1.
>> eth0 - LAN interface. 192.168.2.149
>> eth1 - WAN interface. 192.168.1.149
>> I have setup a simple router using iptables that allows PCs on the LAN
>> to browse the internet via the WAN.
>> I want to limit incoming bandwidth on the LAN interface to about
>> 8mbit/s as when there are massive data transfers initiating from the
>> PCs on the LAN, the system is under heavy load and is not able to
>> execute any applications.
>>
>> I tried to limit the bandwidth using tc's tbf and ingress but it isn't
>> really working well.
>> For example from a PC on the LAN (ip 192.168.2.13, default gw
>> 192.168.2.149)
>> # ping -f -s 1000 192.168.1.3
>> This is correctly limited and ping results show the loss.
>>
>> # ping -f -s 1 192.168.1.3
>> Here the system is under heavy load and is not able to execute any
>> applications. No loss in ping.
>
> This result may indicate that the router's load depends more on the packet
> rate than the bandwidth, which makes sense. So it may be more efficient to
> limit the packet rate instead of the bandwidth.
>
>> Also when I create a udp socket from this PC to an address on the WAN,
>> the rate is not limited.
>> #./udpclient 20000 192.168.1.3 20000 1000
>> This utility sends 20000 packets of 1000 bytes each to 192.168.1.3.
>> Here also the rate is not limited.
>
> I did not understand well whether you want to limit incoming (WAN to LAN) or
> outgoing (LAN to WAN) traffic. The ping test above does not tell whether
> incoming (echo reply) or outgoing (echo request) traffic is limited.
>
> Also, remember that by definition received traffic rate cannot be directly
> controlled. All you can do is drop incoming packets. Due to TCP
> acknowledgement and congestion control mechanism, the sender will interpret
> the packet loss as congestion and reduce the sending rate. But UDP has no
> such mechanism. Anyway, does this really matter ? If the heavy data
> transfers use TCP, it should work. Did you test it ?
>
>
>

Thanks for the reply, I will test with tcp and send the relevant
iptables and tc rules that i have used.

Regards,
Elison

Re: Limiting bandwidth on router

[Elison Niven]

On Sat, Jul 24, 2010 at 9:19 PM, Elison Niven <elison.niven@gmail.com> wrote:
> On Fri, Jul 23, 2010 at 6:53 PM, Pascal Hambourg
> <pascal.mail@plouf.fr.eu.org> wrote:
>> Hello,
>>
>> Elison Niven a écrit :
>>>
>>> I have an ARM based board with 2 ethernet interfaces eth0 and eth1.
>>> eth0 - LAN interface. 192.168.2.149
>>> eth1 - WAN interface. 192.168.1.149
>>> I have setup a simple router using iptables that allows PCs on the LAN
>>> to browse the internet via the WAN.
>>> I want to limit incoming bandwidth on the LAN interface to about
>>> 8mbit/s as when there are massive data transfers initiating from the
>>> PCs on the LAN, the system is under heavy load and is not able to
>>> execute any applications.
>>>
>>> I tried to limit the bandwidth using tc's tbf and ingress but it isn't
>>> really working well.
>>> For example from a PC on the LAN (ip 192.168.2.13, default gw
>>> 192.168.2.149)
>>> # ping -f -s 1000 192.168.1.3
>>> This is correctly limited and ping results show the loss.
>>>
>>> # ping -f -s 1 192.168.1.3
>>> Here the system is under heavy load and is not able to execute any
>>> applications. No loss in ping.
>>
>> This result may indicate that the router's load depends more on the packet
>> rate than the bandwidth, which makes sense. So it may be more efficient to
>> limit the packet rate instead of the bandwidth.
>>
>>> Also when I create a udp socket from this PC to an address on the WAN,
>>> the rate is not limited.
>>> #./udpclient 20000 192.168.1.3 20000 1000
>>> This utility sends 20000 packets of 1000 bytes each to 192.168.1.3.
>>> Here also the rate is not limited.
>>
>> I did not understand well whether you want to limit incoming (WAN to LAN) or
>> outgoing (LAN to WAN) traffic. The ping test above does not tell whether
>> incoming (echo reply) or outgoing (echo request) traffic is limited.

Here I need to limit the outgoing (LAN to WAN) traffic by dropping
incoming packets at the LAN interface.
Also the WAN to LAN traffic (ex. users of PCs on the LAN network
downloading files from the internet available by WAN) should also be
limited to 8mbit/s.

>> Also, remember that by definition received traffic rate cannot be directly
>> controlled. All you can do is drop incoming packets. Due to TCP
>> acknowledgement and congestion control mechanism, the sender will interpret
>> the packet loss as congestion and reduce the sending rate. But UDP has no
>> such mechanism. Anyway, does this really matter ? If the heavy data
>> transfers use TCP, it should work. Did you test it ?

I still have to test for TCP.
Here are the iptables rules I have set up for the router:

LAN=eth0
WAN=eth1
WAN_IP="192.168.1.149"
# Flush everything
iptables -t filter -F
iptables -t filter -F INPUT
iptables -t filter -F OUTPUT
iptables -t filter -F FORWARD
iptables -t nat -F
iptables -t nat -F OUTPUT
iptables -t nat -F PREROUTING
iptables -t nat -F POSTROUTING
# Forwarding
iptables -A FORWARD -i $LAN -j ACCEPT
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
# Postrouting
iptables -t nat -A POSTROUTING -o $WAN -j SNAT --to-source $WAN_IP

And here are the tc rules:
# tbf on WAN
tc qdisc add dev $WAN root tbf rate 8mbit latency 50ms burst 1500
# ingress on LAN
tc qdisc add dev $LAN handle ffff:ingress
tc filter add dev $LAN parent ffff: protocol ip prio 50 \
u32 match ip src 0.0.0.0/0 \
police rate 8mbit burst 1k \
drop flowid :1

Thanks,
Elison