From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Beverley Date: Thu, 08 Feb 2007 21:40:34 +0000 Subject: Re: [LARTC] Re: GPL Software for Small ISP Message-Id: <1170970834.4243.25.camel@andybev.localdomain> MIME-Version: 1 Content-Type: multipart/mixed; boundary="=-CGPAUZlzlcA6xmY566VT" List-Id: References: <1170958346.4260.28.camel@andybev.localdomain> In-Reply-To: <1170958346.4260.28.camel@andybev.localdomain> To: lartc@vger.kernel.org --=-CGPAUZlzlcA6xmY566VT Content-Type: text/plain Content-Transfer-Encoding: 7bit On Thu, 2007-02-08 at 23:01 +0200, Bogdan Hojda wrote: > Andrew Beverley wrote: > >> It is not just browsing ...or HTTP... it is every thing... I want a > >> GPL package for a small ISP > > > > I think you're probably limited in what you could cache other than HTTP. > > > > I suggest you have a look at some of the examples of fair traffic > > shaping using linux. I can send you my script if you like - I have used > > it to share a 1 Mbit link between about 70 people and it works fairly > > well. > > > > Could you send that script to me, please? I have about 150 people > sharing a 2 Mbit link, I'm not satisfied with my old script, and I'm > searching some alternatives. > I've attached it. Let me know if you have any questions. A few notes: - internet link is ppp0, local network is eth0 - local network is on a 10.0.0.0 subnet - I use IFB to shape ingress traffic - edit DOWNLINK and UPLINK variables as required - You'll need to patch your kernel for connlimit and ipset (if you want p2p detection) - I also use Squid as a transparent web cache. Take those rules out if you don't need them. Regards, Andy Beverley --=-CGPAUZlzlcA6xmY566VT Content-Disposition: attachment; filename=traffic-shaper Content-Type: application/x-shellscript; name=traffic-shaper Content-Transfer-Encoding: 7bit #!/bin/bash # Internet = ppp0 # Local net = eth0 # Enable Internet connection sharing echo "1" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/ip_dynaddr # clear out existing setting iptables -P INPUT ACCEPT iptables -F INPUT iptables -P OUTPUT ACCEPT iptables -F OUTPUT iptables -P FORWARD DROP iptables -F FORWARD iptables -t nat -F iptables -t mangle -F iptables -t filter -F # create p2p chain for detecting nasty computers running p2p software iptables -N p2p -t mangle 2>/dev/null iptables -F p2p -t mangle # create ipset for storing ip addresses of p2p users # set timeout to 60 seconds meaning their address will # be cleared after 60 seconds of inactivity of p2p # # The current IP addresses in the IP Set can be monitored # with the 'ipset' comand from the bash prompt # ipset -X p2p ipset -N p2p iptree --timeout 60 # accept all local traffic iptables -t nat -A PREROUTING -i eth0 --destination 10.0.0.0/16 -j ACCEPT # accept ping and ssh incoming FROM Internet networks iptables -t nat -A PREROUTING -p icmp -i ppp0 --icmp-type 8 -j ACCEPT iptables -t nat -A PREROUTING -p ICMP -i ppp0 --icmp-type 11 -j ACCEPT iptables -t nat -A PREROUTING -i ppp0 -p tcp --dport 22 -j ACCEPT # do not allow outgoing SMTP connections in order to prevent spam viruses # this could be changed on a per user basis if required iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 25 -j DROP # Mark traffic for shaping later # The lower the MARK the higher the priority iptables -t mangle -A PREROUTING -i ppp0 -j MARK --set-mark 40 # default mark iptables -t mangle -A PREROUTING -p tcp --sport 80 -i ppp0 -j MARK --set-mark 30 # http iptables -t mangle -A POSTROUTING -p tcp --dport 80 -j MARK --set-mark 30 # http iptables -t mangle -A PREROUTING -p tcp --sport 443 -i ppp0 -j MARK --set-mark 30 # https iptables -t mangle -A PREROUTING -p tcp --dport 443 -i eth0 -j MARK --set-mark 30 # https iptables -t mangle -A PREROUTING -p tcp --sport 22 -i ppp0 -j MARK --set-mark 10 # ssh iptables -t mangle -A POSTROUTING -p tcp --dport 22 -j MARK --set-mark 10 # ssh iptables -t mangle -A POSTROUTING --source 10.0.0.1 -j MARK --set-mark 1 # local traffic iptables -t mangle -A POSTROUTING -p tcp --sport 3128 --source 10.0.0.1 -j MARK --set-mark 30 # squid iptables -t mangle -A PREROUTING -p tcp --sport 993 -i ppp0 -j MARK --set-mark 30 # imap iptables -t mangle -A PREROUTING -p tcp --dport 993 -i eth0 -j MARK --set-mark 30 # imap #Sets high priority for DNS request iptables -t mangle -A POSTROUTING -m udp -p udp --dport 53 -j MARK --set-mark 10 # DNS iptables -t mangle -A POSTROUTING -m tcp -p tcp --dport 53 -j MARK --set-mark 10 # DNS # mark large downloads from squid correctly iptables -t mangle -A POSTROUTING -p tcp --sport 3128 --source 10.0.0.1 -m connbytes \ --connbytes 504857: --connbytes-dir both --connbytes-mode bytes -j MARK --set-mark 40 # Mark large downloads (> 500kb) iptables -t mangle -A PREROUTING -m connbytes --connbytes 504857: --connbytes-dir both \ --connbytes-mode bytes -j MARK --set-mark 40 # To speed up downloads while an upload is going on, put short ACK # packets in the interactive class: iptables -t mangle -A FORWARD -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m length --length :64 -j MARK --set-mark 10 # Look for p2p traffic and add to p2p ipset # p2p traffic is detected using connlimit to look for multiple connections above port 1024 # - this is a sure sign of p2p acitivity! # these users are then added to the ipset and then any traffic by the users on ports above # 1024 is classed as p2p iptables -t mangle -A PREROUTING -p tcp -i eth0 -m conntrack --ctstate INVALID -j DROP # drop invalid connections iptables -t mangle -A PREROUTING -p tcp -i eth0 --dport 1024: -m connlimit --connlimit-above 10 \ -j SET --add-set p2p src # Detects traffic from Users using >5 ports above 1024 and adds the source address to the P2P list. iptables -t mangle -A FORWARD -o eth0 -p tcp -m multiport --sport 1024:65535 -m set --set p2p dst \ -j MARK --set-mark 60 # sets incoming traffic to known P2P users from Ports >1024 to lowest priority iptables -t mangle -A FORWARD -i eth0 -p tcp -m multiport --dport 1024:65535 -m set --set p2p src \ -j MARK --set-mark 60 # sets outgoing traffic from known P2P users to Ports >1024 to lowest priority # force all traffic via web proxy iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128 #blocks outgoing traffic from local host on unused ports iptables -t mangle -A OUTPUT -p tcp --dport 80 -o ppp0 -j ACCEPT # Accept web proxy traffic iptables -t mangle -A OUTPUT -p tcp --dport 25 -o ppp0 -j ACCEPT # Accept outgoing mail iptables -t mangle -A OUTPUT -p tcp --dport 21 -o ppp0 -j ACCEPT # Accept outgoing ftp iptables -t mangle -A OUTPUT -p tcp --dport 22 -o ppp0 -j ACCEPT # Accept outgoing ssh iptables -t mangle -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT # accept related traffic iptables -t mangle -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT iptables -t mangle -A OUTPUT -p icmp -o ppp0 --icmp-type 8 -j ACCEPT #Accept ping commamnds iptables -t mangle -A OUTPUT -p ICMP -o ppp0 --icmp-type 11 -j ACCEPT #Accept ping commands iptables -t mangle -A OUTPUT -m udp -p udp --dport 53 -j ACCEPT # DNS iptables -t mangle -A OUTPUT -o ppp0 -j DROP # Drops all other outputs from the local machine # enables Internet connection sharing iptables -A FORWARD -i ppp0 -o eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT iptables -A FORWARD -i eth0 -o ppp0 -j ACCEPT iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE #################################################################################### # Set up traffic shaping # delete all old stuff first tc qdisc del dev ppp0 root 2> /dev/null > /dev/null tc qdisc del dev ifb0 root 2> /dev/null > /dev/null tc qdisc del dev ifb0 ingress 2> /dev/null > /dev/null tc qdisc del dev ppp0 root 2> /dev/null > /dev/null tc qdisc del dev ifb0 root 2> /dev/null > /dev/null tc qdisc del dev ppp0 ingress 2> /dev/null > /dev/null tc qdisc del dev eth0 root 2> /dev/null > /dev/null tc qdisc del dev eth0 ingress 2> /dev/null > /dev/null # set variables DOWNLINK=3000 UPLINK=450 ################################################################################################# ## downlink via ifb ## # HFSC: # rt = realtime # ls = link sharing # ul = upper limit # sc = service curve ( instead of rt and ls) # Default class 40 tc qdisc add dev ifb0 root handle 1: hfsc default 40 # Main class tc class add dev ifb0 parent 1: classid 1:1 hfsc sc rate ${DOWNLINK} ul rate ${DOWNLINK}kbit # interactive traffic # guarantee realtime full uplink for 50ms, then 1/10 of the uplink tc class add dev ifb0 parent 1:1 classid 1:10 hfsc \ rt m1 ${DOWNLINK}kbit d 50ms m2 $[1*$DOWNLINK/10]kbit \ ls m1 ${DOWNLINK}kbit d 50ms m2 $[3*$DOWNLINK/10]kbit \ ul rate ${DOWNLINK}kbit # other low latency # guarantee full uplink for 200ms, then 1/10 tc class add dev ifb0 parent 1:1 classid 1:20 hfsc \ sc m1 ${DOWNLINK}kbit d 200ms m2 $[1*$DOWNLINK/10]kbit \ ul rate ${DOWNLINK}kbit # web browsing # Don't guarantee anything for the first second, then guarantee 5/10 tc class add dev ifb0 parent 1:1 classid 1:30 hfsc \ sc m1 0 d 1s m2 $[5*$DOWNLINK/10]kbit \ ul rate $[8*$DOWNLINK/10]kbit # default traffic # don't guarantee anything for the first two seconds, then guarantee 1/20 tc class add dev ifb0 parent 1:1 classid 1:40 hfsc \ sc m1 0 d 2s m2 $[1*$DOWNLINK/20]kbit \ ul rate $[8*$DOWNLINK/10]kbit # bad boys # don't guarantee anything for the first 10 seconds, then guarantee 1/20 tc class add dev ifb0 parent 1:1 classid 1:60 hfsc \ sc m1 0 d 10s m2 $[1*$DOWNLINK/20]kbit \ ul rate $[1*$DOWNLINK/10]kbit # Shares traffic equally within each classes tc qdisc add dev ifb0 parent 1:10 handle 10: pfifo tc qdisc add dev ifb0 parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev ifb0 parent 1:30 handle 30: sfq perturb 10 #srr slots 64 limit 1024 classify dst tc qdisc add dev ifb0 parent 1:40 handle 40: sfq perturb 10 #srr slots 64 limit 1024 classify dst tc qdisc add dev ifb0 parent 1:60 handle 60: sfq perturb 10 # Uses marks applied by the Firewall to classify traffic correctly tc filter add dev ifb0 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10 tc filter add dev ifb0 parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20 tc filter add dev ifb0 parent 1:0 prio 0 protocol ip handle 30 fw flowid 1:30 tc filter add dev ifb0 parent 1:0 prio 0 protocol ip handle 40 fw flowid 1:40 tc filter add dev ifb0 parent 1:0 prio 0 protocol ip handle 60 fw flowid 1:60 # Starts dummy IFB device used to shape ingress traffic as egress traffic ifconfig ifb0 up # send all traffic to the ifb0 dummy interface that is not marked as local traffic tc qdisc add dev eth0 root handle 2: htb default 1 tc filter add dev eth0 parent 2: prio 0 protocol ip handle 10 fw flowid 1:10 action mirred egress redirect dev ifb0 tc filter add dev eth0 parent 2: prio 0 protocol ip handle 20 fw flowid 1:20 action mirred egress redirect dev ifb0 tc filter add dev eth0 parent 2: prio 0 protocol ip handle 30 fw flowid 1:30 action mirred egress redirect dev ifb0 tc filter add dev eth0 parent 2: prio 0 protocol ip handle 40 fw flowid 1:40 action mirred egress redirect dev ifb0 tc filter add dev eth0 parent 2: prio 0 protocol ip handle 60 fw flowid 1:60 action mirred egress redirect dev ifb0 ########################################## End of downlink configuration ########################################## ################################################################################################################### ## uplink via ppp0 ## # rt = realtime # ls = link sharing # ul = upper limit # sc = service curve ( instead of rt and ls) # Default class 40 tc qdisc add dev ppp0 root handle 1: hfsc default 40 # Main class tc class add dev ppp0 parent 1: classid 1:1 hfsc sc rate ${UPLINK} ul rate ${UPLINK}kbit # interactive traffic # guarantee realtime full uplink for 50ms, then 1/10 of the uplink tc class add dev ppp0 parent 1:1 classid 1:10 hfsc \ rt m1 ${UPLINK}kbit d 50ms m2 $[1*$UPLINK/10]kbit \ ls m1 ${UPLINK}kbit d 50ms m2 $[3*$UPLINK/10]kbit \ ul rate ${UPLINK}kbit # other low latency # guarantee full uplink for 200ms, then 1/10 tc class add dev ppp0 parent 1:1 classid 1:20 hfsc \ sc m1 ${UPLINK}kbit d 200ms m2 $[1*$UPLINK/10]kbit \ ul rate ${UPLINK}kbit # web browsing # Don't guarantee anything for the first second, then guarantee 5/10 tc class add dev ppp0 parent 1:1 classid 1:30 hfsc \ sc m1 0 d 1s m2 $[5*$UPLINK/10]kbit \ ul rate $[8*$DOWNLINK/10]kbit # default traffic # don't guarantee anything for the first two seconds, then guarantee 1/20 tc class add dev ppp0 parent 1:1 classid 1:40 hfsc \ sc m1 0 d 2s m2 $[1*$UPLINK/20]kbit \ ul rate $[8*$DOWNLINK/10]kbit # bad boys # don't guarantee anything for the first 10 seconds, then guarantee 1/20 tc class add dev ppp0 parent 1:1 classid 1:60 hfsc \ sc m1 0 d 10s m2 $[1*$UPLINK/20]kbit \ ul rate $[1*$UPLINK/10]kbit tc qdisc add dev ppp0 parent 1:10 handle 10: pfifo tc qdisc add dev ppp0 parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev ppp0 parent 1:30 handle 30: sfq perturb 10 tc qdisc add dev ppp0 parent 1:40 handle 40: sfq perturb 10 tc qdisc add dev ppp0 parent 1:60 handle 60: sfq perturb 10 tc filter add dev ppp0 parent 1:0 prio 0 protocol ip handle 10 fw flowid 1:10 tc filter add dev ppp0 parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20 tc filter add dev ppp0 parent 1:0 prio 0 protocol ip handle 30 fw flowid 1:30 tc filter add dev ppp0 parent 1:0 prio 0 protocol ip handle 40 fw flowid 1:40 tc filter add dev ppp0 parent 1:0 prio 0 protocol ip handle 60 fw flowid 1:60 ################################# End of uplink configuration ##################################################### --=-CGPAUZlzlcA6xmY566VT Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc --=-CGPAUZlzlcA6xmY566VT--