RE: Round Robin Load Balancing

RE: Round Robin Load Balancing

[Javier Govea]

Thanx to all of you guys for all ur help. (I just replied one of ur last emails and i
think i sent it by mistake like 3 times to some of you guys, my apologies for
that...sorry, it was a mistake)

I think Nth is what i've been looking for, but i need to test it first. I'm getting a
problem when i use my own compiled version of iptables (even without the Nth patch). My
LAN is not accessing internet, i think traffic is not being masqueraded but i don't know
why. Any ideas on why my compiled verion does not work?? and the redhat version does work????

Here are two problems i found:

1. "depmod -a" is giving this message  (i'm showing only one message, but i'm getting the
same thing for all iptables modules): 

depmod: *** Unresolved symbols in
/lib/modules/2.4.20-8/kernel/net/ipv4/netfilter/ipt_MASQUERADE.o

how do i fix this????????

2. "modprobe ipt_MASQUERADE" is giving me an error (all other iptables modules seems to be
working without a problem):

modprobe: Too deep recursion in module dependencies!
modprobe: Circular dependency? ip_nat_core ip_nat_proto_udp ip_conntrack ip_tables
ipt_MASQUERADE
Aborted (core dumped)

how do i fix this??????

Finally, I actually don't know if any of my two previous errors are my problem. The
iptables that comes with redhat gives me the same error i'm describing in 2. But if i
insert the redhat module with "insmod" then my LAN can access internet. If i insert my own
compiled module I still cannot access the net. 

I'm also getting the same problem i'm describing in 1 with an NTFS module i compiled. But
my NTFS module is working properly. so...i'm all confused...where else can i look to find
my problem?????.....please any help is very much appreciated...

Thanx to all of you guys..
X


> It's the NTH patch. he he p-o-m.. 
> 
> Thanks,
> ____________________________________________
> George Vieira 
>
> 
> There is an extension that says something like every N packets, execute
> this rule. I forgot what it was called though.. *doh*
> 
> Try looking back ~ 1 month ago. I know I saw it there somewhere.
> 
> -----Original Message-----
> From: Javier Govea [mailto:jgovea@magma.ca] 
> Sent: Thursday, July 17, 2003 1:30 PM
> To: Ramin Dousti; Daniel Chemko
> Cc: netfilter@lists.netfilter.org
> Subject: Re: Round Robin Load Balancing
> 
> I undesrtand what you mean about perfect load balancing (i'm not
> actually looking for a
> perfect load balancer) I have two examples below, but first i will
> responde some of the
> questions.
> 
> > Do these two ppp accounts belong to the same ISP? 
> Yes. I have four accounts, all of them with the same ISP
> >Does the ISP drop forign src?
> ?????
> > Is the gateway doing nat? 
> Yes, im using iptables to setup the nat
> > Do you have any preference on one of the ppp's than the other? 
> No
> >Can you bond (mppp)?
> I haven't tried multilink ppp...i will look into this...
> 
> > You could also setup something like BGP to allow multiple routes
> to....
> I don't know if this would be the best approach. I already tried to
> setup BGP and OSPF 
> routes using zebra (<a href='http://zebra.org'>http://zebra.org</a>) and i never made it
work....
> I found a tool called EQLPlus
> (<a
href='http://www.cwareco.com/download/eqlplus.html'>http://www.cwareco.com/download/eqlplus.html</a>)
but i was
> never able to compile it. Has anybody has tried eqlplus before???????
> 
> > If one user makes a request out of line X then the return packet HAS
> to come back      
>  > through line X. So, if one guy sends a huge request taking minutes to
> fulfill, he / she
> > will tie up the line until the job is finished
> 
> Absolutely. I can live with that, but here is my problem. I have 4 ppp
> links on my router
> (which is doing nat). Then if in a host, located in my LAN which
> connected to my router, i
> open four browsers and each browser is pointing to the same site then
> i'm expepecting each
> web page to be requested and returned in a different link. But that
> doesn't happen. Some
> times it does happen but most of the time i get three of the responses
> on one link, one in
> another and the other two links do nothing. Sometimes i get 2,1,1,0 ....
> 
> I did another test...i have website with has in its main web page has
> only 4 images
> (differnt images but all of them of exactely the same size). if i point
> my browser to that
> site, then i'm believe the browser is sending four http requests (one
> for each image),
> well i would expect one image on each link....but again sometimes i get
> the four images on
> the same link...some times i get 2 images in one link...
> 
> So, i don't want a perfect load balancer but i would like to fix the
> problems on my two
> examples... i thought about implementing a round robin algorithm for
> load balancing where
> my first request goes on my first available link, the second one on the
> second available
> link and so on....this idea fixes my problems in my two previous
> examples, but i'm open to
> suggestions....
> 
> any tips, pointer, ideas are all welcome...
> 
> cheers...
> X
> 
> 
> 
> > Absolutely. Perfect load balancing needs to be coordinated on _all_
> the
> > endpoints of the links involved. In this case, 4 endpoints.
> > 
> > For a regular load balancing (which is going to be the case here)
> > we still have lots of unknown variables. Do these two ppp
> > accounts belong to the same ISP? Does the ISP drop forign src?
> > Is the gateway doing nat? Do you have any preference on one of
> > the ppp's than the other? Can you bond (mppp)? And so on.
> > But a fun project, though, for someone who has time...
> > 
> > Ramin
> > 
> > On Tue, Jul 15, 2003 at 12:44:40PM -0700, Daniel Chemko wrote:
> > 
> > > Because of the nature of your setup, you cannot have a perfect equal
> > > load balance setup. This is because you cannot control the inbound
> flow
> > > of data. If one user makes a request out of line X then the return
> > > packet HAS to come back through line X. So, if one guy sends a huge
> > > request taking minutes to fulfill, he / she will tie up the line
> until
> > > the job is finished. The load balancer should be smart enough to not
> > > send any more requests to that line, but you are still seeing the
> line
> > > being monopolized by this single connection, hence it is not
> balanced
> > > over all lines equally.
> > > 
> > > In order to have fair balancing of all lines, I think you need to
> set up
> > > a deal with your ISP to load balance on their end as well.
> > > 
> > > You could also setup something like BGP to allow multiple routes to
> the
> > > same return address, but I am not familiar enough with BGP to be
> much
> > > help in this area. In all likelihood, you are better off with your
> > > current solution or maybe the ISP solution if it is supported by
> them
> > > (more money usually).
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: Javier Govea [mailto:jgovea@magma.ca] 
> > > Sent: Tuesday, July 15, 2003 12:34 PM
> > > To: netfilter@lists.netfilter.org
> > > Subject: Round Robin Load Balancing 
> > > 
> > > Hi,
> > > 
> > > I'm trying to do some load balancing with four ppp connections. Here
> is
> > > what i have: a LAN
> > > connected to a redhat box which has four ppp interfaces. All the
> boxes
> > > in the LAN are
> > > accesing internet through the ppp interfaces in the redhat box. I'm
> > > using iproute2, in my
> > > redhat box, to setup the the four ppp interfaces as my default out
> going
> > > route (as
> > > described in LART <a
> href='<a href='http://lartc.org/howto/index.html&#39;&gt;<a
href='http://lartc.org/howto/index.ht'>http://lartc.org/howto/index.ht</a>'>http://lartc.org/howto/index.html'><a
href='http://lartc.org/howto/index.ht'>http://lartc.org/howto/index.ht</a></a>
> ml</a>) and I'm using
> > > iptables to masquerade
> > > all the traffic comming from the LAN. 
> > > 
> > > My setup is working fine, ie. my LAN can access the net throught the
> > > four ppp interfaces.
> > > My problem is that i don't know how is the load balancing working.
> Some
> > > times one of the
> > > ppp interfaces is used more than the others (and that is my
> problem).
> > > According to LART
> > > the routes are cached, can someone go a bit into more details in
> this
> > > caching thing??? how
> > > does it work?  which particular files in the kernel are doing this?
> 
> > > 
> > > I would like to implement a simple round robing algorithm (with no
> > > caching) for doing the
> > > laod balancing. That is first connection established gose through
> ppp0,
> > > the second
> > > connection on ppp1 and so on. 
> > > 
> > > I could hack iproute2 and/or iptables, but i'm not sure about which
> > > particular files i
> > > should hack in order to implement this round robin algorithm. I
> actually
> > > don't know if
> > > what i want makes any sense
> > > 
> > > Any ideas or pointers are all very well appreaciated.
> > > Thanx to all
> > > X
> > > 
> > > 
> > 
> 
> 

Re: Round Robin Load Balancing

[Vivek Kashyap]

Hi,

I came across the discussion below -- here is what I tried for
load-balancing across multiple interfaces :

1. created a target called RANGEMARK.

-j RANGEMARK --set-rangemark <base>[-limit[.increment][:r|h][,S|O|A]

The idea is to be able to mark the packets in the range base to limit with
increments of <increment>. r implies the values, within the range, are
picked randomly. h is porthash. S implise set the nfmark value, O is OR
and A is AND.

2. Now set the equalize path

ip route add Z src X equalize nexthop via Y1 dev y1 \
			      nexthop via Y2 dev y2 \
			      nexthop via Y3 dev y3


3. Set the iptables rules

iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark
iptables -t mangle -A OUTPUT -p tcp --tcp-flags SYN,RST,FIN SYN -j \
                                    RANGEMARK --set-rangemark 1-4:r,S
iptables -t mangle -A OUTPUT -p tcp --tcp-flags SYN,RST,FIN SYN \
                                     -j CONNMARK --save-mark

4. Must have enabled equal path routes and fwmark based routing in the
kernel


Now, every new connection (SYN flag) will be assigned a random mark by the
RANGEMARK target. The route cache will add the route with the fwmark
included. The next connection to the same destination will likely not get
the same fwmark and so will not match the cached routes and the kernel
will pick a route one of the other equal path routes. The 3rd rule stores
the mark value for a connection. The first rule marks all packets
belonging to the same TCP session with the same mark so that they follow
the same path out.


Vivek

PS. I posted the RANGEMARK patch to netfilter-devel on Nov. 6th.

====================================

>With that regard it doesn't matter if the routing desision has already
been made or not...
I would've thought it does matter as your packet has gone through all the
proper chain paths and now it's about to leave into the ethernet and your
now forcing it back through routing and out again through another
interface.. unless the patch jumps a bit or something...

OK the URL makes the point, as you saw my example was all there was so I
kinda hard coded that what it was supposed to be used in... (PREROUTING)..
I guess it couldn't hurt trying it though.. ;P

Has anybody tried marking the packets and then using iproute2 instead of
the ROUTE patch to go via a different ethernet??

Thanks,
____________________________________________
George Vieira
Systems Manager
georgev@citadelcomputer.com.au

Citadel Computer Systems Pty Ltd
http://www.citadelcomputer.com.au


-----Original Message-----
From: Ramin Dousti [mailto:ramin@cannon.eng.us.uu.net]
Sent: Monday, August 11, 2003 5:26 PM
To: George Vieira
Cc: Javier Govea; netfilter@lists.netfilter.org
Subject: Re: Round Robin Load Balancing


On Mon, Aug 11, 2003 at 08:30:20AM +1000, George Vieira wrote:

> Hate to burst your bubble

Hey, don't worry...

> but isn't the ROUTE module being used a _little_ too late.

No, not really. At least not according to:

http://www.netfilter.org/documentation/pomlist/pom-extra.html#ROUTE

> I mean, it's suppose to reroute to a new device but your using it in
POSTROUTING which means it's to late to reroute it (basically leaving the
interface and out to the internet)...
>

That is the traditional routing, but this module _forces_ the packet to go
wherever you want... With that regard it doesn't matter if the routing
desision has already been made or not...

Ramin

> >From memory, the ROUTE module is supposed to be used in PREROUTING on
the internal interface so that it doesn't hit the routing table yet and
the rule modifies which interface to go out on..
>


__

Vivek Kashyap
Linux Technology Center, IBM

RE: Round Robin Load Balancing

[Javier Govea]

Hi guys,

Well I tried the rules as you suggested, but this rule:

iptables -t nat -A PREROUTING -m connmark --mark 1 -j ROUTE --oif ppp0
...

give me an error: Invalid argument. According to this link:

http://www.netfilter.org/documentation/pomlist/pom-extra.html#ROUTE

the ROUTE target "has to be used inside the mangle table." So I can't use it in the NAT
table. 

The file mentioned by Geroge
(~georgev/iptables-1.2.7a/patch-o-matic/extra/ROUTE.patch.help) has an example using ROUTE
with the NAT table. However this example seems to refer to iptables-1.2.7a. I am using
iptables-1.2.8. I don't think it's a good idea to go back to install an older version.
What do u guys think??

Any more suggestions or ideas????? All tips are very welcome...
Thanx to all
Xavier



On Aug 11, "George Vieira" <georgev@citadelcomputer.com.au> wrote:
> 
> Hate to burst your bubble but isn't the ROUTE module being used a _little_ too late. I
mean, it's suppose to reroute to a new device but your using it in POSTROUTING which means
it's to late to reroute it (basically leaving the interface and out to the internet)...
> 
> From memory, the ROUTE module is supposed to be used in PREROUTING on the internal
interface so that it doesn't hit the routing table yet and the rule modifies which
interface to go out on..
> 
> I just did a search for the file :
~georgev/iptables-1.2.7a/patch-o-matic/extra/ROUTE.patch.help
> --------------------------------------------------------------------------------
> Author: Cédric de Launois <delaunois@info.ucl.ac.be>
> Status: In Development/Works for me
> 
>   This option adds a `ROUTE' target, which allows you to directly resend
>   a received packet through a specified interface, even and especially
>   if the packet IP address is one of the router itself. Those packets
>   are locally delivered and cannot be forwarded to another computer
>   using the standard routing mechanisms.
> 
>   ROUTE target v1.2.7 options:
>     --iface   name            Send the packet directly through iface name.
>     --ifindex index           Send the packet directly through iface index.
> 
>   Example :
>   You want to install a ssh server on a computer inside your network but
>   you also want it to appear exactly as if it was located on the router.
>   A solution is to simply reroute packets with destination port 22 to the
>   computer having the same IP as the router and hosting the ssh service,
>   thanks to this ROUTE target and an ipip tunnel.
> 
>   # iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 22 -j ROUTE --iface tunl1
>   # iptables -A PREROUTING -t nat -i tunl1 --j ROUTE --iface eth0
> --------------------------------------------------------------------------------
> So my guess is that you need to change some lines eg:
> 
> > iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
> > iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ppp1-ip-addr>
> > iptables -t nat -A POSTROUTING -o ppp2 -j SNAT --to-source <ppp2-ip-addr>
> > iptables -t nat -A POSTROUTING -o ppp3 -j SNAT --to-source <ppp3-ip-addr>
> This appears to be OK.
> 
> > iptables -t mangle -A PREROUTING -m nth --every 4 --packet 1 -m state --state new  -j
CONNMARK --set-mark 1
> > iptables -t mangle -A PREROUTING -m nth --every 4 --packet 2 -m state --state new  -j
CONNMARK --set-mark 2
> > iptables -t mangle -A PREROUTING -m nth --every 4 --packet 3 -m state --state new  -j
CONNMARK --set-mark 3
> > iptables -t mangle -A PREROUTING -m nth --every 4 --packet 0 -m state --state new  -j
CONNMARK --set-mark 4
> This _might_ be OK.. as long as it works and marks them.
> 
> > iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 1 -m state --state new  -j
CONNMARK --set-mark 1
> > iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 2 -m state --state new  -j
CONNMARK --set-mark 2
> > iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 3 -m state --state new  -j
CONNMARK --set-mark 3
> > iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 0 -m state --state new  -j
CONNMARK --set-mark 4
> This only gets used by the local machine itself but usefull also if your SQUID and
transparent proxy.
> 
> > iptables -t mangle -A POSTROUTING -m connmark --mark 1 -j ROUTE --oif ppp1
> > iptables -t mangle -A POSTROUTING -m connmark --mark 2 -j ROUTE --oif ppp2
> > iptables -t mangle -A POSTROUTING -m connmark --mark 3 -j ROUTE --oif ppp3
> > iptables -t mangle -A POSTROUTING -m connmark --mark 4 -j ROUTE --oif ppp0
> This should be replaced by the lines below (I think..)..
> 
>  iptables -t nat -A PREROUTING -m connmark --mark 1 -j ROUTE --oif ppp1
>  iptables -t nat -A PREROUTING -m connmark --mark 2 -j ROUTE --oif ppp2
>  iptables -t nat -A PREROUTING -m connmark --mark 3 -j ROUTE --oif ppp3
>  iptables -t nat -A PREROUTING -m connmark --mark 4 -j ROUTE --oif ppp0
> 
> Give this a try but I don't know if all of this is right and there's nothing else
missing... looks OK.
> 
> Also, when testing use telnet and don't use a browser.. makes it easier to debug whereas
a browser pulls up to 20 connections and hard to figure out. Just telnet to a specific
host on the internet and use tcpdump on that host to see where the packets are routing
through..
> 
> Good luck.
> 
> 
> Thanks,
> ____________________________________________
> George Vieira
> Systems Manager
> georgev@citadelcomputer.com.au
> 
> Citadel Computer Systems Pty Ltd
> <a href='http://www.citadelcomputer.com.au'>http://www.citadelcomputer.com.au</a>
> 
> 
> -----Original Message-----
> From: Javier Govea [mailto:jgovea@magma.ca]
> Sent: Monday, August 11, 2003 5:08 AM
> To: Ramin Dousti; Javier Govea
> Cc: netfilter@lists.netfilter.org
> Subject: Re: Round Robin Load Balancing
> 
> 

RE: Round Robin Load Balancing

[George Vieira]

>With that regard it doesn't matter if the routing desision has already been made or not...
I would've thought it does matter as your packet has gone through all the proper chain paths and now it's about to leave into the ethernet and your now forcing it back through routing and out again through another interface.. unless the patch jumps a bit or something...

OK the URL makes the point, as you saw my example was all there was so I kinda hard coded that what it was supposed to be used in... (PREROUTING)..
I guess it couldn't hurt trying it though.. ;P

Has anybody tried marking the packets and then using iproute2 instead of the ROUTE patch to go via a different ethernet??

Thanks,
____________________________________________
George Vieira
Systems Manager
georgev@citadelcomputer.com.au

Citadel Computer Systems Pty Ltd
http://www.citadelcomputer.com.au
 

-----Original Message-----
From: Ramin Dousti [mailto:ramin@cannon.eng.us.uu.net]
Sent: Monday, August 11, 2003 5:26 PM
To: George Vieira
Cc: Javier Govea; netfilter@lists.netfilter.org
Subject: Re: Round Robin Load Balancing


On Mon, Aug 11, 2003 at 08:30:20AM +1000, George Vieira wrote:

> Hate to burst your bubble

Hey, don't worry...

> but isn't the ROUTE module being used a _little_ too late.

No, not really. At least not according to:

http://www.netfilter.org/documentation/pomlist/pom-extra.html#ROUTE

> I mean, it's suppose to reroute to a new device but your using it in POSTROUTING which means it's to late to reroute it (basically leaving the interface and out to the internet)...
> 

That is the traditional routing, but this module _forces_ the packet to go
wherever you want... With that regard it doesn't matter if the routing
desision has already been made or not...

Ramin

> >From memory, the ROUTE module is supposed to be used in PREROUTING on the internal interface so that it doesn't hit the routing table yet and the rule modifies which interface to go out on..
> 

Re: Round Robin Load Balancing

[Ramin Dousti]

On Mon, Aug 11, 2003 at 08:30:20AM +1000, George Vieira wrote:

> Hate to burst your bubble

Hey, don't worry...

> but isn't the ROUTE module being used a _little_ too late.

No, not really. At least not according to:

http://www.netfilter.org/documentation/pomlist/pom-extra.html#ROUTE

> I mean, it's suppose to reroute to a new device but your using it in POSTROUTING which means it's to late to reroute it (basically leaving the interface and out to the internet)...
> 

That is the traditional routing, but this module _forces_ the packet to go
wherever you want... With that regard it doesn't matter if the routing
desision has already been made or not...

Ramin

> >From memory, the ROUTE module is supposed to be used in PREROUTING on the internal interface so that it doesn't hit the routing table yet and the rule modifies which interface to go out on..
> 

RE: Round Robin Load Balancing

[George Vieira]

Hate to burst your bubble but isn't the ROUTE module being used a _little_ too late. I mean, it's suppose to reroute to a new device but your using it in POSTROUTING which means it's to late to reroute it (basically leaving the interface and out to the internet)...

From memory, the ROUTE module is supposed to be used in PREROUTING on the internal interface so that it doesn't hit the routing table yet and the rule modifies which interface to go out on..

I just did a search for the file : ~georgev/iptables-1.2.7a/patch-o-matic/extra/ROUTE.patch.help
--------------------------------------------------------------------------------
Author: Cédric de Launois <delaunois@info.ucl.ac.be>
Status: In Development/Works for me

  This option adds a `ROUTE' target, which allows you to directly resend
  a received packet through a specified interface, even and especially
  if the packet IP address is one of the router itself. Those packets
  are locally delivered and cannot be forwarded to another computer
  using the standard routing mechanisms.

  ROUTE target v1.2.7 options:
    --iface   name            Send the packet directly through iface name.
    --ifindex index           Send the packet directly through iface index.

  Example :
  You want to install a ssh server on a computer inside your network but
  you also want it to appear exactly as if it was located on the router.
  A solution is to simply reroute packets with destination port 22 to the
  computer having the same IP as the router and hosting the ssh service,
  thanks to this ROUTE target and an ipip tunnel.

  # iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 22 -j ROUTE --iface tunl1
  # iptables -A PREROUTING -t nat -i tunl1 --j ROUTE --iface eth0
--------------------------------------------------------------------------------
So my guess is that you need to change some lines eg:

> iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ppp1-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp2 -j SNAT --to-source <ppp2-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp3 -j SNAT --to-source <ppp3-ip-addr>
This appears to be OK.

> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 1 -m state --state new  -j CONNMARK --set-mark 1
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 2 -m state --state new  -j CONNMARK --set-mark 2
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 3 -m state --state new  -j CONNMARK --set-mark 3
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 0 -m state --state new  -j CONNMARK --set-mark 4
This _might_ be OK.. as long as it works and marks them.

> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 1 -m state --state new  -j CONNMARK --set-mark 1
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 2 -m state --state new  -j CONNMARK --set-mark 2
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 3 -m state --state new  -j CONNMARK --set-mark 3
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 0 -m state --state new  -j CONNMARK --set-mark 4
This only gets used by the local machine itself but usefull also if your SQUID and transparent proxy.

> iptables -t mangle -A POSTROUTING -m connmark --mark 1 -j ROUTE --oif ppp1
> iptables -t mangle -A POSTROUTING -m connmark --mark 2 -j ROUTE --oif ppp2
> iptables -t mangle -A POSTROUTING -m connmark --mark 3 -j ROUTE --oif ppp3
> iptables -t mangle -A POSTROUTING -m connmark --mark 4 -j ROUTE --oif ppp0
This should be replaced by the lines below (I think..)..

 iptables -t nat -A PREROUTING -m connmark --mark 1 -j ROUTE --oif ppp1
 iptables -t nat -A PREROUTING -m connmark --mark 2 -j ROUTE --oif ppp2
 iptables -t nat -A PREROUTING -m connmark --mark 3 -j ROUTE --oif ppp3
 iptables -t nat -A PREROUTING -m connmark --mark 4 -j ROUTE --oif ppp0

Give this a try but I don't know if all of this is right and there's nothing else missing... looks OK.

Also, when testing use telnet and don't use a browser.. makes it easier to debug whereas a browser pulls up to 20 connections and hard to figure out. Just telnet to a specific host on the internet and use tcpdump on that host to see where the packets are routing through..

Good luck.


Thanks,
____________________________________________
George Vieira
Systems Manager
georgev@citadelcomputer.com.au

Citadel Computer Systems Pty Ltd
http://www.citadelcomputer.com.au


-----Original Message-----
From: Javier Govea [mailto:jgovea@magma.ca]
Sent: Monday, August 11, 2003 5:08 AM
To: Ramin Dousti; Javier Govea
Cc: netfilter@lists.netfilter.org
Subject: Re: Round Robin Load Balancing

Re: Round Robin Load Balancing

[Javier Govea]

Hi guys, 

I think i sent this message twice by mistake, very sorry about that. This is the email you
want to read, please discard the first email.

And sorry about this late response, i've been playing with the kernel during this time.
Well, i tested the CONNMARK option (the rules i tested are below) for the round robin, but
i still get the same problem, when my browser just doesn't load any webpage and eventually
times out. 

I hacked into the iptables modules: ipt_MASQUERADE and ip_tables and I put some "printk"s
to see what was happening. The "printk"s in the ip_tables module show that the ppp links
are  chosen in a round robin fashion, which is very good. 

The rules I'm showing below use

"-j SNAT --to-source <ppp0-ip-addr>" 

in the NAT table. I however used  

"-j MASQUERADE" 

instead because i believe this two jumps are equivalent and because i put some prints in
the ipt_MASQUERADE module. (However if i'm wrong please correct me) Well the prints from
the MASQUERADE module doesn't show anything, which means that the packets are never been
masqueraded. 

So my browser is timing out because the packets, if they are sent at all, they are not
being masqueraded, so they don't know how to return.

The CONNMARK rules work fine for the round robin but packets are not masqueraded. I need
to somehow connect this two set rules or to force the masqueraded somehow. I think
iptables is getting mixup with all these rules and it doesn't know how to handle them. 

Any ideas or suggestions?? all are very welcome..
Thanx to all of you guys...
Cheers
Xavier

PS Here are the last rules that i tested:
 
> iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ppp1-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp2 -j SNAT --to-source <ppp2-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp3 -j SNAT --to-source <ppp3-ip-addr>
> 
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 1 -m state --state new  -j
CONNMARK --set-mark 1
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 2 -m state --state new  -j
CONNMARK --set-mark 2
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 3 -m state --state new  -j
CONNMARK --set-mark 3
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 0 -m state --state new  -j
CONNMARK --set-mark 4
> 
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 1 -m state --state new  -j
CONNMARK --set-mark 1
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 2 -m state --state new  -j
CONNMARK --set-mark 2
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 3 -m state --state new  -j
CONNMARK --set-mark 3
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 0 -m state --state new  -j
CONNMARK --set-mark 4
> 
> iptables -t mangle -A POSTROUTING -m connmark --mark 1 -j ROUTE --oif ppp1
> iptables -t mangle -A POSTROUTING -m connmark --mark 2 -j ROUTE --oif ppp2
> iptables -t mangle -A POSTROUTING -m connmark --mark 3 -j ROUTE --oif ppp3
> iptables -t mangle -A POSTROUTING -m connmark --mark 4 -j ROUTE --oif ppp0
> 
> Haven't had the chance to test it, though.
> 
> Ramin
> 

Re: Round Robin Load Balancing

[Javier Govea]

Hi guys, sorry about this late response, i've been playing with the kernel during this
time. Well, i tested the CONNMARK option (the rules i tested are below) for the round
robin, but i still get the same problem, when my browser just doesn't load any webpage and
eventually times out. 

I hacked into the iptables modules: ipt_MASQUERADE and ip_tables and put some "printk"s to
see what was happening. The "printk"s in the ip_tables module show that the ppp links are
 chosen in a round robin fashion, which is very good. The rules I'm shoing below use
"-j SNAT --to-source <ppp0-ip-addr>" in the NAT table. I however used  
"-j MASQUERADE" instead because i believe this two jumps are equivalent and because i put
some prints in the ipt_MASQUERADE module. Well the prints from the MASQUERADE module
doesn't show anything, which means that the packets are never been masqueraded. 

So my browser is timing out because the packets, if they are sent at all, they are not
being masqueraded, so they don't know how to return.

The CONNMARK
 
> iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ppp1-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp2 -j SNAT --to-source <ppp2-ip-addr>
> iptables -t nat -A POSTROUTING -o ppp3 -j SNAT --to-source <ppp3-ip-addr>
> 
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 1 -m state --state new  -j
CONNMARK --set-mark 1
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 2 -m state --state new  -j
CONNMARK --set-mark 2
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 3 -m state --state new  -j
CONNMARK --set-mark 3
> iptables -t mangle -A PREROUTING -m nth --every 4 --packet 0 -m state --state new  -j
CONNMARK --set-mark 4
> 
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 1 -m state --state new  -j
CONNMARK --set-mark 1
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 2 -m state --state new  -j
CONNMARK --set-mark 2
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 3 -m state --state new  -j
CONNMARK --set-mark 3
> iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 0 -m state --state new  -j
CONNMARK --set-mark 4
> 
> iptables -t mangle -A POSTROUTING -m connmark --mark 1 -j ROUTE --oif ppp1
> iptables -t mangle -A POSTROUTING -m connmark --mark 2 -j ROUTE --oif ppp2
> iptables -t mangle -A POSTROUTING -m connmark --mark 3 -j ROUTE --oif ppp3
> iptables -t mangle -A POSTROUTING -m connmark --mark 4 -j ROUTE --oif ppp0
> 
> Haven't had the chance to test it, though.
> 
> Ramin
> 
> 

Re: Round Robin Load Balancing

[Ramin Dousti]

On Thu, Jul 31, 2003 at 10:48:53AM -0400, Javier Govea wrote:

> Hi guys,
> 
> Well I tried to use the "nth" patch in the mangle table in both the FORWARD and PREROUTING
> chains and still no success. My browser still cannot surf internet, it just waits and
> eventually times out. The rules i tested are below, please correct them if they are wrong.
> 
> Any more suggestions??? any other ideas????

Yes. CONNMARK the conn's and route them accordingly:

iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ppp1-ip-addr>
iptables -t nat -A POSTROUTING -o ppp2 -j SNAT --to-source <ppp2-ip-addr>
iptables -t nat -A POSTROUTING -o ppp3 -j SNAT --to-source <ppp3-ip-addr>

iptables -t mangle -A PREROUTING -m nth --every 4 --packet 1 -m state --state new  -j CONNMARK --set-mark 1
iptables -t mangle -A PREROUTING -m nth --every 4 --packet 2 -m state --state new  -j CONNMARK --set-mark 2
iptables -t mangle -A PREROUTING -m nth --every 4 --packet 3 -m state --state new  -j CONNMARK --set-mark 3
iptables -t mangle -A PREROUTING -m nth --every 4 --packet 0 -m state --state new  -j CONNMARK --set-mark 4

iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 1 -m state --state new  -j CONNMARK --set-mark 1
iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 2 -m state --state new  -j CONNMARK --set-mark 2
iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 3 -m state --state new  -j CONNMARK --set-mark 3
iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 0 -m state --state new  -j CONNMARK --set-mark 4

iptables -t mangle -A POSTROUTING -m connmark --mark 1 -j ROUTE --oif ppp1
iptables -t mangle -A POSTROUTING -m connmark --mark 2 -j ROUTE --oif ppp2
iptables -t mangle -A POSTROUTING -m connmark --mark 3 -j ROUTE --oif ppp3
iptables -t mangle -A POSTROUTING -m connmark --mark 4 -j ROUTE --oif ppp0

Haven't had the chance to test it, though.

Ramin

RE: Round Robin Load Balancing

[Javier Govea]

Hi guys,

Well I tried to use the "nth" patch in the mangle table in both the FORWARD and PREROUTING
chains and still no success. My browser still cannot surf internet, it just waits and
eventually times out. The rules i tested are below, please correct them if they are wrong.

Any more suggestions??? any other ideas????

Many thanks to all of you guys...
Xavier
PS. I tried th following 3 sets rules:

1. This rule:
iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
...

together with this one (i tested this rule in both chains FORWARD and PREROUTING):

iptables -t mangle -A FORWARD -m nth --every 4 --packet 0 -p --syn -j ROUTE --oif ppp0
...

2. This rule:
iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
...

together with this one (i tested this rule in both chains FORWARD and PREROUTING):

iptables -t mangle -A FORWARD -m nth --every 4 --packet 0 -m state --state new  -j ROUTE
--oif ppp0
...

3.And this rule

iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
...

together with this one (i tested this rule in both chains FORWARD and PREROUTING):

iptables -t mangle -A FORWARD -m nth --every 4 --packet 0 -m state -p tcp --syn --state
new  -j ROUTE --oif ppp0
...



On Jul 29, "George Vieira" <georgev@citadelcomputer.com.au> wrote:
> 
> probably because you should be doing that on the SYN packet ONLY.. not on any packet
because it'll send 1 connection across multiple interfaces...
> 
> Thanks,
> ____________________________________________
> George Vieira
> Systems Manager
> georgev@citadelcomputer.com.au
> 
> Citadel Computer Systems Pty Ltd
> <a href='http://www.citadelcomputer.com.au'>http://www.citadelcomputer.com.au</a>
>  
> 
> -----Original Message-----
> From: Javier Govea [mailto:jgovea@magma.ca]
> Sent: Tuesday, July 29, 2003 7:15 AM
> To: Chris Wilson; Javier Govea
> Cc: Daniel Chemko; George Vieira; netfilter@lists.netfilter.org; Ramin
> Dousti; netfilter@lists.netfilter.org
> Subject: Re: Round Robin Load Balancing
> 
> 
> Hi guys,
> 
> Thanx for all your suggestions, but i have bad news. The rules you guys are suggesting 
> make a lot of sense to me but they don't work. My hosts still cannot surf the net. 
> 
> This rule:
> 
> iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>
> 
> is another form for the masquerade rule:
> 
> iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
> 
> Using either of these two rules my hosts can access internet, but i still have the 
> problem of the load balancing. Some ppp links are used more than others in situations 
> such as loading the same web page in four different browsers.
> 
> As soon as i add the "nth" rules to the mangle table my browsers stop working, they just 
> can contact the web servers and eventually they time out. I'm not sure if by adding any 
> rule to my mangle table the browsers stop, but at least they stop when i add your 
> suggestions to the mangle table.
> 
> Any other ideas, suggestions, tips?? all are very welcome...
> Thanx to all, i really appreacite all your time and interest....
> Xavier
> 
> 
> > 
> > Try something like this:
> > 
> > iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 0 \
> > 	-j ROUTE --oif ppp0
> > iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 1 \
> >         -j ROUTE --oif ppp1
> > ...
> > 
> > iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>
> > iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ip-of-ppp1>
> > ...
> > 
> > Cheers, Chris.
> > -- 
> >    ___ __     _
> >  / __// / ,__(_)_  | Chris Wilson -- UNIX Firewall Lead Developer |
> > / (_ / ,\/ _/ /_ \ | NetServers.co.uk <a 
> href='<a href='http://www.netservers.co.uk&#39;&gt;<a
href='http://www.netservers.co.uk&lt;/a&gt;'>http://www.netservers.co.uk</a></a>'>http://www.netservers.co.uk'><a
href='http://www.netservers.co.uk&lt;/a&gt;'>http://www.netservers.co.uk</a></a></a> |
> > \ _//_/_/_//_/___/ | 21 Signet Court, Cambridge, UK. 01223 576516 |
> > 
> > 
> > 
> 
> 

RE: Round Robin Load Balancing

[Daniel Chemko]

Found a note in the patch:

/* Send the packet. This will also free skb
+		 * Do not go through the POST_ROUTING hook because 
+		 * skb->dst is not set and because it will probably
+		 * get confused by the destination IP address.
+		 */

Maybe try the ROUTE command in FORWARD instead. Hey, why not even try
PREROUTING.

-----Original Message-----
From: Javier Govea [mailto:jgovea@magma.ca] 
Sent: Tuesday, July 29, 2003 8:39 AM
To: George Vieira; Javier Govea
Cc: netfilter@lists.netfilter.org
Subject: RE: Round Robin Load Balancing

Hi guys,

Well, i tried with only the SYN packets (my rules are below) and still
nothing. My hosts
cannot access the net. Any more ideas??? 

Here are are my rules (if they are wrong, please correct me):

iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>
.

iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 0 -p tcp
--syn \
	-j ROUTE --oif ppp0
.

Thanx...
Xavier



On Jul 29, "George Vieira" <georgev@citadelcomputer.com.au> wrote:
> 
> probably because you should be doing that on the SYN packet ONLY.. not
on any packet
because it'll send 1 connection across multiple interfaces...
> 
> Thanks,
> ____________________________________________
> George Vieira
> Systems Manager
> georgev@citadelcomputer.com.au
> 
> Citadel Computer Systems Pty Ltd
> <a
href='http://www.citadelcomputer.com.au'>http://www.citadelcomputer.com.
au</a>
>  
> 
> -----Original Message-----
> From: Javier Govea [mailto:jgovea@magma.ca]
> Sent: Tuesday, July 29, 2003 7:15 AM
> To: Chris Wilson; Javier Govea
> Cc: Daniel Chemko; George Vieira; netfilter@lists.netfilter.org; Ramin
> Dousti; netfilter@lists.netfilter.org
> Subject: Re: Round Robin Load Balancing
> 
> 
> Hi guys,
> 
> Thanx for all your suggestions, but i have bad news. The rules you
guys are suggesting 
> make a lot of sense to me but they don't work. My hosts still cannot
surf the net. 
> 
> This rule:
> 
> iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source
<ip-of-ppp0>
> 
> is another form for the masquerade rule:
> 
> iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
> 
> Using either of these two rules my hosts can access internet, but i
still have the 
> problem of the load balancing. Some ppp links are used more than
others in situations 
> such as loading the same web page in four different browsers.
> 
> As soon as i add the "nth" rules to the mangle table my browsers stop
working, they just 
> can contact the web servers and eventually they time out. I'm not sure
if by adding any 
> rule to my mangle table the browsers stop, but at least they stop when
i add your 
> suggestions to the mangle table.
> 
> Any other ideas, suggestions, tips?? all are very welcome...
> Thanx to all, i really appreacite all your time and interest....
> Xavier
> 
> 
> > 
> > Try something like this:
> > 
> > iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 0 \
> > 	-j ROUTE --oif ppp0
> > iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 1 \
> >         -j ROUTE --oif ppp1
> > ...
> > 
> > iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source
<ip-of-ppp0>
> > iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source
<ip-of-ppp1>
> > ...
> > 
> > Cheers, Chris.
> > -- 
> >    ___ __     _
> >  / __// / ,__(_)_  | Chris Wilson -- UNIX Firewall Lead Developer |
> > / (_ / ,\/ _/ /_ \ | NetServers.co.uk <a 
> href='<a href='http://www.netservers.co.uk&#39;&gt;<a
href='http://www.netservers.co.uk&lt;/a&gt;'>http://www.netservers.co.uk
</a></a>'>http://www.netservers.co.uk'><a
href='http://www.netservers.co.uk&lt;/a&gt;'>http://www.netservers.co.uk
</a></a></a> |
> > \ _//_/_/_//_/___/ | 21 Signet Court, Cambridge, UK. 01223 576516 |
> > 
> > 
> > 
> 
> 

RE: Round Robin Load Balancing

[Javier Govea]

Hi guys,

Well, i tried with only the SYN packets (my rules are below) and still nothing. My hosts
cannot access the net. Any more ideas??? 

Here are are my rules (if they are wrong, please correct me):

iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>
...

iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 0 -p tcp --syn \
	-j ROUTE --oif ppp0
...

Thanx...
Xavier



On Jul 29, "George Vieira" <georgev@citadelcomputer.com.au> wrote:
> 
> probably because you should be doing that on the SYN packet ONLY.. not on any packet
because it'll send 1 connection across multiple interfaces...
> 
> Thanks,
> ____________________________________________
> George Vieira
> Systems Manager
> georgev@citadelcomputer.com.au
> 
> Citadel Computer Systems Pty Ltd
> <a href='http://www.citadelcomputer.com.au'>http://www.citadelcomputer.com.au</a>
>  
> 
> -----Original Message-----
> From: Javier Govea [mailto:jgovea@magma.ca]
> Sent: Tuesday, July 29, 2003 7:15 AM
> To: Chris Wilson; Javier Govea
> Cc: Daniel Chemko; George Vieira; netfilter@lists.netfilter.org; Ramin
> Dousti; netfilter@lists.netfilter.org
> Subject: Re: Round Robin Load Balancing
> 
> 
> Hi guys,
> 
> Thanx for all your suggestions, but i have bad news. The rules you guys are suggesting 
> make a lot of sense to me but they don't work. My hosts still cannot surf the net. 
> 
> This rule:
> 
> iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>
> 
> is another form for the masquerade rule:
> 
> iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE
> 
> Using either of these two rules my hosts can access internet, but i still have the 
> problem of the load balancing. Some ppp links are used more than others in situations 
> such as loading the same web page in four different browsers.
> 
> As soon as i add the "nth" rules to the mangle table my browsers stop working, they just 
> can contact the web servers and eventually they time out. I'm not sure if by adding any 
> rule to my mangle table the browsers stop, but at least they stop when i add your 
> suggestions to the mangle table.
> 
> Any other ideas, suggestions, tips?? all are very welcome...
> Thanx to all, i really appreacite all your time and interest....
> Xavier
> 
> 
> > 
> > Try something like this:
> > 
> > iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 0 \
> > 	-j ROUTE --oif ppp0
> > iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 1 \
> >         -j ROUTE --oif ppp1
> > ...
> > 
> > iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>
> > iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ip-of-ppp1>
> > ...
> > 
> > Cheers, Chris.
> > -- 
> >    ___ __     _
> >  / __// / ,__(_)_  | Chris Wilson -- UNIX Firewall Lead Developer |
> > / (_ / ,\/ _/ /_ \ | NetServers.co.uk <a 
> href='<a href='http://www.netservers.co.uk&#39;&gt;<a
href='http://www.netservers.co.uk&lt;/a&gt;'>http://www.netservers.co.uk</a></a>'>http://www.netservers.co.uk'><a
href='http://www.netservers.co.uk&lt;/a&gt;'>http://www.netservers.co.uk</a></a></a> |
> > \ _//_/_/_//_/___/ | 21 Signet Court, Cambridge, UK. 01223 576516 |
> > 
> > 
> > 
> 
> 

Re: Round Robin Load Balancing

[Javier Govea]

Hi guys,

Thanx for all your suggestions, but i have bad news. The rules you guys are suggesting 
make a lot of sense to me but they don't work. My hosts still cannot surf the net. 

This rule:

iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>

is another form for the masquerade rule:

iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE

Using either of these two rules my hosts can access internet, but i still have the 
problem of the load balancing. Some ppp links are used more than others in situations 
such as loading the same web page in four different browsers.

As soon as i add the "nth" rules to the mangle table my browsers stop working, they just 
can contact the web servers and eventually they time out. I'm not sure if by adding any 
rule to my mangle table the browsers stop, but at least they stop when i add your 
suggestions to the mangle table.

Any other ideas, suggestions, tips?? all are very welcome...
Thanx to all, i really appreacite all your time and interest....
Xavier


> 
> Try something like this:
> 
> iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 0 \
> 	-j ROUTE --oif ppp0
> iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 1 \
>         -j ROUTE --oif ppp1
> ...
> 
> iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>
> iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ip-of-ppp1>
> ...
> 
> Cheers, Chris.
> -- 
>    ___ __     _
>  / __// / ,__(_)_  | Chris Wilson -- UNIX Firewall Lead Developer |
> / (_ / ,\/ _/ /_ \ | NetServers.co.uk <a 
href='http://www.netservers.co.uk'>http://www.netservers.co.uk</a> |
> \ _//_/_/_//_/___/ | 21 Signet Court, Cambridge, UK. 01223 576516 |
> 
> 
> 

Re: Round Robin Load Balancing

[Chris Wilson]

Hi Javier,

Try something like this:

iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 0 \
	-j ROUTE --oif ppp0
iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 1 \
        -j ROUTE --oif ppp1
...

iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ip-of-ppp0>
iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ip-of-ppp1>
...

Cheers, Chris.
-- 
   ___ __     _
 / __// / ,__(_)_  | Chris Wilson -- UNIX Firewall Lead Developer |
/ (_ / ,\/ _/ /_ \ | NetServers.co.uk http://www.netservers.co.uk |
\ _//_/_/_//_/___/ | 21 Signet Court, Cambridge, UK. 01223 576516 |

RE: Round Robin Load Balancing

[Daniel Chemko]

Sorry about the bogus syntax. I was shooting form memory.

If you do the route in mangle, then I can also see the need to change
the source IP with -j MASQUERADE in -nat POSTROUTING.

# Every 4th SYN packet, send the syn down a different pipe (with a bad
IP address for the output)
iptables -t mangle -A POSTROUTING -m nth --every 4 --packet 0 -m state
--state new -j ROUTE --oif ppp0

# Fix the bad IP address from the previous command and setup a conntrack
for the session
iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE


Tell me how this goes.

-----Original Message-----
From: Javier Govea [mailto:jgovea@magma.ca] 
Sent: Sunday, July 27, 2003 10:41 AM
To: Daniel Chemko; Javier Govea
Cc: George Vieira; netfilter@lists.netfilter.org; Ramin Dousti;
netfilter@lists.netfilter.org
Subject: Re: Round Robin Load Balancing

Hi guys,

I tried the following line (i'm using "-j ROUTE" instead of "-m route",
as Daniel wrote
before, because the route patch is actually a target and because "-m
route" was giving me
a an error. But if i'm doing it in the wrong way, please correct me):
 

Re: Round Robin Load Balancing

[Javier Govea]

Hi guys,

I tried the following line (i'm using "-j ROUTE" instead of "-m route", as Daniel wrote
before, because the route patch is actually a target and because "-m route" was giving me
a an error. But if i'm doing it in the wrong way, please correct me):

iptables -t nat -A POSTROUTING -m nth --every 4 --packet 0 -m state --state new -j ROUTE
--oif ppp0

and i got:

iptables: Invalid argument

I think the ROUTE target doesn't work with the nat table. Has anybody use NAT and ROUTE 
together?? I tried also the following rule but i got the same error message:

iptables -t nat -A POSTROUTING -j ROUTE --oif ppp0

However if i use the mangle table then i do not get an error (the rule below works) but my
host cannot access inernet and no wonder, i need the NAT/MASQUERADE  stuff:

iptables -t mangle -A POSTROUTING -j ROUTE --oif ppp0

I also tried:

iptables -t nat -A POSTROUTING -m nth --every 4 --packet 0 -m state --state new -o ppp0 -j
MASQUERADE

with this rule my hosts can access internet if i have only one browser open (and it works
faster than before, when i wasn't using "-m state --state new") but if i open two browsers
then none of them can access internet.

Am i using the wrong rules??? Does anybody has any other ideas??? Can my problem of load
balancing internet traffic actually be solved with some combinations of rules?? or Am I
going in the wrong direction???

Any ideas or suggestions are all very much appreaciated...
Thanx to all of you guys...
X
PS. In my rules above i'm only showing one rule in each case but i'm actually using four
rules (one for each ppp link) in all cases.

> Yeah, it is wrong...
> 
> #1. Only perform this on state 'new' packets. After that, let snat take 
> care of making sure they go through the right interface.
> 
> #2. You need the 'route' patch as well..
> The rules that you define will only work if the packets are going to the 
> correct interface to begin with. To allow every 4th CONNECTION to travel 
> through each interface as you described below, you might want to try:
> 
> iptables -t nat -A POSTROUTING -m nth --every 4 --packet 0 -m state --state new -m route
--oif ppp0
> iptables -t nat -A POSTROUTING -m nth --every 4 --packet 1 -m state --state new -m route
--oif ppp1
> iptables -t nat -A POSTROUTING -m nth --every 4 --packet 2 -m state --state new -m route
--oif ppp2
> iptables -t nat -A POSTROUTING -m nth --every 4 --packet 3 -m state --state new -m route
--oif ppp2
> 
> 
> As described earier, this does not guarantee 100% load balancing, but it 
> will help.
> 
> >I'm not sure if NTH does not work well with the masquerade target (has anybody use NTH and
> >Masquerade succesfully????) or if i'm applying the wrong rule (below are my rules) or if
> >just the NTH patch doesn't really work....
> >
> >any ideas or suggestions are all very welcome....
> >
> >Here are my rules:
> >
> >iptables -t nat -A POSTROUTING -m nth --every 4 --packet 0 -o ppp0 -j MASQUERADE
> >iptables -t nat -A POSTROUTING -m nth --every 4 --packet 1 -o ppp1 -j MASQUERADE
> >iptables -t nat -A POSTROUTING -m nth --every 4 --packet 2 -o ppp2 -j MASQUERADE
> >iptables -t nat -A POSTROUTING -m nth --every 4 --packet 3 -o ppp2 -j MASQUERADE
> >  
> >
> 
> 
> 

Re: Round Robin Load Balancing

[Daniel Chemko]

Yeah, it is wrong...

#1. Only perform this on state 'new' packets. After that, let snat take 
care of making sure they go through the right interface.

#2. You need the 'route' patch as well..
The rules that you define will only work if the packets are going to the 
correct interface to begin with. To allow every 4th CONNECTION to travel 
through each interface as you described below, you might want to try:

iptables -t nat -A POSTROUTING -m nth --every 4 --packet 0 -m state --state new -m route --oif ppp0
iptables -t nat -A POSTROUTING -m nth --every 4 --packet 1 -m state --state new -m route --oif ppp1
iptables -t nat -A POSTROUTING -m nth --every 4 --packet 2 -m state --state new -m route --oif ppp2
iptables -t nat -A POSTROUTING -m nth --every 4 --packet 3 -m state --state new -m route --oif ppp2


As described earier, this does not guarantee 100% load balancing, but it 
will help.

>I'm not sure if NTH does not work well with the masquerade target (has anybody use NTH and
>Masquerade succesfully????) or if i'm applying the wrong rule (below are my rules) or if
>just the NTH patch doesn't really work....
>
>any ideas or suggestions are all very welcome....
>
>Here are my rules:
>
>iptables -t nat -A POSTROUTING -m nth --every 4 --packet 0 -o ppp0 -j MASQUERADE
>iptables -t nat -A POSTROUTING -m nth --every 4 --packet 1 -o ppp1 -j MASQUERADE
>iptables -t nat -A POSTROUTING -m nth --every 4 --packet 2 -o ppp2 -j MASQUERADE
>iptables -t nat -A POSTROUTING -m nth --every 4 --packet 3 -o ppp2 -j MASQUERADE
>  
>

Re: Round Robin Load Balancing

[Ramin Dousti]

On Sat, Jul 26, 2003 at 02:21:37PM -0400, Javier Govea wrote:

> Ok guys, I was finally able to compile and install the NTH patch. But, it's not working
> for me...does anybody use it before succesfully???? 
> 
> Here is again what i'm trying to do:
> In my linux box i'm trying to masquerade some traffic comming from my LAN. The traffic is
> masqueraded before it is forwarded on 4 ppp links to my ISP. I want to load balance my
> traffic on my ppp links. By using the NTH patch, if in one of my hosts i open two browsers
> at the same time, then none of them can load any web page!! if i open only one browser
> then it is able to load a web page (but very slowly). 

If you open two it also works but "very very very" slowly. Basically, Nth is
not what you need, regardless the impression that you get by its name. You
don't want to send packets to different interfaces just because the count of
the packet mod 4 is something. There is much more involed than that specially
with TCP. When you say "it works with one browser but very slow", it's because
from 4 packets being sent out only one is sent the right way. This almost kills
TCP...

Ramin

> 
> I'm not sure if NTH does not work well with the masquerade target (has anybody use NTH and
> Masquerade succesfully????) or if i'm applying the wrong rule (below are my rules) or if
> just the NTH patch doesn't really work....
> 
> any ideas or suggestions are all very welcome....
> 
> Here are my rules:
> 
> iptables -t nat -A POSTROUTING -m nth --every 4 --packet 0 -o ppp0 -j MASQUERADE
> iptables -t nat -A POSTROUTING -m nth --every 4 --packet 1 -o ppp1 -j MASQUERADE
> iptables -t nat -A POSTROUTING -m nth --every 4 --packet 2 -o ppp2 -j MASQUERADE
> iptables -t nat -A POSTROUTING -m nth --every 4 --packet 3 -o ppp2 -j MASQUERADE
> 
> and this the default route i'm setting up with iproute2
> 
> ip route add default nexthop via 192.168.100.101 dev ppp0 weight 1 \
>                      nexthop via 192.168.100.101 dev ppp1 weight 1 \
>                      nexthop via 192.168.100.101 dev ppp2 weight 1 \
>                      nexthop via 192.168.100.101 dev ppp3 weight 1 
> 
> thanx to all
> X
> 
> 
> On Jul 18, "George Vieira" <georgev@citadelcomputer.com.au> wrote:
> > 
> > It's the NTH patch. he he p-o-m.. 
> > 
> > Thanks,
> > ____________________________________________
> > George Vieira
> > Systems Manager
> > georgev@citadelcomputer.com.au
> > 
> > Citadel Computer Systems Pty Ltd
> > <a href='http://www.citadelcomputer.com.au'>http://www.citadelcomputer.com.au</a>
> >  
> > 
> > -----Original Message-----
> > From: Daniel Chemko [mailto:dchemko@smgtec.com]
> > Sent: Friday, July 18, 2003 8:37 AM
> > To: Javier Govea; Ramin Dousti
> > Cc: netfilter@lists.netfilter.org
> > Subject: RE: Round Robin Load Balancing
> > 
> > 
> > There is an extension that says something like every N packets, execute
> > this rule. I forgot what it was called though.. *doh*
> > 
> > 

RE: Round Robin Load Balancing

[Javier Govea]

Ok guys, I was finally able to compile and install the NTH patch. But, it's not working
for me...does anybody use it before succesfully???? 

Here is again what i'm trying to do:
In my linux box i'm trying to masquerade some traffic comming from my LAN. The traffic is
masqueraded before it is forwarded on 4 ppp links to my ISP. I want to load balance my
traffic on my ppp links. By using the NTH patch, if in one of my hosts i open two browsers
at the same time, then none of them can load any web page!! if i open only one browser
then it is able to load a web page (but very slowly). 

I'm not sure if NTH does not work well with the masquerade target (has anybody use NTH and
Masquerade succesfully????) or if i'm applying the wrong rule (below are my rules) or if
just the NTH patch doesn't really work....

any ideas or suggestions are all very welcome....

Here are my rules:

iptables -t nat -A POSTROUTING -m nth --every 4 --packet 0 -o ppp0 -j MASQUERADE
iptables -t nat -A POSTROUTING -m nth --every 4 --packet 1 -o ppp1 -j MASQUERADE
iptables -t nat -A POSTROUTING -m nth --every 4 --packet 2 -o ppp2 -j MASQUERADE
iptables -t nat -A POSTROUTING -m nth --every 4 --packet 3 -o ppp2 -j MASQUERADE

and this the default route i'm setting up with iproute2

ip route add default nexthop via 192.168.100.101 dev ppp0 weight 1 \
                     nexthop via 192.168.100.101 dev ppp1 weight 1 \
                     nexthop via 192.168.100.101 dev ppp2 weight 1 \
                     nexthop via 192.168.100.101 dev ppp3 weight 1 

thanx to all
X


On Jul 18, "George Vieira" <georgev@citadelcomputer.com.au> wrote:
> 
> It's the NTH patch. he he p-o-m.. 
> 
> Thanks,
> ____________________________________________
> George Vieira
> Systems Manager
> georgev@citadelcomputer.com.au
> 
> Citadel Computer Systems Pty Ltd
> <a href='http://www.citadelcomputer.com.au'>http://www.citadelcomputer.com.au</a>
>  
> 
> -----Original Message-----
> From: Daniel Chemko [mailto:dchemko@smgtec.com]
> Sent: Friday, July 18, 2003 8:37 AM
> To: Javier Govea; Ramin Dousti
> Cc: netfilter@lists.netfilter.org
> Subject: RE: Round Robin Load Balancing
> 
> 
> There is an extension that says something like every N packets, execute
> this rule. I forgot what it was called though.. *doh*
> 
> 

Re: Round Robin Load Balancing

[Javier Govea]

Hi to all, 

Well, i was able to fix my problem with "modprobe ipt_MASQUERADE" and "depmod -a". The
solution (perhaps not the best one): to rebuild the whole kernel again...and to rebuild
all modules (including the Nth module, which was the one i really neede to test). I was
only rebuilding the modules, and i guess there was a mistmatch between my modules and the
kernel. Anaway, that problem is fixed....

Cheers...
X

> On Wed, Jul 23, 2003 at 08:58:32PM -0400, Javier Govea wrote:
> > modprobe: Too deep recursion in module dependencies!
> > modprobe: Circular dependency? ip_nat_core ip_nat_proto_udp ip_conntrack ip_tables
> > ipt_MASQUERADE
> > Aborted (core dumped)
> > 
> > how do i fix this??????
> > 
> Just trying to understand, did you point iptables to the kernel source
> you custom rolled?
> -- 
> Jerry M. Howell II
> 

Re: Round Robin Load Balancing

[Ramin Dousti]

You might waht to remove equilize. You most definitely don't want to load-
balance per packet as it might be in conflict with conntrack; just a
suggestion.

Secondly, investigate the nat option of iproute2. As it's from the same
subsystem it might play better with your load-balancing. However, iproute2
nat is stateless and it does not have any helpers for less nat-friendly
protocols.

And to answer to your question about the nat and 2.2, no, this document
is old and when it talks about 2.2, it meant the "new" 2.2 kernel, coming
from 2.0

Ramin



On Wed, Jul 23, 2003 at 08:31:19PM -0400, Javier Govea wrote:

> Hi,
> 
> > ip route add default scope global ["equalize"] nexthop dev ppp0 weight 1 \
> > 	                                       nexthop dev ppp2 weight 1 \
> > 	                                       nexthop dev ppp3 weight 1 \
> > 	                                       nexthop dev ppp4 weight 1
> 
> This is exactely the setup i'm running including the "equalize" keyword. I'm combining
> this with NAT which as i said in previous emails it is not load balancing my traffic. So,
> this seems to answer your question about NAT playing well with this iproute setup. 
> 
> I read the documentation about NAT in IPROUTE
> (http://snafu.freedom.org/linux2.2/docs/ip-cref/ip-cref.html) and it mentions that NAT is
> only avalable for kernels 2.2.x .... am i right??????
> 
> Thanx....
> X
> 
> > You can even do per packet routing ("equalize" keyword should do)
> > which is not recommended while combining with nat. However, I don't
> > know whether conntrack/nat in netfilter plays well with this or not.
> > But even if not, you still can use the "nat" capability of iproute2 itself.
> > 
> > Let us know if it worked.
> > 
> > Ramin
> > 
> > On Thu, Jul 17, 2003 at 04:29:31PM -0400, Javier Govea wrote:
> > 
> > > I undesrtand what you mean about perfect load balancing (i'm not actually looking for a
> > > perfect load balancer) I have two examples below, but first i will responde some of the
> > > questions.
> > > 
> > > > Do these two ppp accounts belong to the same ISP? 
> > > Yes. I have four accounts, all of them with the same ISP
> > > >Does the ISP drop forign src?
> > > ?????
> > > > Is the gateway doing nat? 
> > > Yes, im using iptables to setup the nat
> > > > Do you have any preference on one of the ppp's than the other? 
> > > No
> > > >Can you bond (mppp)?
> > > I haven't tried multilink ppp...i will look into this...
> > > 
> > > > You could also setup something like BGP to allow multiple routes to....
> > > I don't know if this would be the best approach. I already tried to setup BGP and OSPF 
> > > routes using zebra (<a href='http://zebra.org'>http://zebra.org</a>) and i never made
> it work....
> > > I found a tool called EQLPlus (<a
> href='http://www.cwareco.com/download/eqlplus.html'>http://www.cwareco.com/download/eqlplus.html</a>)
> but i was
> > > never able to compile it. Has anybody has tried eqlplus before???????
> > > 
> > > > If one user makes a request out of line X then the return packet HAS to come back      
> > >  > through line X. So, if one guy sends a huge request taking minutes to fulfill, he / she
> > > > will tie up the line until the job is finished
> > > 
> > > Absolutely. I can live with that, but here is my problem. I have 4 ppp links on my router
> > > (which is doing nat). Then if in a host, located in my LAN which connected to my router, i
> > > open four browsers and each browser is pointing to the same site then i'm expepecting each
> > > web page to be requested and returned in a different link. But that doesn't happen. Some
> > > times it does happen but most of the time i get three of the responses on one link, one in
> > > another and the other two links do nothing. Sometimes i get 2,1,1,0 ....
> > > 
> > > I did another test...i have website with has in its main web page has only 4 images
> > > (differnt images but all of them of exactely the same size). if i point my browser to that
> > > site, then i'm believe the browser is sending four http requests (one for each image),
> > > well i would expect one image on each link....but again sometimes i get the four images on
> > > the same link...some times i get 2 images in one link...
> > > 
> > > So, i don't want a perfect load balancer but i would like to fix the problems on my two
> > > examples... i thought about implementing a round robin algorithm for load balancing where
> > > my first request goes on my first available link, the second one on the second available
> > > link and so on....this idea fixes my problems in my two previous examples, but i'm open to
> > > suggestions....
> > > 
> > > any tips, pointer, ideas are all welcome...
> > > 
> > > cheers...
> > > X
> > > 
> > > 
> > > 
> > > > Absolutely. Perfect load balancing needs to be coordinated on _all_ the
> > > > endpoints of the links involved. In this case, 4 endpoints.
> > > > 
> > > > For a regular load balancing (which is going to be the case here)
> > > > we still have lots of unknown variables. Do these two ppp
> > > > accounts belong to the same ISP? Does the ISP drop forign src?
> > > > Is the gateway doing nat? Do you have any preference on one of
> > > > the ppp's than the other? Can you bond (mppp)? And so on.
> > > > But a fun project, though, for someone who has time...
> > > > 
> > > > Ramin
> > > > 
> > > > On Tue, Jul 15, 2003 at 12:44:40PM -0700, Daniel Chemko wrote:
> > > > 
> > > > > Because of the nature of your setup, you cannot have a perfect equal
> > > > > load balance setup. This is because you cannot control the inbound flow
> > > > > of data. If one user makes a request out of line X then the return
> > > > > packet HAS to come back through line X. So, if one guy sends a huge
> > > > > request taking minutes to fulfill, he / she will tie up the line until
> > > > > the job is finished. The load balancer should be smart enough to not
> > > > > send any more requests to that line, but you are still seeing the line
> > > > > being monopolized by this single connection, hence it is not balanced
> > > > > over all lines equally.
> > > > > 
> > > > > In order to have fair balancing of all lines, I think you need to set up
> > > > > a deal with your ISP to load balance on their end as well.
> > > > > 
> > > > > You could also setup something like BGP to allow multiple routes to the
> > > > > same return address, but I am not familiar enough with BGP to be much
> > > > > help in this area. In all likelihood, you are better off with your
> > > > > current solution or maybe the ISP solution if it is supported by them
> > > > > (more money usually).
> > > > > 
> > > > > 
> > > > > 
> > > > > -----Original Message-----
> > > > > From: Javier Govea [mailto:jgovea@magma.ca] 
> > > > > Sent: Tuesday, July 15, 2003 12:34 PM
> > > > > To: netfilter@lists.netfilter.org
> > > > > Subject: Round Robin Load Balancing 
> > > > > 
> > > > > Hi,
> > > > > 
> > > > > I'm trying to do some load balancing with four ppp connections. Here is
> > > > > what i have: a LAN
> > > > > connected to a redhat box which has four ppp interfaces. All the boxes
> > > > > in the LAN are
> > > > > accesing internet through the ppp interfaces in the redhat box. I'm
> > > > > using iproute2, in my
> > > > > redhat box, to setup the the four ppp interfaces as my default out going
> > > > > route (as
> > > > > described in LART <a
> > > href='<a href='http://lartc.org/howto/index.html&#39;&gt;<a
> href='http://lartc.org/howto/index.html&lt;/a&gt;'>http://lartc.org/howto/index.html</a></a>'>http://lartc.org/howto/index.html'><a
> href='http://lartc.org/howto/index.html&lt;/a&gt;'>http://lartc.org/howto/index.html</a></a></a>)
> and I'm using
> > > > > iptables to masquerade
> > > > > all the traffic comming from the LAN. 
> > > > > 
> > > > > My setup is working fine, ie. my LAN can access the net throught the
> > > > > four ppp interfaces.
> > > > > My problem is that i don't know how is the load balancing working. Some
> > > > > times one of the
> > > > > ppp interfaces is used more than the others (and that is my problem).
> > > > > According to LART
> > > > > the routes are cached, can someone go a bit into more details in this
> > > > > caching thing??? how
> > > > > does it work?  which particular files in the kernel are doing this?  
> > > > > 
> > > > > I would like to implement a simple round robing algorithm (with no
> > > > > caching) for doing the
> > > > > laod balancing. That is first connection established gose through ppp0,
> > > > > the second
> > > > > connection on ppp1 and so on. 
> > > > > 
> > > > > I could hack iproute2 and/or iptables, but i'm not sure about which
> > > > > particular files i
> > > > > should hack in order to implement this round robin algorithm. I actually
> > > > > don't know if
> > > > > what i want makes any sense
> > > > > 
> > > > > Any ideas or pointers are all very well appreaciated.
> > > > > Thanx to all
> > > > > X
> > > > > 
> > > > > 
> > > > 
> > 
> > 

Re: Round Robin Load Balancing

[Javier Govea]

Hi,

> ip route add default scope global ["equalize"] nexthop dev ppp0 weight 1 \
> 	                                       nexthop dev ppp2 weight 1 \
> 	                                       nexthop dev ppp3 weight 1 \
> 	                                       nexthop dev ppp4 weight 1

This is exactely the setup i'm running including the "equalize" keyword. I'm combining
this with NAT which as i said in previous emails it is not load balancing my traffic. So,
this seems to answer your question about NAT playing well with this iproute setup. 

I read the documentation about NAT in IPROUTE
(http://snafu.freedom.org/linux2.2/docs/ip-cref/ip-cref.html) and it mentions that NAT is
only avalable for kernels 2.2.x .... am i right??????

Thanx....
X

> You can even do per packet routing ("equalize" keyword should do)
> which is not recommended while combining with nat. However, I don't
> know whether conntrack/nat in netfilter plays well with this or not.
> But even if not, you still can use the "nat" capability of iproute2 itself.
> 
> Let us know if it worked.
> 
> Ramin
> 
> On Thu, Jul 17, 2003 at 04:29:31PM -0400, Javier Govea wrote:
> 
> > I undesrtand what you mean about perfect load balancing (i'm not actually looking for a
> > perfect load balancer) I have two examples below, but first i will responde some of the
> > questions.
> > 
> > > Do these two ppp accounts belong to the same ISP? 
> > Yes. I have four accounts, all of them with the same ISP
> > >Does the ISP drop forign src?
> > ?????
> > > Is the gateway doing nat? 
> > Yes, im using iptables to setup the nat
> > > Do you have any preference on one of the ppp's than the other? 
> > No
> > >Can you bond (mppp)?
> > I haven't tried multilink ppp...i will look into this...
> > 
> > > You could also setup something like BGP to allow multiple routes to....
> > I don't know if this would be the best approach. I already tried to setup BGP and OSPF 
> > routes using zebra (<a href='http://zebra.org'>http://zebra.org</a>) and i never made
it work....
> > I found a tool called EQLPlus (<a
href='http://www.cwareco.com/download/eqlplus.html'>http://www.cwareco.com/download/eqlplus.html</a>)
but i was
> > never able to compile it. Has anybody has tried eqlplus before???????
> > 
> > > If one user makes a request out of line X then the return packet HAS to come back      
> >  > through line X. So, if one guy sends a huge request taking minutes to fulfill, he / she
> > > will tie up the line until the job is finished
> > 
> > Absolutely. I can live with that, but here is my problem. I have 4 ppp links on my router
> > (which is doing nat). Then if in a host, located in my LAN which connected to my router, i
> > open four browsers and each browser is pointing to the same site then i'm expepecting each
> > web page to be requested and returned in a different link. But that doesn't happen. Some
> > times it does happen but most of the time i get three of the responses on one link, one in
> > another and the other two links do nothing. Sometimes i get 2,1,1,0 ....
> > 
> > I did another test...i have website with has in its main web page has only 4 images
> > (differnt images but all of them of exactely the same size). if i point my browser to that
> > site, then i'm believe the browser is sending four http requests (one for each image),
> > well i would expect one image on each link....but again sometimes i get the four images on
> > the same link...some times i get 2 images in one link...
> > 
> > So, i don't want a perfect load balancer but i would like to fix the problems on my two
> > examples... i thought about implementing a round robin algorithm for load balancing where
> > my first request goes on my first available link, the second one on the second available
> > link and so on....this idea fixes my problems in my two previous examples, but i'm open to
> > suggestions....
> > 
> > any tips, pointer, ideas are all welcome...
> > 
> > cheers...
> > X
> > 
> > 
> > 
> > > Absolutely. Perfect load balancing needs to be coordinated on _all_ the
> > > endpoints of the links involved. In this case, 4 endpoints.
> > > 
> > > For a regular load balancing (which is going to be the case here)
> > > we still have lots of unknown variables. Do these two ppp
> > > accounts belong to the same ISP? Does the ISP drop forign src?
> > > Is the gateway doing nat? Do you have any preference on one of
> > > the ppp's than the other? Can you bond (mppp)? And so on.
> > > But a fun project, though, for someone who has time...
> > > 
> > > Ramin
> > > 
> > > On Tue, Jul 15, 2003 at 12:44:40PM -0700, Daniel Chemko wrote:
> > > 
> > > > Because of the nature of your setup, you cannot have a perfect equal
> > > > load balance setup. This is because you cannot control the inbound flow
> > > > of data. If one user makes a request out of line X then the return
> > > > packet HAS to come back through line X. So, if one guy sends a huge
> > > > request taking minutes to fulfill, he / she will tie up the line until
> > > > the job is finished. The load balancer should be smart enough to not
> > > > send any more requests to that line, but you are still seeing the line
> > > > being monopolized by this single connection, hence it is not balanced
> > > > over all lines equally.
> > > > 
> > > > In order to have fair balancing of all lines, I think you need to set up
> > > > a deal with your ISP to load balance on their end as well.
> > > > 
> > > > You could also setup something like BGP to allow multiple routes to the
> > > > same return address, but I am not familiar enough with BGP to be much
> > > > help in this area. In all likelihood, you are better off with your
> > > > current solution or maybe the ISP solution if it is supported by them
> > > > (more money usually).
> > > > 
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Javier Govea [mailto:jgovea@magma.ca] 
> > > > Sent: Tuesday, July 15, 2003 12:34 PM
> > > > To: netfilter@lists.netfilter.org
> > > > Subject: Round Robin Load Balancing 
> > > > 
> > > > Hi,
> > > > 
> > > > I'm trying to do some load balancing with four ppp connections. Here is
> > > > what i have: a LAN
> > > > connected to a redhat box which has four ppp interfaces. All the boxes
> > > > in the LAN are
> > > > accesing internet through the ppp interfaces in the redhat box. I'm
> > > > using iproute2, in my
> > > > redhat box, to setup the the four ppp interfaces as my default out going
> > > > route (as
> > > > described in LART <a
> > href='<a href='http://lartc.org/howto/index.html&#39;&gt;<a
href='http://lartc.org/howto/index.html&lt;/a&gt;'>http://lartc.org/howto/index.html</a></a>'>http://lartc.org/howto/index.html'><a
href='http://lartc.org/howto/index.html&lt;/a&gt;'>http://lartc.org/howto/index.html</a></a></a>)
and I'm using
> > > > iptables to masquerade
> > > > all the traffic comming from the LAN. 
> > > > 
> > > > My setup is working fine, ie. my LAN can access the net throught the
> > > > four ppp interfaces.
> > > > My problem is that i don't know how is the load balancing working. Some
> > > > times one of the
> > > > ppp interfaces is used more than the others (and that is my problem).
> > > > According to LART
> > > > the routes are cached, can someone go a bit into more details in this
> > > > caching thing??? how
> > > > does it work?  which particular files in the kernel are doing this?  
> > > > 
> > > > I would like to implement a simple round robing algorithm (with no
> > > > caching) for doing the
> > > > laod balancing. That is first connection established gose through ppp0,
> > > > the second
> > > > connection on ppp1 and so on. 
> > > > 
> > > > I could hack iproute2 and/or iptables, but i'm not sure about which
> > > > particular files i
> > > > should hack in order to implement this round robin algorithm. I actually
> > > > don't know if
> > > > what i want makes any sense
> > > > 
> > > > Any ideas or pointers are all very well appreaciated.
> > > > Thanx to all
> > > > X
> > > > 
> > > > 
> > > 
> 
> 

Re: Round Robin Load Balancing

[Ramin Dousti]

Try to add a "load balanced" default route distributed among these 4 ppp
interfaces.  Something like this:

ip route add default scope global ["equalize"] nexthop dev ppp0 weight 1 \
	                                       nexthop dev ppp2 weight 1 \
	                                       nexthop dev ppp3 weight 1 \
	                                       nexthop dev ppp4 weight 1

You can even do per packet routing ("equalize" keyword should do)
which is not recommended while combining with nat. However, I don't
know whether conntrack/nat in netfilter plays well with this or not.
But even if not, you still can use the "nat" capability of iproute2 itself.

Let us know if it worked.

Ramin

On Thu, Jul 17, 2003 at 04:29:31PM -0400, Javier Govea wrote:

> I undesrtand what you mean about perfect load balancing (i'm not actually looking for a
> perfect load balancer) I have two examples below, but first i will responde some of the
> questions.
> 
> > Do these two ppp accounts belong to the same ISP? 
> Yes. I have four accounts, all of them with the same ISP
> >Does the ISP drop forign src?
> ?????
> > Is the gateway doing nat? 
> Yes, im using iptables to setup the nat
> > Do you have any preference on one of the ppp's than the other? 
> No
> >Can you bond (mppp)?
> I haven't tried multilink ppp...i will look into this...
> 
> > You could also setup something like BGP to allow multiple routes to....
> I don't know if this would be the best approach. I already tried to setup BGP and OSPF 
> routes using zebra (http://zebra.org) and i never made it work....
> I found a tool called EQLPlus (http://www.cwareco.com/download/eqlplus.html) but i was
> never able to compile it. Has anybody has tried eqlplus before???????
> 
> > If one user makes a request out of line X then the return packet HAS to come back      
>  > through line X. So, if one guy sends a huge request taking minutes to fulfill, he / she
> > will tie up the line until the job is finished
> 
> Absolutely. I can live with that, but here is my problem. I have 4 ppp links on my router
> (which is doing nat). Then if in a host, located in my LAN which connected to my router, i
> open four browsers and each browser is pointing to the same site then i'm expepecting each
> web page to be requested and returned in a different link. But that doesn't happen. Some
> times it does happen but most of the time i get three of the responses on one link, one in
> another and the other two links do nothing. Sometimes i get 2,1,1,0 ....
> 
> I did another test...i have website with has in its main web page has only 4 images
> (differnt images but all of them of exactely the same size). if i point my browser to that
> site, then i'm believe the browser is sending four http requests (one for each image),
> well i would expect one image on each link....but again sometimes i get the four images on
> the same link...some times i get 2 images in one link...
> 
> So, i don't want a perfect load balancer but i would like to fix the problems on my two
> examples... i thought about implementing a round robin algorithm for load balancing where
> my first request goes on my first available link, the second one on the second available
> link and so on....this idea fixes my problems in my two previous examples, but i'm open to
> suggestions....
> 
> any tips, pointer, ideas are all welcome...
> 
> cheers...
> X
> 
> 
> 
> > Absolutely. Perfect load balancing needs to be coordinated on _all_ the
> > endpoints of the links involved. In this case, 4 endpoints.
> > 
> > For a regular load balancing (which is going to be the case here)
> > we still have lots of unknown variables. Do these two ppp
> > accounts belong to the same ISP? Does the ISP drop forign src?
> > Is the gateway doing nat? Do you have any preference on one of
> > the ppp's than the other? Can you bond (mppp)? And so on.
> > But a fun project, though, for someone who has time...
> > 
> > Ramin
> > 
> > On Tue, Jul 15, 2003 at 12:44:40PM -0700, Daniel Chemko wrote:
> > 
> > > Because of the nature of your setup, you cannot have a perfect equal
> > > load balance setup. This is because you cannot control the inbound flow
> > > of data. If one user makes a request out of line X then the return
> > > packet HAS to come back through line X. So, if one guy sends a huge
> > > request taking minutes to fulfill, he / she will tie up the line until
> > > the job is finished. The load balancer should be smart enough to not
> > > send any more requests to that line, but you are still seeing the line
> > > being monopolized by this single connection, hence it is not balanced
> > > over all lines equally.
> > > 
> > > In order to have fair balancing of all lines, I think you need to set up
> > > a deal with your ISP to load balance on their end as well.
> > > 
> > > You could also setup something like BGP to allow multiple routes to the
> > > same return address, but I am not familiar enough with BGP to be much
> > > help in this area. In all likelihood, you are better off with your
> > > current solution or maybe the ISP solution if it is supported by them
> > > (more money usually).
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: Javier Govea [mailto:jgovea@magma.ca] 
> > > Sent: Tuesday, July 15, 2003 12:34 PM
> > > To: netfilter@lists.netfilter.org
> > > Subject: Round Robin Load Balancing 
> > > 
> > > Hi,
> > > 
> > > I'm trying to do some load balancing with four ppp connections. Here is
> > > what i have: a LAN
> > > connected to a redhat box which has four ppp interfaces. All the boxes
> > > in the LAN are
> > > accesing internet through the ppp interfaces in the redhat box. I'm
> > > using iproute2, in my
> > > redhat box, to setup the the four ppp interfaces as my default out going
> > > route (as
> > > described in LART <a
> href='http://lartc.org/howto/index.html'>http://lartc.org/howto/index.html</a>) and I'm using
> > > iptables to masquerade
> > > all the traffic comming from the LAN. 
> > > 
> > > My setup is working fine, ie. my LAN can access the net throught the
> > > four ppp interfaces.
> > > My problem is that i don't know how is the load balancing working. Some
> > > times one of the
> > > ppp interfaces is used more than the others (and that is my problem).
> > > According to LART
> > > the routes are cached, can someone go a bit into more details in this
> > > caching thing??? how
> > > does it work?  which particular files in the kernel are doing this?  
> > > 
> > > I would like to implement a simple round robing algorithm (with no
> > > caching) for doing the
> > > laod balancing. That is first connection established gose through ppp0,
> > > the second
> > > connection on ppp1 and so on. 
> > > 
> > > I could hack iproute2 and/or iptables, but i'm not sure about which
> > > particular files i
> > > should hack in order to implement this round robin algorithm. I actually
> > > don't know if
> > > what i want makes any sense
> > > 
> > > Any ideas or pointers are all very well appreaciated.
> > > Thanx to all
> > > X
> > > 
> > > 
> > 

RE: Round Robin Load Balancing

[George Vieira]

It's the NTH patch. he he p-o-m.. 

Thanks,
____________________________________________
George Vieira
Systems Manager
georgev@citadelcomputer.com.au

Citadel Computer Systems Pty Ltd
http://www.citadelcomputer.com.au
 

-----Original Message-----
From: Daniel Chemko [mailto:dchemko@smgtec.com]
Sent: Friday, July 18, 2003 8:37 AM
To: Javier Govea; Ramin Dousti
Cc: netfilter@lists.netfilter.org
Subject: RE: Round Robin Load Balancing


There is an extension that says something like every N packets, execute
this rule. I forgot what it was called though.. *doh*

RE: Round Robin Load Balancing

[Daniel Chemko]

There is an extension that says something like every N packets, execute
this rule. I forgot what it was called though.. *doh*

Try looking back ~ 1 month ago. I know I saw it there somewhere.

-----Original Message-----
From: Javier Govea [mailto:jgovea@magma.ca] 
Sent: Thursday, July 17, 2003 1:30 PM
To: Ramin Dousti; Daniel Chemko
Cc: netfilter@lists.netfilter.org
Subject: Re: Round Robin Load Balancing

I undesrtand what you mean about perfect load balancing (i'm not
actually looking for a
perfect load balancer) I have two examples below, but first i will
responde some of the
questions.

> Do these two ppp accounts belong to the same ISP? 
Yes. I have four accounts, all of them with the same ISP
>Does the ISP drop forign src?
?????
> Is the gateway doing nat? 
Yes, im using iptables to setup the nat
> Do you have any preference on one of the ppp's than the other? 
No
>Can you bond (mppp)?
I haven't tried multilink ppp...i will look into this...

> You could also setup something like BGP to allow multiple routes
to....
I don't know if this would be the best approach. I already tried to
setup BGP and OSPF 
routes using zebra (http://zebra.org) and i never made it work....
I found a tool called EQLPlus
(http://www.cwareco.com/download/eqlplus.html) but i was
never able to compile it. Has anybody has tried eqlplus before???????

> If one user makes a request out of line X then the return packet HAS
to come back      
 > through line X. So, if one guy sends a huge request taking minutes to
fulfill, he / she
> will tie up the line until the job is finished

Absolutely. I can live with that, but here is my problem. I have 4 ppp
links on my router
(which is doing nat). Then if in a host, located in my LAN which
connected to my router, i
open four browsers and each browser is pointing to the same site then
i'm expepecting each
web page to be requested and returned in a different link. But that
doesn't happen. Some
times it does happen but most of the time i get three of the responses
on one link, one in
another and the other two links do nothing. Sometimes i get 2,1,1,0 ....

I did another test...i have website with has in its main web page has
only 4 images
(differnt images but all of them of exactely the same size). if i point
my browser to that
site, then i'm believe the browser is sending four http requests (one
for each image),
well i would expect one image on each link....but again sometimes i get
the four images on
the same link...some times i get 2 images in one link...

So, i don't want a perfect load balancer but i would like to fix the
problems on my two
examples... i thought about implementing a round robin algorithm for
load balancing where
my first request goes on my first available link, the second one on the
second available
link and so on....this idea fixes my problems in my two previous
examples, but i'm open to
suggestions....

any tips, pointer, ideas are all welcome...

cheers...
X



> Absolutely. Perfect load balancing needs to be coordinated on _all_
the
> endpoints of the links involved. In this case, 4 endpoints.
> 
> For a regular load balancing (which is going to be the case here)
> we still have lots of unknown variables. Do these two ppp
> accounts belong to the same ISP? Does the ISP drop forign src?
> Is the gateway doing nat? Do you have any preference on one of
> the ppp's than the other? Can you bond (mppp)? And so on.
> But a fun project, though, for someone who has time...
> 
> Ramin
> 
> On Tue, Jul 15, 2003 at 12:44:40PM -0700, Daniel Chemko wrote:
> 
> > Because of the nature of your setup, you cannot have a perfect equal
> > load balance setup. This is because you cannot control the inbound
flow
> > of data. If one user makes a request out of line X then the return
> > packet HAS to come back through line X. So, if one guy sends a huge
> > request taking minutes to fulfill, he / she will tie up the line
until
> > the job is finished. The load balancer should be smart enough to not
> > send any more requests to that line, but you are still seeing the
line
> > being monopolized by this single connection, hence it is not
balanced
> > over all lines equally.
> > 
> > In order to have fair balancing of all lines, I think you need to
set up
> > a deal with your ISP to load balance on their end as well.
> > 
> > You could also setup something like BGP to allow multiple routes to
the
> > same return address, but I am not familiar enough with BGP to be
much
> > help in this area. In all likelihood, you are better off with your
> > current solution or maybe the ISP solution if it is supported by
them
> > (more money usually).
> > 
> > 
> > 
> > -----Original Message-----
> > From: Javier Govea [mailto:jgovea@magma.ca] 
> > Sent: Tuesday, July 15, 2003 12:34 PM
> > To: netfilter@lists.netfilter.org
> > Subject: Round Robin Load Balancing 
> > 
> > Hi,
> > 
> > I'm trying to do some load balancing with four ppp connections. Here
is
> > what i have: a LAN
> > connected to a redhat box which has four ppp interfaces. All the
boxes
> > in the LAN are
> > accesing internet through the ppp interfaces in the redhat box. I'm
> > using iproute2, in my
> > redhat box, to setup the the four ppp interfaces as my default out
going
> > route (as
> > described in LART <a
href='http://lartc.org/howto/index.html'>http://lartc.org/howto/index.ht
ml</a>) and I'm using
> > iptables to masquerade
> > all the traffic comming from the LAN. 
> > 
> > My setup is working fine, ie. my LAN can access the net throught the
> > four ppp interfaces.
> > My problem is that i don't know how is the load balancing working.
Some
> > times one of the
> > ppp interfaces is used more than the others (and that is my
problem).
> > According to LART
> > the routes are cached, can someone go a bit into more details in
this
> > caching thing??? how
> > does it work?  which particular files in the kernel are doing this?

> > 
> > I would like to implement a simple round robing algorithm (with no
> > caching) for doing the
> > laod balancing. That is first connection established gose through
ppp0,
> > the second
> > connection on ppp1 and so on. 
> > 
> > I could hack iproute2 and/or iptables, but i'm not sure about which
> > particular files i
> > should hack in order to implement this round robin algorithm. I
actually
> > don't know if
> > what i want makes any sense
> > 
> > Any ideas or pointers are all very well appreaciated.
> > Thanx to all
> > X
> > 
> > 
> 

Re: Round Robin Load Balancing

[Javier Govea]

I undesrtand what you mean about perfect load balancing (i'm not actually looking for a
perfect load balancer) I have two examples below, but first i will responde some of the
questions.

> Do these two ppp accounts belong to the same ISP? 
Yes. I have four accounts, all of them with the same ISP
>Does the ISP drop forign src?
?????
> Is the gateway doing nat? 
Yes, im using iptables to setup the nat
> Do you have any preference on one of the ppp's than the other? 
No
>Can you bond (mppp)?
I haven't tried multilink ppp...i will look into this...

> You could also setup something like BGP to allow multiple routes to....
I don't know if this would be the best approach. I already tried to setup BGP and OSPF 
routes using zebra (http://zebra.org) and i never made it work....
I found a tool called EQLPlus (http://www.cwareco.com/download/eqlplus.html) but i was
never able to compile it. Has anybody has tried eqlplus before???????

> If one user makes a request out of line X then the return packet HAS to come back      
 > through line X. So, if one guy sends a huge request taking minutes to fulfill, he / she
> will tie up the line until the job is finished

Absolutely. I can live with that, but here is my problem. I have 4 ppp links on my router
(which is doing nat). Then if in a host, located in my LAN which connected to my router, i
open four browsers and each browser is pointing to the same site then i'm expepecting each
web page to be requested and returned in a different link. But that doesn't happen. Some
times it does happen but most of the time i get three of the responses on one link, one in
another and the other two links do nothing. Sometimes i get 2,1,1,0 ....

I did another test...i have website with has in its main web page has only 4 images
(differnt images but all of them of exactely the same size). if i point my browser to that
site, then i'm believe the browser is sending four http requests (one for each image),
well i would expect one image on each link....but again sometimes i get the four images on
the same link...some times i get 2 images in one link...

So, i don't want a perfect load balancer but i would like to fix the problems on my two
examples... i thought about implementing a round robin algorithm for load balancing where
my first request goes on my first available link, the second one on the second available
link and so on....this idea fixes my problems in my two previous examples, but i'm open to
suggestions....

any tips, pointer, ideas are all welcome...

cheers...
X



> Absolutely. Perfect load balancing needs to be coordinated on _all_ the
> endpoints of the links involved. In this case, 4 endpoints.
> 
> For a regular load balancing (which is going to be the case here)
> we still have lots of unknown variables. Do these two ppp
> accounts belong to the same ISP? Does the ISP drop forign src?
> Is the gateway doing nat? Do you have any preference on one of
> the ppp's than the other? Can you bond (mppp)? And so on.
> But a fun project, though, for someone who has time...
> 
> Ramin
> 
> On Tue, Jul 15, 2003 at 12:44:40PM -0700, Daniel Chemko wrote:
> 
> > Because of the nature of your setup, you cannot have a perfect equal
> > load balance setup. This is because you cannot control the inbound flow
> > of data. If one user makes a request out of line X then the return
> > packet HAS to come back through line X. So, if one guy sends a huge
> > request taking minutes to fulfill, he / she will tie up the line until
> > the job is finished. The load balancer should be smart enough to not
> > send any more requests to that line, but you are still seeing the line
> > being monopolized by this single connection, hence it is not balanced
> > over all lines equally.
> > 
> > In order to have fair balancing of all lines, I think you need to set up
> > a deal with your ISP to load balance on their end as well.
> > 
> > You could also setup something like BGP to allow multiple routes to the
> > same return address, but I am not familiar enough with BGP to be much
> > help in this area. In all likelihood, you are better off with your
> > current solution or maybe the ISP solution if it is supported by them
> > (more money usually).
> > 
> > 
> > 
> > -----Original Message-----
> > From: Javier Govea [mailto:jgovea@magma.ca] 
> > Sent: Tuesday, July 15, 2003 12:34 PM
> > To: netfilter@lists.netfilter.org
> > Subject: Round Robin Load Balancing 
> > 
> > Hi,
> > 
> > I'm trying to do some load balancing with four ppp connections. Here is
> > what i have: a LAN
> > connected to a redhat box which has four ppp interfaces. All the boxes
> > in the LAN are
> > accesing internet through the ppp interfaces in the redhat box. I'm
> > using iproute2, in my
> > redhat box, to setup the the four ppp interfaces as my default out going
> > route (as
> > described in LART <a
href='http://lartc.org/howto/index.html'>http://lartc.org/howto/index.html</a>) and I'm using
> > iptables to masquerade
> > all the traffic comming from the LAN. 
> > 
> > My setup is working fine, ie. my LAN can access the net throught the
> > four ppp interfaces.
> > My problem is that i don't know how is the load balancing working. Some
> > times one of the
> > ppp interfaces is used more than the others (and that is my problem).
> > According to LART
> > the routes are cached, can someone go a bit into more details in this
> > caching thing??? how
> > does it work?  which particular files in the kernel are doing this?  
> > 
> > I would like to implement a simple round robing algorithm (with no
> > caching) for doing the
> > laod balancing. That is first connection established gose through ppp0,
> > the second
> > connection on ppp1 and so on. 
> > 
> > I could hack iproute2 and/or iptables, but i'm not sure about which
> > particular files i
> > should hack in order to implement this round robin algorithm. I actually
> > don't know if
> > what i want makes any sense
> > 
> > Any ideas or pointers are all very well appreaciated.
> > Thanx to all
> > X
> > 
> > 
> 

Re: Round Robin Load Balancing

[Ramin Dousti]

Absolutely. Perfect load balancing needs to be coordinated on _all_ the
endpoints of the links involved. In this case, 4 endpoints.

For a regular load balancing (which is going to be the case here)
we still have lots of unknown variables. Do these two ppp
accounts belong to the same ISP? Does the ISP drop forign src?
Is the gateway doing nat? Do you have any preference on one of
the ppp's than the other? Can you bond (mppp)? And so on.
But a fun project, though, for someone who has time...

Ramin

On Tue, Jul 15, 2003 at 12:44:40PM -0700, Daniel Chemko wrote:

> Because of the nature of your setup, you cannot have a perfect equal
> load balance setup. This is because you cannot control the inbound flow
> of data. If one user makes a request out of line X then the return
> packet HAS to come back through line X. So, if one guy sends a huge
> request taking minutes to fulfill, he / she will tie up the line until
> the job is finished. The load balancer should be smart enough to not
> send any more requests to that line, but you are still seeing the line
> being monopolized by this single connection, hence it is not balanced
> over all lines equally.
> 
> In order to have fair balancing of all lines, I think you need to set up
> a deal with your ISP to load balance on their end as well.
> 
> You could also setup something like BGP to allow multiple routes to the
> same return address, but I am not familiar enough with BGP to be much
> help in this area. In all likelihood, you are better off with your
> current solution or maybe the ISP solution if it is supported by them
> (more money usually).
> 
> 
> 
> -----Original Message-----
> From: Javier Govea [mailto:jgovea@magma.ca] 
> Sent: Tuesday, July 15, 2003 12:34 PM
> To: netfilter@lists.netfilter.org
> Subject: Round Robin Load Balancing 
> 
> Hi,
> 
> I'm trying to do some load balancing with four ppp connections. Here is
> what i have: a LAN
> connected to a redhat box which has four ppp interfaces. All the boxes
> in the LAN are
> accesing internet through the ppp interfaces in the redhat box. I'm
> using iproute2, in my
> redhat box, to setup the the four ppp interfaces as my default out going
> route (as
> described in LART http://lartc.org/howto/index.html) and I'm using
> iptables to masquerade
> all the traffic comming from the LAN. 
> 
> My setup is working fine, ie. my LAN can access the net throught the
> four ppp interfaces.
> My problem is that i don't know how is the load balancing working. Some
> times one of the
> ppp interfaces is used more than the others (and that is my problem).
> According to LART
> the routes are cached, can someone go a bit into more details in this
> caching thing??? how
> does it work?  which particular files in the kernel are doing this?  
> 
> I would like to implement a simple round robing algorithm (with no
> caching) for doing the
> laod balancing. That is first connection established gose through ppp0,
> the second
> connection on ppp1 and so on. 
> 
> I could hack iproute2 and/or iptables, but i'm not sure about which
> particular files i
> should hack in order to implement this round robin algorithm. I actually
> don't know if
> what i want makes any sense
> 
> Any ideas or pointers are all very well appreaciated.
> Thanx to all
> X
> 
> 

RE: Round Robin Load Balancing

[Daniel Chemko]

Because of the nature of your setup, you cannot have a perfect equal
load balance setup. This is because you cannot control the inbound flow
of data. If one user makes a request out of line X then the return
packet HAS to come back through line X. So, if one guy sends a huge
request taking minutes to fulfill, he / she will tie up the line until
the job is finished. The load balancer should be smart enough to not
send any more requests to that line, but you are still seeing the line
being monopolized by this single connection, hence it is not balanced
over all lines equally.

In order to have fair balancing of all lines, I think you need to set up
a deal with your ISP to load balance on their end as well.

You could also setup something like BGP to allow multiple routes to the
same return address, but I am not familiar enough with BGP to be much
help in this area. In all likelihood, you are better off with your
current solution or maybe the ISP solution if it is supported by them
(more money usually).



-----Original Message-----
From: Javier Govea [mailto:jgovea@magma.ca] 
Sent: Tuesday, July 15, 2003 12:34 PM
To: netfilter@lists.netfilter.org
Subject: Round Robin Load Balancing 

Hi,

I'm trying to do some load balancing with four ppp connections. Here is
what i have: a LAN
connected to a redhat box which has four ppp interfaces. All the boxes
in the LAN are
accesing internet through the ppp interfaces in the redhat box. I'm
using iproute2, in my
redhat box, to setup the the four ppp interfaces as my default out going
route (as
described in LART http://lartc.org/howto/index.html) and I'm using
iptables to masquerade
all the traffic comming from the LAN. 

My setup is working fine, ie. my LAN can access the net throught the
four ppp interfaces.
My problem is that i don't know how is the load balancing working. Some
times one of the
ppp interfaces is used more than the others (and that is my problem).
According to LART
the routes are cached, can someone go a bit into more details in this
caching thing??? how
does it work?  which particular files in the kernel are doing this?  

I would like to implement a simple round robing algorithm (with no
caching) for doing the
laod balancing. That is first connection established gose through ppp0,
the second
connection on ppp1 and so on. 

I could hack iproute2 and/or iptables, but i'm not sure about which
particular files i
should hack in order to implement this round robin algorithm. I actually
don't know if
what i want makes any sense

Any ideas or pointers are all very well appreaciated.
Thanx to all
X

Round Robin Load Balancing

[Javier Govea]

Hi,

I'm trying to do some load balancing with four ppp connections. Here is what i have: a LAN
connected to a redhat box which has four ppp interfaces. All the boxes in the LAN are
accesing internet through the ppp interfaces in the redhat box. I'm using iproute2, in my
redhat box, to setup the the four ppp interfaces as my default out going route (as
described in LART http://lartc.org/howto/index.html) and I'm using iptables to masquerade
all the traffic comming from the LAN. 

My setup is working fine, ie. my LAN can access the net throught the four ppp interfaces.
My problem is that i don't know how is the load balancing working. Some times one of the
ppp interfaces is used more than the others (and that is my problem). According to LART
the routes are cached, can someone go a bit into more details in this caching thing??? how
does it work?  which particular files in the kernel are doing this?  

I would like to implement a simple round robing algorithm (with no caching) for doing the
laod balancing. That is first connection established gose through ppp0, the second
connection on ppp1 and so on. 

I could hack iproute2 and/or iptables, but i'm not sure about which particular files i
should hack in order to implement this round robin algorithm. I actually don't know if
what i want makes any sense

Any ideas or pointers are all very well appreaciated.
Thanx to all
X