redundancy with Adsl modem

redundancy with Adsl modem

[Usuário do Sistema]

Hello everyone,

I'm searching a way of the redundancy with my firewall CentOs Linux
5.5 that is behind from two adsl modem.


well, I wish that when a link adsl comes down ( side from ISP line
adsl no Ethernet cable that connects the modem adsl to my firewall )
the traffic go to for other link adsl.

there is any way to reach this redundancy with Linux iptables ??

any tips is welcome.

thanks

Re: redundancy with Adsl modem

[Andrew Beverley]

On Mon, 2012-01-02 at 23:54 -0200, Usuário do Sistema wrote:
> Hello everyone,
> 
> I'm searching a way of the redundancy with my firewall CentOs Linux
> 5.5 that is behind from two adsl modem.
> 
> 
> well, I wish that when a link adsl comes down ( side from ISP line
> adsl no Ethernet cable that connects the modem adsl to my firewall )
> the traffic go to for other link adsl.
> 
> there is any way to reach this redundancy with Linux iptables ??

I would recommend using LSM[1] and configuring that to run the relevant
iptables/iproute2 commands should an interface fail.

Andy

[1] http://lsm.foobar.fi/

Re: redundancy with Adsl modem

[Usuário do Sistema]

Thanks, I found few documentation about lsm on Internet.

if somebody has some how to about lsm post here please.






thanks




Em 3 de janeiro de 2012 05:14, Andrew Beverley <andy@andybev.com> escreveu:
> On Mon, 2012-01-02 at 23:54 -0200, Usuário do Sistema wrote:
>> Hello everyone,
>>
>> I'm searching a way of the redundancy with my firewall CentOs Linux
>> 5.5 that is behind from two adsl modem.
>>
>>
>> well, I wish that when a link adsl comes down ( side from ISP line
>> adsl no Ethernet cable that connects the modem adsl to my firewall )
>> the traffic go to for other link adsl.
>>
>> there is any way to reach this redundancy with Linux iptables ??
>
> I would recommend using LSM[1] and configuring that to run the relevant
> iptables/iproute2 commands should an interface fail.
>
> Andy
>
> [1] http://lsm.foobar.fi/
>
>

Re: redundancy with Adsl modem

[Andrew Beverley]

On Tue, 2012-01-03 at 13:18 -0200, Usuário do Sistema wrote:
> Thanks, I found few documentation about lsm on Internet.

Yeah, the documentation's not great. I'm hoping to write a man page
sometime soon...

> if somebody has some how to about lsm post here please.
> 

I'm planning to post my configuration to my "PortalShaper" website,
hopefully in the next week or so if you can wait that long.

Andy

Re: redundancy with Adsl modem

[Usuário do Sistema]

Thanks Andrew, when you post in your website please tell us.




Em 3 de janeiro de 2012 21:58, Andrew Beverley <andy@andybev.com> escreveu:
> On Tue, 2012-01-03 at 13:18 -0200, Usuário do Sistema wrote:
>> Thanks, I found few documentation about lsm on Internet.
>
> Yeah, the documentation's not great. I'm hoping to write a man page
> sometime soon...
>
>> if somebody has some how to about lsm post here please.
>>
>
> I'm planning to post my configuration to my "PortalShaper" website,
> hopefully in the next week or so if you can wait that long.
>
> Andy
>
>

Re: redundancy with Adsl modem

[Lloyd Standish]

On Tue, 03 Jan 2012 09:18:03 -0600, Usuário do Sistema <maiconlp@ig.com.br> wrote:

> Thanks, I found few documentation about lsm on Internet.
>
> if somebody has some how to about lsm post here please.
>

Hi,

I don't have a "how-to" document, but I can offer some tips on the use of lsm.

lsm is a powerful, flexible system, since when lsm detects that an interface is down, it executes a user-supplied script with a bunch of parameters about the interface that changed state (either "up" or "down").  That script can in turn run your router script, passing to it whatever parameters you want (usually the interface name that changed state and the new state). I use the same router script used to initialize the router/firewall, but with different parameters.

I found it necessary to save the state of all interfaces in a bit field, saved as a decimal number in a file on disk.  That allows your router script to know which interfaces are up and which are down, so it can reconfigure load-balancing when lsm notifies that an interface has changed state..  (lsm, when it "triggers," only supplies information about the interface that changed state.  It does not pass any information about what interfaces are up and which are down.)

You can use bash's bitwise operators to retrieve and set interface state information in the bitfield.

For load-balancing, I have had excellent results with the "statistic" module in "probability" mode.  With that module, all you need to do to take an interface out of load-balancing (so it will no longer be selected for NEW connections) is to set it's "prob" value to zero, and readjust the prob values of all the other uplinks.  The routing tables and policy routing rules do not need to be changed.

--
Lloyd

Re: redundancy with Adsl modem

[Gáspár Lajos]

Hi all!!

I have made some time ago an iface match that can check the state of an 
interface...
Maybe you can use it. (It is in the xtables-addons)

Swifty.

Re: redundancy with Adsl modem

[Usuário do Sistema]

OK guys, I will test lsm


thanks





Em 4 de janeiro de 2012 07:09, Gáspár Lajos <swifty@freemail.hu> escreveu:
> Hi all!!
>
> I have made some time ago an iface match that can check the state of an
> interface...
> Maybe you can use it. (It is in the xtables-addons)
>
> Swifty.

Re: redundancy with Adsl modem

[Lloyd Standish]

On Wed, 04 Jan 2012 03:09:12 -0600, Gáspár Lajos <swifty@freemail.hu> wrote:

> I have made some time ago an iface match that can check the state of an interface...
> Maybe you can use it. (It is in the xtables-addons)
>Swifty.

Hi Swifty,

I'm sure your iface match is very useful in many circumstances.  However I would like to point out that link status monitor (http://lsm.foobar.fi/) actually evaluates the link quality by pinging an IP (perhaps several hops past the gateway IP), keeping track of the number of lost and late-arriving packets over the last 60 seconds.  If the number of late or dropped packets exceeds a certain (configurable) number, then the link is reported as "down".  The main advantage to this (and the fact that it happens outside of netfilter) is that the firewall can be automatically reconfigured to exclude the failed link from routing.  When the link quality is seen to have improved, the failed link can be included again in the routing decision.

-- 
Lloyd

Re: redundancy with Adsl modem

[Gáspár Lajos]

Hi Lloyd,

Thank you for your comment ! :D
I have never used this monitor, but I am going to try it... :D

2012-01-04 15:08 keltezéssel, Lloyd Standish írta:
> I'm sure your iface match is very useful in many circumstances.  
> However I would like to point out that link status monitor 
> (http://lsm.foobar.fi/) actually evaluates the link quality by pinging 
> an IP (perhaps several hops past the gateway IP), keeping track of the 
> number of lost and late-arriving packets over the last 60 seconds.  If 
> the number of late or dropped packets exceeds a certain (configurable) 
> number, then the link is reported as "down".  The main advantage to 
> this (and the fact that it happens outside of netfilter) is that the 
> firewall can be automatically reconfigured to exclude the failed link 
> from routing.  When the link quality is seen to have improved, the 
> failed link can be included again in the routing decision.

I think that both of these approaches has pros and cons.

Maybe you also know that (in Linux) the kernel chooses the output 
interface depending on the routing table and not the source IP...
So if the ping is not bound to a specific interface then it is "useless"...
(There is an oping utility that can be set up to use a specific interface.)
I do not know LSM but I hope that it is also aware of this.

Besides this, pinging is not always accurate and may lead the 
application think that link quality is dropping down...
Just imagine that the pinged host(s) can be under a DOS attack and the 
reply times can go high...
(Not to mention that the pinging generates traffic and that requires 
resources. Probably not too much resources at all :D)

Other question is that how often/rarely do you ping? If often then it is 
too much traffic. If rarely then do you REALLY KNOW that the interface 
was all the time up?

To repeat myself: I do not know LSM :D

It seems to me that LSM is some kind of line quality checking software...

OTOH my match checks the interface state when the packet is in the queue...
With that info you can mark the packets and let the kernel decide about 
the routing depending on the mark..

But my match does not know anything about the "quality" of the 
connection just about the state of the interface...

Returning to the main question:
If an interface goes down then the associated connections will most 
likely break down...
Without knowing the required "high-availability" services, for example 
you can use "fallback_relay" in postfix; multiple remote lines in 
openvpn, etc. etc. etc.
So maybe the redundancy is not the right word for the main requirement...
I would ask myself: Do I really need redundancy or do I need alternativity?


Swifty

Re: redundancy with Adsl modem

[Usuário do Sistema]

good your tips Gáspár,


I would ask myself: Do I really need redundancy or do I need alternativity?

 yes, I need redundancy. when an line adsl come down all traffic go to
other line adsl. I couldn't have done this on linux machine. if you
has any how to about please post here.


thanks.










Em 4 de janeiro de 2012 16:00, Gáspár Lajos <swifty@freemail.hu> escreveu:
> Hi Lloyd,
>
> Thank you for your comment ! :D
> I have never used this monitor, but I am going to try it... :D
>
> 2012-01-04 15:08 keltezéssel, Lloyd Standish írta:
>
>> I'm sure your iface match is very useful in many circumstances.  However I
>> would like to point out that link status monitor (http://lsm.foobar.fi/)
>> actually evaluates the link quality by pinging an IP (perhaps several hops
>> past the gateway IP), keeping track of the number of lost and late-arriving
>> packets over the last 60 seconds.  If the number of late or dropped packets
>> exceeds a certain (configurable) number, then the link is reported as
>> "down".  The main advantage to this (and the fact that it happens outside of
>> netfilter) is that the firewall can be automatically reconfigured to exclude
>> the failed link from routing.  When the link quality is seen to have
>> improved, the failed link can be included again in the routing decision.
>
>
> I think that both of these approaches has pros and cons.
>
> Maybe you also know that (in Linux) the kernel chooses the output interface
> depending on the routing table and not the source IP...
> So if the ping is not bound to a specific interface then it is "useless"...
> (There is an oping utility that can be set up to use a specific interface.)
> I do not know LSM but I hope that it is also aware of this.
>
> Besides this, pinging is not always accurate and may lead the application
> think that link quality is dropping down...
> Just imagine that the pinged host(s) can be under a DOS attack and the reply
> times can go high...
> (Not to mention that the pinging generates traffic and that requires
> resources. Probably not too much resources at all :D)
>
> Other question is that how often/rarely do you ping? If often then it is too
> much traffic. If rarely then do you REALLY KNOW that the interface was all
> the time up?
>
> To repeat myself: I do not know LSM :D
>
> It seems to me that LSM is some kind of line quality checking software...
>
> OTOH my match checks the interface state when the packet is in the queue...
> With that info you can mark the packets and let the kernel decide about the
> routing depending on the mark..
>
> But my match does not know anything about the "quality" of the connection
> just about the state of the interface...
>
> Returning to the main question:
> If an interface goes down then the associated connections will most likely
> break down...
> Without knowing the required "high-availability" services, for example you
> can use "fallback_relay" in postfix; multiple remote lines in openvpn, etc.
> etc. etc.
> So maybe the redundancy is not the right word for the main requirement...
> I would ask myself: Do I really need redundancy or do I need alternativity?
>
>
> Swifty

Re: redundancy with Adsl modem

[Lloyd Standish]

On Wed, 04 Jan 2012 12:00:53 -0600, Gáspár Lajos <swifty@freemail.hu> wrote:

> I think that both of these approaches has pros and cons.
>Maybe you also know that (in Linux) the kernel chooses the output interface depending on the routing table and not the source IP...
> So if the ping is not bound to a specific interface then it is "useless"...
> (There is an oping utility that can be set up to use a specific interface.)
> I do not know LSM but I hope that it is also aware of this.

lsm ("link status monitor") does direct its pings through the interface specified in the config file, to a "ping IP."  However, to make sure this happens I use a unique ping IP for each interface, with policy routing rules like this:
ip rule add to <pingIP> table T<n>  (where T<n> is the name of a table that routes through a particular interface.)

lsm can of course monitor several interfaces at once.  I am using multiple uplinks to give both Internet connection redundancy and increased total bandwidth for a firewalled LAN.  I have 5 uplinks of 5 Mbit each, throttled to 4 Mbit to prevent queuing at the ISP.  I use a separate routing table for each interface, and handle routing by fw-marking packets with iptables rules, and routing with rules like "ip rule add fwmark <n>..."

>Besides this, pinging is not always accurate and may lead the application think that link quality is dropping down...
> Just imagine that the pinged host(s) can be under a DOS attack and the reply times can go high...

True.  If the "ping IP" can't be pinged or if the ping statistics are poor enough, the interface will be removed from routing.  If the ping IP host fails for any reason, that will give a "false positive."  However, since we have 5 interfaces, it would not be a serious problem for us.

> (Not to mention that the pinging generates traffic and that requires resources. Probably not too much resources at all :D)
>Other question is that how often/rarely do you ping? If often then it is too much traffic. If rarely then do you REALLY KNOW that the interface was all the time up?

I have configured lsm to ping the test IPs every second.  In my configuration, a response is considered "timed out" or "lost" if it takes more than 1000ms to get a reply to a ping.  The system determines that the link quality is too low if there are 7 consecutive lost packets, or 15 lost packets in a 60 second interval.

To consider that a "down" interface should come back up, it looks for 5 or less "lost" (timed out) packets in a 60 second interval.  All these parameters are configurable.  I left the default settings for most things.

>To repeat myself: I do not know LSM
>It seems to me that LSM is some kind of line quality checking software...

Right.

>OTOH my match checks the interface state when the packet is in the queue...
> With that info you can mark the packets and let the kernel decide about the routing depending on the mark..
>But my match does not know anything about the "quality" of the connection just about the state of the interface...
>Returning to the main question:
> If an interface goes down then the associated connections will most likely break down...
> Without knowing the required "high-availability" services, for example you can use "fallback_relay" in postfix; multiple remote lines in openvpn, etc. etc. etc.
> So maybe the redundancy is not the right word for the main requirement...
> I would ask myself: Do I really need redundancy or do I need alternativity?

Again, in our case the primary service is Internet connectivity for an internal LAN, with no services running on any of the outward-facing interfaces.  lsm is perfect for our situation, but it may not be best for others.

-- 
Lloyd Standish
Tropical Health Foods LLC
information on carao for blood health: http://www.bloodhealth.net
Use Suggetions: http://www.bloodhealth.net/files/use-suggestions.htm
selling website: http://www.tropicalhealthfoods.com
order form: https://www.tropicalhealthfoods.com/order.shtml.
New York, USA: 347 352 0058
other countries: +506 8816 1658
time zone CST (GMT-6)

--
Carao fruit is a food product intended to help support blood health.  It is not intended to treat any disease.  Furthermore, statements in this email should not be interpreted as medical advice or counsel.  All statements regarding carao fruit are the responsibility of Tropical Health Foods LLC.

Re: redundancy with Adsl modem

[Lloyd Standish]

On Wed, 04 Jan 2012 14:15:12 -0600, Usuário do Sistema <maiconlp@ig.com.br> wrote:

> yes, I need redundancy. when an line adsl come down all traffic go to
> other line adsl. I couldn't have done this on linux machine. if you
> has any how to about please post here.


One simple way to do this is to use the gateway failover that is built-in to Linux.  I haven't done this, since I wanted more in-depth link quality monitoring, but I understand that all you have to do is to set up a default route for each of your interfaces, and then reduce the gc_timeout value.  From http://www.muug.mb.ca/pipermail/roundtable/2005-May/000872.html:

"Enabling failover routing

After you have configured your network, the next step is to enable
failover routing on your Linux box, so that if the first route dies the
router will automatically switch over to the next route. To do so,
you'll need to add the default gateway routes provided to you by your
ISPs for both your network cards:

# route add default gw 61.16.130.97 dev eth0
# route add default gw 200.15.110.90 dev eth1

Finally, modify the /proc/sys/net/ipv4/route/gc_timeout file. This file
contains a numerical value that denotes the time in seconds after which
the kernel declares a route to be inactive and automatically switches to
the other route if available. Change its default value of 300 to some
smaller value, say 10 or 15. Save the changes and exit.

# echo "10" > /proc/sys/net/ipv4/route/gc_timeout

Now your Linux machine is ready to serve as a failover router,
automatically and quickly switching to the secondary route every time
the primary route fails."

-- 
Lloyd

Re: redundancy with Adsl modem

[Lloyd Standish]

Sorry, I forgot to remove my business email signature from my last post.  It was accidental.
--
Lloyd

Re: redundancy with Adsl modem

[Gáspár Lajos]

Hi,

>   yes, I need redundancy. when an line adsl come down all traffic go to
> other line adsl. I couldn't have done this on linux machine. if you
> has any how to about please post here.

How do you want to do that???

If you have two connections then you have two external IP-s...
If one goes down then you can not simply use the other one...

Just think about these steps:

In normal connection:

1. a local host (A - 192.168.1.1) sends a packet from LAN to an external 
host (C - 8.8.8.8)
2. the firewall applies source NAT to the packet and sends it out as if 
it were coming from the firewall's first external IP (B1 - 1.1.1.1)
3. C sends back a reply to B1
4. B1 forwards it with destination NAT to A.
5. go to step 1. or close the connection

If the line goes down between step 2 and step 3 then the reply packet 
will never get back to B1 and A (because the line is down) !!!
If the line goes down after step 4 then you can send it out on the 
second extrenal IP (B2 - 2.2.2.2) but C will see it as a NEW connection !!!

So I think that your request is impossible on IP level.
However you can use a proxy or a mail server, etc. etc. etc.

Swifty

Re: redundancy with Adsl modem

[Lloyd Standish]

On Fri, 06 Jan 2012 11:12:03 -0600, Gáspár Lajos <swifty@freemail.hu> wrote:

> If the line goes down between step 2 and step 3 then the reply packet will never get back to B1 and A (because the line is down) !!!
> If the line goes down after step 4 then you can send it out on the second extrenal IP (B2 - 2.2.2.2) but C will see it as a NEW connection !!!
>So I think that your request is impossible on IP level.
> However you can use a proxy or a mail server, etc. etc. etc.

I think any sort of "uplink redundancy" such as "Usuario de Sistema" is looking for must be on the connection level, unless one is connecting to a server using some sort of link bonding or multilink protocol, which has to be supported on the gateway computer.  I have never done that.

My suggestions to Usuario for uplink redundancy assume that if an interface/uplink goes down, the connection *will* be broken, but any following connections will avoid the failed interface/uplink.  I think that is the best that can usually be done to provide Internet connection redundancy.  In my scenario, load-balancing is also performed by selecting the interface for NEW connection from the LAN to Internet at random.

-- 
Lloyd

Re: redundancy with Adsl modem

[Usuário do Sistema]

Gáspár, my issue is an redundancy from ISP side. think that my two
connections with Internet is by line ADSL. so I'm thinking at the
contingency when an line adsl go down. I'm not regarding my connection
by ethernet cable between my modem ADSl and my firewall.

so....I have think how to monitor the adsl line as have ever comment before.


thanks.











Em 6 de janeiro de 2012 15:12, Gáspár Lajos <swifty@freemail.hu> escreveu:
> Hi,
>
>
>>  yes, I need redundancy. when an line adsl come down all traffic go to
>> other line adsl. I couldn't have done this on linux machine. if you
>> has any how to about please post here.
>
>
> How do you want to do that???
>
> If you have two connections then you have two external IP-s...
> If one goes down then you can not simply use the other one...
>
> Just think about these steps:
>
> In normal connection:
>
> 1. a local host (A - 192.168.1.1) sends a packet from LAN to an external
> host (C - 8.8.8.8)
> 2. the firewall applies source NAT to the packet and sends it out as if it
> were coming from the firewall's first external IP (B1 - 1.1.1.1)
> 3. C sends back a reply to B1
> 4. B1 forwards it with destination NAT to A.
> 5. go to step 1. or close the connection
>
> If the line goes down between step 2 and step 3 then the reply packet will
> never get back to B1 and A (because the line is down) !!!
> If the line goes down after step 4 then you can send it out on the second
> extrenal IP (B2 - 2.2.2.2) but C will see it as a NEW connection !!!
>
> So I think that your request is impossible on IP level.
> However you can use a proxy or a mail server, etc. etc. etc.
>
> Swifty

Re: redundancy with Adsl modem

[Gáspár Lajos]

Hi,


It is a bit late reply... :D

2012-01-10 01:20 keltezéssel, Usuário do Sistema írta:
> Gáspár, my issue is an redundancy from ISP side. think that my two
> connections with Internet is by line ADSL. so I'm thinking at the
> contingency when an line adsl go down. I'm not regarding my connection
> by ethernet cable between my modem ADSl and my firewall.
Just to mention: my iface target is independent of the type of the 
interface... So you can check the ethernet interface and the pppoe 
interface too !!!

I really do not understand your situation...

You have two ADSL lines at the same ISP, right?
Do they have fix IP? Both on them?

Because, you going to need a very big help from your ISP to keep the 
redundancy...

> so....I have think how to monitor the adsl line as have ever comment before.
>
>
> thanks.

Swifty

Re: redundancy with Adsl modem

[Usuário do Sistema]

Hello,

You have two ADSL lines at the same ISP, right?
Do they have fix IP? Both on them?

yes, Both have fix IP.


thanks






Em 17 de janeiro de 2012 14:09, Gáspár Lajos <swifty@freemail.hu> escreveu:
> Hi,
>
>
> It is a bit late reply... :D
>
> 2012-01-10 01:20 keltezéssel, Usuário do Sistema írta:
>
>> Gáspár, my issue is an redundancy from ISP side. think that my two
>> connections with Internet is by line ADSL. so I'm thinking at the
>> contingency when an line adsl go down. I'm not regarding my connection
>> by ethernet cable between my modem ADSl and my firewall.
>
> Just to mention: my iface target is independent of the type of the
> interface... So you can check the ethernet interface and the pppoe interface
> too !!!
>
> I really do not understand your situation...
>
> You have two ADSL lines at the same ISP, right?
> Do they have fix IP? Both on them?
>
> Because, you going to need a very big help from your ISP to keep the
> redundancy...
>
>
>> so....I have think how to monitor the adsl line as have ever comment
>> before.
>>
>>
>> thanks.
>
>
> Swifty