[LARTC] Multihomed Masquerading, routing and iptables

[LARTC] Multihomed Masquerading, routing and iptables

[Gordan Bobic]

Hi.

I have a networking problem that is driving me nuts at the moment. I
have a multi homed network: Cable + DSL.

The problem I have is that although I am 99% sure that I have the
routing table rules set up correctly, for some reason
masqueraded/NATed traffic doesn't go out of the correct interface.
i.e. I am getting traffic leaving eth2 with the source IP header set
to eth3 and vice versa.

There are 3 network interfaces:

eth0 (internal)
eth2 (DSL)
eth3 (Cable)

(eth1 is unused at present)

Here is my iptables setup (/etc/sysconfig/iptables):
################################
# Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003
*nat
:PREROUTING ACCEPT [0:0]
# Port forwarding to an internal machine
-A PREROUTING -i eth2 -d 217.79.103.2 -p tcp -m tcp --dport 18001 -j
DNAT --to-destination 192.168.0.10:18001
-A PREROUTING -i eth3 -d 62.252.21.17 -p tcp -m tcp --dport 18001 -j
DNAT --to-destination 192.168.0.10:18001
# SSH Port Forwarding
-A PREROUTING -i eth2 -d 217.79.103.3 -p tcp -m tcp --dport 22 -j DNAT
--to-destination 192.168.0.10:22
:POSTROUTING ACCEPT [0:0]
# IP Masquerading Traffic From eth2 and eth3
-A POSTROUTING -o eth2 -j MASQUERADE
-A POSTROUTING -o eth3 -j MASQUERADE
:OUTPUT ACCEPT [0:0]
COMMIT
# Completed on Sat Dec 27 10:47:54 2003
# Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
-A FORWARD -i eth0 -o eth2 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state
--state NEW,ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eth0 -o eth3 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state
--state NEW,ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eth2 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state
--state ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eth3 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state
--state ESTABLISHED,RELATED -j ACCEPT
:OUTPUT ACCEPT [0:0]
COMMIT
# Completed on Sat Dec 27 10:47:54 2003
###################################

Additionally, here is the script I use to set up the multi homed
routing:

####################################
# Add ip rules for routing
ip rule add from 217.79.103.0/29        table Griffin
ip rule add from 62.252.21.17           table NTL

# Add routing rules for specific interfaces to insure connectivity
ip route add to default via 217.79.103.1        dev eth2 table Griffin
ip route add to default via 62.252.21.254       dev eth3 table NTL

ip route add to 217.79.103.0/29 dev eth2 table Griffin
ip route add to 62.252.21.0/24  dev eth3 table NTL

# Default route is multi homed
ip route add to default                                         \
        nexthop via 217.79.103.1        dev eth2 weight 1       \
        nexthop via 62.252.21.254       dev eth3 weight 1

# Commit routing changes
ip route flush cache
#############################

However, looking at tcpdump output from eth2:
11:19:27.153771 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
217.81.134.183.57626: R 0:0(0) ack 2502579442 win 0 (DF)
11:19:30.212427 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
217.81.134.183.57626: R 0:0(0) ack 1 win 0 (DF)
11:20:23.928900 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
217.81.134.183.58367: R 0:0(0) ack 2551899092 win 0 (DF)

This is wrong because cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com is
62.252.21.17, which is the IP address of eth3.

Similarly, tcpdump from eth3 says things like:
11:18:32.787404 217.79.103.2.adsl.griffin.net.uk.18001 >
p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 4066315873 win 0 (DF)
11:18:35.683228 217.79.103.2.adsl.griffin.net.uk.18001 >
p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF)
11:18:41.744790 217.79.103.2.adsl.griffin.net.uk.18001 >
p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF)

This is again wrong, because 217.79.103.2.adsl.griffin.net.uk is the
IP address of eth2.

I am pretty sure the IP rules I set up should work. They assign all
packets with source IP of a particular interface to a routing table
that is routed out via the correct gateway. However, some packets
(from what I have been able to tell, only the masqueraded packets,
but the test was not exhaustive) get sent out of the wrong interface.

Can anybody see a problem with this setup?

TIA.

Gordan
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Rio Martin]

Dear Bobic,
I am sure you havent read Lartc Document clearly.
Find inside the document, "iproute2" 
Those are clue for setting up local area network to connect using two or more 
connections to ISP.

Regards,
Rio Martin.


On Wednesday 31 December 2003 23:49, Gordan Bobic wrote:
> Hi.
> I have a networking problem that is driving me nuts at the moment. I
> have a multi homed network: Cable + DSL.
> The problem I have is that although I am 99% sure that I have the
> routing table rules set up correctly, for some reason
> masqueraded/NATed traffic doesn't go out of the correct interface.
> i.e. I am getting traffic leaving eth2 with the source IP header set
> to eth3 and vice versa.
> There are 3 network interfaces:
> eth0 (internal)
> eth2 (DSL)
> eth3 (Cable)
> (eth1 is unused at present)
>
> Here is my iptables setup (/etc/sysconfig/iptables):
> ################################
> # Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003
> *nat
>
> :PREROUTING ACCEPT [0:0]
>
> # Port forwarding to an internal machine
> -A PREROUTING -i eth2 -d 217.79.103.2 -p tcp -m tcp --dport 18001 -j
> DNAT --to-destination 192.168.0.10:18001
> -A PREROUTING -i eth3 -d 62.252.21.17 -p tcp -m tcp --dport 18001 -j
> DNAT --to-destination 192.168.0.10:18001
> # SSH Port Forwarding
> -A PREROUTING -i eth2 -d 217.79.103.3 -p tcp -m tcp --dport 22 -j DNAT
> --to-destination 192.168.0.10:22
>
> :POSTROUTING ACCEPT [0:0]
>
> # IP Masquerading Traffic From eth2 and eth3
> -A POSTROUTING -o eth2 -j MASQUERADE
> -A POSTROUTING -o eth3 -j MASQUERADE
>
> :OUTPUT ACCEPT [0:0]
>
> COMMIT
> # Completed on Sat Dec 27 10:47:54 2003
> # Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003
> *filter
>
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
>
> -A FORWARD -i eth0 -o eth2 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state
> --state NEW,ESTABLISHED,RELATED -j ACCEPT
> -A FORWARD -i eth0 -o eth3 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state
> --state NEW,ESTABLISHED,RELATED -j ACCEPT
> -A FORWARD -i eth2 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state
> --state ESTABLISHED,RELATED -j ACCEPT
> -A FORWARD -i eth3 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state
> --state ESTABLISHED,RELATED -j ACCEPT
>
> :OUTPUT ACCEPT [0:0]
>
> COMMIT
> # Completed on Sat Dec 27 10:47:54 2003
> ###################################
>
> Additionally, here is the script I use to set up the multi homed
> routing:
>
> ####################################
> # Add ip rules for routing
> ip rule add from 217.79.103.0/29        table Griffin
> ip rule add from 62.252.21.17           table NTL
>
> # Add routing rules for specific interfaces to insure connectivity
> ip route add to default via 217.79.103.1        dev eth2 table Griffin
> ip route add to default via 62.252.21.254       dev eth3 table NTL
>
> ip route add to 217.79.103.0/29 dev eth2 table Griffin
> ip route add to 62.252.21.0/24  dev eth3 table NTL
>
> # Default route is multi homed
> ip route add to default                                         \
>         nexthop via 217.79.103.1        dev eth2 weight 1       \
>         nexthop via 62.252.21.254       dev eth3 weight 1
>
> # Commit routing changes
> ip route flush cache
> #############################
>
> However, looking at tcpdump output from eth2:
> 11:19:27.153771 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
> 217.81.134.183.57626: R 0:0(0) ack 2502579442 win 0 (DF)
> 11:19:30.212427 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
> 217.81.134.183.57626: R 0:0(0) ack 1 win 0 (DF)
> 11:20:23.928900 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
> 217.81.134.183.58367: R 0:0(0) ack 2551899092 win 0 (DF)
>
> This is wrong because cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com is
> 62.252.21.17, which is the IP address of eth3.
>
> Similarly, tcpdump from eth3 says things like:
> 11:18:32.787404 217.79.103.2.adsl.griffin.net.uk.18001 >
> p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 4066315873 win 0 (DF)
> 11:18:35.683228 217.79.103.2.adsl.griffin.net.uk.18001 >
> p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF)
> 11:18:41.744790 217.79.103.2.adsl.griffin.net.uk.18001 >
> p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF)
>
> This is again wrong, because 217.79.103.2.adsl.griffin.net.uk is the
> IP address of eth2.
>
> I am pretty sure the IP rules I set up should work. They assign all
> packets with source IP of a particular interface to a routing table
> that is routed out via the correct gateway. However, some packets
> (from what I have been able to tell, only the masqueraded packets,
> but the test was not exhaustive) get sent out of the wrong interface.
>
> Can anybody see a problem with this setup?
>
> TIA.
>
> Gordan
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Rio Martin]

Ooops ..
Sorry, i havent read the entire email sent to the list by Bobic. 
My mistake. 

Bobic having the problem similar to what i got with one of my server running 
kernel-2.4.20. All the interface i have are under the same brand (Realtek), 
eth0 would be for clients, eth1 for DSLCable, eth2 for Wireless 2.4Ghz.
Weirdly, several of my clients set up correctly to use both eth1 and eth2, but 
there are many clients having the wrong route packets just as Bobic.

This problem can be solved if i change to use SNAT instead of MASQUERADE. Try 
it Bobic.

This Masquerade problem didnt appeared under my Linux 2.4.21

Regards,
Rio Martin.

On Monday 05 January 2004 09:04, Rio Martin wrote:
> Dear Bobic,
> I am sure you havent read Lartc Document clearly.
> Find inside the document, "iproute2"
> Those are clue for setting up local area network to connect using two or
> more connections to ISP.
>
> Regards,
> Rio Martin.
>
> On Wednesday 31 December 2003 23:49, Gordan Bobic wrote:
> > Hi.
> > I have a networking problem that is driving me nuts at the moment. I
> > have a multi homed network: Cable + DSL.
> > The problem I have is that although I am 99% sure that I have the
> > routing table rules set up correctly, for some reason
> > masqueraded/NATed traffic doesn't go out of the correct interface.
> > i.e. I am getting traffic leaving eth2 with the source IP header set
> > to eth3 and vice versa.
> > There are 3 network interfaces:
> > eth0 (internal)
> > eth2 (DSL)
> > eth3 (Cable)
> > (eth1 is unused at present)
> >
> > Here is my iptables setup (/etc/sysconfig/iptables):
> > ################################
> > # Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003
> > *nat
> >
> > :PREROUTING ACCEPT [0:0]
> >
> > # Port forwarding to an internal machine
> > -A PREROUTING -i eth2 -d 217.79.103.2 -p tcp -m tcp --dport 18001 -j
> > DNAT --to-destination 192.168.0.10:18001
> > -A PREROUTING -i eth3 -d 62.252.21.17 -p tcp -m tcp --dport 18001 -j
> > DNAT --to-destination 192.168.0.10:18001
> > # SSH Port Forwarding
> > -A PREROUTING -i eth2 -d 217.79.103.3 -p tcp -m tcp --dport 22 -j DNAT
> > --to-destination 192.168.0.10:22
> >
> > :POSTROUTING ACCEPT [0:0]
> >
> > # IP Masquerading Traffic From eth2 and eth3
> > -A POSTROUTING -o eth2 -j MASQUERADE
> > -A POSTROUTING -o eth3 -j MASQUERADE
> >
> > :OUTPUT ACCEPT [0:0]
> >
> > COMMIT
> > # Completed on Sat Dec 27 10:47:54 2003
> > # Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003
> > *filter
> >
> > :INPUT ACCEPT [0:0]
> > :FORWARD ACCEPT [0:0]
> >
> > -A FORWARD -i eth0 -o eth2 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state
> > --state NEW,ESTABLISHED,RELATED -j ACCEPT
> > -A FORWARD -i eth0 -o eth3 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state
> > --state NEW,ESTABLISHED,RELATED -j ACCEPT
> > -A FORWARD -i eth2 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state
> > --state ESTABLISHED,RELATED -j ACCEPT
> > -A FORWARD -i eth3 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state
> > --state ESTABLISHED,RELATED -j ACCEPT
> >
> > :OUTPUT ACCEPT [0:0]
> >
> > COMMIT
> > # Completed on Sat Dec 27 10:47:54 2003
> > ###################################
> >
> > Additionally, here is the script I use to set up the multi homed
> > routing:
> >
> > ####################################
> > # Add ip rules for routing
> > ip rule add from 217.79.103.0/29        table Griffin
> > ip rule add from 62.252.21.17           table NTL
> >
> > # Add routing rules for specific interfaces to insure connectivity
> > ip route add to default via 217.79.103.1        dev eth2 table Griffin
> > ip route add to default via 62.252.21.254       dev eth3 table NTL
> >
> > ip route add to 217.79.103.0/29 dev eth2 table Griffin
> > ip route add to 62.252.21.0/24  dev eth3 table NTL
> >
> > # Default route is multi homed
> > ip route add to default                                         \
> >         nexthop via 217.79.103.1        dev eth2 weight 1       \
> >         nexthop via 62.252.21.254       dev eth3 weight 1
> >
> > # Commit routing changes
> > ip route flush cache
> > #############################
> >
> > However, looking at tcpdump output from eth2:
> > 11:19:27.153771 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
> > 217.81.134.183.57626: R 0:0(0) ack 2502579442 win 0 (DF)
> > 11:19:30.212427 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
> > 217.81.134.183.57626: R 0:0(0) ack 1 win 0 (DF)
> > 11:20:23.928900 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
> > 217.81.134.183.58367: R 0:0(0) ack 2551899092 win 0 (DF)
> >
> > This is wrong because cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com is
> > 62.252.21.17, which is the IP address of eth3.
> >
> > Similarly, tcpdump from eth3 says things like:
> > 11:18:32.787404 217.79.103.2.adsl.griffin.net.uk.18001 >
> > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 4066315873 win 0 (DF)
> > 11:18:35.683228 217.79.103.2.adsl.griffin.net.uk.18001 >
> > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF)
> > 11:18:41.744790 217.79.103.2.adsl.griffin.net.uk.18001 >
> > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF)
> >
> > This is again wrong, because 217.79.103.2.adsl.griffin.net.uk is the
> > IP address of eth2.
> >
> > I am pretty sure the IP rules I set up should work. They assign all
> > packets with source IP of a particular interface to a routing table
> > that is routed out via the correct gateway. However, some packets
> > (from what I have been able to tell, only the masqueraded packets,
> > but the test was not exhaustive) get sent out of the wrong interface.
> >
> > Can anybody see a problem with this setup?
> >
> > TIA.
> >
> > Gordan
> > _______________________________________________
> > LARTC mailing list / LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Gordan Bobic]

On Monday 05 Jan 2004 02:55, Rio Martin wrote:

> Bobic having the problem similar to what i got with one of my server
> running kernel-2.4.20. All the interface i have are under the same brand
> (Realtek), eth0 would be for clients, eth1 for DSLCable, eth2 for Wireless
> 2.4Ghz. Weirdly, several of my clients set up correctly to use both eth1
> and eth2, but there are many clients having the wrong route packets just as
> Bobic.
>
> This problem can be solved if i change to use SNAT instead of MASQUERADE.
> Try it Bobic.

Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other 
things?

> This Masquerade problem didnt appeared under my Linux 2.4.21

So, it is a 2.4.20 kernel problem? I was hoping to avoid home-brewing a kernel 
again like I did back with my old setup (RH 7.0, 2.2 kernel), and stick with 
a standard release kernel.

Thanks.

Gordan
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Artūras Šlajus]

Gordan Bobic wrote:

> Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other 
> things?
-j SNAT your_ip
man iptables


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Gordan Bobic]

On Monday 05 Jan 2004 11:28, Artūras Šlajus wrote:
> Gordan Bobic wrote:
> > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other
> > things?
>
> -j SNAT your_ip

Or rather -j SNAT --to-source your_ip. I get it. I'll check if that works 
better than masquerading.

Thanks.

Gordan
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Gordan Bobic]

On Monday 05 Jan 2004 11:54, Gordan Bobic wrote:
> On Monday 05 Jan 2004 11:28, Artūras Šlajus wrote:
> > Gordan Bobic wrote:
> > > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other
> > > things?
> >
> > -j SNAT your_ip
>
> Or rather -j SNAT --to-source your_ip. I get it. I'll check if that works
> better than masquerading.

Just tried it - no difference. Packets still come out with source IP address 
not matching the interface. :-(

Gordan
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[andybr]

Hi all,

I dont understando what the problem is, can you explain 
so i can help you out.

[]´s
Anderson

> On Monday 05 Jan 2004 02:55, Rio Martin wrote:
> 
> > Bobic having the problem similar to what i got with o
ne of my server
> > running kernel-
2.4.20. All the interface i have are under the same brand
> > (Realtek), eth0 would be for clients, eth1 for DSLCab
le, eth2 for Wireless
> > 2.4Ghz. Weirdly, several of my clients set up correct
ly to use both eth1
> > and eth2, but there are many clients having the wrong
 route packets just as
> > Bobic.
> >
> > This problem can be solved if i change to use SNAT in
stead of MASQUERADE.
> > Try it Bobic.
> 
> Hmm. Just replace -j MASQUERADE with -
j SNAT? Will that not break other 
> things?
> 
> > This Masquerade problem didnt appeared under my Linux
 2.4.21
> 
> So, it is a 2.4.20 kernel problem? I was hoping to avoi
d home-brewing a kernel 
> again like I did back with my old setup (RH 7.0, 2.2 ke
rnel), and stick with 
> a standard release kernel.
> 
> Thanks.
> 
> Gordan
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: ht
tp://lartc.org/
> 

 
__________________________________________________________________________
Acabe com aquelas janelinhas que pulam na sua tela.
AntiPop-up UOL - É grátis!
http://antipopup.uol.com.br/


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Gordan Bobic]

On Monday 05 Jan 2004 8:32 pm, andybr wrote:
> Hi all,
>
> I dont understando what the problem is, can you explain
> so i can help you out.

2 interfaces, eth2 and eth3. ip rules ip route are set up correctly. 
Yet, for some reason, there are packets coming out of eth2 with the IP 
address of eth3 and vice versa. This seems to be related mainly to 
forwarded/NATed connections (which tend to break quite quickly). It is 
almost as if the connection tracking code in the kernel is broken.

I am not pretty certain it is a kernel bug in the shipping RH9 kernel. 
There was an update released today by RH, but I'm not holding much hope 
it will behave any differently. But it doesn't matter, as I have a 
fresh patched kernel that will hopefully work to try in a bit.

Thanks for the help.

Gordan
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Rio Martin]

On Monday 05 January 2004 19:06, Gordan Bobic wrote:
> On Monday 05 Jan 2004 11:54, Gordan Bobic wrote:
> > On Monday 05 Jan 2004 11:28, Artūras Šlajus wrote:
> > > Gordan Bobic wrote:
> > > > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break
> > > > other things?
> > > -j SNAT your_ip
> > Or rather -j SNAT --to-source your_ip. I get it. I'll check if that works
> > better than masquerading.
> Just tried it - no difference. Packets still come out with source IP
> address not matching the interface. :-(

Try it switch manually, first you set up without iproute. Remove all the 
tables you have created and flush it. Try with ISP1 first. Do SNAT --to 
ip.of.ISP1
Is it work? Okay, now switch to the ISP2. Do SNAT --to ip.of.ISP2. 
It should be work, otherwise something wrong with the kernel or iptables you 
had on your machine.

Finish this step first, report back to the list.

Regards,
Rio Martin.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[Gordan Bobic]

On Tuesday 06 Jan 2004 01:49, Rio Martin wrote:
> > > > > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break
> > > > > other things?
> > > >
> > > > -j SNAT your_ip
> > >
> > > Or rather -j SNAT --to-source your_ip. I get it. I'll check if that
> > > works better than masquerading.
> >
> > Just tried it - no difference. Packets still come out with source IP
> > address not matching the interface. :-(
>
> Try it switch manually, first you set up without iproute. Remove all the
> tables you have created and flush it. Try with ISP1 first. Do SNAT --to
> ip.of.ISP1
> Is it work? Okay, now switch to the ISP2. Do SNAT --to ip.of.ISP2.
> It should be work, otherwise something wrong with the kernel or iptables
> you had on your machine.
>
> Finish this step first, report back to the list.

If one of the default routes is removed, everything works OK. However, if 
there are two default routes, packets get misdirected. ChangeLog for 2.4.21 
lists a few conntrack bug fixes, which I suspect to be the cause of this. 
Basically, the non-deterministic default route selection/rotation seems to 
take precedence over maintaining the same interface for serving a particular 
established connection through the firewall.

I'm compiling a new clean 2.4.24 with the jumbo routes patch at the moment, 
which will hopefully fix things. I'm hoping to try it out tonight. And BTW, 
the latest RH9 kernel released yesterday (2.4.20-28.9 IIRC), is still broken 
as far as routing is concerned.

Gordan
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multihomed Masquerading, routing and iptables

[R. Steve McKown]

On Tuesday 06 January 2004 02:25 am, Gordan Bobic wrote:
> If one of the default routes is removed, everything works OK. However, if
> there are two default routes, packets get misdirected. ChangeLog for 2.4.21
> lists a few conntrack bug fixes, which I suspect to be the cause of this.
> Basically, the non-deterministic default route selection/rotation seems to
> take precedence over maintaining the same interface for serving a
> particular established connection through the firewall.

You are right.  This is because the routing core is often queried more than 
once to set up a usable route cache entry for a given connection/session.  
Have a look at ip_route_connect() in linux/include/net/route.h as an example:

static inline int ip_route_connect(struct rtable **rp, u32 dst, u32 src, u32 
tos, int oif)
{
        int err;
        err = ip_route_output(rp, dst, src, tos, oif);
        if (err || (dst && src))
                return err;
        dst = (*rp)->rt_dst;
        src = (*rp)->rt_src;
        ip_rt_put(*rp);
        *rp = NULL;
        return ip_route_output(rp, dst, src, tos, oif);
}

Consider when this function is called with src=0, which happens for locally 
generated output (SNAT is similar I believe).  The first ip_route_output() 
call returns a pointer to a route cache entry, which includes a src ip in 
(*rp)->rt_src.  The first route cache entry doesn't work for us, because its 
'key' has src=0 and so won't match subsequent traffic.  So a second 
ip_route_output() is called using the new src as part of its key.  The new 
key matches no existing route cache and as a result the default multipath 
route is again consulted and a nexthop is determined.  This latter process 
does not use src in its processing so there is no guarantee that the nexthop 
returned is the same as that returned by the first query.  Hence, src ip is 
not guaranteed to match outbound interface.

Julian Anastasov's patches, noted earlier in this thread, provide a solution 
to this problem.  He allows for additional route rules and route tables that 
are matched by the second route query in preference to the default route so 
the src ip and outbound interface can be forced to be consistent.

I'm still pretty new to all this, so I hope Julian or someone else can correct 
any errors I have made.  The example above is in the non-NAT case of locally 
generated traffic, but I believe it's representative of what happens in the 
SNAT case as well.

> I'm compiling a new clean 2.4.24 with the jumbo routes patch at the moment,
> which will hopefully fix things. I'm hoping to try it out tonight. And BTW,
> the latest RH9 kernel released yesterday (2.4.20-28.9 IIRC), is still
> broken as far as routing is concerned.

I haven't looked at RedHat's route patch; it'd be killer if they solved this 
without requiring the additional route rules and tables setups as required by 
Julian's patches.  Let us know the outcome, would you?

The reason for this behavior makes sense from a code perspective, but not IMO 
from a route administration perspective.  I have a patch in its infancy that 
attempts to address this problem without requiring extra route administration 
(rules and tables).  It works in the non-nat case, but there is still much 
more testing to go before it's worth publishing.  If it survives the next few 
weeks of testing, I'd be happy to pass it on to anyone else who might be 
interested in playing with it.

Best Regards,
Steve

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/