Routing traffic over two gateways by fwmark

Routing traffic over two gateways by fwmark

[Matt Killock]

Hello,

I am having trouble re-using scripts that worked in kernel 2.6.24 with a 
more recent 3.16.0 series kernel. The essence of what I am trying to do 
is this:

iptables mark all packets arriving from anywhere on interface 1 with 
'10'
iptables mark all packets arriving from anywhere on interface 2 with 
'20'

iptables DNAT incoming traffic on interface 1 on port 1111 to internal 
server via interface 3
iptables DNAT incoming traffic on interface 2 on port 1111 to internal 
server via interface 3

ip rule from all fwmark 10 table T1
ip rule from all fwmark 20 table T2

default gateway in table T1 is through interface 1
default gateway in table T2 is through interface 2

With the view to make everything that arrives on a particular interface 
leave via that interface. The above approach worked fine in kernel 2.6 
but does not seem to work any more. With no default gateway in the main 
routing table, the packets arrive on the external interfaces but do not 
appear on the LAN interface, almost as if DNAT is having no effect at 
all. One of the two DNAT rules works if I add a default gateway to the 
main routing table, but then it appears that the policy routing tables 
or firewall marks are ignored.

Should the above approach be possible still? If not, then could someone 
advise what approach I should use nowadays please?

Matt

Re: Routing traffic over two gateways by fwmark

[Pascal Hambourg]

Matt Killock a écrit :
> 
> I am having trouble re-using scripts that worked in kernel 2.6.24 with a 
> more recent 3.16.0 series kernel. The essence of what I am trying to do 
> is this:
> 
> iptables mark all packets arriving from anywhere on interface 1 with 
> '10'
> iptables mark all packets arriving from anywhere on interface 2 with 
> '20'
> 
> iptables DNAT incoming traffic on interface 1 on port 1111 to internal 
> server via interface 3
> iptables DNAT incoming traffic on interface 2 on port 1111 to internal 
> server via interface 3
> 
> ip rule from all fwmark 10 table T1
> ip rule from all fwmark 20 table T2
> 
> default gateway in table T1 is through interface 1
> default gateway in table T2 is through interface 2
> 
> With the view to make everything that arrives on a particular interface 
> leave via that interface. The above approach worked fine in kernel 2.6 

I don't think so. You must mark packets from the server arriving from
the server at interface 3, not packets arriving at interface 1 and 2.
Packets arriving at interface 1 and 2 are all routed via interface 3, so
they have no need for advanced routing.

> but does not seem to work any more. With no default gateway in the main 
> routing table, the packets arrive on the external interfaces but do not 
> appear on the LAN interface, almost as if DNAT is having no effect at 
> all.

Check the value of rp_filter for interface 1, interface 2 and all.
Since some kernel version the logic has changed from AND to MAX.

> One of the two DNAT rules works if I add a default gateway to the 
> main routing table, but then it appears that the policy routing tables 
> or firewall marks are ignored.

You must mark packets arriving from the server at interface 3. For this
you must mark the incoming connections (CONNMARK) and transfer the
connection mark to the packets arriving at interface 3.

Another method not requiring connection marking is to use the original
destination address (-m conntrack --ctorigdst) to mark the packets.

Re: Routing traffic over two gateways by fwmark

[Matt Killock]

On 2015-04-01 23:13, Pascal Hambourg wrote:
> Check the value of rp_filter for interface 1, interface 2 and all.

Ah ha! Yes, the rp_filter was enabled for all those interfaces. 
Disabling it on all has made it work again! Hoorah!

Many thanks!

Matt