Multiple link, policy routing and link not in defaut route...

Multiple link, policy routing and link not in defaut route...

[Marco Gaiarin]

Sorry for my prevuìious post, i've done some extensive tests and now i
have more info, i hope at least.

Situation: multiple link firewall, with 4 link to the internet,
'balanced' via route base balancing and policy routing (AKA, iptables
mark).

Situation:

 root@tank:~# ip route show
 default 
	nexthop via 81.174.0.21  dev ppp0 weight 30
	nexthop via 88.37.116.137  dev vlan192 weight 7
	nexthop via 10.5.248.253  dev vlan193 weight 100

there's a fourth link, but i want to 'reserve' it for some specific
traffic. I've excluded from the default route pool, so.

But the specific table routing is here:

 root@tank:~# ip rule show
 0:	from all lookup local 
 32758:	from all fwmark 0x40/0xf0 lookup FWFibra 
 32759:	from 37.186.212.162 lookup FWFibra 
 32760:	from all fwmark 0x30/0xf0 lookup FWFTTC 
 32761:	from 10.5.248.254 lookup FWFTTC 
 32762:	from all fwmark 0x20/0xf0 lookup EOLO 
 32763:	from 88.147.114.200 lookup EOLO 
 32764:	from all fwmark 0x10/0xf0 lookup TI7 
 32765:	from 88.37.116.142 lookup TI7 
 32766:	from all lookup main 
 32767:	from all lookup default 

 root@tank:~# ip route show table FWFibra
 default via 37.186.212.161 dev vlan249 
 10.5.0.0/21 dev eth0 scope link 
 10.5.8.0/22 dev eth0.3 scope link 
 37.186.212.160/30 dev vlan249 scope link src 37.186.212.162 
 127.0.0.0/8 dev lo scope link 

I've added a policy routing test:

 /sbin/iptables -t mangle -A PREROUTING -i eth0 -s 10.5.0.0/21 -d 173.194.79.109 -p icmp -m mark --mark 0x0/0xf0 -j MARK --set-mark 64/0x00f0

i can see the rule match, the ping from an internal host go ouside, come
back but nothing arrived on internal host.
I've double checked forward chains and NAT tables, and all seems OK.


After a bit of fiddling, i've tried to add the fourth line to the
'default route' pool, eg:

 root@tank:~# ip route show
 default 
	nexthop via 81.174.0.21  dev ppp0 weight 30
	nexthop via 88.37.116.137  dev vlan192 weight 7
	nexthop via 10.5.248.253  dev vlan193 weight 100
	nexthop via 37.186.212.161  dev vlan249 weight 10

and now the policy routing works as expected.


Why the interface need to be in 'default route'? Thanks.

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)

Re: Multiple link, policy routing and link not in defaut route...

[Marco Gaiarin]

> Why the interface need to be in 'default route'? Thanks.

As sugested by a private reply, i've disabled 'rp_filter' and packet
flow correctly.

AFAI've understood, packet get routed correctly to the intended
interface, but when reply come back the reverse path filter interpret
it as 'impossible' (because there's no a forward route, and this is
true indeed), and filter it away.


There's some 'smarter' way, or fine-grained way, or i have to disable
rp_filter as the only option?


Thanks.

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)

Re: Multiple link, policy routing and link not in defaut route...

[cronolog+lartc]

Hi,

Rather than disable rp_filter (by setting to 0 on all interfaces I presume),
what about setting it to 2 for Loose mode instead, and only on the affected
interfaces, so only those interfaces change behaviour?

Loose mode would allow the packet as long as there is a valid route on any
interface, instead of the specific interface it comes in.  So as long as a
default route exists anywhere, the packet should pass.

Potentially this opens up the interface to spoofed traffic, as it would now
allow traffic with source IP belonging to subnets on your private interfaces,
because obviously you would have routes to those too.  But that can be solved
easily with iptables rules.  Generally I block all packets with source in all
private IP ranges on Internet-facing interfaces, with exceptions if necessary
e.g. for external DMZ etc.  (I do similar for outbound too, no need to pollute
Internet with traffic destined to private IP space, though it might not as
simple if you are behind CG-NAT...)

If it works, you would need to look into sysctl config for your specific
system to make it persistent across reboot.

Regards,

Ali



On 05/02/2021 16:42, Marco Gaiarin wrote:
>> Why the interface need to be in 'default route'? Thanks.
> As sugested by a private reply, i've disabled 'rp_filter' and packet
> flow correctly.
>
> AFAI've understood, packet get routed correctly to the intended
> interface, but when reply come back the reverse path filter interpret
> it as 'impossible' (because there's no a forward route, and this is
> true indeed), and filter it away.
>
>
> There's some 'smarter' way, or fine-grained way, or i have to disable
> rp_filter as the only option?
>
>
> Thanks.
>

Re: Multiple link, policy routing and link not in defaut route...

[Marco Gaiarin]

Mandi! cronolog+lartc
  In chel di` si favelave...

> Rather than disable rp_filter (by setting to 0 on all interfaces I presume),
> what about setting it to 2 for Loose mode instead, and only on the affected
> interfaces, so only those interfaces change behaviour?
> Loose mode would allow the packet as long as there is a valid route on any
> interface, instead of the specific interface it comes in.  So as long as a
> default route exists anywhere, the packet should pass.

Bingo!


> Potentially this opens up the interface to spoofed traffic, as it would now
> allow traffic with source IP belonging to subnets on your private interfaces,
> because obviously you would have routes to those too.  But that can be solved
> easily with iptables rules.  Generally I block all packets with source in all
> private IP ranges on Internet-facing interfaces, with exceptions if necessary
> e.g. for external DMZ etc.

I do exactly the same things. To at least have notices, i've also
enabled 'log_martians'.


Many thanks!!!

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)

Re: Multiple link, policy routing and link not in defaut route...

[Jay Vosburgh]

Marco Gaiarin <gaio@sv.lnf.it> wrote:

>> Why the interface need to be in 'default route'? Thanks.
>
>As sugested by a private reply, i've disabled 'rp_filter' and packet
>flow correctly.
>
>AFAI've understood, packet get routed correctly to the intended
>interface, but when reply come back the reverse path filter interpret
>it as 'impossible' (because there's no a forward route, and this is
>true indeed), and filter it away.
>
>
>There's some 'smarter' way, or fine-grained way, or i have to disable
>rp_filter as the only option?

	Another possibility is that, because you're using fwmark in the
routing, you're running afoul of the src_valid_mark sysctl.

	By default (src_valid_mark = 0), fwmark is not checked when
performing rp_filter reverse path route lookups.  Enabling
net.ipv4.conf.*.src_valid_mark will cause the fwmark to be utilized for
the reverse path lookup.

	-J

---
	-Jay Vosburgh, jay.vosburgh@canonical.com

Re: Multiple link, policy routing and link not in defaut route...

[Marco Gaiarin]

Mandi! Jay Vosburgh
  In chel di` si favelave...

> 	Another possibility is that, because you're using fwmark in the
> routing, you're running afoul of the src_valid_mark sysctl.
> 	By default (src_valid_mark = 0), fwmark is not checked when
> performing rp_filter reverse path route lookups.  Enabling
> net.ipv4.conf.*.src_valid_mark will cause the fwmark to be utilized for
> the reverse path lookup.

Again, Bingo!

And effectively this solution seems too much betetr to me, in respect
to rp_filter=0 or =2.


Thanks!!!

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''          http://www.lanostrafamiglia.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
      http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)