Query regarding IPV6 multipath (ECMP) routes in Linux

Query regarding IPV6 multipath (ECMP) routes in Linux

[Preethi Ramachandra]

Hi,

I want to understand the IPV4 and IPV6 multipath(ECMP) routes behaviour in Linux. 

I tried configuring IPV4 and IPV6 multipath routes on Linux using ip route add command. Linux versions used in this testing are 
 1. Linux Kernel: 3.14.52-rt50-WR7.0.0.9_ovp
 2. Linux Kernel: 4.8.28-WR2.2.1_standard

IPV4 multipath route addition and display:
ip route add 1.1.1.0/24 nexthop via 192.168.1.254 nexthop via 192.168.0.254

ip route show 
1.1.1.0/24
        nexthop via 192.168.1.254  dev virbr0 weight 1
        nexthop via 192.168.0.254  dev eth0 weight 1

IPV6 multipath route addition and display:
ip route add 2001:d98:0:e101::1/128  nexthop via 3100:db8:c70:a20:c2bf:a7ff:fed2:1111 nexthop via 2102:db8:c70:a20::2

ip -6 route show
2001:d98:0:e101::1 via 3100:db8:c70:a20:c2bf:a7ff:fed2:1111 dev eth0  metric 1024
2001:d98:0:e101::1 via 2102:db8:c70:a20::2 dev eth0  metric 1024

In IPV4 a single route is added with multipath nexthops. However, in IPV6 , multipath routes are added as separate single routes with each multipath NH. I wanted to understand why is this behaviour change in Linux for IPV6 multipath routes. Why wasn’t same IPV4 approach followed in this case?

The code path to add IPV6 multipath routes is 

inet6_rtm_newroute()
  rtm_to_fib6_config()
  ip6_route_multipath_add()
    	list_for_each_entry(nh, &rt6_nh_list, next) {
		err = __ip6_ins_rt(nh->rt6_info, &cfg->fc_nlinfo, &nh->mxc);
		/* nh->rt6_info is used or freed at this point, reset to NULL*/
		nh->rt6_info = NULL;
		if (err) {
			if (replace && nhn)
				ip6_print_replace_route_err(&rt6_nh_list);
			err_nh = nh;
			goto add_errout;
		}

		/* Because each route is added like a single route we remove
		 * these flags after the first nexthop: if there is a collision,
		 * we have already failed to add the first nexthop:
		 * fib6_add_rt2node() has rejected it; when replacing, old
		 * nexthops have been replaced by first new, the rest should
		 * be added to it.
		 */
		cfg->fc_nlinfo.nlh->nlmsg_flags &= ~(NLM_F_EXCL |                    <<<< This flag is unset here to allow duplicate routes in fib6_add_rt2node() function.
						     NLM_F_REPLACE);
		nhn++;
	}
	

Thanks,
Preethi

Re: Query regarding IPV6 multipath (ECMP) routes in Linux

[Preethi Ramachandra]

+ Reji

On 15/04/19, 3:02 PM, "Preethi Ramachandra" <preethir@juniper.net> wrote:

    Hi,
    
    I want to understand the IPV4 and IPV6 multipath(ECMP) routes behaviour in Linux. 
    
    I tried configuring IPV4 and IPV6 multipath routes on Linux using ip route add command. Linux versions used in this testing are 
     1. Linux Kernel: 3.14.52-rt50-WR7.0.0.9_ovp
     2. Linux Kernel: 4.8.28-WR2.2.1_standard
    
    IPV4 multipath route addition and display:
    ip route add 1.1.1.0/24 nexthop via 192.168.1.254 nexthop via 192.168.0.254
    
    ip route show 
    1.1.1.0/24
            nexthop via 192.168.1.254  dev virbr0 weight 1
            nexthop via 192.168.0.254  dev eth0 weight 1
    
    IPV6 multipath route addition and display:
    ip route add 2001:d98:0:e101::1/128  nexthop via 3100:db8:c70:a20:c2bf:a7ff:fed2:1111 nexthop via 2102:db8:c70:a20::2
    
    ip -6 route show
    2001:d98:0:e101::1 via 3100:db8:c70:a20:c2bf:a7ff:fed2:1111 dev eth0  metric 1024
    2001:d98:0:e101::1 via 2102:db8:c70:a20::2 dev eth0  metric 1024
    
    In IPV4 a single route is added with multipath nexthops. However, in IPV6 , multipath routes are added as separate single routes with each multipath NH. I wanted to understand why is this behaviour change in Linux for IPV6 multipath routes. Why wasn’t same IPV4 approach followed in this case?
    
    The code path to add IPV6 multipath routes is 
    
    inet6_rtm_newroute()
      rtm_to_fib6_config()
      ip6_route_multipath_add()
        	list_for_each_entry(nh, &rt6_nh_list, next) {
    		err = __ip6_ins_rt(nh->rt6_info, &cfg->fc_nlinfo, &nh->mxc);
    		/* nh->rt6_info is used or freed at this point, reset to NULL*/
    		nh->rt6_info = NULL;
    		if (err) {
    			if (replace && nhn)
    				ip6_print_replace_route_err(&rt6_nh_list);
    			err_nh = nh;
    			goto add_errout;
    		}
    
    		/* Because each route is added like a single route we remove
    		 * these flags after the first nexthop: if there is a collision,
    		 * we have already failed to add the first nexthop:
    		 * fib6_add_rt2node() has rejected it; when replacing, old
    		 * nexthops have been replaced by first new, the rest should
    		 * be added to it.
    		 */
    		cfg->fc_nlinfo.nlh->nlmsg_flags &= ~(NLM_F_EXCL |                    <<<< This flag is unset here to allow duplicate routes in fib6_add_rt2node() function.
    						     NLM_F_REPLACE);
    		nhn++;
    	}
    	
    
    Thanks,
    Preethi
    
    
    
    
    
    
    
    
    
    
    

Re: Query regarding IPV6 multipath (ECMP) routes in Linux

[David Ahern]

On 4/15/19 3:32 AM, Preethi Ramachandra wrote:
> 
> In IPV4 a single route is added with multipath nexthops. However, in IPV6 , multipath routes are added as separate single routes with each multipath NH. I wanted to understand why is this behaviour change in Linux for IPV6 multipath routes. Why wasn’t same IPV4 approach followed in this case?

IPv4 support for multipath existed first. IPv6 is fairly new.

If you move forward a few kernel versions there is a bit more
consistency between IPv4 and IPv6, but still there are annoying
differences and limitations due to IPv6 code guessing whether to make
gateway routes multipath or not (see rt6_qualify_for_ecmp).