Multicast routing: wrong output interface selected unless VRF default route is added

Multicast routing: wrong output interface selected unless VRF default route is added

[Qiwei Wen]

[-- Attachment #1: Type: text/plain, Size: 1978 bytes --]

Hi all,

While experimenting with FRRouting, I observed the following
behaviour. I'm not sure whether it's intended or not.

In a virtual machine set up as a multicast router, I added two
networks, created a VRF, and enslaved interfaces to both networks to
the VRF, like so:

ip link add blue type vrf table 1001
ip link set eth0 master blue
ip link set eth1 master blue

I then set up PIM on the router VM (FRR configs attached) and started
the multicast sender and receiver processes on two other VMs. The
mroutes came up as expected (ip show mroute table 1001), but no
packets came to the receiver. I added the following debug message to
ipmr_queue_xmit, just before the NF_HOOK macro:

+    pr_info("calling NF_HOOK! vif->dev is %s,"
+            " dev is %s, skb->dev is %s\n",
+            vif->dev->name, dev->name, skb->dev->name);

and I found that "dev", the selected output interface, is in fact the
output interface of the main table (unicast) default route. Running
tcpdump on that (very wrong) output interface confirmed this.

I then went back to networking/vrf.txt, and found that I forgot to do this:

ip route add table 1001 unreachable default metric 4278198272

after this step, multicast routing began to work correctly.

Further debugging-by-printk lead to these observations:
1. Using the main table (without VRFs), multicast routing works fine
with or without the default unicast route; but in the function "
ip_route_output_key_hash_rcu", the call to "fib_lookup" in fact fails
with -101, "network unreachable".
2. Using the VRF table 1001, the kernel stops routing multicast
packets to the wrong interface once the unreachable default route is
added. "fib_lookup" continues to fail, but with -113, "host
unreachable".

My questions are:
1. is fib_lookup supposed to work with multicast daddr? If so, has
multicast routing been working for the wrong reason?
2. Why does the addition of a unicast default route affect multicast
routing behaviour?

-Dave

[-- Attachment #2: frr-runn --]
[-- Type: application/octet-stream, Size: 295 bytes --]

!
frr version 7.3
frr defaults traditional
hostname debian
log syslog informational
no ipv6 forwarding
service integrated-vtysh-config
!
vrf blue
 ip pim rp 10.1.2.2 224.0.0.0/4
 exit-vrf
!
interface enp0s8 vrf blue
 ip igmp
 ip pim
!
interface enp0s9 vrf blue
 ip igmp
 ip pim
!
line vty
!
end

Re: Multicast routing: wrong output interface selected unless VRF default route is added

[Ido Schimmel]

On Mon, Jun 22, 2020 at 09:19:10AM +1000, Qiwei Wen wrote:
> Hi all,
> 
> While experimenting with FRRouting, I observed the following
> behaviour. I'm not sure whether it's intended or not.
> 
> In a virtual machine set up as a multicast router, I added two
> networks, created a VRF, and enslaved interfaces to both networks to
> the VRF, like so:
> 
> ip link add blue type vrf table 1001
> ip link set eth0 master blue
> ip link set eth1 master blue
> 
> I then set up PIM on the router VM (FRR configs attached) and started
> the multicast sender and receiver processes on two other VMs. The
> mroutes came up as expected (ip show mroute table 1001), but no
> packets came to the receiver. I added the following debug message to
> ipmr_queue_xmit, just before the NF_HOOK macro:
> 
> +    pr_info("calling NF_HOOK! vif->dev is %s,"
> +            " dev is %s, skb->dev is %s\n",
> +            vif->dev->name, dev->name, skb->dev->name);
> 
> and I found that "dev", the selected output interface, is in fact the
> output interface of the main table (unicast) default route. Running
> tcpdump on that (very wrong) output interface confirmed this.
> 
> I then went back to networking/vrf.txt, and found that I forgot to do this:
> 
> ip route add table 1001 unreachable default metric 4278198272
> 
> after this step, multicast routing began to work correctly.
> 
> Further debugging-by-printk lead to these observations:
> 1. Using the main table (without VRFs), multicast routing works fine
> with or without the default unicast route; but in the function "
> ip_route_output_key_hash_rcu", the call to "fib_lookup" in fact fails
> with -101, "network unreachable".
> 2. Using the VRF table 1001, the kernel stops routing multicast
> packets to the wrong interface once the unreachable default route is
> added. "fib_lookup" continues to fail, but with -113, "host
> unreachable".
> 
> My questions are:
> 1. is fib_lookup supposed to work with multicast daddr? If so, has
> multicast routing been working for the wrong reason?
> 2. Why does the addition of a unicast default route affect multicast
> routing behaviour?

I believe this was discussed in the past. See:
https://lore.kernel.org/netdev/20200115191920.GA1490933@splinter/#t

Re: Multicast routing: wrong output interface selected unless VRF default route is added

[David Ahern]

On 6/23/20 11:38 AM, Ido Schimmel wrote:
>>
>> My questions are:
>> 1. is fib_lookup supposed to work with multicast daddr? If so, has
>> multicast routing been working for the wrong reason?

yes; it is a quirk with Linux routing. see the code after fib_lookup in
ip_route_output_key_hash_rcu. perf can show you the call chain that gets
to that lookup if you want the details.

>> 2. Why does the addition of a unicast default route affect multicast
>> routing behaviour?
> 
> I believe this was discussed in the past. See:
> https://lore.kernel.org/netdev/20200115191920.GA1490933@splinter/#t
>