IPv6 multicast with VRF

IPv6 multicast with VRF

[Vladimir Oltean]

Hi,

I don't have experience with either IPv6 multicast or VRF, yet I need to
send some IPv6 multicast packets from a device enslaved to a VRF, and I
don't really know what's wrong with the routing table setup.

The system is configured in the following way:

 ip link set dev eth0 up

 # The kernel kindly creates a ff00::/8 route for IPv6 multicast traffic
 # in the local table, and I think this is what makes multicast route
 # lookups find the egress device.
 ip -6 route show table local
local ::1 dev lo proto kernel metric 0 pref medium
local fe80::204:9fff:fe05:f4ab dev eth0 proto kernel metric 0 pref medium
multicast ff00::/8 dev eth0 proto kernel metric 256 pref medium

 ip -6 route get ff02::1
multicast ff02::1 dev eth0 table local proto kernel src fe80::204:9fff:fe05:f4ab metric 256 pref medium

 ip link add dev vrf0 type vrf table 3 && ip link set dev vrf0 up

 ip -4 route add table 3 unreachable default metric 4278198272

 ip -6 route add table 3 unreachable default metric 4278198272

 ip link set dev eth0 master vrf0

The problem seems to be that, although the "ff00::/8 dev eth0" route
migrates from table 255 to table 3, route lookups after this point fail
to find it and return -ENETUNREACH (ip6_null_entry).

 ip -6 route show table local
local ::1 dev lo proto kernel metric 0 pref medium

 ip -6 route show table main
::1 dev lo proto kernel metric 256 pref medium

 ip -6 route show table 3
local fe80::204:9fff:fe05:f4ab dev eth0 proto kernel metric 0 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
multicast ff00::/8 dev eth0 proto kernel metric 256 pref medium
unreachable default dev lo metric 4278198272 pref medium

 ip -6 route get ff02::1
RTNETLINK answers: Network is unreachable

 ip -6 route get vrf vrf0 ff02::1
RTNETLINK answers: Network is unreachable

I'm not exactly sure what is missing?

Re: IPv6 multicast with VRF

[David Ahern]

On 4/20/22 10:54 AM, Vladimir Oltean wrote:
> Hi,
> 
> I don't have experience with either IPv6 multicast or VRF, yet I need to
> send some IPv6 multicast packets from a device enslaved to a VRF, and I
> don't really know what's wrong with the routing table setup.
> 
> The system is configured in the following way:
> 
>  ip link set dev eth0 up
> 
>  # The kernel kindly creates a ff00::/8 route for IPv6 multicast traffic
>  # in the local table, and I think this is what makes multicast route
>  # lookups find the egress device.
>  ip -6 route show table local
> local ::1 dev lo proto kernel metric 0 pref medium
> local fe80::204:9fff:fe05:f4ab dev eth0 proto kernel metric 0 pref medium
> multicast ff00::/8 dev eth0 proto kernel metric 256 pref medium
> 
>  ip -6 route get ff02::1
> multicast ff02::1 dev eth0 table local proto kernel src fe80::204:9fff:fe05:f4ab metric 256 pref medium
> 
>  ip link add dev vrf0 type vrf table 3 && ip link set dev vrf0 up
> 
>  ip -4 route add table 3 unreachable default metric 4278198272
> 
>  ip -6 route add table 3 unreachable default metric 4278198272
> 
>  ip link set dev eth0 master vrf0
> 
> The problem seems to be that, although the "ff00::/8 dev eth0" route
> migrates from table 255 to table 3, route lookups after this point fail
> to find it and return -ENETUNREACH (ip6_null_entry).
> 
>  ip -6 route show table local
> local ::1 dev lo proto kernel metric 0 pref medium
> 
>  ip -6 route show table main
> ::1 dev lo proto kernel metric 256 pref medium
> 
>  ip -6 route show table 3
> local fe80::204:9fff:fe05:f4ab dev eth0 proto kernel metric 0 pref medium
> fe80::/64 dev eth0 proto kernel metric 256 pref medium
> multicast ff00::/8 dev eth0 proto kernel metric 256 pref medium
> unreachable default dev lo metric 4278198272 pref medium
> 
>  ip -6 route get ff02::1
> RTNETLINK answers: Network is unreachable
> 
>  ip -6 route get vrf vrf0 ff02::1
> RTNETLINK answers: Network is unreachable
> 
> I'm not exactly sure what is missing?

Did you adjust the FIB rules? See the documentation in the kernel repo.

And add a device scope to the `get`. e.g.,

    ip -6 route get ff02::1%eth0

Re: IPv6 multicast with VRF

[Vladimir Oltean]

On Wed, Apr 20, 2022 at 12:59:45PM -0600, David Ahern wrote:
> Did you adjust the FIB rules? See the documentation in the kernel repo.

Sorry, I don't understand what you mean by "adjusting". I tried various
forms of adding an IPv6 multicast route on eth0, to multiple tables,
some routes more generic and some more specific, and none seem to match
when eth0 is under a VRF, for a reason I don't really know. This does
not occur with IPv4 multicast, by the way.

By documentation I think you mean Documentation/networking/vrf.rst.
I went through it but I didn't notice something that would make me
realize what the issue is.

> And add a device scope to the `get`. e.g.,
> 
>     ip -6 route get ff02::1%eth0

I'm probably not understanding this, because:

 ip -6 route get ff02::1%eth0
Error: inet6 prefix is expected rather than "ff02::1%eth0".

Re: IPv6 multicast with VRF

[David Ahern]

On 4/20/22 1:18 PM, Vladimir Oltean wrote:
> On Wed, Apr 20, 2022 at 12:59:45PM -0600, David Ahern wrote:
>> Did you adjust the FIB rules? See the documentation in the kernel repo.
> 
> Sorry, I don't understand what you mean by "adjusting". I tried various
> forms of adding an IPv6 multicast route on eth0, to multiple tables,
> some routes more generic and some more specific, and none seem to match
> when eth0 is under a VRF, for a reason I don't really know. This does
> not occur with IPv4 multicast, by the way.
> 
> By documentation I think you mean Documentation/networking/vrf.rst.
> I went through it but I didn't notice something that would make me
> realize what the issue is.

try this:
    https://static.sched.com/hosted_files/ossna2017/fe/vrf-tutorial-oss.pdf
slide 79 and on

> 
>> And add a device scope to the `get`. e.g.,
>>
>>     ip -6 route get ff02::1%eth0
> 
> I'm probably not understanding this, because:
> 
>  ip -6 route get ff02::1%eth0
> Error: inet6 prefix is expected rather than "ff02::1%eth0".

ip -6 ro get oif eth0 ff02::1

(too many syntax differences between tools)

Re: IPv6 multicast with VRF

[Vladimir Oltean]

On Wed, Apr 20, 2022 at 02:40:53PM -0600, David Ahern wrote:
> On 4/20/22 1:18 PM, Vladimir Oltean wrote:
> > On Wed, Apr 20, 2022 at 12:59:45PM -0600, David Ahern wrote:
> >> Did you adjust the FIB rules? See the documentation in the kernel repo.
> >
> > Sorry, I don't understand what you mean by "adjusting". I tried various
> > forms of adding an IPv6 multicast route on eth0, to multiple tables,
> > some routes more generic and some more specific, and none seem to match
> > when eth0 is under a VRF, for a reason I don't really know. This does
> > not occur with IPv4 multicast, by the way.
> >
> > By documentation I think you mean Documentation/networking/vrf.rst.
> > I went through it but I didn't notice something that would make me
> > realize what the issue is.
>
> try this:
>     https://static.sched.com/hosted_files/ossna2017/fe/vrf-tutorial-oss.pdf
> slide 79 and on

Yeah, that worked. Well, now I know what vrf_prepare() and vrf_cleanup()
from tools/testing/selfteste/net/forwarding/lib.sh are for, I guess..

Thanks for helping and for sharing the presentation.

> >> And add a device scope to the `get`. e.g.,
> >>
> >>     ip -6 route get ff02::1%eth0
> >
> > I'm probably not understanding this, because:
> >
> >  ip -6 route get ff02::1%eth0
> > Error: inet6 prefix is expected rather than "ff02::1%eth0".
>
> ip -6 ro get oif eth0 ff02::1
>
> (too many syntax differences between tools)

Could you explain why specifying the oif is needed here? If I don't do
it, I still can't find the route. Either that, or what would an
application need to do to find the route from the VRF FIB?

 ip -6 route get vrf vrf0 ff02::1
RTNETLINK answers: Network is unreachable
 ip -6 route get vrf vrf0 ff02::1 oif eth0
multicast ff02::1 dev eth0 table 3 proto kernel src 2001:db8:1::1 metric 256 pref medium

For some context, the multicast application I'm trying to get running in
a VRF is mcjoin (https://github.com/troglobit/mcjoin). It will send
packets as long as the interface only has a link-local IPv6 address.
As long as I add a global IPv6 address *and* the netdev is in the VRF
(basically the circumstances from the forwarding selftests), sendto()
fails with -ENETUNREACH.

 ip vrf exec vrf0 mcjoin -s -o -i eth0 ff0e::1 -c 1
Sending IPv6 multicast on eth0 addr, fe80::201:2ff:fe03:401 ifindex: 10, sd: 6
*,ff0e::1: invalid 0     delay 0     gaps 0     reorder 0     dupes 0     bytes 100           packets 1

Total: 1 packets

vs:

 ip addr add 2001:db8:1::1/64 dev eth0
 ip vrf exec vrf0 mcjoin -s -o -i eth0 ff0e::1 -c 1
Sending IPv6 multicast on eth0 addr, 2001:db8:1::1 ifindex: 10, sd: 6
Failed sending mcast to ff2e::1: Network is unreachable
*,ff2e::1: invalid 0     delay 0     gaps 1     reorder 0     dupes 0     bytes 0             packets 0

Total: 0 packets

Re: IPv6 multicast with VRF

[David Ahern]

On 4/21/22 3:24 AM, Vladimir Oltean wrote:
>>>  ip -6 route get ff02::1%eth0
>>> Error: inet6 prefix is expected rather than "ff02::1%eth0".
>>
>> ip -6 ro get oif eth0 ff02::1
>>
>> (too many syntax differences between tools)
> 
> Could you explain why specifying the oif is needed here? If I don't do

multicast and linklocal are local to a device, so you need to specify
which interface to use.

> it, I still can't find the route. Either that, or what would an
> application need to do to find the route from the VRF FIB?

applications bind their sockets to a VRF device or a port device.