* Re: Different behavior wrt VRF and no VRF - packet Tx
2021-04-15 19:41 ` David Ahern
@ 2021-04-16 3:57 ` Bala Sajja
2021-04-18 17:40 ` David Ahern
0 siblings, 1 reply; 5+ messages in thread
From: Bala Sajja @ 2021-04-16 3:57 UTC (permalink / raw)
To: David Ahern; +Cc: netdev
Hi David,
please find the ip link show output(for ifindex) and ping and
its corresponding perf fib events output. OIF seems MGMT(ifindex 5)
always, not enslaved interfaces ?
ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
master MGMT state UP mode DEFAULT group default qlen 1000
link/ether 08:00:27:ee:c2:f8 brd ff:ff:ff:ff:ff:ff
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
master MGMT state UP mode DEFAULT group default qlen 1000
link/ether 08:00:27:0e:75:05 brd ff:ff:ff:ff:ff:ff
4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UP mode DEFAULT group default qlen 1000
link/ether 08:00:27:08:6c:37 brd ff:ff:ff:ff:ff:ff
5: MGMT: <NOARP,MASTER,UP,LOWER_UP> mtu 65536 qdisc noqueue state UP
mode DEFAULT group default qlen 1000
link/ether c2:08:e9:2b:8a:a4 brd ff:ff:ff:ff:ff:ff
6: LOMGMT: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master
MGMT state UNKNOWN mode DEFAULT group default qlen 1000
link/ether c2:04:90:b4:2d:9e brd ff:ff:ff:ff:ff:ff
ping 1.1.1.100 -I enp0s3 -c 1
sudo perf report
Samples: 27 of event 'fib:fib_table_lookup', Event count (approx.): 27
Overhead Trace output
11.11% table 1 oif 0 iif 0 proto 0 0.0.0.0/0 -> 2.2.2.100/0 tos 0
scope 0 flags 0 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0
◆
7.41% table 254 oif 0 iif 1 proto 0 0.0.0.0/0 -> 192.168.1.11/0
tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src 192.168.1.2 err 0
▒
7.41% table 255 oif 0 iif 1 proto 0 0.0.0.0/0 -> 192.168.1.11/0
tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
▒
3.70% table 1 oif 0 iif 0 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 0 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0
▒
3.70% table 1 oif 5 iif 1 proto 1 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0
▒
3.70% table 1 oif 5 iif 1 proto 1 0.0.0.0/0 -> 2.2.2.100/0 tos 0
scope 0 flags 4 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0
▒
3.70% table 1 oif 5 iif 1 proto 1 2.2.2.100/0 -> 2.2.2.100/0 tos 0
scope 0 flags 5 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0
▒
3.70% table 1 oif 5 iif 1 proto 17 0.0.0.0/36297 -> 1.1.1.100/1025
tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0
▒
3.70% table 1 oif 5 iif 1 proto 17 2.2.2.100/36297 ->
1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src
2.2.2.100 err 0 ▒
3.70% table 254 oif 0 iif 1 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 0 ==> dev enp0s9 gw 192.168.1.1 src 192.168.1.2 err 0
▒
3.70% table 254 oif 0 iif 1 proto 0 192.168.1.2/0 ->
192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src
192.168.1.2 err 0 ▒
3.70% table 254 oif 0 iif 4 proto 0 1.1.1.100/0 -> 2.2.2.100/0 tos
0 scope 0 flags 0 ==> dev enp0s9 gw 192.168.1.1 src 192.168.1.2 err 0
▒
3.70% table 254 oif 0 iif 4 proto 0 192.168.1.11/0 ->
192.168.1.8/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src
192.168.1.2 err 0 ▒
3.70% table 255 oif 0 iif 1 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
▒
3.70% table 255 oif 0 iif 1 proto 0 192.168.1.2/0 ->
192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11 ▒
3.70% table 255 oif 0 iif 4 proto 0 1.1.1.100/0 -> 2.2.2.100/0 tos
0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
▒
3.70% table 255 oif 0 iif 4 proto 0 192.168.1.11/0 ->
192.168.1.8/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11 ▒
3.70% table 255 oif 0 iif 4 proto 0 192.168.1.12/0 ->
192.168.1.2/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src
192.168.1.2 err 0 ▒
3.70% table 255 oif 5 iif 1 proto 1 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
▒
3.70% table 255 oif 5 iif 1 proto 1 0.0.0.0/0 -> 2.2.2.100/0 tos 0
scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
▒
3.70% table 255 oif 5 iif 1 proto 1 2.2.2.100/0 -> 2.2.2.100/0 tos
0 scope 0 flags 5 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
▒
3.70% table 255 oif 5 iif 1 proto 17 0.0.0.0/36297 ->
1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11 ▒
3.70% table 255 oif 5 iif 1 proto 17 2.2.2.100/36297 ->
1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11 ▒
ping 1.1.1.100 -I enp0s8 -c 1
sudo perf report
Samples: 27 of event 'fib:fib_table_lookup', Event count (approx.): 27
Overhead Trace output
11.11% table 1 oif 0 iif 0 proto 0 0.0.0.0/0 -> 2.2.2.100/0 tos 0
scope 0 flags 0 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0
3.70% table 1 oif 0 iif 0 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 0 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0
3.70% table 1 oif 5 iif 1 proto 1 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0
3.70% table 1 oif 5 iif 1 proto 1 0.0.0.0/0 -> 2.2.2.100/0 tos 0
scope 0 flags 4 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0
3.70% table 1 oif 5 iif 1 proto 1 2.2.2.100/0 -> 2.2.2.100/0 tos 0
scope 0 flags 5 ==> dev LOMGMT gw 0.0.0.0 src 2.2.2.100 err 0
3.70% table 1 oif 5 iif 1 proto 17 0.0.0.0/51188 -> 1.1.1.100/1025
tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src 2.2.2.100 err 0
3.70% table 1 oif 5 iif 1 proto 17 2.2.2.100/51188 ->
1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev enp0s3 gw 0.0.0.0 src
2.2.2.100 err 0
3.70% table 254 oif 0 iif 1 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 0 ==> dev enp0s9 gw 192.168.1.1 src 192.168.1.2 err 0
3.70% table 254 oif 0 iif 1 proto 0 192.168.1.2/0 ->
192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src
192.168.1.2 err 0
3.70% table 254 oif 0 iif 4 proto 0 1.1.1.100/0 -> 2.2.2.100/0 tos
0 scope 0 flags 0 ==> dev enp0s9 gw 192.168.1.1 src 192.168.1.2 err 0
3.70% table 254 oif 0 iif 4 proto 0 192.168.1.1/0 ->
192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src
192.168.1.2 err 0
3.70% table 254 oif 0 iif 4 proto 0 192.168.1.11/0 ->
192.168.1.8/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src
192.168.1.2 err 0
3.70% table 254 oif 0 iif 4 proto 0 192.168.1.8/0 ->
192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src
192.168.1.2 err 0
3.70% table 255 oif 0 iif 1 proto 0 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
3.70% table 255 oif 0 iif 1 proto 0 192.168.1.2/0 ->
192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11
3.70% table 255 oif 0 iif 4 proto 0 1.1.1.100/0 -> 2.2.2.100/0 tos
0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
3.70% table 255 oif 0 iif 4 proto 0 192.168.1.1/0 ->
192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11
3.70% table 255 oif 0 iif 4 proto 0 192.168.1.11/0 ->
192.168.1.8/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11
3.70% table 255 oif 0 iif 4 proto 0 192.168.1.12/0 ->
192.168.1.2/0 tos 0 scope 0 flags 0 ==> dev enp0s9 gw 0.0.0.0 src
192.168.1.2 err 0
3.70% table 255 oif 0 iif 4 proto 0 192.168.1.8/0 ->
192.168.1.12/0 tos 0 scope 0 flags 0 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11
3.70% table 255 oif 5 iif 1 proto 1 0.0.0.0/0 -> 1.1.1.100/0 tos 0
scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
3.70% table 255 oif 5 iif 1 proto 1 0.0.0.0/0 -> 2.2.2.100/0 tos 0
scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
3.70% table 255 oif 5 iif 1 proto 1 2.2.2.100/0 -> 2.2.2.100/0 tos
0 scope 0 flags 5 ==> dev - gw 0.0.0.0 src 0.0.0.0 err -11
3.70% table 255 oif 5 iif 1 proto 17 0.0.0.0/51188 ->
1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11
3.70% table 255 oif 5 iif 1 proto 17 2.2.2.100/51188 ->
1.1.1.100/1025 tos 0 scope 0 flags 4 ==> dev - gw 0.0.0.0 src 0.0.0.0
err -11
Regards,
Bala.
On Fri, Apr 16, 2021 at 1:11 AM David Ahern <dsahern@gmail.com> wrote:
>
> On 4/15/21 12:15 AM, Bala Sajja wrote:
> > When interfaces are not part of VRF and below ip address config is
> > done on these interfaces, ping with -I (interface) option, we see
> > packets transmitting out of the right interfaces.
> >
> > ip addr add 2.2.2.100 peer 1.1.1.100/32 dev enp0s3
> > ip addr add 2.2.2.100 peer 1.1.1.100/32 dev enp0s8
> >
> > ping 1.1.1.100 -I enp0s3 , packet always goes out of enp0s3
> > ping 1.1.1.100 -I enp0s8, packet always goes out of enp0s8
> >
> > When interfaces are enslaved to VRF as below and ip are configured
> > on these interfaces, packets go out of one interface only.
> >
> > ip link add MGMT type vrf table 1
> > ip link set dev MGMT up
> > ip link set dev enp0s3 up
> > ip link set dev enp0s3 master MGMT
> > ip link set dev enp0s8 up
> > ip link set dev enp0s8 master MGMT
> > ip link set dev enp0s9 up
> >
> > ip addr add 2.2.2.100 peer 1.1.1.100/32 dev enp0s3
> > ip addr add 2.2.2.100 peer 1.1.1.100/32 dev enp0s8
> >
> > ping 1.1.1.100 -I enp0s3 , packet always goes out of enp0s3
> > ping 1.1.1.100 -I enp0s8, packet always goes out of enp0s3
> >
> >
>
> I believe this use case will not work since the FIB lookup loses the
> original device binding (the -I argument). take a look at the FIB lookup
> argument and result:
>
> perf record -e fib:*
> perf script
^ permalink raw reply [flat|nested] 5+ messages in thread