Re: Veth pair swallow packets for XDP_TX operation

[William Tu <u9012063@gmail.com>]

On Thu, Jan 16, 2020 at 1:13 AM Toshiaki Makita
<toshiaki.makita1@gmail.com> wrote:
>
> Hi Hanlin,
>
> On 2020/01/16 7:35, Hanlin Shi wrote:
> > Hi community,
> >
> > I’m prototyping an XDP program, and the hit issues with XDP_TX operation on veth device. The following code snippet is working as expected on 4.15.0-54-generic, but is NOT working on 4.20.17-042017-lowlatency (I got the kernel here: https://kernel.ubuntu.com/~kernel-ppa/mainline/v4.20.17/).
> >
> > Here’s my setup: I created a veth pair (namely veth1 and veth2), and put them in two namespaces (namely ns1 and ns2). I assigned address 60.0.0.1 on veth1 and 60.0.0.2 on veth2, set the device as the default interface in its namespace respectively (e.g. in ns1, do “ip r set default dev veth1”). Then in ns1, I ping 60.0.0.2, and tcpdump on veth1’s RX for ICMP.
> >
> > Before loading any XDP program on veth2, I can see ICMP replies on veth1 interface. I load a program which do “XDP_TX” for all packets on veth2. I expect to see the same ICMP packet being returned, but I saw nothing.
> >
> > I added some debugging message in the XDP program so I’m sure that the packet is processed on veth2, but on veth1, even with promisc mode on, I cannot see any ICMP packets or even ARP packets. In my understanding, 4.15 is using generic XDP mode where 4.20 is using native XDP mode for veth, so I guess there’s something wrong with veth native XDP and need some helps on fixing the issue.
>
> You need to load a dummy program to receive packets from peer XDP_TX when using native veth XDP.
>
> The dummy program is something like this:
> int xdp_pass(struct xdp_md *ctx) {
>         return XDP_PASS;
> }
> And load this program on "veth1".
>
> For more information please refer to this slides.
> https://netdevconf.info/0x13/session.html?talk-veth-xdp
>
> Also there is a working example here.
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/test_xdp_veth.sh
>
> Toshiaki Makita
>
> >
> > Please let me know if you need help on reproducing the issue.
> >
> > Thanks,
> > Hanlin
> >
> > PS: here’s the src code for the XDP program:
> > #include <stddef.h>
> > #include <string.h>
> > #include <linux/if_vlan.h>
> > #include <stdbool.h>
> > #include <bpf/bpf_endian.h>
> > #include <linux/if_ether.h>
> > #include <linux/ip.h>
> > #include <linux/tcp.h>
> > #include <linux/udp.h>
> > #include <linux/in.h>#define DEBUG
> > #include "bpf_helpers.h"
> >
> > SEC("xdp")
> > int loadbal(struct xdp_md *ctx) {
> >    bpf_printk("got packet, direct return\n");
> >    return XDP_TX;
> > }char _license[] SEC("license") = "GPL";
> >
> > "bpf_helpers.h" can be found here: https://github.com/dropbox/goebpf/raw/master/bpf_helpers.h
> >

Hi Hanlin,

I tested XDP_TX and the case you mentioned, and it works as OK on my 5.3 kernel.
I used the script to tested, can you give it a try?

#!/bin/bash
ip netns add at_ns0
ip link add p0 type veth peer name afxdp-p0
ip link set p0 netns at_ns0
ip addr add 10.1.1.2/24 dev afxdp-p0
ip link set dev afxdp-p0 up

ip netns exec at_ns0 sh << NS_EXEC_HEREDOC
ip addr add "10.1.1.1/24" dev p0
ip link set dev p0 up
NS_EXEC_HEREDOC

tcpdump -l -n -i afxdp-p0 -w /tmp/t.pcap icmp &
ping -c 100 10.1.1.1 &

# the xdp program return XDP_TX
ip netns exec at_ns0 ip link set dev p0 xdp obj xdp1_kern.o sec xdp1

$ tcpdump -r /tmp/t.pcap
13:24:59.925165 IP 10.1.1.1 > 10.1.1.2: ICMP echo reply, id 31521, seq
3, length 64
13:25:00.949240 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 4, length 64
13:25:00.949273 IP 10.1.1.1 > 10.1.1.2: ICMP echo reply, id 31521, seq
4, length 64
13:25:01.972369 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 5, length 64
13:25:01.972402 IP 10.1.1.1 > 10.1.1.2: ICMP echo reply, id 31521, seq
5, length 64
load the XDP_TX progam...
13:25:02.995996 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 6, length 64
13:25:04.021256 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 7, length 64
13:25:05.044943 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 8, length 64
13:25:06.069341 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 9, length 64
13:25:07.093121 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 10, length 64
13:25:08.115943 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 11, length 64
13:25:09.141542 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 31521,
seq 12, length 64

William