* [PATCH 0/1] Pass correct tun device's RX queue number to XDP program
@ 2020-04-10 16:20 Gilberto Bertin
2020-04-10 16:20 ` [PATCH 1/1] net: tun: record RX queue in skb before do_xdp_generic() Gilberto Bertin
0 siblings, 1 reply; 3+ messages in thread
From: Gilberto Bertin @ 2020-04-10 16:20 UTC (permalink / raw)
To: bpf, jasowang; +Cc: Gilberto Bertin
Hi,
I'm trying to run XDP on top of a tun device in multi queue mode as that allows
me to easily play on my laptop with an interface that supports multiple queues.
While testing it I found an issue: regardless of the the tun fd (i.e. RX queue)
I'm writing packets to, the XDP program thinks all packets are coming from RX
queue 0.
I'd like to propose a fix for that (see next mail) but let me first share some
context.
Here's what I'm using to setup a tun device with multiple RX/TX queues and write
some test packets into all of its RX queues:
struct ifreq ifr = {
.ifr_flags = IFF_TAP | IFF_NO_PI | IFF_MULTI_QUEUE, .ifr_name = "mytun0"
};
int fds[4];
for (int i = 0; i < 4; i++) {
if ((fds[i] = open("/dev/net/tun", O_RDWR)) < 0)
return -1;
if (ioctl(fds[i], TUNSETIFF, (void *)&ifr))
return -1;
}
while (1) {
for (int i = 0 ; i < 4; i++)
write(fds[i], "\x12\x34\x56\x78\x90\xab\x12\x34\x56\x67\x90\xab\x08\x00", 14);
sleep(2);
}
Next I reproduced the problem with this XDP program attached to the tun device
created with the previous code snippet:
$ cat xdp_kern.c
#include <linux/bpf.h>
#include <bpf_helpers.h>
SEC("prog")
unsigned int xdp_prog(struct xdp_md *xdp) {
bpf_printk("rx queue: %d\n", xdp->rx_queue_index);
return XDP_PASS;
}
char _license[] SEC("license") = "GPL";
$ sudo ip link set dev mytun0 xdpdrv obj ./xdp_kern.o
$ sudo bpftool prog tracelog
a.out-22189 [003] .... 14444.594609: 0: rx queue: 0
a.out-22189 [003] .... 14444.594645: 0: rx queue: 0
a.out-22189 [003] .... 14444.594653: 0: rx queue: 0
a.out-22189 [003] .... 14444.594661: 0: rx queue: 0
Here the XDP program is reporting all packets coming from RX queue 0 when they
should have been coming from RX queues 0..3.
After looking into the tun driver I ended up writing this stap script which
reveals the issue:
$ cat rx_queue.stap
probe kernel.function("tun_get_user") {
printf("tun_get_user(): tfile->xdp_rxq.queue_index: %d\n", $tfile->xdp_rxq->queue_index);
}
probe kernel.function("netif_receive_generic_xdp") {
printf("netif_receive_generic_xdp(): skb->queue_mapping: %d\n", $skb->queue_mapping);
printf("\n");
}
$ sudo stap ./rx_queue.stap
tun_get_user(): tfile->xdp_rxq.queue_index: 0
netif_receive_generic_xdp(): skb->queue_mapping: 0
tun_get_user(): tfile->xdp_rxq.queue_index: 1
netif_receive_generic_xdp(): skb->queue_mapping: 0
tun_get_user(): tfile->xdp_rxq.queue_index: 2
netif_receive_generic_xdp(): skb->queue_mapping: 0
tun_get_user(): tfile->xdp_rxq.queue_index: 3
netif_receive_generic_xdp(): skb->queue_mapping: 0
It looks like the tun driver (specifically tun_get_user()) is not passing
correctly the information about the RX queue to do_xdp_generic().
Setting the queue_mapping field in the skb with skb_record_rx_queue() seems to
fix the issue.
Same test with the patched kernel:
sudo bpftool prog tracelog
a.out-791 [004] .... 4826.499356: 0: rx queue: 0
a.out-791 [004] .... 4826.499532: 0: rx queue: 1
a.out-791 [004] .... 4826.499541: 0: rx queue: 2
a.out-791 [004] .... 4826.499549: 0: rx queue: 3
While at it, I also fixed tun_xdp_one(), which was calling skb_record_rx_queue()
after invoking do_xdp_generic().
Gilberto Bertin (1):
net: tun: record RX queue in skb before do_xdp_generic()
drivers/net/tun.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
2.20.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH 1/1] net: tun: record RX queue in skb before do_xdp_generic()
2020-04-10 16:20 [PATCH 0/1] Pass correct tun device's RX queue number to XDP program Gilberto Bertin
@ 2020-04-10 16:20 ` Gilberto Bertin
2020-04-13 3:59 ` David Miller
0 siblings, 1 reply; 3+ messages in thread
From: Gilberto Bertin @ 2020-04-10 16:20 UTC (permalink / raw)
To: bpf, jasowang; +Cc: Gilberto Bertin
This allows netif_receive_generic_xdp() to correctly determine the RX
queue from which the skb is coming, so that the context passed to the
XDP program will contain the correct RX queue index.
Signed-off-by: Gilberto Bertin <me@jibi.io>
---
drivers/net/tun.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index 228fe449dc6d..42b5d8740987 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -1886,6 +1886,7 @@ static ssize_t tun_get_user(struct tun_struct *tun, struct tun_file *tfile,
skb_reset_network_header(skb);
skb_probe_transport_header(skb);
+ skb_record_rx_queue(skb, tfile->queue_index);
if (skb_xdp) {
struct bpf_prog *xdp_prog;
@@ -2457,6 +2458,7 @@ static int tun_xdp_one(struct tun_struct *tun,
skb->protocol = eth_type_trans(skb, tun->dev);
skb_reset_network_header(skb);
skb_probe_transport_header(skb);
+ skb_record_rx_queue(skb, tfile->queue_index);
if (skb_xdp) {
err = do_xdp_generic(xdp_prog, skb);
@@ -2468,7 +2470,6 @@ static int tun_xdp_one(struct tun_struct *tun,
!tfile->detached)
rxhash = __skb_get_hash_symmetric(skb);
- skb_record_rx_queue(skb, tfile->queue_index);
netif_receive_skb(skb);
/* No need for get_cpu_ptr() here since this function is
--
2.20.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH 1/1] net: tun: record RX queue in skb before do_xdp_generic()
2020-04-10 16:20 ` [PATCH 1/1] net: tun: record RX queue in skb before do_xdp_generic() Gilberto Bertin
@ 2020-04-13 3:59 ` David Miller
0 siblings, 0 replies; 3+ messages in thread
From: David Miller @ 2020-04-13 3:59 UTC (permalink / raw)
To: me; +Cc: bpf, jasowang
From: Gilberto Bertin <me@jibi.io>
Date: Fri, 10 Apr 2020 18:20:59 +0200
> This allows netif_receive_generic_xdp() to correctly determine the RX
> queue from which the skb is coming, so that the context passed to the
> XDP program will contain the correct RX queue index.
>
> Signed-off-by: Gilberto Bertin <me@jibi.io>
Applied and queued up for -stable, thanks.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-04-13 3:59 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-10 16:20 [PATCH 0/1] Pass correct tun device's RX queue number to XDP program Gilberto Bertin
2020-04-10 16:20 ` [PATCH 1/1] net: tun: record RX queue in skb before do_xdp_generic() Gilberto Bertin
2020-04-13 3:59 ` David Miller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).