* [PATCH bpf-next 0/4] samples/bpf: xdpsock app enhancements
@ 2021-11-24 9:18 Ong Boon Leong
2021-11-24 9:18 ` [PATCH bpf-next 1/4] samples/bpf: xdpsock: add VLAN support for Tx-only operation Ong Boon Leong
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: Ong Boon Leong @ 2021-11-24 9:18 UTC (permalink / raw)
To: bpf, netdev
Cc: bjorn, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov,
Daniel Borkmann, David S . Miller, Jakub Kicinski,
Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh,
Ong Boon Leong
This patch series adds below capabilities to xdpsock app:-
1/4: Add VLAN tag (ID & Priority) to the generated Tx-Only frames.
2/4: Add DMAC and SMAC setting to the generated Tx-Only frames.
If parameters are not set, the previous DMAC and SMAC are used.
3/4: Add cyclic transmission (cycle-time) setting for Tx-only operation.
This option can be used together with batch size and packet count.
4/4: Add time-out with retries to Tx complete cleaning process to prevent
unsuccessful XDP Transmission from causing the Tx cleaning process
from polling indefinitely.
With above enhancements to xdpsock, we can know run concurrent VLAN-tagged
XDP TX streams in periodic Tx with batch size fashion.
For examples:
DMAC (-G) = fa:8d:f1:e2:0b:e8
SMAC (-H) = ce:17:07:17:3e:3a
VLAN ID (-J) = 1
VLAN Pri (-K) = 1-3
Tx Queue (-q) = 1-3
Cycle Time in us (-T) = 1000
Batch (-b) = 16
Packet Count = 1000000
Sending Board
=============
Terminal-1:
$ xdpsock -i enp0s29f1 -t -N -z -H ce:17:07:17:3e:3a -G fa:8d:f1:e2:0b:e8 -V -J 1 -K 1 -q 1 -T 1000 -b 16 -C 1000000 -x
Terminal-2:
$ xdpsock -i enp0s29f1 -t -N -z -H ce:17:07:17:3e:3a -G fa:8d:f1:e2:0b:e8 -V -J 1 -K 2 -q 2 -T 1000 -b 16 -C 1000000 -x
Terminal-3:
$ xdpsock -i enp0s29f1 -t -N -z -H ce:17:07:17:3e:3a -G fa:8d:f1:e2:0b:e8 -V -J 1 -K 3 -q 3 -T 1000 -b 16 -C 1000000 -x
Receiving Board
===============
Terminal-1:
$ xdpsock -i enp0s29f1 -r -N -z -q 1
Terminal-2:
$ xdpsock -i enp0s29f1 -r -N -z -q 2
Terminal-3:
$ xdpsock -i enp0s29f1 -r -N -z -q 3
Thanks
Ong Boon Leong (4):
samples/bpf: xdpsock: add VLAN support for Tx-only operation
samples/bpf: xdpsock: add Dest and Src MAC setting for Tx-only
operation
samples/bpf: xdpsock: add period cycle time to Tx operation
samples/bpf: xdpsock: add time-out for cleaning Tx
samples/bpf/xdpsock_user.c | 154 ++++++++++++++++++++++++++++++++-----
1 file changed, 134 insertions(+), 20 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH bpf-next 1/4] samples/bpf: xdpsock: add VLAN support for Tx-only operation
2021-11-24 9:18 [PATCH bpf-next 0/4] samples/bpf: xdpsock app enhancements Ong Boon Leong
@ 2021-11-24 9:18 ` Ong Boon Leong
2021-11-27 6:48 ` Song Liu
2021-11-24 9:18 ` [PATCH bpf-next 2/4] samples/bpf: xdpsock: add Dest and Src MAC setting " Ong Boon Leong
` (2 subsequent siblings)
3 siblings, 1 reply; 14+ messages in thread
From: Ong Boon Leong @ 2021-11-24 9:18 UTC (permalink / raw)
To: bpf, netdev
Cc: bjorn, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov,
Daniel Borkmann, David S . Miller, Jakub Kicinski,
Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh,
Ong Boon Leong
In multi-queue environment testing, the support for VLAN-tag based
steering is useful. So, this patch adds the capability to add
VLAN tag (VLAN ID and Priority) to the generated Tx frame.
To set the VLAN ID=10 and Priority=2 for Tx only through TxQ=3:
$ xdpsock -i eth0 -t -N -z -q 3 -V -J 10 -K 2
If VLAN ID (-J) and Priority (-K) is set, it default to
VLAN ID = 1
VLAN Priority = 0.
For example, VLAN-tagged Tx only, xdp copy mode through TxQ=1:
$ xdpsock -i eth0 -t -N -c -q 1 -V
Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
---
samples/bpf/xdpsock_user.c | 90 +++++++++++++++++++++++++++++++-------
1 file changed, 75 insertions(+), 15 deletions(-)
diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
index 49d7a6ad7e3..e09fabecd69 100644
--- a/samples/bpf/xdpsock_user.c
+++ b/samples/bpf/xdpsock_user.c
@@ -53,6 +53,12 @@
#define DEBUG_HEXDUMP 0
+#define VLAN_PRIO_MASK 0xe000 /* Priority Code Point */
+#define VLAN_PRIO_SHIFT 13
+#define VLAN_VID_MASK 0x0fff /* VLAN Identifier */
+#define VLAN_VID__DEFAULT 1
+#define VLAN_PRI__DEFAULT 0
+
typedef __u64 u64;
typedef __u32 u32;
typedef __u16 u16;
@@ -78,6 +84,9 @@ static u32 opt_batch_size = 64;
static int opt_pkt_count;
static u16 opt_pkt_size = MIN_PKT_SIZE;
static u32 opt_pkt_fill_pattern = 0x12345678;
+static bool opt_vlan_tag;
+static u16 opt_pkt_vlan_id = VLAN_VID__DEFAULT;
+static u16 opt_pkt_vlan_pri = VLAN_PRI__DEFAULT;
static bool opt_extra_stats;
static bool opt_quiet;
static bool opt_app_stats;
@@ -98,6 +107,14 @@ static u32 prog_id;
static bool opt_busy_poll;
static bool opt_reduced_cap;
+struct vlan_ethhdr {
+ unsigned char h_dest[6];
+ unsigned char h_source[6];
+ __be16 h_vlan_proto;
+ __be16 h_vlan_TCI;
+ __be16 h_vlan_encapsulated_proto;
+};
+
struct xsk_ring_stats {
unsigned long rx_npkts;
unsigned long tx_npkts;
@@ -737,11 +754,13 @@ static inline u16 udp_csum(u32 saddr, u32 daddr, u32 len,
#define ETH_FCS_SIZE 4
-#define PKT_HDR_SIZE (sizeof(struct ethhdr) + sizeof(struct iphdr) + \
+#define ETH_HDR_SIZE (opt_vlan_tag ? sizeof(struct vlan_ethhdr) : \
+ sizeof(struct ethhdr))
+#define PKT_HDR_SIZE (ETH_HDR_SIZE + sizeof(struct iphdr) + \
sizeof(struct udphdr))
#define PKT_SIZE (opt_pkt_size - ETH_FCS_SIZE)
-#define IP_PKT_SIZE (PKT_SIZE - sizeof(struct ethhdr))
+#define IP_PKT_SIZE (PKT_SIZE - ETH_HDR_SIZE)
#define UDP_PKT_SIZE (IP_PKT_SIZE - sizeof(struct iphdr))
#define UDP_PKT_DATA_SIZE (UDP_PKT_SIZE - sizeof(struct udphdr))
@@ -749,17 +768,42 @@ static u8 pkt_data[XSK_UMEM__DEFAULT_FRAME_SIZE];
static void gen_eth_hdr_data(void)
{
- struct udphdr *udp_hdr = (struct udphdr *)(pkt_data +
- sizeof(struct ethhdr) +
- sizeof(struct iphdr));
- struct iphdr *ip_hdr = (struct iphdr *)(pkt_data +
- sizeof(struct ethhdr));
- struct ethhdr *eth_hdr = (struct ethhdr *)pkt_data;
-
- /* ethernet header */
- memcpy(eth_hdr->h_dest, "\x3c\xfd\xfe\x9e\x7f\x71", ETH_ALEN);
- memcpy(eth_hdr->h_source, "\xec\xb1\xd7\x98\x3a\xc0", ETH_ALEN);
- eth_hdr->h_proto = htons(ETH_P_IP);
+ struct udphdr *udp_hdr;
+ struct iphdr *ip_hdr;
+
+ if (opt_vlan_tag) {
+ struct vlan_ethhdr *veth_hdr = (struct vlan_ethhdr *)pkt_data;
+ u16 vlan_tci = 0;
+
+ udp_hdr = (struct udphdr *)(pkt_data +
+ sizeof(struct vlan_ethhdr) +
+ sizeof(struct iphdr));
+ ip_hdr = (struct iphdr *)(pkt_data +
+ sizeof(struct vlan_ethhdr));
+
+ /* ethernet & VLAN header */
+ memcpy(veth_hdr->h_dest, "\x3c\xfd\xfe\x9e\x7f\x71", ETH_ALEN);
+ memcpy(veth_hdr->h_source, "\xec\xb1\xd7\x98\x3a\xc0", ETH_ALEN);
+ veth_hdr->h_vlan_proto = htons(ETH_P_8021Q);
+ vlan_tci = opt_pkt_vlan_id & VLAN_VID_MASK;
+ vlan_tci |= (opt_pkt_vlan_pri << VLAN_PRIO_SHIFT) & VLAN_PRIO_MASK;
+ veth_hdr->h_vlan_TCI = htons(vlan_tci);
+ veth_hdr->h_vlan_encapsulated_proto = htons(ETH_P_IP);
+ } else {
+ struct ethhdr *eth_hdr = (struct ethhdr *)pkt_data;
+
+ udp_hdr = (struct udphdr *)(pkt_data +
+ sizeof(struct ethhdr) +
+ sizeof(struct iphdr));
+ ip_hdr = (struct iphdr *)(pkt_data +
+ sizeof(struct ethhdr));
+
+ /* ethernet header */
+ memcpy(eth_hdr->h_dest, "\x3c\xfd\xfe\x9e\x7f\x71", ETH_ALEN);
+ memcpy(eth_hdr->h_source, "\xec\xb1\xd7\x98\x3a\xc0", ETH_ALEN);
+ eth_hdr->h_proto = htons(ETH_P_IP);
+ }
+
/* IP header */
ip_hdr->version = IPVERSION;
@@ -917,6 +961,9 @@ static struct option long_options[] = {
{"tx-pkt-count", required_argument, 0, 'C'},
{"tx-pkt-size", required_argument, 0, 's'},
{"tx-pkt-pattern", required_argument, 0, 'P'},
+ {"tx-vlan", no_argument, 0, 'V'},
+ {"tx-vlan-id", required_argument, 0, 'J'},
+ {"tx-vlan-pri", required_argument, 0, 'K'},
{"extra-stats", no_argument, 0, 'x'},
{"quiet", no_argument, 0, 'Q'},
{"app-stats", no_argument, 0, 'a'},
@@ -957,6 +1004,9 @@ static void usage(const char *prog)
" (Default: %d bytes)\n"
" Min size: %d, Max size %d.\n"
" -P, --tx-pkt-pattern=nPacket fill pattern. Default: 0x%x\n"
+ " -V, --tx-vlan Send VLAN tagged packets (For -t|--txonly)\n"
+ " -J, --tx-vlan-id=n Tx VLAN ID [1-4095]. Default: %d (For -V|--tx-vlan)\n"
+ " -K, --tx-vlan-pri=n Tx VLAN Priority [0-7]. Default: %d (For -V|--tx-vlan)\n"
" -x, --extra-stats Display extra statistics.\n"
" -Q, --quiet Do not display any stats.\n"
" -a, --app-stats Display application (syscall) statistics.\n"
@@ -966,7 +1016,8 @@ static void usage(const char *prog)
"\n";
fprintf(stderr, str, prog, XSK_UMEM__DEFAULT_FRAME_SIZE,
opt_batch_size, MIN_PKT_SIZE, MIN_PKT_SIZE,
- XSK_UMEM__DEFAULT_FRAME_SIZE, opt_pkt_fill_pattern);
+ XSK_UMEM__DEFAULT_FRAME_SIZE, opt_pkt_fill_pattern,
+ VLAN_VID__DEFAULT, VLAN_PRI__DEFAULT);
exit(EXIT_FAILURE);
}
@@ -978,7 +1029,7 @@ static void parse_command_line(int argc, char **argv)
opterr = 0;
for (;;) {
- c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:xQaI:BR",
+ c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:VJ:K:xQaI:BR",
long_options, &option_index);
if (c == -1)
break;
@@ -1059,6 +1110,15 @@ static void parse_command_line(int argc, char **argv)
case 'P':
opt_pkt_fill_pattern = strtol(optarg, NULL, 16);
break;
+ case 'V':
+ opt_vlan_tag = true;
+ break;
+ case 'J':
+ opt_pkt_vlan_id = atoi(optarg);
+ break;
+ case 'K':
+ opt_pkt_vlan_pri = atoi(optarg);
+ break;
case 'x':
opt_extra_stats = 1;
break;
--
2.25.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH bpf-next 2/4] samples/bpf: xdpsock: add Dest and Src MAC setting for Tx-only operation
2021-11-24 9:18 [PATCH bpf-next 0/4] samples/bpf: xdpsock app enhancements Ong Boon Leong
2021-11-24 9:18 ` [PATCH bpf-next 1/4] samples/bpf: xdpsock: add VLAN support for Tx-only operation Ong Boon Leong
@ 2021-11-24 9:18 ` Ong Boon Leong
2021-11-27 6:40 ` Song Liu
2021-11-27 9:51 ` Jesper Dangaard Brouer
2021-11-24 9:18 ` [PATCH bpf-next 3/4] samples/bpf: xdpsock: add period cycle time to Tx operation Ong Boon Leong
2021-11-24 9:18 ` [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx Ong Boon Leong
3 siblings, 2 replies; 14+ messages in thread
From: Ong Boon Leong @ 2021-11-24 9:18 UTC (permalink / raw)
To: bpf, netdev
Cc: bjorn, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov,
Daniel Borkmann, David S . Miller, Jakub Kicinski,
Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh,
Ong Boon Leong
To set Dest MAC address (-G|--tx-dmac) only:
$ xdpsock -i eth0 -t -N -z -G aa:bb:cc:dd:ee:ff
To set Source MAC address (-H|--tx-smac) only:
$ xdpsock -i eth0 -t -N -z -H 11:22:33:44:55:66
To set both Dest and Source MAC address:
$ xdpsock -i eth0 -t -N -z -G aa:bb:cc:dd:ee:ff \
-H 11:22:33:44:55:66
The default Dest and Source MAC address remain the same as before.
Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
---
samples/bpf/xdpsock_user.c | 36 +++++++++++++++++++++++++++++++-----
1 file changed, 31 insertions(+), 5 deletions(-)
diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
index e09fabecd69..691f442bbb2 100644
--- a/samples/bpf/xdpsock_user.c
+++ b/samples/bpf/xdpsock_user.c
@@ -14,6 +14,7 @@
#include <arpa/inet.h>
#include <locale.h>
#include <net/ethernet.h>
+#include <netinet/ether.h>
#include <net/if.h>
#include <poll.h>
#include <pthread.h>
@@ -87,6 +88,10 @@ static u32 opt_pkt_fill_pattern = 0x12345678;
static bool opt_vlan_tag;
static u16 opt_pkt_vlan_id = VLAN_VID__DEFAULT;
static u16 opt_pkt_vlan_pri = VLAN_PRI__DEFAULT;
+static struct ether_addr opt_txdmac = {{ 0x3c, 0xfd, 0xfe,
+ 0x9e, 0x7f, 0x71 }};
+static struct ether_addr opt_txsmac = {{ 0xec, 0xb1, 0xd7,
+ 0x98, 0x3a, 0xc0 }};
static bool opt_extra_stats;
static bool opt_quiet;
static bool opt_app_stats;
@@ -782,8 +787,9 @@ static void gen_eth_hdr_data(void)
sizeof(struct vlan_ethhdr));
/* ethernet & VLAN header */
- memcpy(veth_hdr->h_dest, "\x3c\xfd\xfe\x9e\x7f\x71", ETH_ALEN);
- memcpy(veth_hdr->h_source, "\xec\xb1\xd7\x98\x3a\xc0", ETH_ALEN);
+
+ memcpy(veth_hdr->h_dest, &opt_txdmac, ETH_ALEN);
+ memcpy(veth_hdr->h_source, &opt_txsmac, ETH_ALEN);
veth_hdr->h_vlan_proto = htons(ETH_P_8021Q);
vlan_tci = opt_pkt_vlan_id & VLAN_VID_MASK;
vlan_tci |= (opt_pkt_vlan_pri << VLAN_PRIO_SHIFT) & VLAN_PRIO_MASK;
@@ -799,8 +805,8 @@ static void gen_eth_hdr_data(void)
sizeof(struct ethhdr));
/* ethernet header */
- memcpy(eth_hdr->h_dest, "\x3c\xfd\xfe\x9e\x7f\x71", ETH_ALEN);
- memcpy(eth_hdr->h_source, "\xec\xb1\xd7\x98\x3a\xc0", ETH_ALEN);
+ memcpy(eth_hdr->h_dest, &opt_txdmac, ETH_ALEN);
+ memcpy(eth_hdr->h_source, &opt_txsmac, ETH_ALEN);
eth_hdr->h_proto = htons(ETH_P_IP);
}
@@ -964,6 +970,8 @@ static struct option long_options[] = {
{"tx-vlan", no_argument, 0, 'V'},
{"tx-vlan-id", required_argument, 0, 'J'},
{"tx-vlan-pri", required_argument, 0, 'K'},
+ {"tx-dmac", required_argument, 0, 'G'},
+ {"tx-smac", required_argument, 0, 'H'},
{"extra-stats", no_argument, 0, 'x'},
{"quiet", no_argument, 0, 'Q'},
{"app-stats", no_argument, 0, 'a'},
@@ -1007,6 +1015,8 @@ static void usage(const char *prog)
" -V, --tx-vlan Send VLAN tagged packets (For -t|--txonly)\n"
" -J, --tx-vlan-id=n Tx VLAN ID [1-4095]. Default: %d (For -V|--tx-vlan)\n"
" -K, --tx-vlan-pri=n Tx VLAN Priority [0-7]. Default: %d (For -V|--tx-vlan)\n"
+ " -G, --tx-dmac=<MAC> Dest MAC addr of TX frame in aa:bb:cc:dd:ee:ff format (For -V|--tx-vlan)\n"
+ " -H, --tx-smac=<MAC> Src MAC addr of TX frame in aa:bb:cc:dd:ee:ff format (For -V|--tx-vlan)\n"
" -x, --extra-stats Display extra statistics.\n"
" -Q, --quiet Do not display any stats.\n"
" -a, --app-stats Display application (syscall) statistics.\n"
@@ -1029,7 +1039,7 @@ static void parse_command_line(int argc, char **argv)
opterr = 0;
for (;;) {
- c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:VJ:K:xQaI:BR",
+ c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:VJ:K:G:H:xQaI:BR",
long_options, &option_index);
if (c == -1)
break;
@@ -1119,6 +1129,22 @@ static void parse_command_line(int argc, char **argv)
case 'K':
opt_pkt_vlan_pri = atoi(optarg);
break;
+ case 'G':
+ if (!ether_aton_r(optarg,
+ (struct ether_addr *)&opt_txdmac)) {
+ fprintf(stderr, "Invalid dmac address:%s\n",
+ optarg);
+ usage(basename(argv[0]));
+ }
+ break;
+ case 'H':
+ if (!ether_aton_r(optarg,
+ (struct ether_addr *)&opt_txsmac)) {
+ fprintf(stderr, "Invalid smac address:%s\n",
+ optarg);
+ usage(basename(argv[0]));
+ }
+ break;
case 'x':
opt_extra_stats = 1;
break;
--
2.25.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH bpf-next 3/4] samples/bpf: xdpsock: add period cycle time to Tx operation
2021-11-24 9:18 [PATCH bpf-next 0/4] samples/bpf: xdpsock app enhancements Ong Boon Leong
2021-11-24 9:18 ` [PATCH bpf-next 1/4] samples/bpf: xdpsock: add VLAN support for Tx-only operation Ong Boon Leong
2021-11-24 9:18 ` [PATCH bpf-next 2/4] samples/bpf: xdpsock: add Dest and Src MAC setting " Ong Boon Leong
@ 2021-11-24 9:18 ` Ong Boon Leong
2021-11-27 6:52 ` Song Liu
2021-11-27 10:41 ` Jesper Dangaard Brouer
2021-11-24 9:18 ` [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx Ong Boon Leong
3 siblings, 2 replies; 14+ messages in thread
From: Ong Boon Leong @ 2021-11-24 9:18 UTC (permalink / raw)
To: bpf, netdev
Cc: bjorn, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov,
Daniel Borkmann, David S . Miller, Jakub Kicinski,
Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh,
Ong Boon Leong
Tx cycle time is in micro-seconds unit. By combining the batch size (-b M)
and Tx cycle time (-T|--tx-cycle N), xdpsock now can transmit batch-size of
packets every N-us periodically.
For example to transmit 1 packet each 1ms cycle time for total of 2000000
packets:
$ xdpsock -i eth0 -T -N -z -T 1000 -b 1 -C 2000000
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 1000 1996872
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 1000 1997872
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 1000 1998872
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 1000 1999872
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 128 2000000
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 0.00
rx 0 0
tx 0 2000000
Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
---
samples/bpf/xdpsock_user.c | 36 +++++++++++++++++++++++++++++++-----
1 file changed, 31 insertions(+), 5 deletions(-)
diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
index 691f442bbb2..61d4063f11a 100644
--- a/samples/bpf/xdpsock_user.c
+++ b/samples/bpf/xdpsock_user.c
@@ -111,6 +111,7 @@ static u32 opt_num_xsks = 1;
static u32 prog_id;
static bool opt_busy_poll;
static bool opt_reduced_cap;
+static unsigned long opt_cycle_time;
struct vlan_ethhdr {
unsigned char h_dest[6];
@@ -173,6 +174,8 @@ struct xsk_socket_info {
struct xsk_app_stats app_stats;
struct xsk_driver_stats drv_stats;
u32 outstanding_tx;
+ unsigned long prev_tx_time;
+ unsigned long tx_cycle_time;
};
static int num_socks;
@@ -972,6 +975,7 @@ static struct option long_options[] = {
{"tx-vlan-pri", required_argument, 0, 'K'},
{"tx-dmac", required_argument, 0, 'G'},
{"tx-smac", required_argument, 0, 'H'},
+ {"tx-cycle", required_argument, 0, 'T'},
{"extra-stats", no_argument, 0, 'x'},
{"quiet", no_argument, 0, 'Q'},
{"app-stats", no_argument, 0, 'a'},
@@ -1017,6 +1021,7 @@ static void usage(const char *prog)
" -K, --tx-vlan-pri=n Tx VLAN Priority [0-7]. Default: %d (For -V|--tx-vlan)\n"
" -G, --tx-dmac=<MAC> Dest MAC addr of TX frame in aa:bb:cc:dd:ee:ff format (For -V|--tx-vlan)\n"
" -H, --tx-smac=<MAC> Src MAC addr of TX frame in aa:bb:cc:dd:ee:ff format (For -V|--tx-vlan)\n"
+ " -T, --tx-cycle=n Tx cycle time in micro-seconds (For -t|--txonly).\n"
" -x, --extra-stats Display extra statistics.\n"
" -Q, --quiet Do not display any stats.\n"
" -a, --app-stats Display application (syscall) statistics.\n"
@@ -1039,7 +1044,7 @@ static void parse_command_line(int argc, char **argv)
opterr = 0;
for (;;) {
- c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:VJ:K:G:H:xQaI:BR",
+ c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:VJ:K:G:H:T:xQaI:BR",
long_options, &option_index);
if (c == -1)
break;
@@ -1145,6 +1150,10 @@ static void parse_command_line(int argc, char **argv)
usage(basename(argv[0]));
}
break;
+ case 'T':
+ opt_cycle_time = atoi(optarg);
+ opt_cycle_time *= 1000;
+ break;
case 'x':
opt_extra_stats = 1;
break;
@@ -1350,16 +1359,25 @@ static void rx_drop_all(void)
}
}
-static void tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
+static int tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
{
u32 idx;
unsigned int i;
+ if (xsk->tx_cycle_time) {
+ unsigned long now = get_nsecs();
+
+ if ((now - xsk->prev_tx_time) < xsk->tx_cycle_time)
+ return 0;
+
+ xsk->prev_tx_time = now;
+ }
+
while (xsk_ring_prod__reserve(&xsk->tx, batch_size, &idx) <
batch_size) {
complete_tx_only(xsk, batch_size);
if (benchmark_done)
- return;
+ return 0;
}
for (i = 0; i < batch_size; i++) {
@@ -1375,6 +1393,8 @@ static void tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
*frame_nb += batch_size;
*frame_nb %= NUM_FRAMES;
complete_tx_only(xsk, batch_size);
+
+ return batch_size;
}
static inline int get_batch_size(int pkt_cnt)
@@ -1407,6 +1427,7 @@ static void complete_tx_only_all(void)
static void tx_only_all(void)
{
struct pollfd fds[MAX_SOCKS] = {};
+ unsigned long now = get_nsecs();
u32 frame_nb[MAX_SOCKS] = {};
int pkt_cnt = 0;
int i, ret;
@@ -1414,10 +1435,15 @@ static void tx_only_all(void)
for (i = 0; i < num_socks; i++) {
fds[0].fd = xsk_socket__fd(xsks[i]->xsk);
fds[0].events = POLLOUT;
+ if (opt_cycle_time) {
+ xsks[i]->prev_tx_time = now;
+ xsks[i]->tx_cycle_time = opt_cycle_time;
+ }
}
while ((opt_pkt_count && pkt_cnt < opt_pkt_count) || !opt_pkt_count) {
int batch_size = get_batch_size(pkt_cnt);
+ int tx_cnt = 0;
if (opt_poll) {
for (i = 0; i < num_socks; i++)
@@ -1431,9 +1457,9 @@ static void tx_only_all(void)
}
for (i = 0; i < num_socks; i++)
- tx_only(xsks[i], &frame_nb[i], batch_size);
+ tx_cnt += tx_only(xsks[i], &frame_nb[i], batch_size);
- pkt_cnt += batch_size;
+ pkt_cnt += tx_cnt;
if (benchmark_done)
break;
--
2.25.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx
2021-11-24 9:18 [PATCH bpf-next 0/4] samples/bpf: xdpsock app enhancements Ong Boon Leong
` (2 preceding siblings ...)
2021-11-24 9:18 ` [PATCH bpf-next 3/4] samples/bpf: xdpsock: add period cycle time to Tx operation Ong Boon Leong
@ 2021-11-24 9:18 ` Ong Boon Leong
2021-11-27 6:45 ` Song Liu
2021-11-27 9:48 ` Jesper Dangaard Brouer
3 siblings, 2 replies; 14+ messages in thread
From: Ong Boon Leong @ 2021-11-24 9:18 UTC (permalink / raw)
To: bpf, netdev
Cc: bjorn, Magnus Karlsson, Jonathan Lemon, Alexei Starovoitov,
Daniel Borkmann, David S . Miller, Jakub Kicinski,
Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, KP Singh,
Ong Boon Leong
When user sets tx-pkt-count and in case where there are invalid Tx frame,
the complete_tx_only_all() process polls indefinitely. So, this patch
adds a time-out mechanism into the process so that the application
can terminate automatically after it retries 3*polling interval duration.
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 136383 1000000
rx dropped 0 0
rx invalid 0 0
tx invalid 35 245
rx queue full 0 0
fill ring empty 0 1
tx ring empty 957 7011
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 0 1000000
rx dropped 0 0
rx invalid 0 0
tx invalid 0 245
rx queue full 0 0
fill ring empty 0 1
tx ring empty 1 7012
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 0 1000000
rx dropped 0 0
rx invalid 0 0
tx invalid 0 245
rx queue full 0 0
fill ring empty 0 1
tx ring empty 1 7013
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 0 1000000
rx dropped 0 0
rx invalid 0 0
tx invalid 0 245
rx queue full 0 0
fill ring empty 0 1
tx ring empty 1 7014
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 1.00
rx 0 0
tx 0 1000000
rx dropped 0 0
rx invalid 0 0
tx invalid 0 245
rx queue full 0 0
fill ring empty 0 1
tx ring empty 0 7014
sock0@enp0s29f1:2 txonly xdp-drv
pps pkts 0.00
rx 0 0
tx 0 1000000
rx dropped 0 0
rx invalid 0 0
tx invalid 0 245
rx queue full 0 0
fill ring empty 0 1
tx ring empty 0 7014
Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
---
samples/bpf/xdpsock_user.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
index 61d4063f11a..9c3311329ec 100644
--- a/samples/bpf/xdpsock_user.c
+++ b/samples/bpf/xdpsock_user.c
@@ -1410,6 +1410,7 @@ static inline int get_batch_size(int pkt_cnt)
static void complete_tx_only_all(void)
{
+ u32 retries = 3;
bool pending;
int i;
@@ -1421,7 +1422,8 @@ static void complete_tx_only_all(void)
pending = !!xsks[i]->outstanding_tx;
}
}
- } while (pending);
+ sleep(opt_interval);
+ } while (pending && retries-- > 0);
}
static void tx_only_all(void)
--
2.25.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next 2/4] samples/bpf: xdpsock: add Dest and Src MAC setting for Tx-only operation
2021-11-24 9:18 ` [PATCH bpf-next 2/4] samples/bpf: xdpsock: add Dest and Src MAC setting " Ong Boon Leong
@ 2021-11-27 6:40 ` Song Liu
2021-11-27 9:51 ` Jesper Dangaard Brouer
1 sibling, 0 replies; 14+ messages in thread
From: Song Liu @ 2021-11-27 6:40 UTC (permalink / raw)
To: Ong Boon Leong
Cc: bpf, Networking, bjorn, Magnus Karlsson, Jonathan Lemon,
Alexei Starovoitov, Daniel Borkmann, David S . Miller,
Jakub Kicinski, Jesper Dangaard Brouer, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh
On Wed, Nov 24, 2021 at 1:21 AM Ong Boon Leong <boon.leong.ong@intel.com> wrote:
>
> To set Dest MAC address (-G|--tx-dmac) only:
> $ xdpsock -i eth0 -t -N -z -G aa:bb:cc:dd:ee:ff
>
> To set Source MAC address (-H|--tx-smac) only:
> $ xdpsock -i eth0 -t -N -z -H 11:22:33:44:55:66
>
> To set both Dest and Source MAC address:
> $ xdpsock -i eth0 -t -N -z -G aa:bb:cc:dd:ee:ff \
> -H 11:22:33:44:55:66
>
> The default Dest and Source MAC address remain the same as before.
>
> Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
Acked-by: Song Liu <songliubraving@fb.com>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx
2021-11-24 9:18 ` [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx Ong Boon Leong
@ 2021-11-27 6:45 ` Song Liu
2021-11-29 1:14 ` Ong, Boon Leong
2021-11-27 9:48 ` Jesper Dangaard Brouer
1 sibling, 1 reply; 14+ messages in thread
From: Song Liu @ 2021-11-27 6:45 UTC (permalink / raw)
To: Ong Boon Leong
Cc: bpf, Networking, bjorn, Magnus Karlsson, Jonathan Lemon,
Alexei Starovoitov, Daniel Borkmann, David S . Miller,
Jakub Kicinski, Jesper Dangaard Brouer, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh
On Wed, Nov 24, 2021 at 1:22 AM Ong Boon Leong <boon.leong.ong@intel.com> wrote:
>
> When user sets tx-pkt-count and in case where there are invalid Tx frame,
> the complete_tx_only_all() process polls indefinitely. So, this patch
> adds a time-out mechanism into the process so that the application
> can terminate automatically after it retries 3*polling interval duration.
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 136383 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 35 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 957 7011
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 1 7012
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 1 7013
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 1 7014
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 0 7014
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 0.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 0 7014
I am not following why we need examples above in the commit log.
>
> Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
> ---
> samples/bpf/xdpsock_user.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
> index 61d4063f11a..9c3311329ec 100644
> --- a/samples/bpf/xdpsock_user.c
> +++ b/samples/bpf/xdpsock_user.c
> @@ -1410,6 +1410,7 @@ static inline int get_batch_size(int pkt_cnt)
>
> static void complete_tx_only_all(void)
> {
> + u32 retries = 3;
Shall we make the retry value configurable? And maybe make it a timeout
value in seconds?
> bool pending;
> int i;
>
> @@ -1421,7 +1422,8 @@ static void complete_tx_only_all(void)
> pending = !!xsks[i]->outstanding_tx;
> }
> }
> - } while (pending);
> + sleep(opt_interval);
> + } while (pending && retries-- > 0);
> }
>
> static void tx_only_all(void)
> --
> 2.25.1
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next 1/4] samples/bpf: xdpsock: add VLAN support for Tx-only operation
2021-11-24 9:18 ` [PATCH bpf-next 1/4] samples/bpf: xdpsock: add VLAN support for Tx-only operation Ong Boon Leong
@ 2021-11-27 6:48 ` Song Liu
0 siblings, 0 replies; 14+ messages in thread
From: Song Liu @ 2021-11-27 6:48 UTC (permalink / raw)
To: Ong Boon Leong
Cc: bpf, Networking, bjorn, Magnus Karlsson, Jonathan Lemon,
Alexei Starovoitov, Daniel Borkmann, David S . Miller,
Jakub Kicinski, Jesper Dangaard Brouer, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh
On Wed, Nov 24, 2021 at 1:21 AM Ong Boon Leong <boon.leong.ong@intel.com> wrote:
>
> In multi-queue environment testing, the support for VLAN-tag based
> steering is useful. So, this patch adds the capability to add
> VLAN tag (VLAN ID and Priority) to the generated Tx frame.
>
> To set the VLAN ID=10 and Priority=2 for Tx only through TxQ=3:
> $ xdpsock -i eth0 -t -N -z -q 3 -V -J 10 -K 2
>
> If VLAN ID (-J) and Priority (-K) is set, it default to
> VLAN ID = 1
> VLAN Priority = 0.
>
> For example, VLAN-tagged Tx only, xdp copy mode through TxQ=1:
> $ xdpsock -i eth0 -t -N -c -q 1 -V
>
> Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
Acked-by: Song Liu <songliubraving@fb.com>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next 3/4] samples/bpf: xdpsock: add period cycle time to Tx operation
2021-11-24 9:18 ` [PATCH bpf-next 3/4] samples/bpf: xdpsock: add period cycle time to Tx operation Ong Boon Leong
@ 2021-11-27 6:52 ` Song Liu
2021-11-27 10:41 ` Jesper Dangaard Brouer
1 sibling, 0 replies; 14+ messages in thread
From: Song Liu @ 2021-11-27 6:52 UTC (permalink / raw)
To: Ong Boon Leong
Cc: bpf, Networking, bjorn, Magnus Karlsson, Jonathan Lemon,
Alexei Starovoitov, Daniel Borkmann, David S . Miller,
Jakub Kicinski, Jesper Dangaard Brouer, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh
On Wed, Nov 24, 2021 at 1:22 AM Ong Boon Leong <boon.leong.ong@intel.com> wrote:
>
> Tx cycle time is in micro-seconds unit. By combining the batch size (-b M)
> and Tx cycle time (-T|--tx-cycle N), xdpsock now can transmit batch-size of
> packets every N-us periodically.
>
> For example to transmit 1 packet each 1ms cycle time for total of 2000000
> packets:
>
> $ xdpsock -i eth0 -T -N -z -T 1000 -b 1 -C 2000000
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 1000 1996872
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 1000 1997872
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 1000 1998872
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 1000 1999872
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 128 2000000
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 0.00
> rx 0 0
> tx 0 2000000
>
> Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
Acked-by: Song Liu <songliubraving@fb.com>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx
2021-11-24 9:18 ` [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx Ong Boon Leong
2021-11-27 6:45 ` Song Liu
@ 2021-11-27 9:48 ` Jesper Dangaard Brouer
2021-11-29 1:17 ` Ong, Boon Leong
1 sibling, 1 reply; 14+ messages in thread
From: Jesper Dangaard Brouer @ 2021-11-27 9:48 UTC (permalink / raw)
To: Ong Boon Leong, bpf, netdev
Cc: brouer, bjorn, Magnus Karlsson, Jonathan Lemon,
Alexei Starovoitov, Daniel Borkmann, David S . Miller,
Jakub Kicinski, Jesper Dangaard Brouer, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh
On 24/11/2021 10.18, Ong Boon Leong wrote:
> When user sets tx-pkt-count and in case where there are invalid Tx frame,
> the complete_tx_only_all() process polls indefinitely. So, this patch
> adds a time-out mechanism into the process so that the application
> can terminate automatically after it retries 3*polling interval duration.
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 136383 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 35 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 957 7011
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 1 7012
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 1 7013
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 1 7014
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 0 7014
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 0.00
> rx 0 0
> tx 0 1000000
> rx dropped 0 0
> rx invalid 0 0
> tx invalid 0 245
> rx queue full 0 0
> fill ring empty 0 1
> tx ring empty 0 7014
>
> Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
> ---
> samples/bpf/xdpsock_user.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
> index 61d4063f11a..9c3311329ec 100644
> --- a/samples/bpf/xdpsock_user.c
> +++ b/samples/bpf/xdpsock_user.c
> @@ -1410,6 +1410,7 @@ static inline int get_batch_size(int pkt_cnt)
>
> static void complete_tx_only_all(void)
> {
> + u32 retries = 3;
> bool pending;
> int i;
>
> @@ -1421,7 +1422,8 @@ static void complete_tx_only_all(void)
> pending = !!xsks[i]->outstanding_tx;
> }
> }
> - } while (pending);
> + sleep(opt_interval);
Why/how is this connected with the 'opt_interval' ?
(Which is used by the pthtread 'poller' dumping stats)
> + } while (pending && retries-- > 0);
> }
>
> static void tx_only_all(void)
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next 2/4] samples/bpf: xdpsock: add Dest and Src MAC setting for Tx-only operation
2021-11-24 9:18 ` [PATCH bpf-next 2/4] samples/bpf: xdpsock: add Dest and Src MAC setting " Ong Boon Leong
2021-11-27 6:40 ` Song Liu
@ 2021-11-27 9:51 ` Jesper Dangaard Brouer
1 sibling, 0 replies; 14+ messages in thread
From: Jesper Dangaard Brouer @ 2021-11-27 9:51 UTC (permalink / raw)
To: Ong Boon Leong, bpf, netdev
Cc: brouer, bjorn, Magnus Karlsson, Alexei Starovoitov,
Daniel Borkmann, David S . Miller, Jakub Kicinski,
Jesper Dangaard Brouer, John Fastabend, Andrii Nakryiko,
Song Liu, Yonghong Song
On 24/11/2021 10.18, Ong Boon Leong wrote:
> To set Dest MAC address (-G|--tx-dmac) only:
> $ xdpsock -i eth0 -t -N -z -G aa:bb:cc:dd:ee:ff
>
> To set Source MAC address (-H|--tx-smac) only:
> $ xdpsock -i eth0 -t -N -z -H 11:22:33:44:55:66
>
> To set both Dest and Source MAC address:
> $ xdpsock -i eth0 -t -N -z -G aa:bb:cc:dd:ee:ff \
> -H 11:22:33:44:55:66
>
> The default Dest and Source MAC address remain the same as before.
>
> Signed-off-by: Ong Boon Leong<boon.leong.ong@intel.com>
> ---
> samples/bpf/xdpsock_user.c | 36 +++++++++++++++++++++++++++++++-----
> 1 file changed, 31 insertions(+), 5 deletions(-)
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH bpf-next 3/4] samples/bpf: xdpsock: add period cycle time to Tx operation
2021-11-24 9:18 ` [PATCH bpf-next 3/4] samples/bpf: xdpsock: add period cycle time to Tx operation Ong Boon Leong
2021-11-27 6:52 ` Song Liu
@ 2021-11-27 10:41 ` Jesper Dangaard Brouer
1 sibling, 0 replies; 14+ messages in thread
From: Jesper Dangaard Brouer @ 2021-11-27 10:41 UTC (permalink / raw)
To: Ong Boon Leong, bpf, netdev
Cc: brouer, bjorn, Magnus Karlsson, Jonathan Lemon,
Alexei Starovoitov, Daniel Borkmann, David S . Miller,
Jakub Kicinski, Jesper Dangaard Brouer, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh
On 24/11/2021 10.18, Ong Boon Leong wrote:
> Tx cycle time is in micro-seconds unit. By combining the batch size (-b M)
> and Tx cycle time (-T|--tx-cycle N), xdpsock now can transmit batch-size of
> packets every N-us periodically.
Does this also work for --poll mode (which is a wakeup mode) ?
> For example to transmit 1 packet each 1ms cycle time for total of 2000000
> packets:
>
> $ xdpsock -i eth0 -T -N -z -T 1000 -b 1 -C 2000000
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 1000 1996872
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 1000 1997872
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 1000 1998872
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 1000 1999872
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 1.00
> rx 0 0
> tx 128 2000000
>
> sock0@enp0s29f1:2 txonly xdp-drv
> pps pkts 0.00
> rx 0 0
> tx 0 2000000
>
> Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
> ---
> samples/bpf/xdpsock_user.c | 36 +++++++++++++++++++++++++++++++-----
> 1 file changed, 31 insertions(+), 5 deletions(-)
>
> diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
> index 691f442bbb2..61d4063f11a 100644
> --- a/samples/bpf/xdpsock_user.c
> +++ b/samples/bpf/xdpsock_user.c
> @@ -111,6 +111,7 @@ static u32 opt_num_xsks = 1;
> static u32 prog_id;
> static bool opt_busy_poll;
> static bool opt_reduced_cap;
> +static unsigned long opt_cycle_time;
>
> struct vlan_ethhdr {
> unsigned char h_dest[6];
> @@ -173,6 +174,8 @@ struct xsk_socket_info {
> struct xsk_app_stats app_stats;
> struct xsk_driver_stats drv_stats;
> u32 outstanding_tx;
> + unsigned long prev_tx_time;
> + unsigned long tx_cycle_time;
> };
>
> static int num_socks;
> @@ -972,6 +975,7 @@ static struct option long_options[] = {
> {"tx-vlan-pri", required_argument, 0, 'K'},
> {"tx-dmac", required_argument, 0, 'G'},
> {"tx-smac", required_argument, 0, 'H'},
> + {"tx-cycle", required_argument, 0, 'T'},
> {"extra-stats", no_argument, 0, 'x'},
> {"quiet", no_argument, 0, 'Q'},
> {"app-stats", no_argument, 0, 'a'},
> @@ -1017,6 +1021,7 @@ static void usage(const char *prog)
> " -K, --tx-vlan-pri=n Tx VLAN Priority [0-7]. Default: %d (For -V|--tx-vlan)\n"
> " -G, --tx-dmac=<MAC> Dest MAC addr of TX frame in aa:bb:cc:dd:ee:ff format (For -V|--tx-vlan)\n"
> " -H, --tx-smac=<MAC> Src MAC addr of TX frame in aa:bb:cc:dd:ee:ff format (For -V|--tx-vlan)\n"
> + " -T, --tx-cycle=n Tx cycle time in micro-seconds (For -t|--txonly).\n"
> " -x, --extra-stats Display extra statistics.\n"
> " -Q, --quiet Do not display any stats.\n"
> " -a, --app-stats Display application (syscall) statistics.\n"
> @@ -1039,7 +1044,7 @@ static void parse_command_line(int argc, char **argv)
> opterr = 0;
>
> for (;;) {
> - c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:VJ:K:G:H:xQaI:BR",
> + c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:VJ:K:G:H:T:xQaI:BR",
> long_options, &option_index);
> if (c == -1)
> break;
> @@ -1145,6 +1150,10 @@ static void parse_command_line(int argc, char **argv)
> usage(basename(argv[0]));
> }
> break;
> + case 'T':
> + opt_cycle_time = atoi(optarg);
> + opt_cycle_time *= 1000;
Converting to nanosec, right(?).
> + break;
> case 'x':
> opt_extra_stats = 1;
> break;
> @@ -1350,16 +1359,25 @@ static void rx_drop_all(void)
> }
> }
>
> -static void tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
> +static int tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
> {
> u32 idx;
> unsigned int i;
>
> + if (xsk->tx_cycle_time) {
> + unsigned long now = get_nsecs();
> +
> + if ((now - xsk->prev_tx_time) < xsk->tx_cycle_time)
> + return 0;
So, this test is actively spinning until the time is reached, spending
100% CPU time on this. I guess we can have this as a test for most
accurate transmit (cyclic period) with AF_XDP.
Do you have a use-case for this?
I have a customer use-case, but my customer don't want to actively spin.
My plan is to use clock_nanosleep() and wakeup slightly before the
target time and then we can spin shortly for the Tx time slot.
I will need to code this up for the customer soon anyway... perhaps we
can extend your code with this idea?
I have coded the period cycle Tx with UDP packets, here[1], if you like
to see some code using clock_nanosleep(). Next step (for me) is doing
this for AF_XDP (likely in my example[2].
[1]
https://github.com/netoptimizer/network-testing/blob/master/src/udp_pacer.c
[2]
https://github.com/xdp-project/bpf-examples/tree/master/AF_XDP-interaction
> +
> + xsk->prev_tx_time = now;
Would it be valuable to know how-much we shoot "over" the tx_cycle_time?
For my use-case, I will be monitoring the other-side receiving the
packets (and using HW RX-time) to evaluate how accurate my sender is. In
this case, I would like to know if my software "knew" if was not 100%
accurate.
> + }
> +
> while (xsk_ring_prod__reserve(&xsk->tx, batch_size, &idx) <
> batch_size) {
> complete_tx_only(xsk, batch_size);
> if (benchmark_done)
> - return;
> + return 0;
> }
I wonder if this step can introduce jitter/delay before the actual Tx
happens?
I mean, the real transmit cannot happen before xsk_ring_prod__submit()
is called. If the cycles spend are exactly the same, it doesn't matter
if you tx_cycle_time timestamp is done above.
Here you have a potential call to complete_tx_only(), which can
introduce variance for your period.
I will suggest moving the TX completion handling, so it doesn't
interfere with accurate TX.
>
> for (i = 0; i < batch_size; i++) {
> @@ -1375,6 +1393,8 @@ static void tx_only(struct xsk_socket_info *xsk, u32 *frame_nb, int batch_size)
> *frame_nb += batch_size;
> *frame_nb %= NUM_FRAMES;
> complete_tx_only(xsk, batch_size);
> +
> + return batch_size;
> }
>
> static inline int get_batch_size(int pkt_cnt)
> @@ -1407,6 +1427,7 @@ static void complete_tx_only_all(void)
> static void tx_only_all(void)
> {
> struct pollfd fds[MAX_SOCKS] = {};
> + unsigned long now = get_nsecs();
> u32 frame_nb[MAX_SOCKS] = {};
> int pkt_cnt = 0;
> int i, ret;
> @@ -1414,10 +1435,15 @@ static void tx_only_all(void)
> for (i = 0; i < num_socks; i++) {
> fds[0].fd = xsk_socket__fd(xsks[i]->xsk);
> fds[0].events = POLLOUT;
> + if (opt_cycle_time) {
> + xsks[i]->prev_tx_time = now;
> + xsks[i]->tx_cycle_time = opt_cycle_time;
> + }
> }
>
> while ((opt_pkt_count && pkt_cnt < opt_pkt_count) || !opt_pkt_count) {
> int batch_size = get_batch_size(pkt_cnt);
> + int tx_cnt = 0;
>
> if (opt_poll) {
> for (i = 0; i < num_socks; i++)
> @@ -1431,9 +1457,9 @@ static void tx_only_all(void)
> }
>
> for (i = 0; i < num_socks; i++)
> - tx_only(xsks[i], &frame_nb[i], batch_size);
> + tx_cnt += tx_only(xsks[i], &frame_nb[i], batch_size);
>
> - pkt_cnt += batch_size;
> + pkt_cnt += tx_cnt;
>
> if (benchmark_done)
> break;
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx
2021-11-27 6:45 ` Song Liu
@ 2021-11-29 1:14 ` Ong, Boon Leong
0 siblings, 0 replies; 14+ messages in thread
From: Ong, Boon Leong @ 2021-11-29 1:14 UTC (permalink / raw)
To: Song Liu
Cc: bpf, Networking, bjorn, Karlsson, Magnus, Jonathan Lemon,
Alexei Starovoitov, Daniel Borkmann, David S . Miller,
Jakub Kicinski, Jesper Dangaard Brouer, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh
>On Wed, Nov 24, 2021 at 1:22 AM Ong Boon Leong
><boon.leong.ong@intel.com> wrote:
>>
>> When user sets tx-pkt-count and in case where there are invalid Tx frame,
>> the complete_tx_only_all() process polls indefinitely. So, this patch
>> adds a time-out mechanism into the process so that the application
>> can terminate automatically after it retries 3*polling interval duration.
>>
>> sock0@enp0s29f1:2 txonly xdp-drv
>> pps pkts 1.00
>> rx 0 0
>> tx 136383 1000000
>> rx dropped 0 0
>> rx invalid 0 0
>> tx invalid 35 245
>> rx queue full 0 0
>> fill ring empty 0 1
>> tx ring empty 957 7011
>>
>> sock0@enp0s29f1:2 txonly xdp-drv
>> pps pkts 1.00
>> rx 0 0
>> tx 0 1000000
>> rx dropped 0 0
>> rx invalid 0 0
>> tx invalid 0 245
>> rx queue full 0 0
>> fill ring empty 0 1
>> tx ring empty 1 7012
>>
>> sock0@enp0s29f1:2 txonly xdp-drv
>> pps pkts 1.00
>> rx 0 0
>> tx 0 1000000
>> rx dropped 0 0
>> rx invalid 0 0
>> tx invalid 0 245
>> rx queue full 0 0
>> fill ring empty 0 1
>> tx ring empty 1 7013
>>
>> sock0@enp0s29f1:2 txonly xdp-drv
>> pps pkts 1.00
>> rx 0 0
>> tx 0 1000000
>> rx dropped 0 0
>> rx invalid 0 0
>> tx invalid 0 245
>> rx queue full 0 0
>> fill ring empty 0 1
>> tx ring empty 1 7014
>>
>> sock0@enp0s29f1:2 txonly xdp-drv
>> pps pkts 1.00
>> rx 0 0
>> tx 0 1000000
>> rx dropped 0 0
>> rx invalid 0 0
>> tx invalid 0 245
>> rx queue full 0 0
>> fill ring empty 0 1
>> tx ring empty 0 7014
>>
>> sock0@enp0s29f1:2 txonly xdp-drv
>> pps pkts 0.00
>> rx 0 0
>> tx 0 1000000
>> rx dropped 0 0
>> rx invalid 0 0
>> tx invalid 0 245
>> rx queue full 0 0
>> fill ring empty 0 1
>> tx ring empty 0 7014
>
>I am not following why we need examples above in the commit log.
I pasted the log to demonstrate the behavior of the time-out, that is
aligned with the stats poller. Will remove in next rev.
>
>>
>> Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
>> ---
>> samples/bpf/xdpsock_user.c | 4 +++-
>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
>> index 61d4063f11a..9c3311329ec 100644
>> --- a/samples/bpf/xdpsock_user.c
>> +++ b/samples/bpf/xdpsock_user.c
>> @@ -1410,6 +1410,7 @@ static inline int get_batch_size(int pkt_cnt)
>>
>> static void complete_tx_only_all(void)
>> {
>> + u32 retries = 3;
>
>Shall we make the retry value configurable? And maybe make it a timeout
>value in seconds?
OK, next rev will have seconds granularity. Ok, I can make it configurable.
^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx
2021-11-27 9:48 ` Jesper Dangaard Brouer
@ 2021-11-29 1:17 ` Ong, Boon Leong
0 siblings, 0 replies; 14+ messages in thread
From: Ong, Boon Leong @ 2021-11-29 1:17 UTC (permalink / raw)
To: Jesper Dangaard Brouer, bpf, netdev
Cc: brouer, bjorn, Karlsson, Magnus, Jonathan Lemon,
Alexei Starovoitov, Daniel Borkmann, David S . Miller,
Jakub Kicinski, Jesper Dangaard Brouer, John Fastabend,
Andrii Nakryiko, Martin KaFai Lau, Song Liu, Yonghong Song,
KP Singh
>> static void complete_tx_only_all(void)
>> {
>> + u32 retries = 3;
>> bool pending;
>> int i;
>>
>> @@ -1421,7 +1422,8 @@ static void complete_tx_only_all(void)
>> pending = !!xsks[i]->outstanding_tx;
>> }
>> }
>> - } while (pending);
>> + sleep(opt_interval);
>
>Why/how is this connected with the 'opt_interval' ?
>
>(Which is used by the pthtread 'poller' dumping stats)
>
The original thought was to use the poller interval since
it is what user would experience from terminal. In next rev,
I plan to make it configurable with second granularity as
suggested by Song Liu.
Thanks
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2021-11-29 1:19 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-24 9:18 [PATCH bpf-next 0/4] samples/bpf: xdpsock app enhancements Ong Boon Leong
2021-11-24 9:18 ` [PATCH bpf-next 1/4] samples/bpf: xdpsock: add VLAN support for Tx-only operation Ong Boon Leong
2021-11-27 6:48 ` Song Liu
2021-11-24 9:18 ` [PATCH bpf-next 2/4] samples/bpf: xdpsock: add Dest and Src MAC setting " Ong Boon Leong
2021-11-27 6:40 ` Song Liu
2021-11-27 9:51 ` Jesper Dangaard Brouer
2021-11-24 9:18 ` [PATCH bpf-next 3/4] samples/bpf: xdpsock: add period cycle time to Tx operation Ong Boon Leong
2021-11-27 6:52 ` Song Liu
2021-11-27 10:41 ` Jesper Dangaard Brouer
2021-11-24 9:18 ` [PATCH bpf-next 4/4] samples/bpf: xdpsock: add time-out for cleaning Tx Ong Boon Leong
2021-11-27 6:45 ` Song Liu
2021-11-29 1:14 ` Ong, Boon Leong
2021-11-27 9:48 ` Jesper Dangaard Brouer
2021-11-29 1:17 ` Ong, Boon Leong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).