[PATCH bpf-next 00/10] AF_XDP ZC selftests

[PATCH bpf-next 00/10] AF_XDP ZC selftests

[Maciej Fijalkowski]

Hi!

This set makes it possible to test ice driver with test suite that
xdpxceiver provides. In order to do it, device needs to be turned on to
loopback mode, so in first patch knob is introduced that allows user to
toggle loopback mode. Furthermore, ethtool's loopback test is fixed for
ice (patch 2 and 3). Besides turning device into loopback mode, I was
limiting queue count to 1, so that none of the steering filters are
needed.

Rest of the work is related to xdpxceiver. Currently it is assumed that
underlying device supports native XDP. It is true for veth, but might
not be for other device that might be used with xdpxceiver once this
patch set land. So, patch 4 adds a logic to find out if underlying
device supports XDP so that TEST_MODE_DRV can be used for test suite.
Patch 5 restores close() on netns fd that was removed by accident.

In patch 6, default Rx pkt stream is added so physical device testing
will be able to use shared UMEM in a way that Tx will utilize first half
of buffer space and Rx second one. Then, patch 7 adds support for running
xdpxceiver on physical devices.

Patch 8 and 9 prepares xdpxceiver to address ZC drivers that clean
lazily Tx descriptors (and therefore produce XSK descs to CQ) so that
pacing algorithm works fine.

Patch 10 finally adds new TEST_MODE_ZC for testing zero copy AF_XDP
driver support.

This work already allowed us to spot and fix two bugs in AF_XDP kernel
side ([0], [1]).

[0]: https://lore.kernel.org/bpf/20220425153745.481322-1-maciej.fijalkowski@intel.com/
[1]: https://lore.kernel.org/bpf/20220607142200.576735-1-maciej.fijalkowski@intel.com/

Thank you.


Maciej Fijalkowski (10):
  ice: introduce priv-flag for toggling loopback mode
  ice: check DD bit on Rx descriptor rather than (EOP | RS)
  ice: do not setup vlan for loopback VSI
  selftests: xsk: query for native XDP support
  selftests: xsk: add missing close() on netns fd
  selftests: xsk: introduce default Rx pkt stream
  selftests: xsk: add support for executing tests on physical device
  selftests: xsk: rely on pkts_in_flight in wait_for_tx_completion()
  selftests: xsk: remove struct xsk_socket_info::outstanding_tx
  selftests: xsk: add support for zero copy testing

 drivers/net/ethernet/intel/ice/ice.h         |   1 +
 drivers/net/ethernet/intel/ice/ice_ethtool.c |  19 +-
 drivers/net/ethernet/intel/ice/ice_main.c    |   8 +-
 tools/testing/selftests/bpf/test_xsk.sh      |  52 ++-
 tools/testing/selftests/bpf/xdpxceiver.c     | 404 +++++++++++++------
 tools/testing/selftests/bpf/xdpxceiver.h     |  10 +-
 6 files changed, 352 insertions(+), 142 deletions(-)

-- 
2.27.0

[PATCH bpf-next 01/10] ice: introduce priv-flag for toggling loopback mode

[Maciej Fijalkowski]

Add a knob that will allow user to turn the underlying net device into
loopback mode. The use case for this will be the AF_XDP ZC tests. Once
the device is in loopback mode, then it will be possible from AF_XDP
perspective to see if zero copy implementations in drivers work
properly.

The code for interaction with admin queue is reused from ethtool's
loopback test.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 drivers/net/ethernet/intel/ice/ice.h         |  1 +
 drivers/net/ethernet/intel/ice/ice_ethtool.c | 17 +++++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 60453b3b8d23..90c066f3782b 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -487,6 +487,7 @@ enum ice_pf_flags {
 	ICE_FLAG_PLUG_AUX_DEV,
 	ICE_FLAG_MTU_CHANGED,
 	ICE_FLAG_GNSS,			/* GNSS successfully initialized */
+	ICE_FLAG_LOOPBACK,
 	ICE_PF_FLAGS_NBITS		/* must be last */
 };
 
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index 1e71b70f0e52..cfc3c5e36907 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -166,6 +166,7 @@ static const struct ice_priv_flag ice_gstrings_priv_flags[] = {
 	ICE_PRIV_FLAG("mdd-auto-reset-vf", ICE_FLAG_MDD_AUTO_RESET_VF),
 	ICE_PRIV_FLAG("vf-vlan-pruning", ICE_FLAG_VF_VLAN_PRUNING),
 	ICE_PRIV_FLAG("legacy-rx", ICE_FLAG_LEGACY_RX),
+	ICE_PRIV_FLAG("loopback", ICE_FLAG_LOOPBACK),
 };
 
 #define ICE_PRIV_FLAG_ARRAY_SIZE	ARRAY_SIZE(ice_gstrings_priv_flags)
@@ -1288,6 +1289,22 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)
 			ice_up(vsi);
 		}
 	}
+
+	if (test_bit(ICE_FLAG_LOOPBACK, change_flags)) {
+		if (!test_bit(ICE_FLAG_LOOPBACK, orig_flags)) {
+			/* Enable MAC loopback in firmware */
+			if (ice_aq_set_mac_loopback(&pf->hw, true, NULL)) {
+				dev_err(dev, "Failed to enable loopback\n");
+				ret = -ENXIO;
+			}
+		} else {
+			/* Disable MAC loopback in firmware */
+			if (ice_aq_set_mac_loopback(&pf->hw, false, NULL)) {
+				dev_err(dev, "Failed to disable loopback\n");
+				ret = -ENXIO;
+			}
+		}
+	}
 	/* don't allow modification of this flag when a single VF is in
 	 * promiscuous mode because it's not supported
 	 */
-- 
2.27.0

[PATCH bpf-next 02/10] ice: check DD bit on Rx descriptor rather than (EOP | RS)

[Maciej Fijalkowski]

Tx side sets EOP and RS bits on descriptors to indicate that a
particular descriptor is the last one and needs to generate an irq when
it was sent. These bits should not be checked on completion path
regardless whether it's the Tx or the Rx. DD bit serves this purpose and
it indicates that a particular descriptor is either for Rx or was
successfully Txed.

Look at DD bit being set in ice_lbtest_receive_frames() instead of EOP
and RS pair.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index cfc3c5e36907..3d595345cdfa 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -659,7 +659,7 @@ static int ice_lbtest_receive_frames(struct ice_rx_ring *rx_ring)
 		rx_desc = ICE_RX_DESC(rx_ring, i);
 
 		if (!(rx_desc->wb.status_error0 &
-		    cpu_to_le16(ICE_TX_DESC_CMD_EOP | ICE_TX_DESC_CMD_RS)))
+		    cpu_to_le16(BIT(ICE_RX_FLEX_DESC_STATUS0_DD_S))))
 			continue;
 
 		rx_buf = &rx_ring->rx_buf[i];
-- 
2.27.0

[PATCH bpf-next 03/10] ice: do not setup vlan for loopback VSI

[Maciej Fijalkowski]

Currently loopback test is failing due to the error returned from
ice_vsi_vlan_setup(). Skip calling it when preparing loopback VSI.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 drivers/net/ethernet/intel/ice/ice_main.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index e1cae253412c..08b44e43bcfa 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -5997,10 +5997,12 @@ int ice_vsi_cfg(struct ice_vsi *vsi)
 	if (vsi->netdev) {
 		ice_set_rx_mode(vsi->netdev);
 
-		err = ice_vsi_vlan_setup(vsi);
+		if (vsi->type != ICE_VSI_LB) {
+			err = ice_vsi_vlan_setup(vsi);
 
-		if (err)
-			return err;
+			if (err)
+				return err;
+		}
 	}
 	ice_vsi_cfg_dcb_rings(vsi);
 
-- 
2.27.0

[PATCH bpf-next 04/10] selftests: xsk: query for native XDP support

[Maciej Fijalkowski]

Currently, xdpxceiver assumes that underlying device supports XDP in
native mode - it is fine by now since tests can run only on a veth pair.
Future commit is going to allow running test suite against physical
devices, so let us query the device if it is capable of running XDP
programs in native mode. This way xdpxceiver will not try to run
TEST_MODE_DRV if device being tested is not supporting it.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 tools/testing/selftests/bpf/xdpxceiver.c | 38 ++++++++++++++++++++++--
 1 file changed, 36 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
index e5992a6b5e09..da8098f1b655 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.c
+++ b/tools/testing/selftests/bpf/xdpxceiver.c
@@ -98,6 +98,8 @@
 #include <unistd.h>
 #include <stdatomic.h>
 #include <bpf/xsk.h>
+#include <bpf/bpf.h>
+#include <linux/filter.h>
 #include "xdpxceiver.h"
 #include "../kselftest.h"
 
@@ -1605,10 +1607,39 @@ static void ifobject_delete(struct ifobject *ifobj)
 	free(ifobj);
 }
 
+static bool is_xdp_supported(struct ifobject *ifobject)
+{
+	int flags = XDP_FLAGS_DRV_MODE;
+
+	LIBBPF_OPTS(bpf_link_create_opts, opts, .flags = flags);
+	struct bpf_insn insns[2] = {
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_EXIT_INSN()
+	};
+	int ifindex = if_nametoindex(ifobject->ifname);
+	int prog_fd, insn_cnt = ARRAY_SIZE(insns);
+	bool ret = false;
+	int err;
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, NULL);
+	if (prog_fd < 0)
+		return ret;
+
+	err = bpf_xdp_attach(ifindex, prog_fd, flags, NULL);
+
+	if (!err) {
+		ret = true;
+		bpf_xdp_detach(ifindex, flags, NULL);
+	}
+
+	return ret;
+}
+
 int main(int argc, char **argv)
 {
 	struct pkt_stream *pkt_stream_default;
 	struct ifobject *ifobj_tx, *ifobj_rx;
+	int modes = TEST_MODE_SKB + 1;
 	u32 i, j, failed_tests = 0;
 	struct test_spec test;
 
@@ -1636,15 +1667,18 @@ int main(int argc, char **argv)
 	init_iface(ifobj_rx, MAC2, MAC1, IP2, IP1, UDP_PORT2, UDP_PORT1,
 		   worker_testapp_validate_rx);
 
+	if (is_xdp_supported(ifobj_tx))
+		modes++;
+
 	test_spec_init(&test, ifobj_tx, ifobj_rx, 0);
 	pkt_stream_default = pkt_stream_generate(ifobj_tx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
 	if (!pkt_stream_default)
 		exit_with_error(ENOMEM);
 	test.pkt_stream_default = pkt_stream_default;
 
-	ksft_set_plan(TEST_MODE_MAX * TEST_TYPE_MAX);
+	ksft_set_plan(modes * TEST_TYPE_MAX);
 
-	for (i = 0; i < TEST_MODE_MAX; i++)
+	for (i = 0; i < modes; i++)
 		for (j = 0; j < TEST_TYPE_MAX; j++) {
 			test_spec_init(&test, ifobj_tx, ifobj_rx, i);
 			run_pkt_test(&test, i, j);
-- 
2.27.0

[PATCH bpf-next 05/10] selftests: xsk: add missing close() on netns fd

[Maciej Fijalkowski]

Commit 1034b03e54ac ("selftests: xsk: Simplify cleanup of ifobjects") removed
close on netns fd, which is not correct, so let us restore it.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 tools/testing/selftests/bpf/xdpxceiver.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
index da8098f1b655..2499075fad82 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.c
+++ b/tools/testing/selftests/bpf/xdpxceiver.c
@@ -1591,6 +1591,8 @@ static struct ifobject *ifobject_create(void)
 	if (!ifobj->umem)
 		goto out_umem;
 
+	ifobj->ns_fd = -1;
+
 	return ifobj;
 
 out_umem:
@@ -1602,6 +1604,8 @@ static struct ifobject *ifobject_create(void)
 
 static void ifobject_delete(struct ifobject *ifobj)
 {
+	if (ifobj->ns_fd != -1)
+		close(ifobj->ns_fd);
 	free(ifobj->umem);
 	free(ifobj->xsk_arr);
 	free(ifobj);
-- 
2.27.0

[PATCH bpf-next 06/10] selftests: xsk: introduce default Rx pkt stream

[Maciej Fijalkowski]

In order to prepare xdpxceiver for physical device testing, let us
introduce default Rx pkt stream. Reason for doing it is that physical
device testing will use a UMEM with a doubled size where half of it will
be used by Tx and other half by Rx. This means that pkt addresses will
differ for Tx and Rx streams. Rx thread will initialize the
xsk_umem_info::base_addr that is added here so that pkt_set(), when
working on Rx UMEM will add this offset and second half of UMEM space
will be used. Note that currently base_addr is 0 on both sides. Future
commit will do the mentioned initialization.

Previously, veth based testing worked on separate UMEMs, so single
default stream was fine.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 tools/testing/selftests/bpf/xdpxceiver.c | 74 +++++++++++++++---------
 tools/testing/selftests/bpf/xdpxceiver.h |  4 +-
 2 files changed, 51 insertions(+), 27 deletions(-)

diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
index 2499075fad82..ad6c92c31026 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.c
+++ b/tools/testing/selftests/bpf/xdpxceiver.c
@@ -428,15 +428,16 @@ static void __test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
 		ifobj->use_poll = false;
 		ifobj->use_fill_ring = true;
 		ifobj->release_rx = true;
-		ifobj->pkt_stream = test->pkt_stream_default;
 		ifobj->validation_func = NULL;
 
 		if (i == 0) {
 			ifobj->rx_on = false;
 			ifobj->tx_on = true;
+			ifobj->pkt_stream = test->tx_pkt_stream_default;
 		} else {
 			ifobj->rx_on = true;
 			ifobj->tx_on = false;
+			ifobj->pkt_stream = test->rx_pkt_stream_default;
 		}
 
 		memset(ifobj->umem, 0, sizeof(*ifobj->umem));
@@ -460,12 +461,15 @@ static void __test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
 static void test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
 			   struct ifobject *ifobj_rx, enum test_mode mode)
 {
-	struct pkt_stream *pkt_stream;
+	struct pkt_stream *tx_pkt_stream;
+	struct pkt_stream *rx_pkt_stream;
 	u32 i;
 
-	pkt_stream = test->pkt_stream_default;
+	tx_pkt_stream = test->tx_pkt_stream_default;
+	rx_pkt_stream = test->rx_pkt_stream_default;
 	memset(test, 0, sizeof(*test));
-	test->pkt_stream_default = pkt_stream;
+	test->tx_pkt_stream_default = tx_pkt_stream;
+	test->rx_pkt_stream_default = rx_pkt_stream;
 
 	for (i = 0; i < MAX_INTERFACES; i++) {
 		struct ifobject *ifobj = i ? ifobj_rx : ifobj_tx;
@@ -526,16 +530,17 @@ static void pkt_stream_delete(struct pkt_stream *pkt_stream)
 static void pkt_stream_restore_default(struct test_spec *test)
 {
 	struct pkt_stream *tx_pkt_stream = test->ifobj_tx->pkt_stream;
+	struct pkt_stream *rx_pkt_stream = test->ifobj_rx->pkt_stream;
 
-	if (tx_pkt_stream != test->pkt_stream_default) {
+	if (tx_pkt_stream != test->tx_pkt_stream_default) {
 		pkt_stream_delete(test->ifobj_tx->pkt_stream);
-		test->ifobj_tx->pkt_stream = test->pkt_stream_default;
+		test->ifobj_tx->pkt_stream = test->tx_pkt_stream_default;
 	}
 
-	if (test->ifobj_rx->pkt_stream != test->pkt_stream_default &&
-	    test->ifobj_rx->pkt_stream != tx_pkt_stream)
+	if (rx_pkt_stream != test->rx_pkt_stream_default) {
 		pkt_stream_delete(test->ifobj_rx->pkt_stream);
-	test->ifobj_rx->pkt_stream = test->pkt_stream_default;
+		test->ifobj_rx->pkt_stream = test->rx_pkt_stream_default;
+	}
 }
 
 static struct pkt_stream *__pkt_stream_alloc(u32 nb_pkts)
@@ -558,7 +563,7 @@ static struct pkt_stream *__pkt_stream_alloc(u32 nb_pkts)
 
 static void pkt_set(struct xsk_umem_info *umem, struct pkt *pkt, u64 addr, u32 len)
 {
-	pkt->addr = addr;
+	pkt->addr = addr + umem->base_addr;
 	pkt->len = len;
 	if (len > umem->frame_size - XDP_PACKET_HEADROOM - MIN_PKT_SIZE * 2 - umem->frame_headroom)
 		pkt->valid = false;
@@ -597,22 +602,29 @@ static void pkt_stream_replace(struct test_spec *test, u32 nb_pkts, u32 pkt_len)
 
 	pkt_stream = pkt_stream_generate(test->ifobj_tx->umem, nb_pkts, pkt_len);
 	test->ifobj_tx->pkt_stream = pkt_stream;
+	pkt_stream = pkt_stream_generate(test->ifobj_rx->umem, nb_pkts, pkt_len);
 	test->ifobj_rx->pkt_stream = pkt_stream;
 }
 
-static void pkt_stream_replace_half(struct test_spec *test, u32 pkt_len, int offset)
+static void __pkt_stream_replace_half(struct ifobject *ifobj, u32 pkt_len,
+				      int offset)
 {
-	struct xsk_umem_info *umem = test->ifobj_tx->umem;
+	struct xsk_umem_info *umem = ifobj->umem;
 	struct pkt_stream *pkt_stream;
 	u32 i;
 
-	pkt_stream = pkt_stream_clone(umem, test->pkt_stream_default);
-	for (i = 1; i < test->pkt_stream_default->nb_pkts; i += 2)
+	pkt_stream = pkt_stream_clone(umem, ifobj->pkt_stream);
+	for (i = 1; i < ifobj->pkt_stream->nb_pkts; i += 2)
 		pkt_set(umem, &pkt_stream->pkts[i],
 			(i % umem->num_frames) * umem->frame_size + offset, pkt_len);
 
-	test->ifobj_tx->pkt_stream = pkt_stream;
-	test->ifobj_rx->pkt_stream = pkt_stream;
+	ifobj->pkt_stream = pkt_stream;
+}
+
+static void pkt_stream_replace_half(struct test_spec *test, u32 pkt_len, int offset)
+{
+	__pkt_stream_replace_half(test->ifobj_tx, pkt_len, offset);
+	__pkt_stream_replace_half(test->ifobj_rx, pkt_len, offset);
 }
 
 static void pkt_stream_receive_half(struct test_spec *test)
@@ -654,7 +666,8 @@ static struct pkt *pkt_generate(struct ifobject *ifobject, u32 pkt_nb)
 	return pkt;
 }
 
-static void pkt_stream_generate_custom(struct test_spec *test, struct pkt *pkts, u32 nb_pkts)
+static void __pkt_stream_generate_custom(struct ifobject *ifobj,
+					 struct pkt *pkts, u32 nb_pkts)
 {
 	struct pkt_stream *pkt_stream;
 	u32 i;
@@ -663,15 +676,20 @@ static void pkt_stream_generate_custom(struct test_spec *test, struct pkt *pkts,
 	if (!pkt_stream)
 		exit_with_error(ENOMEM);
 
-	test->ifobj_tx->pkt_stream = pkt_stream;
-	test->ifobj_rx->pkt_stream = pkt_stream;
-
 	for (i = 0; i < nb_pkts; i++) {
-		pkt_stream->pkts[i].addr = pkts[i].addr;
+		pkt_stream->pkts[i].addr = pkts[i].addr + ifobj->umem->base_addr;
 		pkt_stream->pkts[i].len = pkts[i].len;
 		pkt_stream->pkts[i].payload = i;
 		pkt_stream->pkts[i].valid = pkts[i].valid;
 	}
+
+	ifobj->pkt_stream = pkt_stream;
+}
+
+static void pkt_stream_generate_custom(struct test_spec *test, struct pkt *pkts, u32 nb_pkts)
+{
+	__pkt_stream_generate_custom(test->ifobj_tx, pkts, nb_pkts);
+	__pkt_stream_generate_custom(test->ifobj_rx, pkts, nb_pkts);
 }
 
 static void pkt_dump(void *pkt, u32 len)
@@ -1641,7 +1659,8 @@ static bool is_xdp_supported(struct ifobject *ifobject)
 
 int main(int argc, char **argv)
 {
-	struct pkt_stream *pkt_stream_default;
+	struct pkt_stream *rx_pkt_stream_default;
+	struct pkt_stream *tx_pkt_stream_default;
 	struct ifobject *ifobj_tx, *ifobj_rx;
 	int modes = TEST_MODE_SKB + 1;
 	u32 i, j, failed_tests = 0;
@@ -1675,10 +1694,12 @@ int main(int argc, char **argv)
 		modes++;
 
 	test_spec_init(&test, ifobj_tx, ifobj_rx, 0);
-	pkt_stream_default = pkt_stream_generate(ifobj_tx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
-	if (!pkt_stream_default)
+	tx_pkt_stream_default = pkt_stream_generate(ifobj_tx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
+	rx_pkt_stream_default = pkt_stream_generate(ifobj_rx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
+	if (!tx_pkt_stream_default || !rx_pkt_stream_default)
 		exit_with_error(ENOMEM);
-	test.pkt_stream_default = pkt_stream_default;
+	test.tx_pkt_stream_default = tx_pkt_stream_default;
+	test.rx_pkt_stream_default = rx_pkt_stream_default;
 
 	ksft_set_plan(modes * TEST_TYPE_MAX);
 
@@ -1692,7 +1713,8 @@ int main(int argc, char **argv)
 				failed_tests++;
 		}
 
-	pkt_stream_delete(pkt_stream_default);
+	pkt_stream_delete(tx_pkt_stream_default);
+	pkt_stream_delete(rx_pkt_stream_default);
 	ifobject_delete(ifobj_tx);
 	ifobject_delete(ifobj_rx);
 
diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
index 8f672b0fe0e1..ccfc829b2e5e 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.h
+++ b/tools/testing/selftests/bpf/xdpxceiver.h
@@ -95,6 +95,7 @@ struct xsk_umem_info {
 	u32 frame_headroom;
 	void *buffer;
 	u32 frame_size;
+	u32 base_addr;
 	bool unaligned_mode;
 };
 
@@ -155,7 +156,8 @@ struct ifobject {
 struct test_spec {
 	struct ifobject *ifobj_tx;
 	struct ifobject *ifobj_rx;
-	struct pkt_stream *pkt_stream_default;
+	struct pkt_stream *tx_pkt_stream_default;
+	struct pkt_stream *rx_pkt_stream_default;
 	u16 total_steps;
 	u16 current_step;
 	u16 nb_sockets;
-- 
2.27.0

[PATCH bpf-next 07/10] selftests: xsk: add support for executing tests on physical device

[Maciej Fijalkowski]

Currently, architecture of xdpxceiver is designed strictly for
conducting veth based tests. Veth pair is created together with a
network namespace and one of the veth interfaces is moved to the
mentioned netns. Then, separate threads for Tx and Rx are spawned which
will utilize described setup.

Infrastructure described in the paragraph above can not be used for
testing AF_XDP support on physical devices. That testing will be
conducted on a single network interface and same queue. Xdpxceiver
needs to be extended to distinguish between veth tests and physical
interface tests.

Since same iface/queue id pair will be used by both Tx/Rx threads for
physical device testing, Tx thread, which happen to run after the Rx
thread, is going to create XSK socket with shared umem flag. In order to
track this setting throughout the lifetime of spawned threads, introduce
'shared_umem' boolean variable to struct ifobject and set it to true
when xdpxceiver is run against physical device. In such case, UMEM size
needs to be doubled, so half of it will be used by Rx thread and other
half by Tx thread. For two step based test types, value of XSKMAP
element under key 0 has to be updated as there is now another socket for
the second step. Also, to avoid race conditions when destroying XSK
resources, move this activity to the main thread after spawned Rx and Tx
threads have finished its job. This way it is possible to gracefully
remove shared umem without introducing synchronization mechanisms.

To run xsk selftests suite on physical device, append "-i $IFACE" when
invoking test_xsk.sh. For veth based tests, simply skip it. When "-i
$IFACE" is in place, under the hood test_xsk.sh will use $IFACE for both
interfaces supplied to xdpxceiver, which in turn will interpret that
this execution of test suite is for a physical device.

Note that currently this makes it possible only to test SKB and DRV mode
(in case underlying device has native XDP support). ZC testing support
is added in a later patch.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 tools/testing/selftests/bpf/test_xsk.sh  |  52 +++++--
 tools/testing/selftests/bpf/xdpxceiver.c | 189 ++++++++++++++---------
 tools/testing/selftests/bpf/xdpxceiver.h |   1 +
 3 files changed, 156 insertions(+), 86 deletions(-)

diff --git a/tools/testing/selftests/bpf/test_xsk.sh b/tools/testing/selftests/bpf/test_xsk.sh
index 567500299231..19b24cce5414 100755
--- a/tools/testing/selftests/bpf/test_xsk.sh
+++ b/tools/testing/selftests/bpf/test_xsk.sh
@@ -73,14 +73,20 @@
 #
 # Run and dump packet contents:
 #   sudo ./test_xsk.sh -D
+#
+# Run test suite for physical device in loopback mode
+#   sudo ./test_xsk.sh -i IFACE
 
 . xsk_prereqs.sh
 
-while getopts "vD" flag
+ETH=""
+
+while getopts "vDi:" flag
 do
 	case "${flag}" in
 		v) verbose=1;;
 		D) dump_pkts=1;;
+		i) ETH=${OPTARG};;
 	esac
 done
 
@@ -132,18 +138,25 @@ setup_vethPairs() {
 	ip link set ${VETH0} up
 }
 
-validate_root_exec
-validate_veth_support ${VETH0}
-validate_ip_utility
-setup_vethPairs
-
-retval=$?
-if [ $retval -ne 0 ]; then
-	test_status $retval "${TEST_NAME}"
-	cleanup_exit ${VETH0} ${VETH1} ${NS1}
-	exit $retval
+if [ ! -z $ETH ]; then
+	VETH0=${ETH}
+	VETH1=${ETH}
+	NS1=""
+else
+	validate_root_exec
+	validate_veth_support ${VETH0}
+	validate_ip_utility
+	setup_vethPairs
+
+	retval=$?
+	if [ $retval -ne 0 ]; then
+		test_status $retval "${TEST_NAME}"
+		cleanup_exit ${VETH0} ${VETH1} ${NS1}
+		exit $retval
+	fi
 fi
 
+
 if [[ $verbose -eq 1 ]]; then
 	ARGS+="-v "
 fi
@@ -152,26 +165,33 @@ if [[ $dump_pkts -eq 1 ]]; then
 	ARGS="-D "
 fi
 
+retval=$?
 test_status $retval "${TEST_NAME}"
 
 ## START TESTS
 
 statusList=()
 
-TEST_NAME="XSK_SELFTESTS_SOFTIRQ"
+TEST_NAME="XSK_SELFTESTS_${VETH0}_SOFTIRQ"
 
 execxdpxceiver
 
-cleanup_exit ${VETH0} ${VETH1} ${NS1}
-TEST_NAME="XSK_SELFTESTS_BUSY_POLL"
+if [ -z $ETH ]; then
+	cleanup_exit ${VETH0} ${VETH1} ${NS1}
+fi
+TEST_NAME="XSK_SELFTESTS_${VETH0}_BUSY_POLL"
 busy_poll=1
 
-setup_vethPairs
+if [ -z $ETH ]; then
+	setup_vethPairs
+fi
 execxdpxceiver
 
 ## END TESTS
 
-cleanup_exit ${VETH0} ${VETH1} ${NS1}
+if [ -z $ETH ]; then
+	cleanup_exit ${VETH0} ${VETH1} ${NS1}
+fi
 
 failures=0
 echo -e "\nSummary:"
diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
index ad6c92c31026..316f1dd338fc 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.c
+++ b/tools/testing/selftests/bpf/xdpxceiver.c
@@ -296,8 +296,8 @@ static void enable_busy_poll(struct xsk_socket_info *xsk)
 		exit_with_error(errno);
 }
 
-static int xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_info *umem,
-				struct ifobject *ifobject, bool shared)
+static int __xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_info *umem,
+				  struct ifobject *ifobject, bool shared)
 {
 	struct xsk_socket_config cfg = {};
 	struct xsk_ring_cons *rxr;
@@ -443,6 +443,9 @@ static void __test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
 		memset(ifobj->umem, 0, sizeof(*ifobj->umem));
 		ifobj->umem->num_frames = DEFAULT_UMEM_BUFFERS;
 		ifobj->umem->frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE;
+		if (ifobj->shared_umem && ifobj->rx_on)
+			ifobj->umem->base_addr = DEFAULT_UMEM_BUFFERS *
+				XSK_UMEM__DEFAULT_FRAME_SIZE;
 
 		for (j = 0; j < MAX_SOCKETS; j++) {
 			memset(&ifobj->xsk_arr[j], 0, sizeof(ifobj->xsk_arr[j]));
@@ -1101,19 +1104,85 @@ static int validate_tx_invalid_descs(struct ifobject *ifobject)
 	return TEST_PASS;
 }
 
+static void xsk_configure_socket(struct test_spec *test, struct ifobject *ifobject,
+				 struct xsk_umem_info *umem, bool tx)
+{
+	int i, ret;
+
+	for (i = 0; i < test->nb_sockets; i++) {
+		bool shared = (ifobject->shared_umem && tx) ? true : !!i;
+		u32 ctr = 0;
+
+		while (ctr++ < SOCK_RECONF_CTR) {
+			ret = __xsk_configure_socket(&ifobject->xsk_arr[i], umem,
+						     ifobject, shared);
+			if (!ret)
+				break;
+
+			/* Retry if it fails as xsk_socket__create() is asynchronous */
+			if (ctr >= SOCK_RECONF_CTR)
+				exit_with_error(-ret);
+			usleep(USLEEP_MAX);
+		}
+		if (ifobject->busy_poll)
+			enable_busy_poll(&ifobject->xsk_arr[i]);
+	}
+}
+
+static void thread_common_ops_tx(struct test_spec *test, struct ifobject *ifobject)
+{
+	xsk_configure_socket(test, ifobject, test->ifobj_rx->umem, true);
+	ifobject->xsk = &ifobject->xsk_arr[0];
+	ifobject->xsk_map_fd = test->ifobj_rx->xsk_map_fd;
+	memcpy(ifobject->umem, test->ifobj_rx->umem, sizeof(struct xsk_umem_info));
+}
+
+static void xsk_populate_fill_ring(struct xsk_umem_info *umem, struct pkt_stream *pkt_stream)
+{
+	u32 idx = 0, i, buffers_to_fill;
+	int ret;
+
+	if (umem->num_frames < XSK_RING_PROD__DEFAULT_NUM_DESCS)
+		buffers_to_fill = umem->num_frames;
+	else
+		buffers_to_fill = XSK_RING_PROD__DEFAULT_NUM_DESCS;
+
+	ret = xsk_ring_prod__reserve(&umem->fq, buffers_to_fill, &idx);
+	if (ret != buffers_to_fill)
+		exit_with_error(ENOSPC);
+	for (i = 0; i < buffers_to_fill; i++) {
+		u64 addr;
+
+		if (pkt_stream->use_addr_for_fill) {
+			struct pkt *pkt = pkt_stream_get_pkt(pkt_stream, i);
+
+			if (!pkt)
+				break;
+			addr = pkt->addr;
+		} else {
+			addr = i * umem->frame_size;
+		}
+
+		*xsk_ring_prod__fill_addr(&umem->fq, idx++) = addr;
+	}
+	xsk_ring_prod__submit(&umem->fq, buffers_to_fill);
+}
+
 static void thread_common_ops(struct test_spec *test, struct ifobject *ifobject)
 {
 	u64 umem_sz = ifobject->umem->num_frames * ifobject->umem->frame_size;
 	int mmap_flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE;
 	int ret, ifindex;
 	void *bufs;
-	u32 i;
 
 	ifobject->ns_fd = switch_namespace(ifobject->nsname);
 
 	if (ifobject->umem->unaligned_mode)
 		mmap_flags |= MAP_HUGETLB;
 
+	if (ifobject->shared_umem)
+		umem_sz *= 2;
+
 	bufs = mmap(NULL, umem_sz, PROT_READ | PROT_WRITE, mmap_flags, -1, 0);
 	if (bufs == MAP_FAILED)
 		exit_with_error(errno);
@@ -1122,24 +1191,9 @@ static void thread_common_ops(struct test_spec *test, struct ifobject *ifobject)
 	if (ret)
 		exit_with_error(-ret);
 
-	for (i = 0; i < test->nb_sockets; i++) {
-		u32 ctr = 0;
-
-		while (ctr++ < SOCK_RECONF_CTR) {
-			ret = xsk_configure_socket(&ifobject->xsk_arr[i], ifobject->umem,
-						   ifobject, !!i);
-			if (!ret)
-				break;
-
-			/* Retry if it fails as xsk_socket__create() is asynchronous */
-			if (ctr >= SOCK_RECONF_CTR)
-				exit_with_error(-ret);
-			usleep(USLEEP_MAX);
-		}
+	xsk_populate_fill_ring(ifobject->umem, ifobject->pkt_stream);
 
-		if (ifobject->busy_poll)
-			enable_busy_poll(&ifobject->xsk_arr[i]);
-	}
+	xsk_configure_socket(test, ifobject, ifobject->umem, false);
 
 	ifobject->xsk = &ifobject->xsk_arr[0];
 
@@ -1159,22 +1213,18 @@ static void thread_common_ops(struct test_spec *test, struct ifobject *ifobject)
 		exit_with_error(-ret);
 }
 
-static void testapp_cleanup_xsk_res(struct ifobject *ifobj)
-{
-	print_verbose("Destroying socket\n");
-	xsk_socket__delete(ifobj->xsk->xsk);
-	munmap(ifobj->umem->buffer, ifobj->umem->num_frames * ifobj->umem->frame_size);
-	xsk_umem__delete(ifobj->umem->umem);
-}
-
 static void *worker_testapp_validate_tx(void *arg)
 {
 	struct test_spec *test = (struct test_spec *)arg;
 	struct ifobject *ifobject = test->ifobj_tx;
 	int err;
 
-	if (test->current_step == 1)
-		thread_common_ops(test, ifobject);
+	if (test->current_step == 1) {
+		if (!ifobject->shared_umem)
+			thread_common_ops(test, ifobject);
+		else
+			thread_common_ops_tx(test, ifobject);
+	}
 
 	print_verbose("Sending %d packets on interface %s\n", ifobject->pkt_stream->nb_pkts,
 		      ifobject->ifname);
@@ -1185,53 +1235,23 @@ static void *worker_testapp_validate_tx(void *arg)
 	if (err)
 		report_failure(test);
 
-	if (test->total_steps == test->current_step || err)
-		testapp_cleanup_xsk_res(ifobject);
 	pthread_exit(NULL);
 }
 
-static void xsk_populate_fill_ring(struct xsk_umem_info *umem, struct pkt_stream *pkt_stream)
-{
-	u32 idx = 0, i, buffers_to_fill;
-	int ret;
-
-	if (umem->num_frames < XSK_RING_PROD__DEFAULT_NUM_DESCS)
-		buffers_to_fill = umem->num_frames;
-	else
-		buffers_to_fill = XSK_RING_PROD__DEFAULT_NUM_DESCS;
-
-	ret = xsk_ring_prod__reserve(&umem->fq, buffers_to_fill, &idx);
-	if (ret != buffers_to_fill)
-		exit_with_error(ENOSPC);
-	for (i = 0; i < buffers_to_fill; i++) {
-		u64 addr;
-
-		if (pkt_stream->use_addr_for_fill) {
-			struct pkt *pkt = pkt_stream_get_pkt(pkt_stream, i);
-
-			if (!pkt)
-				break;
-			addr = pkt->addr;
-		} else {
-			addr = i * umem->frame_size;
-		}
-
-		*xsk_ring_prod__fill_addr(&umem->fq, idx++) = addr;
-	}
-	xsk_ring_prod__submit(&umem->fq, buffers_to_fill);
-}
-
 static void *worker_testapp_validate_rx(void *arg)
 {
 	struct test_spec *test = (struct test_spec *)arg;
 	struct ifobject *ifobject = test->ifobj_rx;
 	struct pollfd fds = { };
+	int id = 0;
 	int err;
 
-	if (test->current_step == 1)
+	if (test->current_step == 1) {
 		thread_common_ops(test, ifobject);
-
-	xsk_populate_fill_ring(ifobject->umem, ifobject->pkt_stream);
+	} else {
+		bpf_map_delete_elem(ifobject->xsk_map_fd, &id);
+		xsk_socket__update_xskmap(ifobject->xsk->xsk, ifobject->xsk_map_fd);
+	}
 
 	fds.fd = xsk_socket__fd(ifobject->xsk->xsk);
 	fds.events = POLLIN;
@@ -1249,11 +1269,20 @@ static void *worker_testapp_validate_rx(void *arg)
 		pthread_mutex_unlock(&pacing_mutex);
 	}
 
-	if (test->total_steps == test->current_step || err)
-		testapp_cleanup_xsk_res(ifobject);
 	pthread_exit(NULL);
 }
 
+static void testapp_clean_xsk_umem(struct ifobject *ifobj)
+{
+	u64 umem_sz = ifobj->umem->num_frames * ifobj->umem->frame_size;
+
+	if (ifobj->shared_umem)
+		umem_sz *= 2;
+
+	xsk_umem__delete(ifobj->umem->umem);
+	munmap(ifobj->umem->buffer, umem_sz);
+}
+
 static int testapp_validate_traffic(struct test_spec *test)
 {
 	struct ifobject *ifobj_tx = test->ifobj_tx;
@@ -1280,6 +1309,14 @@ static int testapp_validate_traffic(struct test_spec *test)
 	pthread_join(t1, NULL);
 	pthread_join(t0, NULL);
 
+	if (test->total_steps == test->current_step || test->fail) {
+		xsk_socket__delete(ifobj_tx->xsk->xsk);
+		xsk_socket__delete(ifobj_rx->xsk->xsk);
+		testapp_clean_xsk_umem(ifobj_rx);
+		if (!ifobj_tx->shared_umem)
+			testapp_clean_xsk_umem(ifobj_tx);
+	}
+
 	return !!test->fail;
 }
 
@@ -1359,9 +1396,9 @@ static void testapp_headroom(struct test_spec *test)
 static void testapp_stats_rx_dropped(struct test_spec *test)
 {
 	test_spec_set_name(test, "STAT_RX_DROPPED");
+	pkt_stream_replace_half(test, MIN_PKT_SIZE * 4, 0);
 	test->ifobj_rx->umem->frame_headroom = test->ifobj_rx->umem->frame_size -
 		XDP_PACKET_HEADROOM - MIN_PKT_SIZE * 3;
-	pkt_stream_replace_half(test, MIN_PKT_SIZE * 4, 0);
 	pkt_stream_receive_half(test);
 	test->ifobj_rx->validation_func = validate_rx_dropped;
 	testapp_validate_traffic(test);
@@ -1484,6 +1521,11 @@ static void testapp_invalid_desc(struct test_spec *test)
 		pkts[7].valid = false;
 	}
 
+	if (test->ifobj_tx->shared_umem) {
+		pkts[4].addr += UMEM_SIZE;
+		pkts[5].addr += UMEM_SIZE;
+	}
+
 	pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts));
 	testapp_validate_traffic(test);
 	pkt_stream_restore_default(test);
@@ -1624,7 +1666,6 @@ static void ifobject_delete(struct ifobject *ifobj)
 {
 	if (ifobj->ns_fd != -1)
 		close(ifobj->ns_fd);
-	free(ifobj->umem);
 	free(ifobj->xsk_arr);
 	free(ifobj);
 }
@@ -1665,6 +1706,7 @@ int main(int argc, char **argv)
 	int modes = TEST_MODE_SKB + 1;
 	u32 i, j, failed_tests = 0;
 	struct test_spec test;
+	bool shared_umem;
 
 	/* Use libbpf 1.0 API mode */
 	libbpf_set_strict_mode(LIBBPF_STRICT_ALL);
@@ -1679,6 +1721,10 @@ int main(int argc, char **argv)
 	setlocale(LC_ALL, "");
 
 	parse_command_line(ifobj_tx, ifobj_rx, argc, argv);
+	shared_umem = !strcmp(ifobj_tx->ifname, ifobj_rx->ifname);
+
+	ifobj_tx->shared_umem = shared_umem;
+	ifobj_rx->shared_umem = shared_umem;
 
 	if (!validate_interface(ifobj_tx) || !validate_interface(ifobj_rx)) {
 		usage(basename(argv[0]));
@@ -1715,6 +1761,9 @@ int main(int argc, char **argv)
 
 	pkt_stream_delete(tx_pkt_stream_default);
 	pkt_stream_delete(rx_pkt_stream_default);
+	free(ifobj_rx->umem);
+	if (!ifobj_tx->shared_umem)
+		free(ifobj_tx->umem);
 	ifobject_delete(ifobj_tx);
 	ifobject_delete(ifobj_rx);
 
diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
index ccfc829b2e5e..b7aa6c7cf2be 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.h
+++ b/tools/testing/selftests/bpf/xdpxceiver.h
@@ -149,6 +149,7 @@ struct ifobject {
 	bool busy_poll;
 	bool use_fill_ring;
 	bool release_rx;
+	bool shared_umem;
 	u8 dst_mac[ETH_ALEN];
 	u8 src_mac[ETH_ALEN];
 };
-- 
2.27.0

[PATCH bpf-next 08/10] selftests: xsk: rely on pkts_in_flight in wait_for_tx_completion()

[Maciej Fijalkowski]

Some of the drivers that implement support for AF_XDP Zero Copy (like
ice) can have lazy approach for cleaning Tx descriptors. For ZC, when
descriptor is cleaned, it is placed onto AF_XDP completion queue. This
means that current implementation of wait_for_tx_completion() in
xdpxceiver can get onto infinite loop, as some of the descriptors can
never reach CQ.

This function can be changed to rely on pkts_in_flight instead.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 tools/testing/selftests/bpf/xdpxceiver.c | 3 ++-
 tools/testing/selftests/bpf/xdpxceiver.h | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
index 316f1dd338fc..c9385690af09 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.c
+++ b/tools/testing/selftests/bpf/xdpxceiver.c
@@ -965,7 +965,7 @@ static int __send_pkts(struct ifobject *ifobject, u32 *pkt_nb)
 
 static void wait_for_tx_completion(struct xsk_socket_info *xsk)
 {
-	while (xsk->outstanding_tx)
+	while (pkts_in_flight)
 		complete_pkts(xsk, BATCH_SIZE);
 }
 
@@ -1269,6 +1269,7 @@ static void *worker_testapp_validate_rx(void *arg)
 		pthread_mutex_unlock(&pacing_mutex);
 	}
 
+	pkts_in_flight = 0;
 	pthread_exit(NULL);
 }
 
diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
index b7aa6c7cf2be..f364a92675f8 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.h
+++ b/tools/testing/selftests/bpf/xdpxceiver.h
@@ -170,6 +170,6 @@ pthread_barrier_t barr;
 pthread_mutex_t pacing_mutex = PTHREAD_MUTEX_INITIALIZER;
 pthread_cond_t pacing_cond = PTHREAD_COND_INITIALIZER;
 
-int pkts_in_flight;
+volatile int pkts_in_flight;
 
 #endif				/* XDPXCEIVER_H */
-- 
2.27.0

[PATCH bpf-next 09/10] selftests: xsk: remove struct xsk_socket_info::outstanding_tx

[Maciej Fijalkowski]

Previous change makes xsk->outstanding_tx a dead code, so let's remove
it.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 tools/testing/selftests/bpf/xdpxceiver.c | 20 +++-----------------
 tools/testing/selftests/bpf/xdpxceiver.h |  1 -
 2 files changed, 3 insertions(+), 18 deletions(-)

diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
index c9385690af09..a2aa652d0bb8 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.c
+++ b/tools/testing/selftests/bpf/xdpxceiver.c
@@ -815,7 +815,7 @@ static void kick_rx(struct xsk_socket_info *xsk)
 		exit_with_error(errno);
 }
 
-static int complete_pkts(struct xsk_socket_info *xsk, int batch_size)
+static void complete_pkts(struct xsk_socket_info *xsk, int batch_size)
 {
 	unsigned int rcvd;
 	u32 idx;
@@ -824,20 +824,8 @@ static int complete_pkts(struct xsk_socket_info *xsk, int batch_size)
 		kick_tx(xsk);
 
 	rcvd = xsk_ring_cons__peek(&xsk->umem->cq, batch_size, &idx);
-	if (rcvd) {
-		if (rcvd > xsk->outstanding_tx) {
-			u64 addr = *xsk_ring_cons__comp_addr(&xsk->umem->cq, idx + rcvd - 1);
-
-			ksft_print_msg("[%s] Too many packets completed\n", __func__);
-			ksft_print_msg("Last completion address: %llx\n", addr);
-			return TEST_FAILURE;
-		}
-
+	if (rcvd)
 		xsk_ring_cons__release(&xsk->umem->cq, rcvd);
-		xsk->outstanding_tx -= rcvd;
-	}
-
-	return TEST_PASS;
 }
 
 static int receive_pkts(struct ifobject *ifobj, struct pollfd *fds)
@@ -955,9 +943,7 @@ static int __send_pkts(struct ifobject *ifobject, u32 *pkt_nb)
 	pthread_mutex_unlock(&pacing_mutex);
 
 	xsk_ring_prod__submit(&xsk->tx, i);
-	xsk->outstanding_tx += valid_pkts;
-	if (complete_pkts(xsk, i))
-		return TEST_FAILURE;
+	complete_pkts(xsk, i);
 
 	usleep(10);
 	return TEST_PASS;
diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
index f364a92675f8..12b792004163 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.h
+++ b/tools/testing/selftests/bpf/xdpxceiver.h
@@ -104,7 +104,6 @@ struct xsk_socket_info {
 	struct xsk_ring_prod tx;
 	struct xsk_umem_info *umem;
 	struct xsk_socket *xsk;
-	u32 outstanding_tx;
 	u32 rxqsize;
 };
 
-- 
2.27.0

[PATCH bpf-next 10/10] selftests: xsk: add support for zero copy testing

[Maciej Fijalkowski]

Introduce new mode to xdpxceiver responsible for testing AF_XDP zero
copy support of driver that serves underlying physical device. When
setting up test suite, determine whether driver has ZC support or not by
trying to bind XSK ZC socket to the interface. If it succeeded,
interpret it as ZC support being in place and do softirq and busy poll
tests for zero copy mode.

Note that Rx dropped tests are skipped since ZC path is not touching
rx_dropped stat at all.

Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 tools/testing/selftests/bpf/xdpxceiver.c | 80 ++++++++++++++++++++++--
 tools/testing/selftests/bpf/xdpxceiver.h |  2 +
 2 files changed, 77 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
index a2aa652d0bb8..beef8d694fa6 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.c
+++ b/tools/testing/selftests/bpf/xdpxceiver.c
@@ -124,9 +124,20 @@ static void __exit_with_error(int error, const char *file, const char *func, int
 }
 
 #define exit_with_error(error) __exit_with_error(error, __FILE__, __func__, __LINE__)
-
-#define mode_string(test) (test)->ifobj_tx->xdp_flags & XDP_FLAGS_SKB_MODE ? "SKB" : "DRV"
 #define busy_poll_string(test) (test)->ifobj_tx->busy_poll ? "BUSY-POLL " : ""
+static char *mode_string(struct test_spec *test)
+{
+	switch (test->mode) {
+	case TEST_MODE_SKB:
+		return "SKB";
+	case TEST_MODE_DRV:
+		return "DRV";
+	case TEST_MODE_ZC:
+		return "ZC";
+	default:
+		return "BOGUS";
+	}
+}
 
 static void report_failure(struct test_spec *test)
 {
@@ -317,6 +328,53 @@ static int __xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_i
 	return xsk_socket__create(&xsk->xsk, ifobject->ifname, 0, umem->umem, rxr, txr, &cfg);
 }
 
+static bool ifobj_zc_avail(struct ifobject *ifobject)
+{
+	size_t umem_sz = DEFAULT_UMEM_BUFFERS * XSK_UMEM__DEFAULT_FRAME_SIZE;
+	int mmap_flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE;
+	struct xsk_socket_info *xsk;
+	struct xsk_umem_info *umem;
+	bool zc_avail = false;
+	void *bufs;
+	int ret;
+
+	bufs = mmap(NULL, umem_sz, PROT_READ | PROT_WRITE, mmap_flags, -1, 0);
+	if (bufs == MAP_FAILED)
+		exit_with_error(errno);
+
+	umem = calloc(1, sizeof(struct xsk_umem_info));
+	if (!umem) {
+		munmap(bufs, umem_sz);
+		exit_with_error(-ENOMEM);
+	}
+	umem->frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE;
+	ret = xsk_configure_umem(umem, bufs, umem_sz);
+	if (ret)
+		exit_with_error(-ret);
+
+	xsk = calloc(1, sizeof(struct xsk_socket_info));
+	if (!xsk)
+		goto out;
+	ifobject->xdp_flags = 0;
+	ifobject->xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST;
+	ifobject->xdp_flags |= XDP_FLAGS_DRV_MODE;
+	ifobject->bind_flags = XDP_USE_NEED_WAKEUP | XDP_ZEROCOPY;
+	ifobject->rx_on = true;
+	xsk->rxqsize = XSK_RING_CONS__DEFAULT_NUM_DESCS;
+	ret = __xsk_configure_socket(xsk, umem, ifobject, false);
+	if (!ret)
+		zc_avail = true;
+
+	ifobject->xdp_flags = 0;
+	xsk_socket__delete(xsk->xsk);
+	free(xsk);
+out:
+	munmap(umem->buffer, umem_sz);
+	xsk_umem__delete(umem->umem);
+	free(umem);
+	return zc_avail;
+}
+
 static struct option long_options[] = {
 	{"interface", required_argument, 0, 'i'},
 	{"busy-poll", no_argument, 0, 'b'},
@@ -483,9 +541,14 @@ static void test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
 		else
 			ifobj->xdp_flags |= XDP_FLAGS_DRV_MODE;
 
-		ifobj->bind_flags = XDP_USE_NEED_WAKEUP | XDP_COPY;
+		ifobj->bind_flags = XDP_USE_NEED_WAKEUP;
+		if (mode == TEST_MODE_ZC)
+			ifobj->bind_flags |= XDP_ZEROCOPY;
+		else
+			ifobj->bind_flags |= XDP_COPY;
 	}
 
+	test->mode = mode;
 	__test_spec_init(test, ifobj_tx, ifobj_rx);
 }
 
@@ -1543,6 +1606,10 @@ static void run_pkt_test(struct test_spec *test, enum test_mode mode, enum test_
 {
 	switch (type) {
 	case TEST_TYPE_STATS_RX_DROPPED:
+		if (mode == TEST_MODE_ZC) {
+			ksft_test_result_skip("Can not run RX_DROPPED test for ZC mode\n");
+			return;
+		}
 		testapp_stats_rx_dropped(test);
 		break;
 	case TEST_TYPE_STATS_TX_INVALID_DESCS:
@@ -1723,8 +1790,11 @@ int main(int argc, char **argv)
 	init_iface(ifobj_rx, MAC2, MAC1, IP2, IP1, UDP_PORT2, UDP_PORT1,
 		   worker_testapp_validate_rx);
 
-	if (is_xdp_supported(ifobj_tx))
-		modes++;
+	if (is_xdp_supported(ifobj_tx)) {
+		modes++;
+		if (ifobj_zc_avail(ifobj_tx))
+			modes++;
+	}
 
 	test_spec_init(&test, ifobj_tx, ifobj_rx, 0);
 	tx_pkt_stream_default = pkt_stream_generate(ifobj_tx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
index 12b792004163..a86331c6b0c5 100644
--- a/tools/testing/selftests/bpf/xdpxceiver.h
+++ b/tools/testing/selftests/bpf/xdpxceiver.h
@@ -61,6 +61,7 @@
 enum test_mode {
 	TEST_MODE_SKB,
 	TEST_MODE_DRV,
+	TEST_MODE_ZC,
 	TEST_MODE_MAX
 };
 
@@ -162,6 +163,7 @@ struct test_spec {
 	u16 current_step;
 	u16 nb_sockets;
 	bool fail;
+	enum test_mode mode;
 	char name[MAX_TEST_NAME_SIZE];
 };
 
-- 
2.27.0

Re: [PATCH bpf-next 01/10] ice: introduce priv-flag for toggling loopback mode

[Alexander Lobakin]

From: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Date: Fri, 10 Jun 2022 17:09:14 +0200

> Add a knob that will allow user to turn the underlying net device into
> loopback mode. The use case for this will be the AF_XDP ZC tests. Once
> the device is in loopback mode, then it will be possible from AF_XDP
> perspective to see if zero copy implementations in drivers work
> properly.
> 
> The code for interaction with admin queue is reused from ethtool's
> loopback test.
> 
> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> ---
>  drivers/net/ethernet/intel/ice/ice.h         |  1 +
>  drivers/net/ethernet/intel/ice/ice_ethtool.c | 17 +++++++++++++++++
>  2 files changed, 18 insertions(+)
> 
> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> index 60453b3b8d23..90c066f3782b 100644
> --- a/drivers/net/ethernet/intel/ice/ice.h
> +++ b/drivers/net/ethernet/intel/ice/ice.h
> @@ -487,6 +487,7 @@ enum ice_pf_flags {
>  	ICE_FLAG_PLUG_AUX_DEV,
>  	ICE_FLAG_MTU_CHANGED,
>  	ICE_FLAG_GNSS,			/* GNSS successfully initialized */
> +	ICE_FLAG_LOOPBACK,

I'do for %NETIF_F_LOOPBACK, should work in here, too :)

>  	ICE_PF_FLAGS_NBITS		/* must be last */
>  };
>  
> diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
> index 1e71b70f0e52..cfc3c5e36907 100644
> --- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
> +++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
> @@ -166,6 +166,7 @@ static const struct ice_priv_flag ice_gstrings_priv_flags[] = {
>  	ICE_PRIV_FLAG("mdd-auto-reset-vf", ICE_FLAG_MDD_AUTO_RESET_VF),
>  	ICE_PRIV_FLAG("vf-vlan-pruning", ICE_FLAG_VF_VLAN_PRUNING),
>  	ICE_PRIV_FLAG("legacy-rx", ICE_FLAG_LEGACY_RX),
> +	ICE_PRIV_FLAG("loopback", ICE_FLAG_LOOPBACK),
>  };
>  
>  #define ICE_PRIV_FLAG_ARRAY_SIZE	ARRAY_SIZE(ice_gstrings_priv_flags)
> @@ -1288,6 +1289,22 @@ static int ice_set_priv_flags(struct net_device *netdev, u32 flags)
>  			ice_up(vsi);
>  		}
>  	}
> +
> +	if (test_bit(ICE_FLAG_LOOPBACK, change_flags)) {
> +		if (!test_bit(ICE_FLAG_LOOPBACK, orig_flags)) {
> +			/* Enable MAC loopback in firmware */
> +			if (ice_aq_set_mac_loopback(&pf->hw, true, NULL)) {
> +				dev_err(dev, "Failed to enable loopback\n");
> +				ret = -ENXIO;
> +			}
> +		} else {
> +			/* Disable MAC loopback in firmware */
> +			if (ice_aq_set_mac_loopback(&pf->hw, false, NULL)) {
> +				dev_err(dev, "Failed to disable loopback\n");
> +				ret = -ENXIO;
> +			}
> +		}
> +	}
>  	/* don't allow modification of this flag when a single VF is in
>  	 * promiscuous mode because it's not supported
>  	 */
> -- 
> 2.27.0

Thanks,
Olek

Re: [PATCH bpf-next 04/10] selftests: xsk: query for native XDP support

[Magnus Karlsson]

On Fri, Jun 10, 2022 at 5:15 PM Maciej Fijalkowski
<maciej.fijalkowski@intel.com> wrote:
>
> Currently, xdpxceiver assumes that underlying device supports XDP in
> native mode - it is fine by now since tests can run only on a veth pair.
> Future commit is going to allow running test suite against physical
> devices, so let us query the device if it is capable of running XDP
> programs in native mode. This way xdpxceiver will not try to run
> TEST_MODE_DRV if device being tested is not supporting it.
>
> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> ---
>  tools/testing/selftests/bpf/xdpxceiver.c | 38 ++++++++++++++++++++++--
>  1 file changed, 36 insertions(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
> index e5992a6b5e09..da8098f1b655 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.c
> +++ b/tools/testing/selftests/bpf/xdpxceiver.c
> @@ -98,6 +98,8 @@
>  #include <unistd.h>
>  #include <stdatomic.h>
>  #include <bpf/xsk.h>
> +#include <bpf/bpf.h>
> +#include <linux/filter.h>
>  #include "xdpxceiver.h"
>  #include "../kselftest.h"
>
> @@ -1605,10 +1607,39 @@ static void ifobject_delete(struct ifobject *ifobj)
>         free(ifobj);
>  }
>
> +static bool is_xdp_supported(struct ifobject *ifobject)
> +{
> +       int flags = XDP_FLAGS_DRV_MODE;
> +
> +       LIBBPF_OPTS(bpf_link_create_opts, opts, .flags = flags);
> +       struct bpf_insn insns[2] = {
> +               BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
> +               BPF_EXIT_INSN()
> +       };
> +       int ifindex = if_nametoindex(ifobject->ifname);
> +       int prog_fd, insn_cnt = ARRAY_SIZE(insns);
> +       bool ret = false;
> +       int err;
> +
> +       prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, NULL);
> +       if (prog_fd < 0)
> +               return ret;
> +
> +       err = bpf_xdp_attach(ifindex, prog_fd, flags, NULL);
> +
> +       if (!err) {
> +               ret = true;
> +               bpf_xdp_detach(ifindex, flags, NULL);
> +       }
> +
> +       return ret;

Think it would be clearer if you got rid of the bool ret and just
wrote "return false" and "return true" where applicable.

> +}
> +
>  int main(int argc, char **argv)
>  {
>         struct pkt_stream *pkt_stream_default;
>         struct ifobject *ifobj_tx, *ifobj_rx;
> +       int modes = TEST_MODE_SKB + 1;

Why not keep it a u32? A nit in any way.

>         u32 i, j, failed_tests = 0;
>         struct test_spec test;
>
> @@ -1636,15 +1667,18 @@ int main(int argc, char **argv)
>         init_iface(ifobj_rx, MAC2, MAC1, IP2, IP1, UDP_PORT2, UDP_PORT1,
>                    worker_testapp_validate_rx);
>
> +       if (is_xdp_supported(ifobj_tx))
> +               modes++;
> +
>         test_spec_init(&test, ifobj_tx, ifobj_rx, 0);
>         pkt_stream_default = pkt_stream_generate(ifobj_tx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
>         if (!pkt_stream_default)
>                 exit_with_error(ENOMEM);
>         test.pkt_stream_default = pkt_stream_default;
>
> -       ksft_set_plan(TEST_MODE_MAX * TEST_TYPE_MAX);
> +       ksft_set_plan(modes * TEST_TYPE_MAX);
>
> -       for (i = 0; i < TEST_MODE_MAX; i++)
> +       for (i = 0; i < modes; i++)
>                 for (j = 0; j < TEST_TYPE_MAX; j++) {
>                         test_spec_init(&test, ifobj_tx, ifobj_rx, i);
>                         run_pkt_test(&test, i, j);
> --
> 2.27.0
>

Re: [PATCH bpf-next 05/10] selftests: xsk: add missing close() on netns fd

[Magnus Karlsson]

On Fri, Jun 10, 2022 at 5:15 PM Maciej Fijalkowski
<maciej.fijalkowski@intel.com> wrote:
>
> Commit 1034b03e54ac ("selftests: xsk: Simplify cleanup of ifobjects") removed
> close on netns fd, which is not correct, so let us restore it.

Acked-by: Magnus Karlsson <magnus.karlsson@intel.com>

> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> ---
>  tools/testing/selftests/bpf/xdpxceiver.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
> index da8098f1b655..2499075fad82 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.c
> +++ b/tools/testing/selftests/bpf/xdpxceiver.c
> @@ -1591,6 +1591,8 @@ static struct ifobject *ifobject_create(void)
>         if (!ifobj->umem)
>                 goto out_umem;
>
> +       ifobj->ns_fd = -1;
> +
>         return ifobj;
>
>  out_umem:
> @@ -1602,6 +1604,8 @@ static struct ifobject *ifobject_create(void)
>
>  static void ifobject_delete(struct ifobject *ifobj)
>  {
> +       if (ifobj->ns_fd != -1)
> +               close(ifobj->ns_fd);
>         free(ifobj->umem);
>         free(ifobj->xsk_arr);
>         free(ifobj);
> --
> 2.27.0
>

Re: [PATCH bpf-next 06/10] selftests: xsk: introduce default Rx pkt stream

[Magnus Karlsson]

On Fri, Jun 10, 2022 at 5:15 PM Maciej Fijalkowski
<maciej.fijalkowski@intel.com> wrote:
>
> In order to prepare xdpxceiver for physical device testing, let us
> introduce default Rx pkt stream. Reason for doing it is that physical
> device testing will use a UMEM with a doubled size where half of it will
> be used by Tx and other half by Rx. This means that pkt addresses will
> differ for Tx and Rx streams. Rx thread will initialize the
> xsk_umem_info::base_addr that is added here so that pkt_set(), when
> working on Rx UMEM will add this offset and second half of UMEM space
> will be used. Note that currently base_addr is 0 on both sides. Future
> commit will do the mentioned initialization.
>
> Previously, veth based testing worked on separate UMEMs, so single
> default stream was fine.

Make sense.

Acked-by: Magnus Karlsson <magnus.karlsson@intel.com>

> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> ---
>  tools/testing/selftests/bpf/xdpxceiver.c | 74 +++++++++++++++---------
>  tools/testing/selftests/bpf/xdpxceiver.h |  4 +-
>  2 files changed, 51 insertions(+), 27 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
> index 2499075fad82..ad6c92c31026 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.c
> +++ b/tools/testing/selftests/bpf/xdpxceiver.c
> @@ -428,15 +428,16 @@ static void __test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
>                 ifobj->use_poll = false;
>                 ifobj->use_fill_ring = true;
>                 ifobj->release_rx = true;
> -               ifobj->pkt_stream = test->pkt_stream_default;
>                 ifobj->validation_func = NULL;
>
>                 if (i == 0) {
>                         ifobj->rx_on = false;
>                         ifobj->tx_on = true;
> +                       ifobj->pkt_stream = test->tx_pkt_stream_default;
>                 } else {
>                         ifobj->rx_on = true;
>                         ifobj->tx_on = false;
> +                       ifobj->pkt_stream = test->rx_pkt_stream_default;
>                 }
>
>                 memset(ifobj->umem, 0, sizeof(*ifobj->umem));
> @@ -460,12 +461,15 @@ static void __test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
>  static void test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
>                            struct ifobject *ifobj_rx, enum test_mode mode)
>  {
> -       struct pkt_stream *pkt_stream;
> +       struct pkt_stream *tx_pkt_stream;
> +       struct pkt_stream *rx_pkt_stream;
>         u32 i;
>
> -       pkt_stream = test->pkt_stream_default;
> +       tx_pkt_stream = test->tx_pkt_stream_default;
> +       rx_pkt_stream = test->rx_pkt_stream_default;
>         memset(test, 0, sizeof(*test));
> -       test->pkt_stream_default = pkt_stream;
> +       test->tx_pkt_stream_default = tx_pkt_stream;
> +       test->rx_pkt_stream_default = rx_pkt_stream;
>
>         for (i = 0; i < MAX_INTERFACES; i++) {
>                 struct ifobject *ifobj = i ? ifobj_rx : ifobj_tx;
> @@ -526,16 +530,17 @@ static void pkt_stream_delete(struct pkt_stream *pkt_stream)
>  static void pkt_stream_restore_default(struct test_spec *test)
>  {
>         struct pkt_stream *tx_pkt_stream = test->ifobj_tx->pkt_stream;
> +       struct pkt_stream *rx_pkt_stream = test->ifobj_rx->pkt_stream;
>
> -       if (tx_pkt_stream != test->pkt_stream_default) {
> +       if (tx_pkt_stream != test->tx_pkt_stream_default) {
>                 pkt_stream_delete(test->ifobj_tx->pkt_stream);
> -               test->ifobj_tx->pkt_stream = test->pkt_stream_default;
> +               test->ifobj_tx->pkt_stream = test->tx_pkt_stream_default;
>         }
>
> -       if (test->ifobj_rx->pkt_stream != test->pkt_stream_default &&
> -           test->ifobj_rx->pkt_stream != tx_pkt_stream)
> +       if (rx_pkt_stream != test->rx_pkt_stream_default) {
>                 pkt_stream_delete(test->ifobj_rx->pkt_stream);
> -       test->ifobj_rx->pkt_stream = test->pkt_stream_default;
> +               test->ifobj_rx->pkt_stream = test->rx_pkt_stream_default;
> +       }
>  }
>
>  static struct pkt_stream *__pkt_stream_alloc(u32 nb_pkts)
> @@ -558,7 +563,7 @@ static struct pkt_stream *__pkt_stream_alloc(u32 nb_pkts)
>
>  static void pkt_set(struct xsk_umem_info *umem, struct pkt *pkt, u64 addr, u32 len)
>  {
> -       pkt->addr = addr;
> +       pkt->addr = addr + umem->base_addr;
>         pkt->len = len;
>         if (len > umem->frame_size - XDP_PACKET_HEADROOM - MIN_PKT_SIZE * 2 - umem->frame_headroom)
>                 pkt->valid = false;
> @@ -597,22 +602,29 @@ static void pkt_stream_replace(struct test_spec *test, u32 nb_pkts, u32 pkt_len)
>
>         pkt_stream = pkt_stream_generate(test->ifobj_tx->umem, nb_pkts, pkt_len);
>         test->ifobj_tx->pkt_stream = pkt_stream;
> +       pkt_stream = pkt_stream_generate(test->ifobj_rx->umem, nb_pkts, pkt_len);
>         test->ifobj_rx->pkt_stream = pkt_stream;
>  }
>
> -static void pkt_stream_replace_half(struct test_spec *test, u32 pkt_len, int offset)
> +static void __pkt_stream_replace_half(struct ifobject *ifobj, u32 pkt_len,
> +                                     int offset)
>  {
> -       struct xsk_umem_info *umem = test->ifobj_tx->umem;
> +       struct xsk_umem_info *umem = ifobj->umem;
>         struct pkt_stream *pkt_stream;
>         u32 i;
>
> -       pkt_stream = pkt_stream_clone(umem, test->pkt_stream_default);
> -       for (i = 1; i < test->pkt_stream_default->nb_pkts; i += 2)
> +       pkt_stream = pkt_stream_clone(umem, ifobj->pkt_stream);
> +       for (i = 1; i < ifobj->pkt_stream->nb_pkts; i += 2)
>                 pkt_set(umem, &pkt_stream->pkts[i],
>                         (i % umem->num_frames) * umem->frame_size + offset, pkt_len);
>
> -       test->ifobj_tx->pkt_stream = pkt_stream;
> -       test->ifobj_rx->pkt_stream = pkt_stream;
> +       ifobj->pkt_stream = pkt_stream;
> +}
> +
> +static void pkt_stream_replace_half(struct test_spec *test, u32 pkt_len, int offset)
> +{
> +       __pkt_stream_replace_half(test->ifobj_tx, pkt_len, offset);
> +       __pkt_stream_replace_half(test->ifobj_rx, pkt_len, offset);
>  }
>
>  static void pkt_stream_receive_half(struct test_spec *test)
> @@ -654,7 +666,8 @@ static struct pkt *pkt_generate(struct ifobject *ifobject, u32 pkt_nb)
>         return pkt;
>  }
>
> -static void pkt_stream_generate_custom(struct test_spec *test, struct pkt *pkts, u32 nb_pkts)
> +static void __pkt_stream_generate_custom(struct ifobject *ifobj,
> +                                        struct pkt *pkts, u32 nb_pkts)
>  {
>         struct pkt_stream *pkt_stream;
>         u32 i;
> @@ -663,15 +676,20 @@ static void pkt_stream_generate_custom(struct test_spec *test, struct pkt *pkts,
>         if (!pkt_stream)
>                 exit_with_error(ENOMEM);
>
> -       test->ifobj_tx->pkt_stream = pkt_stream;
> -       test->ifobj_rx->pkt_stream = pkt_stream;
> -
>         for (i = 0; i < nb_pkts; i++) {
> -               pkt_stream->pkts[i].addr = pkts[i].addr;
> +               pkt_stream->pkts[i].addr = pkts[i].addr + ifobj->umem->base_addr;
>                 pkt_stream->pkts[i].len = pkts[i].len;
>                 pkt_stream->pkts[i].payload = i;
>                 pkt_stream->pkts[i].valid = pkts[i].valid;
>         }
> +
> +       ifobj->pkt_stream = pkt_stream;
> +}
> +
> +static void pkt_stream_generate_custom(struct test_spec *test, struct pkt *pkts, u32 nb_pkts)
> +{
> +       __pkt_stream_generate_custom(test->ifobj_tx, pkts, nb_pkts);
> +       __pkt_stream_generate_custom(test->ifobj_rx, pkts, nb_pkts);
>  }
>
>  static void pkt_dump(void *pkt, u32 len)
> @@ -1641,7 +1659,8 @@ static bool is_xdp_supported(struct ifobject *ifobject)
>
>  int main(int argc, char **argv)
>  {
> -       struct pkt_stream *pkt_stream_default;
> +       struct pkt_stream *rx_pkt_stream_default;
> +       struct pkt_stream *tx_pkt_stream_default;
>         struct ifobject *ifobj_tx, *ifobj_rx;
>         int modes = TEST_MODE_SKB + 1;
>         u32 i, j, failed_tests = 0;
> @@ -1675,10 +1694,12 @@ int main(int argc, char **argv)
>                 modes++;
>
>         test_spec_init(&test, ifobj_tx, ifobj_rx, 0);
> -       pkt_stream_default = pkt_stream_generate(ifobj_tx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
> -       if (!pkt_stream_default)
> +       tx_pkt_stream_default = pkt_stream_generate(ifobj_tx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
> +       rx_pkt_stream_default = pkt_stream_generate(ifobj_rx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
> +       if (!tx_pkt_stream_default || !rx_pkt_stream_default)
>                 exit_with_error(ENOMEM);
> -       test.pkt_stream_default = pkt_stream_default;
> +       test.tx_pkt_stream_default = tx_pkt_stream_default;
> +       test.rx_pkt_stream_default = rx_pkt_stream_default;
>
>         ksft_set_plan(modes * TEST_TYPE_MAX);
>
> @@ -1692,7 +1713,8 @@ int main(int argc, char **argv)
>                                 failed_tests++;
>                 }
>
> -       pkt_stream_delete(pkt_stream_default);
> +       pkt_stream_delete(tx_pkt_stream_default);
> +       pkt_stream_delete(rx_pkt_stream_default);
>         ifobject_delete(ifobj_tx);
>         ifobject_delete(ifobj_rx);
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
> index 8f672b0fe0e1..ccfc829b2e5e 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.h
> +++ b/tools/testing/selftests/bpf/xdpxceiver.h
> @@ -95,6 +95,7 @@ struct xsk_umem_info {
>         u32 frame_headroom;
>         void *buffer;
>         u32 frame_size;
> +       u32 base_addr;
>         bool unaligned_mode;
>  };
>
> @@ -155,7 +156,8 @@ struct ifobject {
>  struct test_spec {
>         struct ifobject *ifobj_tx;
>         struct ifobject *ifobj_rx;
> -       struct pkt_stream *pkt_stream_default;
> +       struct pkt_stream *tx_pkt_stream_default;
> +       struct pkt_stream *rx_pkt_stream_default;
>         u16 total_steps;
>         u16 current_step;
>         u16 nb_sockets;
> --
> 2.27.0
>

Re: [PATCH bpf-next 07/10] selftests: xsk: add support for executing tests on physical device

[Magnus Karlsson]

On Fri, Jun 10, 2022 at 5:36 PM Maciej Fijalkowski
<maciej.fijalkowski@intel.com> wrote:
>
> Currently, architecture of xdpxceiver is designed strictly for
> conducting veth based tests. Veth pair is created together with a
> network namespace and one of the veth interfaces is moved to the
> mentioned netns. Then, separate threads for Tx and Rx are spawned which
> will utilize described setup.
>
> Infrastructure described in the paragraph above can not be used for
> testing AF_XDP support on physical devices. That testing will be
> conducted on a single network interface and same queue. Xdpxceiver
> needs to be extended to distinguish between veth tests and physical
> interface tests.
>
> Since same iface/queue id pair will be used by both Tx/Rx threads for
> physical device testing, Tx thread, which happen to run after the Rx
> thread, is going to create XSK socket with shared umem flag. In order to
> track this setting throughout the lifetime of spawned threads, introduce
> 'shared_umem' boolean variable to struct ifobject and set it to true
> when xdpxceiver is run against physical device. In such case, UMEM size
> needs to be doubled, so half of it will be used by Rx thread and other
> half by Tx thread. For two step based test types, value of XSKMAP
> element under key 0 has to be updated as there is now another socket for
> the second step. Also, to avoid race conditions when destroying XSK
> resources, move this activity to the main thread after spawned Rx and Tx
> threads have finished its job. This way it is possible to gracefully
> remove shared umem without introducing synchronization mechanisms.
>
> To run xsk selftests suite on physical device, append "-i $IFACE" when
> invoking test_xsk.sh. For veth based tests, simply skip it. When "-i
> $IFACE" is in place, under the hood test_xsk.sh will use $IFACE for both
> interfaces supplied to xdpxceiver, which in turn will interpret that
> this execution of test suite is for a physical device.
>
> Note that currently this makes it possible only to test SKB and DRV mode
> (in case underlying device has native XDP support). ZC testing support
> is added in a later patch.

Good one! Nice to see that we can test real NICs with this support.

Acked-by: Magnus Karlsson <magnus.karlsson@intel.com>

> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> ---
>  tools/testing/selftests/bpf/test_xsk.sh  |  52 +++++--
>  tools/testing/selftests/bpf/xdpxceiver.c | 189 ++++++++++++++---------
>  tools/testing/selftests/bpf/xdpxceiver.h |   1 +
>  3 files changed, 156 insertions(+), 86 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/test_xsk.sh b/tools/testing/selftests/bpf/test_xsk.sh
> index 567500299231..19b24cce5414 100755
> --- a/tools/testing/selftests/bpf/test_xsk.sh
> +++ b/tools/testing/selftests/bpf/test_xsk.sh
> @@ -73,14 +73,20 @@
>  #
>  # Run and dump packet contents:
>  #   sudo ./test_xsk.sh -D
> +#
> +# Run test suite for physical device in loopback mode
> +#   sudo ./test_xsk.sh -i IFACE
>
>  . xsk_prereqs.sh
>
> -while getopts "vD" flag
> +ETH=""
> +
> +while getopts "vDi:" flag
>  do
>         case "${flag}" in
>                 v) verbose=1;;
>                 D) dump_pkts=1;;
> +               i) ETH=${OPTARG};;
>         esac
>  done
>
> @@ -132,18 +138,25 @@ setup_vethPairs() {
>         ip link set ${VETH0} up
>  }
>
> -validate_root_exec
> -validate_veth_support ${VETH0}
> -validate_ip_utility
> -setup_vethPairs
> -
> -retval=$?
> -if [ $retval -ne 0 ]; then
> -       test_status $retval "${TEST_NAME}"
> -       cleanup_exit ${VETH0} ${VETH1} ${NS1}
> -       exit $retval
> +if [ ! -z $ETH ]; then
> +       VETH0=${ETH}
> +       VETH1=${ETH}
> +       NS1=""
> +else
> +       validate_root_exec
> +       validate_veth_support ${VETH0}
> +       validate_ip_utility
> +       setup_vethPairs
> +
> +       retval=$?
> +       if [ $retval -ne 0 ]; then
> +               test_status $retval "${TEST_NAME}"
> +               cleanup_exit ${VETH0} ${VETH1} ${NS1}
> +               exit $retval
> +       fi
>  fi
>
> +
>  if [[ $verbose -eq 1 ]]; then
>         ARGS+="-v "
>  fi
> @@ -152,26 +165,33 @@ if [[ $dump_pkts -eq 1 ]]; then
>         ARGS="-D "
>  fi
>
> +retval=$?
>  test_status $retval "${TEST_NAME}"
>
>  ## START TESTS
>
>  statusList=()
>
> -TEST_NAME="XSK_SELFTESTS_SOFTIRQ"
> +TEST_NAME="XSK_SELFTESTS_${VETH0}_SOFTIRQ"
>
>  execxdpxceiver
>
> -cleanup_exit ${VETH0} ${VETH1} ${NS1}
> -TEST_NAME="XSK_SELFTESTS_BUSY_POLL"
> +if [ -z $ETH ]; then
> +       cleanup_exit ${VETH0} ${VETH1} ${NS1}
> +fi
> +TEST_NAME="XSK_SELFTESTS_${VETH0}_BUSY_POLL"
>  busy_poll=1
>
> -setup_vethPairs
> +if [ -z $ETH ]; then
> +       setup_vethPairs
> +fi
>  execxdpxceiver
>
>  ## END TESTS
>
> -cleanup_exit ${VETH0} ${VETH1} ${NS1}
> +if [ -z $ETH ]; then
> +       cleanup_exit ${VETH0} ${VETH1} ${NS1}
> +fi
>
>  failures=0
>  echo -e "\nSummary:"
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
> index ad6c92c31026..316f1dd338fc 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.c
> +++ b/tools/testing/selftests/bpf/xdpxceiver.c
> @@ -296,8 +296,8 @@ static void enable_busy_poll(struct xsk_socket_info *xsk)
>                 exit_with_error(errno);
>  }
>
> -static int xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_info *umem,
> -                               struct ifobject *ifobject, bool shared)
> +static int __xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_info *umem,
> +                                 struct ifobject *ifobject, bool shared)
>  {
>         struct xsk_socket_config cfg = {};
>         struct xsk_ring_cons *rxr;
> @@ -443,6 +443,9 @@ static void __test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
>                 memset(ifobj->umem, 0, sizeof(*ifobj->umem));
>                 ifobj->umem->num_frames = DEFAULT_UMEM_BUFFERS;
>                 ifobj->umem->frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE;
> +               if (ifobj->shared_umem && ifobj->rx_on)
> +                       ifobj->umem->base_addr = DEFAULT_UMEM_BUFFERS *
> +                               XSK_UMEM__DEFAULT_FRAME_SIZE;
>
>                 for (j = 0; j < MAX_SOCKETS; j++) {
>                         memset(&ifobj->xsk_arr[j], 0, sizeof(ifobj->xsk_arr[j]));
> @@ -1101,19 +1104,85 @@ static int validate_tx_invalid_descs(struct ifobject *ifobject)
>         return TEST_PASS;
>  }
>
> +static void xsk_configure_socket(struct test_spec *test, struct ifobject *ifobject,
> +                                struct xsk_umem_info *umem, bool tx)
> +{
> +       int i, ret;
> +
> +       for (i = 0; i < test->nb_sockets; i++) {
> +               bool shared = (ifobject->shared_umem && tx) ? true : !!i;
> +               u32 ctr = 0;
> +
> +               while (ctr++ < SOCK_RECONF_CTR) {
> +                       ret = __xsk_configure_socket(&ifobject->xsk_arr[i], umem,
> +                                                    ifobject, shared);
> +                       if (!ret)
> +                               break;
> +
> +                       /* Retry if it fails as xsk_socket__create() is asynchronous */
> +                       if (ctr >= SOCK_RECONF_CTR)
> +                               exit_with_error(-ret);
> +                       usleep(USLEEP_MAX);
> +               }
> +               if (ifobject->busy_poll)
> +                       enable_busy_poll(&ifobject->xsk_arr[i]);
> +       }
> +}
> +
> +static void thread_common_ops_tx(struct test_spec *test, struct ifobject *ifobject)
> +{
> +       xsk_configure_socket(test, ifobject, test->ifobj_rx->umem, true);
> +       ifobject->xsk = &ifobject->xsk_arr[0];
> +       ifobject->xsk_map_fd = test->ifobj_rx->xsk_map_fd;
> +       memcpy(ifobject->umem, test->ifobj_rx->umem, sizeof(struct xsk_umem_info));
> +}
> +
> +static void xsk_populate_fill_ring(struct xsk_umem_info *umem, struct pkt_stream *pkt_stream)
> +{
> +       u32 idx = 0, i, buffers_to_fill;
> +       int ret;
> +
> +       if (umem->num_frames < XSK_RING_PROD__DEFAULT_NUM_DESCS)
> +               buffers_to_fill = umem->num_frames;
> +       else
> +               buffers_to_fill = XSK_RING_PROD__DEFAULT_NUM_DESCS;
> +
> +       ret = xsk_ring_prod__reserve(&umem->fq, buffers_to_fill, &idx);
> +       if (ret != buffers_to_fill)
> +               exit_with_error(ENOSPC);
> +       for (i = 0; i < buffers_to_fill; i++) {
> +               u64 addr;
> +
> +               if (pkt_stream->use_addr_for_fill) {
> +                       struct pkt *pkt = pkt_stream_get_pkt(pkt_stream, i);
> +
> +                       if (!pkt)
> +                               break;
> +                       addr = pkt->addr;
> +               } else {
> +                       addr = i * umem->frame_size;
> +               }
> +
> +               *xsk_ring_prod__fill_addr(&umem->fq, idx++) = addr;
> +       }
> +       xsk_ring_prod__submit(&umem->fq, buffers_to_fill);
> +}
> +
>  static void thread_common_ops(struct test_spec *test, struct ifobject *ifobject)
>  {
>         u64 umem_sz = ifobject->umem->num_frames * ifobject->umem->frame_size;
>         int mmap_flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE;
>         int ret, ifindex;
>         void *bufs;
> -       u32 i;
>
>         ifobject->ns_fd = switch_namespace(ifobject->nsname);
>
>         if (ifobject->umem->unaligned_mode)
>                 mmap_flags |= MAP_HUGETLB;
>
> +       if (ifobject->shared_umem)
> +               umem_sz *= 2;
> +
>         bufs = mmap(NULL, umem_sz, PROT_READ | PROT_WRITE, mmap_flags, -1, 0);
>         if (bufs == MAP_FAILED)
>                 exit_with_error(errno);
> @@ -1122,24 +1191,9 @@ static void thread_common_ops(struct test_spec *test, struct ifobject *ifobject)
>         if (ret)
>                 exit_with_error(-ret);
>
> -       for (i = 0; i < test->nb_sockets; i++) {
> -               u32 ctr = 0;
> -
> -               while (ctr++ < SOCK_RECONF_CTR) {
> -                       ret = xsk_configure_socket(&ifobject->xsk_arr[i], ifobject->umem,
> -                                                  ifobject, !!i);
> -                       if (!ret)
> -                               break;
> -
> -                       /* Retry if it fails as xsk_socket__create() is asynchronous */
> -                       if (ctr >= SOCK_RECONF_CTR)
> -                               exit_with_error(-ret);
> -                       usleep(USLEEP_MAX);
> -               }
> +       xsk_populate_fill_ring(ifobject->umem, ifobject->pkt_stream);
>
> -               if (ifobject->busy_poll)
> -                       enable_busy_poll(&ifobject->xsk_arr[i]);
> -       }
> +       xsk_configure_socket(test, ifobject, ifobject->umem, false);
>
>         ifobject->xsk = &ifobject->xsk_arr[0];
>
> @@ -1159,22 +1213,18 @@ static void thread_common_ops(struct test_spec *test, struct ifobject *ifobject)
>                 exit_with_error(-ret);
>  }
>
> -static void testapp_cleanup_xsk_res(struct ifobject *ifobj)
> -{
> -       print_verbose("Destroying socket\n");
> -       xsk_socket__delete(ifobj->xsk->xsk);
> -       munmap(ifobj->umem->buffer, ifobj->umem->num_frames * ifobj->umem->frame_size);
> -       xsk_umem__delete(ifobj->umem->umem);
> -}
> -
>  static void *worker_testapp_validate_tx(void *arg)
>  {
>         struct test_spec *test = (struct test_spec *)arg;
>         struct ifobject *ifobject = test->ifobj_tx;
>         int err;
>
> -       if (test->current_step == 1)
> -               thread_common_ops(test, ifobject);
> +       if (test->current_step == 1) {
> +               if (!ifobject->shared_umem)
> +                       thread_common_ops(test, ifobject);
> +               else
> +                       thread_common_ops_tx(test, ifobject);
> +       }
>
>         print_verbose("Sending %d packets on interface %s\n", ifobject->pkt_stream->nb_pkts,
>                       ifobject->ifname);
> @@ -1185,53 +1235,23 @@ static void *worker_testapp_validate_tx(void *arg)
>         if (err)
>                 report_failure(test);
>
> -       if (test->total_steps == test->current_step || err)
> -               testapp_cleanup_xsk_res(ifobject);
>         pthread_exit(NULL);
>  }
>
> -static void xsk_populate_fill_ring(struct xsk_umem_info *umem, struct pkt_stream *pkt_stream)
> -{
> -       u32 idx = 0, i, buffers_to_fill;
> -       int ret;
> -
> -       if (umem->num_frames < XSK_RING_PROD__DEFAULT_NUM_DESCS)
> -               buffers_to_fill = umem->num_frames;
> -       else
> -               buffers_to_fill = XSK_RING_PROD__DEFAULT_NUM_DESCS;
> -
> -       ret = xsk_ring_prod__reserve(&umem->fq, buffers_to_fill, &idx);
> -       if (ret != buffers_to_fill)
> -               exit_with_error(ENOSPC);
> -       for (i = 0; i < buffers_to_fill; i++) {
> -               u64 addr;
> -
> -               if (pkt_stream->use_addr_for_fill) {
> -                       struct pkt *pkt = pkt_stream_get_pkt(pkt_stream, i);
> -
> -                       if (!pkt)
> -                               break;
> -                       addr = pkt->addr;
> -               } else {
> -                       addr = i * umem->frame_size;
> -               }
> -
> -               *xsk_ring_prod__fill_addr(&umem->fq, idx++) = addr;
> -       }
> -       xsk_ring_prod__submit(&umem->fq, buffers_to_fill);
> -}
> -
>  static void *worker_testapp_validate_rx(void *arg)
>  {
>         struct test_spec *test = (struct test_spec *)arg;
>         struct ifobject *ifobject = test->ifobj_rx;
>         struct pollfd fds = { };
> +       int id = 0;
>         int err;
>
> -       if (test->current_step == 1)
> +       if (test->current_step == 1) {
>                 thread_common_ops(test, ifobject);
> -
> -       xsk_populate_fill_ring(ifobject->umem, ifobject->pkt_stream);
> +       } else {
> +               bpf_map_delete_elem(ifobject->xsk_map_fd, &id);
> +               xsk_socket__update_xskmap(ifobject->xsk->xsk, ifobject->xsk_map_fd);
> +       }
>
>         fds.fd = xsk_socket__fd(ifobject->xsk->xsk);
>         fds.events = POLLIN;
> @@ -1249,11 +1269,20 @@ static void *worker_testapp_validate_rx(void *arg)
>                 pthread_mutex_unlock(&pacing_mutex);
>         }
>
> -       if (test->total_steps == test->current_step || err)
> -               testapp_cleanup_xsk_res(ifobject);
>         pthread_exit(NULL);
>  }
>
> +static void testapp_clean_xsk_umem(struct ifobject *ifobj)
> +{
> +       u64 umem_sz = ifobj->umem->num_frames * ifobj->umem->frame_size;
> +
> +       if (ifobj->shared_umem)
> +               umem_sz *= 2;
> +
> +       xsk_umem__delete(ifobj->umem->umem);
> +       munmap(ifobj->umem->buffer, umem_sz);
> +}
> +
>  static int testapp_validate_traffic(struct test_spec *test)
>  {
>         struct ifobject *ifobj_tx = test->ifobj_tx;
> @@ -1280,6 +1309,14 @@ static int testapp_validate_traffic(struct test_spec *test)
>         pthread_join(t1, NULL);
>         pthread_join(t0, NULL);
>
> +       if (test->total_steps == test->current_step || test->fail) {
> +               xsk_socket__delete(ifobj_tx->xsk->xsk);
> +               xsk_socket__delete(ifobj_rx->xsk->xsk);
> +               testapp_clean_xsk_umem(ifobj_rx);
> +               if (!ifobj_tx->shared_umem)
> +                       testapp_clean_xsk_umem(ifobj_tx);
> +       }
> +
>         return !!test->fail;
>  }
>
> @@ -1359,9 +1396,9 @@ static void testapp_headroom(struct test_spec *test)
>  static void testapp_stats_rx_dropped(struct test_spec *test)
>  {
>         test_spec_set_name(test, "STAT_RX_DROPPED");
> +       pkt_stream_replace_half(test, MIN_PKT_SIZE * 4, 0);
>         test->ifobj_rx->umem->frame_headroom = test->ifobj_rx->umem->frame_size -
>                 XDP_PACKET_HEADROOM - MIN_PKT_SIZE * 3;
> -       pkt_stream_replace_half(test, MIN_PKT_SIZE * 4, 0);
>         pkt_stream_receive_half(test);
>         test->ifobj_rx->validation_func = validate_rx_dropped;
>         testapp_validate_traffic(test);
> @@ -1484,6 +1521,11 @@ static void testapp_invalid_desc(struct test_spec *test)
>                 pkts[7].valid = false;
>         }
>
> +       if (test->ifobj_tx->shared_umem) {
> +               pkts[4].addr += UMEM_SIZE;
> +               pkts[5].addr += UMEM_SIZE;
> +       }
> +
>         pkt_stream_generate_custom(test, pkts, ARRAY_SIZE(pkts));
>         testapp_validate_traffic(test);
>         pkt_stream_restore_default(test);
> @@ -1624,7 +1666,6 @@ static void ifobject_delete(struct ifobject *ifobj)
>  {
>         if (ifobj->ns_fd != -1)
>                 close(ifobj->ns_fd);
> -       free(ifobj->umem);
>         free(ifobj->xsk_arr);
>         free(ifobj);
>  }
> @@ -1665,6 +1706,7 @@ int main(int argc, char **argv)
>         int modes = TEST_MODE_SKB + 1;
>         u32 i, j, failed_tests = 0;
>         struct test_spec test;
> +       bool shared_umem;
>
>         /* Use libbpf 1.0 API mode */
>         libbpf_set_strict_mode(LIBBPF_STRICT_ALL);
> @@ -1679,6 +1721,10 @@ int main(int argc, char **argv)
>         setlocale(LC_ALL, "");
>
>         parse_command_line(ifobj_tx, ifobj_rx, argc, argv);
> +       shared_umem = !strcmp(ifobj_tx->ifname, ifobj_rx->ifname);
> +
> +       ifobj_tx->shared_umem = shared_umem;
> +       ifobj_rx->shared_umem = shared_umem;
>
>         if (!validate_interface(ifobj_tx) || !validate_interface(ifobj_rx)) {
>                 usage(basename(argv[0]));
> @@ -1715,6 +1761,9 @@ int main(int argc, char **argv)
>
>         pkt_stream_delete(tx_pkt_stream_default);
>         pkt_stream_delete(rx_pkt_stream_default);
> +       free(ifobj_rx->umem);
> +       if (!ifobj_tx->shared_umem)
> +               free(ifobj_tx->umem);
>         ifobject_delete(ifobj_tx);
>         ifobject_delete(ifobj_rx);
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
> index ccfc829b2e5e..b7aa6c7cf2be 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.h
> +++ b/tools/testing/selftests/bpf/xdpxceiver.h
> @@ -149,6 +149,7 @@ struct ifobject {
>         bool busy_poll;
>         bool use_fill_ring;
>         bool release_rx;
> +       bool shared_umem;
>         u8 dst_mac[ETH_ALEN];
>         u8 src_mac[ETH_ALEN];
>  };
> --
> 2.27.0
>

Re: [PATCH bpf-next 08/10] selftests: xsk: rely on pkts_in_flight in wait_for_tx_completion()

[Magnus Karlsson]

On Fri, Jun 10, 2022 at 5:40 PM Maciej Fijalkowski
<maciej.fijalkowski@intel.com> wrote:
>
> Some of the drivers that implement support for AF_XDP Zero Copy (like
> ice) can have lazy approach for cleaning Tx descriptors. For ZC, when
> descriptor is cleaned, it is placed onto AF_XDP completion queue. This
> means that current implementation of wait_for_tx_completion() in
> xdpxceiver can get onto infinite loop, as some of the descriptors can
> never reach CQ.
>
> This function can be changed to rely on pkts_in_flight instead.

Acked-by: Magnus Karlsson <magnus.karlsson@intel.com>

> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> ---
>  tools/testing/selftests/bpf/xdpxceiver.c | 3 ++-
>  tools/testing/selftests/bpf/xdpxceiver.h | 2 +-
>  2 files changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
> index 316f1dd338fc..c9385690af09 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.c
> +++ b/tools/testing/selftests/bpf/xdpxceiver.c
> @@ -965,7 +965,7 @@ static int __send_pkts(struct ifobject *ifobject, u32 *pkt_nb)
>
>  static void wait_for_tx_completion(struct xsk_socket_info *xsk)
>  {
> -       while (xsk->outstanding_tx)
> +       while (pkts_in_flight)
>                 complete_pkts(xsk, BATCH_SIZE);
>  }
>
> @@ -1269,6 +1269,7 @@ static void *worker_testapp_validate_rx(void *arg)
>                 pthread_mutex_unlock(&pacing_mutex);
>         }
>
> +       pkts_in_flight = 0;
>         pthread_exit(NULL);
>  }
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
> index b7aa6c7cf2be..f364a92675f8 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.h
> +++ b/tools/testing/selftests/bpf/xdpxceiver.h
> @@ -170,6 +170,6 @@ pthread_barrier_t barr;
>  pthread_mutex_t pacing_mutex = PTHREAD_MUTEX_INITIALIZER;
>  pthread_cond_t pacing_cond = PTHREAD_COND_INITIALIZER;
>
> -int pkts_in_flight;
> +volatile int pkts_in_flight;
>
>  #endif                         /* XDPXCEIVER_H */
> --
> 2.27.0
>

Re: [PATCH bpf-next 09/10] selftests: xsk: remove struct xsk_socket_info::outstanding_tx

[Magnus Karlsson]

On Fri, Jun 10, 2022 at 5:31 PM Maciej Fijalkowski
<maciej.fijalkowski@intel.com> wrote:
>
> Previous change makes xsk->outstanding_tx a dead code, so let's remove
> it.

Acked-by: Magnus Karlsson <magnus.karlsson@intel.com>

> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> ---
>  tools/testing/selftests/bpf/xdpxceiver.c | 20 +++-----------------
>  tools/testing/selftests/bpf/xdpxceiver.h |  1 -
>  2 files changed, 3 insertions(+), 18 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
> index c9385690af09..a2aa652d0bb8 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.c
> +++ b/tools/testing/selftests/bpf/xdpxceiver.c
> @@ -815,7 +815,7 @@ static void kick_rx(struct xsk_socket_info *xsk)
>                 exit_with_error(errno);
>  }
>
> -static int complete_pkts(struct xsk_socket_info *xsk, int batch_size)
> +static void complete_pkts(struct xsk_socket_info *xsk, int batch_size)
>  {
>         unsigned int rcvd;
>         u32 idx;
> @@ -824,20 +824,8 @@ static int complete_pkts(struct xsk_socket_info *xsk, int batch_size)
>                 kick_tx(xsk);
>
>         rcvd = xsk_ring_cons__peek(&xsk->umem->cq, batch_size, &idx);
> -       if (rcvd) {
> -               if (rcvd > xsk->outstanding_tx) {
> -                       u64 addr = *xsk_ring_cons__comp_addr(&xsk->umem->cq, idx + rcvd - 1);
> -
> -                       ksft_print_msg("[%s] Too many packets completed\n", __func__);
> -                       ksft_print_msg("Last completion address: %llx\n", addr);
> -                       return TEST_FAILURE;
> -               }
> -
> +       if (rcvd)
>                 xsk_ring_cons__release(&xsk->umem->cq, rcvd);
> -               xsk->outstanding_tx -= rcvd;
> -       }
> -
> -       return TEST_PASS;
>  }
>
>  static int receive_pkts(struct ifobject *ifobj, struct pollfd *fds)
> @@ -955,9 +943,7 @@ static int __send_pkts(struct ifobject *ifobject, u32 *pkt_nb)
>         pthread_mutex_unlock(&pacing_mutex);
>
>         xsk_ring_prod__submit(&xsk->tx, i);
> -       xsk->outstanding_tx += valid_pkts;
> -       if (complete_pkts(xsk, i))
> -               return TEST_FAILURE;
> +       complete_pkts(xsk, i);
>
>         usleep(10);
>         return TEST_PASS;
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
> index f364a92675f8..12b792004163 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.h
> +++ b/tools/testing/selftests/bpf/xdpxceiver.h
> @@ -104,7 +104,6 @@ struct xsk_socket_info {
>         struct xsk_ring_prod tx;
>         struct xsk_umem_info *umem;
>         struct xsk_socket *xsk;
> -       u32 outstanding_tx;
>         u32 rxqsize;
>  };
>
> --
> 2.27.0
>

Re: [PATCH bpf-next 10/10] selftests: xsk: add support for zero copy testing

[Magnus Karlsson]

On Fri, Jun 10, 2022 at 5:20 PM Maciej Fijalkowski
<maciej.fijalkowski@intel.com> wrote:
>
> Introduce new mode to xdpxceiver responsible for testing AF_XDP zero
> copy support of driver that serves underlying physical device. When
> setting up test suite, determine whether driver has ZC support or not by
> trying to bind XSK ZC socket to the interface. If it succeeded,
> interpret it as ZC support being in place and do softirq and busy poll
> tests for zero copy mode.
>
> Note that Rx dropped tests are skipped since ZC path is not touching
> rx_dropped stat at all.

Great to be able to test zero-copy drivers. Thank you! Some small
comments below.

> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> ---
>  tools/testing/selftests/bpf/xdpxceiver.c | 80 ++++++++++++++++++++++--
>  tools/testing/selftests/bpf/xdpxceiver.h |  2 +
>  2 files changed, 77 insertions(+), 5 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.c b/tools/testing/selftests/bpf/xdpxceiver.c
> index a2aa652d0bb8..beef8d694fa6 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.c
> +++ b/tools/testing/selftests/bpf/xdpxceiver.c
> @@ -124,9 +124,20 @@ static void __exit_with_error(int error, const char *file, const char *func, int
>  }
>
>  #define exit_with_error(error) __exit_with_error(error, __FILE__, __func__, __LINE__)
> -
> -#define mode_string(test) (test)->ifobj_tx->xdp_flags & XDP_FLAGS_SKB_MODE ? "SKB" : "DRV"
>  #define busy_poll_string(test) (test)->ifobj_tx->busy_poll ? "BUSY-POLL " : ""
> +static char *mode_string(struct test_spec *test)
> +{
> +       switch (test->mode) {
> +       case TEST_MODE_SKB:
> +               return "SKB";
> +       case TEST_MODE_DRV:
> +               return "DRV";
> +       case TEST_MODE_ZC:
> +               return "ZC";
> +       default:
> +               return "BOGUS";
> +       }
> +}
>
>  static void report_failure(struct test_spec *test)
>  {
> @@ -317,6 +328,53 @@ static int __xsk_configure_socket(struct xsk_socket_info *xsk, struct xsk_umem_i
>         return xsk_socket__create(&xsk->xsk, ifobject->ifname, 0, umem->umem, rxr, txr, &cfg);
>  }
>
> +static bool ifobj_zc_avail(struct ifobject *ifobject)
> +{
> +       size_t umem_sz = DEFAULT_UMEM_BUFFERS * XSK_UMEM__DEFAULT_FRAME_SIZE;
> +       int mmap_flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE;
> +       struct xsk_socket_info *xsk;
> +       struct xsk_umem_info *umem;
> +       bool zc_avail = false;
> +       void *bufs;
> +       int ret;
> +
> +       bufs = mmap(NULL, umem_sz, PROT_READ | PROT_WRITE, mmap_flags, -1, 0);
> +       if (bufs == MAP_FAILED)
> +               exit_with_error(errno);
> +
> +       umem = calloc(1, sizeof(struct xsk_umem_info));
> +       if (!umem) {
> +               munmap(bufs, umem_sz);
> +               exit_with_error(-ENOMEM);
> +       }
> +       umem->frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE;
> +       ret = xsk_configure_umem(umem, bufs, umem_sz);
> +       if (ret)
> +               exit_with_error(-ret);
> +
> +       xsk = calloc(1, sizeof(struct xsk_socket_info));
> +       if (!xsk)
> +               goto out;
> +       ifobject->xdp_flags = 0;

This is redundant.

> +       ifobject->xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST;
> +       ifobject->xdp_flags |= XDP_FLAGS_DRV_MODE;
> +       ifobject->bind_flags = XDP_USE_NEED_WAKEUP | XDP_ZEROCOPY;
> +       ifobject->rx_on = true;
> +       xsk->rxqsize = XSK_RING_CONS__DEFAULT_NUM_DESCS;
> +       ret = __xsk_configure_socket(xsk, umem, ifobject, false);
> +       if (!ret)
> +               zc_avail = true;
> +
> +       ifobject->xdp_flags = 0;

Why this assignment?

> +       xsk_socket__delete(xsk->xsk);
> +       free(xsk);
> +out:
> +       munmap(umem->buffer, umem_sz);
> +       xsk_umem__delete(umem->umem);
> +       free(umem);
> +       return zc_avail;
> +}
> +
>  static struct option long_options[] = {
>         {"interface", required_argument, 0, 'i'},
>         {"busy-poll", no_argument, 0, 'b'},
> @@ -483,9 +541,14 @@ static void test_spec_init(struct test_spec *test, struct ifobject *ifobj_tx,
>                 else
>                         ifobj->xdp_flags |= XDP_FLAGS_DRV_MODE;
>
> -               ifobj->bind_flags = XDP_USE_NEED_WAKEUP | XDP_COPY;
> +               ifobj->bind_flags = XDP_USE_NEED_WAKEUP;
> +               if (mode == TEST_MODE_ZC)
> +                       ifobj->bind_flags |= XDP_ZEROCOPY;
> +               else
> +                       ifobj->bind_flags |= XDP_COPY;
>         }
>
> +       test->mode = mode;
>         __test_spec_init(test, ifobj_tx, ifobj_rx);
>  }
>
> @@ -1543,6 +1606,10 @@ static void run_pkt_test(struct test_spec *test, enum test_mode mode, enum test_
>  {
>         switch (type) {
>         case TEST_TYPE_STATS_RX_DROPPED:
> +               if (mode == TEST_MODE_ZC) {
> +                       ksft_test_result_skip("Can not run RX_DROPPED test for ZC mode\n");
> +                       return;
> +               }
>                 testapp_stats_rx_dropped(test);
>                 break;
>         case TEST_TYPE_STATS_TX_INVALID_DESCS:
> @@ -1723,8 +1790,11 @@ int main(int argc, char **argv)
>         init_iface(ifobj_rx, MAC2, MAC1, IP2, IP1, UDP_PORT2, UDP_PORT1,
>                    worker_testapp_validate_rx);
>
> -       if (is_xdp_supported(ifobj_tx))
> -               modes++;
> +       if (is_xdp_supported(ifobj_tx)) {
> +               modes++;
> +               if (ifobj_zc_avail(ifobj_tx))
> +                       modes++;
> +       }
>
>         test_spec_init(&test, ifobj_tx, ifobj_rx, 0);
>         tx_pkt_stream_default = pkt_stream_generate(ifobj_tx->umem, DEFAULT_PKT_CNT, PKT_SIZE);
> diff --git a/tools/testing/selftests/bpf/xdpxceiver.h b/tools/testing/selftests/bpf/xdpxceiver.h
> index 12b792004163..a86331c6b0c5 100644
> --- a/tools/testing/selftests/bpf/xdpxceiver.h
> +++ b/tools/testing/selftests/bpf/xdpxceiver.h
> @@ -61,6 +61,7 @@
>  enum test_mode {
>         TEST_MODE_SKB,
>         TEST_MODE_DRV,
> +       TEST_MODE_ZC,
>         TEST_MODE_MAX
>  };
>
> @@ -162,6 +163,7 @@ struct test_spec {
>         u16 current_step;
>         u16 nb_sockets;
>         bool fail;
> +       enum test_mode mode;
>         char name[MAX_TEST_NAME_SIZE];
>  };
>
> --
> 2.27.0
>