[PATCH bpf 0/3] XSK related fixes

[PATCH bpf 0/3] XSK related fixes

[Maciej Fijalkowski]

Cameron reported [0] that on fresh bpf-next he could not run multiple
xdpsock instances in Tx-only mode on single network interface with i40e
driver.

Turns out that Maxim's series [1] which was adding RCU protection around
ndo_xsk_wakeup added check against the __I40E_CONFIG_BUSY being set on
pf->state within i40e_xsk_wakeup() - if it's set, return -ENETDOWN.
Since this bit is set per PF when UMEM is being enabled/disabled, the
situation Cameron stumbled upon was that when he launched second xdpsock
instance, second UMEM was being registered, hence set __I40E_CONFIG_BUSY
which is now observed by first xdpsock and therefore xdpsock's kick_tx()
gets -ENETDOWN as errno.

-ENETDOWN currently is not allowed in kick_tx(), so we were exiting the
first application. Such exit means also XDP program being unloaded and
its dedicated resources, which caused an -ENXIO being return in the
second xdpsock instance.

Let's fix the issue from both sides - protect ourselves from future
xdpsock crashes by allowing for -ENETDOWN errno being set in kick_tx()
(patch 3) and from driver side, return -EAGAIN for the case where PF is
busy (patch 1).

Remove also doubled variable from xdpsock_user.c (patch 2).

Note that ixgbe seems not to be affected since UMEM registration sets
the busy/disable bit per ring, not per PF.

Thanks!
Maciej

[0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
[1]: https://lore.kernel.org/netdev/20191217162023.16011-1-maximmi@mellanox.com/

Maciej Fijalkowski (3):
  i40e: Relax i40e_xsk_wakeup's return value when PF is busy
  samples: bpf: drop doubled variable declaration in xdpsock
  samples: bpf: allow for -ENETDOWN in xdpsock

 drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +-
 samples/bpf/xdpsock_user.c                 | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

-- 
2.20.1

[PATCH bpf 1/3] i40e: Relax i40e_xsk_wakeup's return value when PF is busy

[Maciej Fijalkowski]

Return -EAGAIN instead of -ENETDOWN to provide a slightly milder
information to user space so that an application will know to retry the
syscall when __I40E_CONFIG_BUSY bit is set on pf->state.

Fixes: b3873a5be757 ("net/i40e: Fix concurrency issues between config flow and XSK")
Acked-by: Björn Töpel <bjorn.topel@intel.com>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
index 42058fad6..0b7d29192 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
@@ -791,7 +791,7 @@ int i40e_xsk_wakeup(struct net_device *dev, u32 queue_id, u32 flags)
 	struct i40e_ring *ring;
 
 	if (test_bit(__I40E_CONFIG_BUSY, pf->state))
-		return -ENETDOWN;
+		return -EAGAIN;
 
 	if (test_bit(__I40E_VSI_DOWN, vsi->state))
 		return -ENETDOWN;
-- 
2.20.1

[PATCH bpf 2/3] samples: bpf: drop doubled variable declaration in xdpsock

[Maciej Fijalkowski]

Seems that by accident there is a doubled declaration of global variable
opt_xdp_bind_flags in xdpsock_user.c. The second one is uninitialized so
compiler was simply ignoring it.

To keep things clean, drop the doubled variable.

Fixes: c543f5469822 ("samples/bpf: add unaligned chunks mode support to xdpsock")
Acked-by: Björn Töpel <bjorn.topel@intel.com>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 samples/bpf/xdpsock_user.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
index 0b5acd722..bab7a850e 100644
--- a/samples/bpf/xdpsock_user.c
+++ b/samples/bpf/xdpsock_user.c
@@ -83,7 +83,6 @@ static u32 opt_xdp_bind_flags = XDP_USE_NEED_WAKEUP;
 static u32 opt_umem_flags;
 static int opt_unaligned_chunks;
 static int opt_mmap_flags;
-static u32 opt_xdp_bind_flags;
 static int opt_xsk_frame_size = XSK_UMEM__DEFAULT_FRAME_SIZE;
 static int opt_timeout = 1000;
 static bool opt_need_wakeup = true;
-- 
2.20.1

[PATCH bpf 3/3] samples: bpf: allow for -ENETDOWN in xdpsock

[Maciej Fijalkowski]

ndo_xsk_wakeup() can return -ENETDOWN and there's no particular reason
to bail the whole application out on that case. Let's check in kick_tx()
whether errno was set to mentioned value and basically allow application
to further process frames.

Fixes: 248c7f9c0e21 ("samples/bpf: convert xdpsock to use libbpf for AF_XDP access")
Reported-by: Cameron Elliott <cameron@cameronelliott.com>
Acked-by: Björn Töpel <bjorn.topel@intel.com>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
---
 samples/bpf/xdpsock_user.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c
index bab7a850e..c91e91362 100644
--- a/samples/bpf/xdpsock_user.c
+++ b/samples/bpf/xdpsock_user.c
@@ -788,7 +788,8 @@ static void kick_tx(struct xsk_socket_info *xsk)
 	int ret;
 
 	ret = sendto(xsk_socket__fd(xsk->xsk), NULL, 0, MSG_DONTWAIT, NULL, 0);
-	if (ret >= 0 || errno == ENOBUFS || errno == EAGAIN || errno == EBUSY)
+	if (ret >= 0 || errno == ENOBUFS || errno == EAGAIN ||
+	    errno == EBUSY || errno == ENETDOWN)
 		return;
 	exit_with_error(errno);
 }
-- 
2.20.1

Re: [PATCH bpf 0/3] XSK related fixes

[Daniel Borkmann]

On Wed, Feb 05, 2020 at 05:58:31AM +0100, Maciej Fijalkowski wrote:
> Cameron reported [0] that on fresh bpf-next he could not run multiple
> xdpsock instances in Tx-only mode on single network interface with i40e
> driver.
> 
> Turns out that Maxim's series [1] which was adding RCU protection around
> ndo_xsk_wakeup added check against the __I40E_CONFIG_BUSY being set on
> pf->state within i40e_xsk_wakeup() - if it's set, return -ENETDOWN.
> Since this bit is set per PF when UMEM is being enabled/disabled, the
> situation Cameron stumbled upon was that when he launched second xdpsock
> instance, second UMEM was being registered, hence set __I40E_CONFIG_BUSY
> which is now observed by first xdpsock and therefore xdpsock's kick_tx()
> gets -ENETDOWN as errno.
> 
> -ENETDOWN currently is not allowed in kick_tx(), so we were exiting the
> first application. Such exit means also XDP program being unloaded and
> its dedicated resources, which caused an -ENXIO being return in the
> second xdpsock instance.
> 
> Let's fix the issue from both sides - protect ourselves from future
> xdpsock crashes by allowing for -ENETDOWN errno being set in kick_tx()
> (patch 3) and from driver side, return -EAGAIN for the case where PF is
> busy (patch 1).
> 
> Remove also doubled variable from xdpsock_user.c (patch 2).
> 
> Note that ixgbe seems not to be affected since UMEM registration sets
> the busy/disable bit per ring, not per PF.
> 
> Thanks!
> Maciej
> 
> [0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
> [1]: https://lore.kernel.org/netdev/20191217162023.16011-1-maximmi@mellanox.com/

Applied, thanks!

Re: [PATCH bpf 0/3] XSK related fixes

[Maxim Mikityanskiy]

On 2020-02-05 06:58, Maciej Fijalkowski wrote:
> Cameron reported [0] that on fresh bpf-next he could not run multiple
> xdpsock instances in Tx-only mode on single network interface with i40e
> driver.
> 
> Turns out that Maxim's series [1] which was adding RCU protection around
> ndo_xsk_wakeup added check against the __I40E_CONFIG_BUSY being set on
> pf->state within i40e_xsk_wakeup() - if it's set, return -ENETDOWN.
> Since this bit is set per PF when UMEM is being enabled/disabled, the
> situation Cameron stumbled upon was that when he launched second xdpsock
> instance, second UMEM was being registered, hence set __I40E_CONFIG_BUSY
> which is now observed by first xdpsock and therefore xdpsock's kick_tx()
> gets -ENETDOWN as errno.
> 
> -ENETDOWN currently is not allowed in kick_tx(), so we were exiting the
> first application. Such exit means also XDP program being unloaded and
> its dedicated resources, which caused an -ENXIO being return in the
> second xdpsock instance.
> 
> Let's fix the issue from both sides - protect ourselves from future
> xdpsock crashes by allowing for -ENETDOWN errno being set in kick_tx()
> (patch 3) and from driver side, return -EAGAIN for the case where PF is
> busy (patch 1).
> 
> Remove also doubled variable from xdpsock_user.c (patch 2).
> 
> Note that ixgbe seems not to be affected since UMEM registration sets
> the busy/disable bit per ring, not per PF.
> 
> Thanks!
> Maciej
> 
> [0]: https://www.spinics.net/lists/xdp-newbies/msg01558.html
> [1]: https://lore.kernel.org/netdev/20191217162023.16011-1-maximmi@mellanox.com/
> 
> Maciej Fijalkowski (3):
>    i40e: Relax i40e_xsk_wakeup's return value when PF is busy
>    samples: bpf: drop doubled variable declaration in xdpsock
>    samples: bpf: allow for -ENETDOWN in xdpsock
> 
>   drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +-
>   samples/bpf/xdpsock_user.c                 | 4 ++--
>   2 files changed, 3 insertions(+), 3 deletions(-)
> 

Acked-by: Maxim Mikityanskiy <maximmi@mellanox.com>

Though it's already merged (that was too fast).