[PATCH net-next 00/16] mlx5 xsk updates part3 2022-09-30

[Saeed Mahameed <saeed@kernel.org>]

From: Saeed Mahameed <saeedm@nvidia.com>

The gist of this 4 part series is in this patchset's last patch

This series contains performance optimizations. XSK starts using the
batching allocator, and XSK data path gets separated from the regular
RX, allowing to drop some branches not relevant for non-XSK use cases.
Some minor optimizations for indirect calls and need_wakeup are also
included.

Other than that, this series adds a few features to the mlx5e
implementation of XSK:

1. XDP metadata support on XSK RQs.

2. RSS contexts support for XSK RQs.

3. Some other optimizations 

4. Last but not least, change the queuing scheme, so that XSK RQs no longer
use higher indices, but replace the regular RQs.

Maxim Says:
==========

In the initial implementation of XSK in mlx5e, XSK RQs coexisted with
regular RQs in the same channel. The main idea was to allow RSS work the
same for regular traffic, without need to reconfigure RSS to exclude XSK
queues.

However, this scheme didn't prove to be beneficial, mainly because of
incompatibility with other vendors. Some tools don't properly support
using higher indices for XSK queues, some tools get confused with the
double amount of RQs exposed in sysfs. Some use cases are purely XSK,
and allocating the same amount of unused regular RQs is a waste of
resources.

This commit changes the queuing scheme to the standard one, where XSK
RQs replace regular RQs on the channels where XSK sockets are open. Two
RQs still exist in the channel to allow failsafe disable of XSK, but
only one is exposed at a time. The next commit will achieve the desired
memory save by flushing the buffers when the regular RQ is unused.

As the result of this transition:

1. It's possible to use RSS contexts over XSK RQs.

2. It's possible to dedicate all queues to XSK.

3. When XSK RQs coexist with regular RQs, the admin should make sure no
unwanted traffic goes into XSK RQs by either excluding them from RSS or
settings up the XDP program to return XDP_PASS for non-XSK traffic.

4. When using a mixed fleet of mlx5e devices and other netdevs, the same
configuration can be applied. If the application supports the fallback
to copy mode on unsupported drivers, it will work too.

==========

Part 4 will include some final xsk optimizations and minor improvements

part 1: https://lore.kernel.org/netdev/20220927203611.244301-1-saeed@kernel.org/
part 2: https://lore.kernel.org/netdev/20220929072156.93299-1-saeed@kernel.org/

Maxim Mikityanskiy (16):
  net/mlx5e: xsk: Use mlx5e_trigger_napi_icosq for XSK wakeup
  net/mlx5e: xsk: Drop the check for XSK state in mlx5e_xsk_wakeup
  net/mlx5e: Introduce wqe_index_mask for legacy RQ
  net/mlx5e: Make the wqe_index_mask calculation more exact
  net/mlx5e: Use partial batches in legacy RQ
  net/mlx5e: xsk: Use partial batches in legacy RQ with XSK
  net/mlx5e: Remove the outer loop when allocating legacy RQ WQEs
  net/mlx5e: xsk: Split out WQE allocation for legacy XSK RQ
  net/mlx5e: xsk: Use xsk_buff_alloc_batch on legacy RQ
  net/mlx5e: xsk: Use xsk_buff_alloc_batch on striding RQ
  net/mlx5e: Use non-XSK page allocator in SHAMPO
  net/mlx5e: Call mlx5e_page_release_dynamic directly where possible
  net/mlx5e: Optimize RQ page deallocation
  net/mlx5e: xsk: Support XDP metadata on XSK RQs
  net/mlx5e: Introduce the mlx5e_flush_rq function
  net/mlx5e: xsk: Use queue indices starting from 0 for XSK queues

 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  10 +-
 .../ethernet/mellanox/mlx5/core/en/channels.c |  29 ++-
 .../ethernet/mellanox/mlx5/core/en/channels.h |   3 +-
 .../ethernet/mellanox/mlx5/core/en/params.c   |  44 +++-
 .../ethernet/mellanox/mlx5/core/en/params.h   |  32 ---
 .../mellanox/mlx5/core/en/reporter_rx.c       |  23 +-
 .../ethernet/mellanox/mlx5/core/en/rx_res.c   | 118 ++--------
 .../ethernet/mellanox/mlx5/core/en/rx_res.h   |   9 +-
 .../net/ethernet/mellanox/mlx5/core/en/txrx.h |   7 +
 .../ethernet/mellanox/mlx5/core/en/xsk/pool.c |  17 +-
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.c   | 176 ++++++++++++++-
 .../ethernet/mellanox/mlx5/core/en/xsk/rx.h   |   3 +
 .../mellanox/mlx5/core/en/xsk/setup.c         |   4 +-
 .../ethernet/mellanox/mlx5/core/en/xsk/tx.c   |  12 +-
 .../mellanox/mlx5/core/en_fs_ethtool.c        |  13 +-
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  52 +++--
 .../net/ethernet/mellanox/mlx5/core/en_rep.c  |   3 -
 .../net/ethernet/mellanox/mlx5/core/en_rx.c   | 211 +++++++-----------
 .../ethernet/mellanox/mlx5/core/ipoib/ipoib.c |   1 -
 .../mellanox/mlx5/core/ipoib/ipoib_vlan.c     |   1 -
 drivers/net/ethernet/mellanox/mlx5/core/wq.h  |   2 +-
 21 files changed, 385 insertions(+), 385 deletions(-)

-- 
2.37.3