[PATCH net-next 0/3] i40e, ice, ixgbe: optimize for XDP_REDIRECT in xsk path

[PATCH net-next 0/3] i40e, ice, ixgbe: optimize for XDP_REDIRECT in xsk path

[Magnus Karlsson]

Optimize run_xdp_zc() for the XDP program verdict being XDP_REDIRECT
in the zsk zero-copy path. This path is only used when having AF_XDP
zero-copy on and in that case most packets will be directed to user
space. This provides around 100k extra packets in throughput on my
server when running l2fwd in xdpsock.

Thanks: Magnus

Magnus Karlsson (3):
  i40: optimize for XDP_REDIRECT in xsk path
  ixgbe: optimize for XDP_REDIRECT in xsk path
  ice: optimize for XDP_REDIRECT in xsk path

 drivers/net/ethernet/intel/i40e/i40e_xsk.c   | 11 +++++++----
 drivers/net/ethernet/intel/ice/ice_xsk.c     | 12 ++++++++----
 drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 11 +++++++----
 3 files changed, 22 insertions(+), 12 deletions(-)


base-commit: 6b4f503186b73e3da24c6716c8c7ea903e6b74d4
--
2.29.0

[PATCH net-next 1/3] i40: optimize for XDP_REDIRECT in xsk path

[Magnus Karlsson]

From: Magnus Karlsson <magnus.karlsson@intel.com>

Optimize i40e_run_xdp_zc() for the XDP program verdict being
XDP_REDIRECT in the zsk zero-copy path. This path is only used when
having AF_XDP zero-copy on and in that case most packets will be
directed to user space. This provides a little over 100k extra packets
in throughput on my server when running l2fwd in xdpsock.

Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
---
 drivers/net/ethernet/intel/i40e/i40e_xsk.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
index 67febc7b6798..fbf76c67d77d 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
@@ -159,6 +159,13 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
 	xdp_prog = READ_ONCE(rx_ring->xdp_prog);
 	act = bpf_prog_run_xdp(xdp_prog, xdp);
 
+	if (likely(act == XDP_REDIRECT)) {
+		err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
+		result = !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED;
+		rcu_read_unlock();
+		return result;
+	}
+
 	switch (act) {
 	case XDP_PASS:
 		break;
@@ -166,10 +173,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp)
 		xdp_ring = rx_ring->vsi->xdp_rings[rx_ring->queue_index];
 		result = i40e_xmit_xdp_tx_ring(xdp, xdp_ring);
 		break;
-	case XDP_REDIRECT:
-		err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
-		result = !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED;
-		break;
 	default:
 		bpf_warn_invalid_xdp_action(act);
 		fallthrough;
-- 
2.29.0

[PATCH net-next 2/3] ixgbe: optimize for XDP_REDIRECT in xsk path

[Magnus Karlsson]

From: Magnus Karlsson <magnus.karlsson@intel.com>

Optimize ixgbe_run_xdp_zc() for the XDP program verdict being
XDP_REDIRECT in the zsk zero-copy path. This path is only used when
having AF_XDP zero-copy on and in that case most packets will be
directed to user space. This provides a little under 100k extra
packets in throughput on my server when running l2fwd in xdpsock.

Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
index 3771857cf887..91ad5b902673 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
@@ -104,6 +104,13 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
 	xdp_prog = READ_ONCE(rx_ring->xdp_prog);
 	act = bpf_prog_run_xdp(xdp_prog, xdp);
 
+	if (likely(act == XDP_REDIRECT)) {
+		err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
+		result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED;
+		rcu_read_unlock();
+		return result;
+	}
+
 	switch (act) {
 	case XDP_PASS:
 		break;
@@ -115,10 +122,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter,
 		}
 		result = ixgbe_xmit_xdp_ring(adapter, xdpf);
 		break;
-	case XDP_REDIRECT:
-		err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
-		result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED;
-		break;
 	default:
 		bpf_warn_invalid_xdp_action(act);
 		fallthrough;
-- 
2.29.0

[PATCH net-next 3/3] ice: optimize for XDP_REDIRECT in xsk path

[Magnus Karlsson]

From: Magnus Karlsson <magnus.karlsson@intel.com>

Optimize ice_run_xdp_zc() for the XDP program verdict being
XDP_REDIRECT in the zsk zero-copy path. This path is only used when
having AF_XDP zero-copy on and in that case most packets will be
directed to user space. This provides a little over 100k extra packets
in throughput on my server when running l2fwd in xdpsock.

Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
---
 drivers/net/ethernet/intel/ice/ice_xsk.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index 797886524054..3b180e52112f 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -521,6 +521,14 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
 	}
 
 	act = bpf_prog_run_xdp(xdp_prog, xdp);
+
+	if (likely(act == XDP_REDIRECT)) {
+		err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
+		result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED;
+		rcu_read_unlock();
+		return result;
+	}
+
 	switch (act) {
 	case XDP_PASS:
 		break;
@@ -528,10 +536,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp)
 		xdp_ring = rx_ring->vsi->xdp_rings[rx_ring->q_index];
 		result = ice_xmit_xdp_buff(xdp, xdp_ring);
 		break;
-	case XDP_REDIRECT:
-		err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
-		result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED;
-		break;
 	default:
 		bpf_warn_invalid_xdp_action(act);
 		fallthrough;
-- 
2.29.0

Re: [PATCH net-next 0/3] i40e, ice, ixgbe: optimize for XDP_REDIRECT in xsk path

[Maciej Fijalkowski]

On Wed, Dec 02, 2020 at 04:07:21PM +0100, Magnus Karlsson wrote:
> Optimize run_xdp_zc() for the XDP program verdict being XDP_REDIRECT
> in the zsk zero-copy path. This path is only used when having AF_XDP
> zero-copy on and in that case most packets will be directed to user
> space. This provides around 100k extra packets in throughput on my
> server when running l2fwd in xdpsock.
> 
> Thanks: Magnus

For series:
Acked-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>

You only ate 'e' from i40e subject line.

> 
> Magnus Karlsson (3):
>   i40: optimize for XDP_REDIRECT in xsk path
>   ixgbe: optimize for XDP_REDIRECT in xsk path
>   ice: optimize for XDP_REDIRECT in xsk path
> 
>  drivers/net/ethernet/intel/i40e/i40e_xsk.c   | 11 +++++++----
>  drivers/net/ethernet/intel/ice/ice_xsk.c     | 12 ++++++++----
>  drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 11 +++++++----
>  3 files changed, 22 insertions(+), 12 deletions(-)
> 
> 
> base-commit: 6b4f503186b73e3da24c6716c8c7ea903e6b74d4
> --
> 2.29.0

Re: [PATCH net-next 0/3] i40e, ice, ixgbe: optimize for XDP_REDIRECT in xsk path

[Magnus Karlsson]

On Wed, Dec 2, 2020 at 9:49 PM Maciej Fijalkowski
<maciej.fijalkowski@intel.com> wrote:
>
> On Wed, Dec 02, 2020 at 04:07:21PM +0100, Magnus Karlsson wrote:
> > Optimize run_xdp_zc() for the XDP program verdict being XDP_REDIRECT
> > in the zsk zero-copy path. This path is only used when having AF_XDP
> > zero-copy on and in that case most packets will be directed to user
> > space. This provides around 100k extra packets in throughput on my
> > server when running l2fwd in xdpsock.
> >
> > Thanks: Magnus
>
> For series:
> Acked-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
>
> You only ate 'e' from i40e subject line.

Sorry, you are right. Tony, would you please be so kind to add this
missing "e" in the commit message before you send the pull request?

Thanks: Magnus

> >
> > Magnus Karlsson (3):
> >   i40: optimize for XDP_REDIRECT in xsk path
> >   ixgbe: optimize for XDP_REDIRECT in xsk path
> >   ice: optimize for XDP_REDIRECT in xsk path
> >
> >  drivers/net/ethernet/intel/i40e/i40e_xsk.c   | 11 +++++++----
> >  drivers/net/ethernet/intel/ice/ice_xsk.c     | 12 ++++++++----
> >  drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 11 +++++++----
> >  3 files changed, 22 insertions(+), 12 deletions(-)
> >
> >
> > base-commit: 6b4f503186b73e3da24c6716c8c7ea903e6b74d4
> > --
> > 2.29.0

Re: [PATCH net-next 0/3] i40e, ice, ixgbe: optimize for XDP_REDIRECT in xsk path

[Nguyen, Anthony L]

On Thu, 2020-12-03 at 07:58 +0100, Magnus Karlsson wrote:
> On Wed, Dec 2, 2020 at 9:49 PM Maciej Fijalkowski
> <maciej.fijalkowski@intel.com> wrote:
> > 
> > On Wed, Dec 02, 2020 at 04:07:21PM +0100, Magnus Karlsson wrote:
> > > Optimize run_xdp_zc() for the XDP program verdict being
> > > XDP_REDIRECT
> > > in the zsk zero-copy path. This path is only used when having
> > > AF_XDP
> > > zero-copy on and in that case most packets will be directed to
> > > user
> > > space. This provides around 100k extra packets in throughput on
> > > my
> > > server when running l2fwd in xdpsock.
> > > 
> > > Thanks: Magnus
> > 
> > For series:
> > Acked-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
> > 
> > You only ate 'e' from i40e subject line.
> 
> Sorry, you are right. Tony, would you please be so kind to add this
> missing "e" in the commit message before you send the pull request?
> 
> Thanks: Magnus

I can do that.

Thanks,
Tony

> > > 
> > > Magnus Karlsson (3):
> > >   i40: optimize for XDP_REDIRECT in xsk path
> > >   ixgbe: optimize for XDP_REDIRECT in xsk path
> > >   ice: optimize for XDP_REDIRECT in xsk path
> > > 
> > >  drivers/net/ethernet/intel/i40e/i40e_xsk.c   | 11 +++++++----
> > >  drivers/net/ethernet/intel/ice/ice_xsk.c     | 12 ++++++++----
> > >  drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 11 +++++++----
> > >  3 files changed, 22 insertions(+), 12 deletions(-)
> > > 
> > > 
> > > base-commit: 6b4f503186b73e3da24c6716c8c7ea903e6b74d4
> > > --
> > > 2.29.0

RE: [Intel-wired-lan] [PATCH net-next 1/3] i40: optimize for XDP_REDIRECT in xsk path

[Kuruvinakunnel, George]

> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Magnus
> Karlsson
> Sent: Wednesday, December 2, 2020 8:37 PM
> To: Karlsson, Magnus <magnus.karlsson@intel.com>; Topel, Bjorn
> <bjorn.topel@intel.com>; intel-wired-lan@lists.osuosl.org; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>
> Cc: netdev@vger.kernel.org; maciejromanfijalkowski@gmail.com
> Subject: [Intel-wired-lan] [PATCH net-next 1/3] i40: optimize for XDP_REDIRECT
> in xsk path
> 
> From: Magnus Karlsson <magnus.karlsson@intel.com>
> 
> Optimize i40e_run_xdp_zc() for the XDP program verdict being XDP_REDIRECT
> in the zsk zero-copy path. This path is only used when having AF_XDP zero-copy
> on and in that case most packets will be directed to user space. This provides a
> little over 100k extra packets in throughput on my server when running l2fwd in
> xdpsock.
> 
> Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
> ---
>  drivers/net/ethernet/intel/i40e/i40e_xsk.c | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
> 

Tested-by: George Kuruvinakunnel <george.kuruvinakunnel@intel.com>

RE: [Intel-wired-lan] [PATCH net-next 3/3] ice: optimize for XDP_REDIRECT in xsk path

[Kuruvinakunnel, George]

> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Magnus
> Karlsson
> Sent: Wednesday, December 2, 2020 8:37 PM
> To: Karlsson, Magnus <magnus.karlsson@intel.com>; Topel, Bjorn
> <bjorn.topel@intel.com>; intel-wired-lan@lists.osuosl.org; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>
> Cc: netdev@vger.kernel.org; maciejromanfijalkowski@gmail.com
> Subject: [Intel-wired-lan] [PATCH net-next 3/3] ice: optimize for XDP_REDIRECT in
> xsk path
> 
> From: Magnus Karlsson <magnus.karlsson@intel.com>
> 
> Optimize ice_run_xdp_zc() for the XDP program verdict being XDP_REDIRECT in
> the zsk zero-copy path. This path is only used when having AF_XDP zero-copy on
> and in that case most packets will be directed to user space. This provides a little
> over 100k extra packets in throughput on my server when running l2fwd in
> xdpsock.
> 
> Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
> ---
>  drivers/net/ethernet/intel/ice/ice_xsk.c | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 

Tested-by: George Kuruvinakunnel <george.kuruvinakunnel@intel.com>

RE: [Intel-wired-lan] [PATCH net-next 2/3] ixgbe: optimize for XDP_REDIRECT in xsk path

[Jambekar, Vishakha]

> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of
> Magnus Karlsson
> Sent: Wednesday, December 2, 2020 8:37 PM
> To: Karlsson, Magnus <magnus.karlsson@intel.com>; Topel, Bjorn
> <bjorn.topel@intel.com>; intel-wired-lan@lists.osuosl.org; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>
> Cc: netdev@vger.kernel.org; maciejromanfijalkowski@gmail.com
> Subject: [Intel-wired-lan] [PATCH net-next 2/3] ixgbe: optimize for
> XDP_REDIRECT in xsk path
> 
> From: Magnus Karlsson <magnus.karlsson@intel.com>
> 
> Optimize ixgbe_run_xdp_zc() for the XDP program verdict being XDP_REDIRECT
> in the zsk zero-copy path. This path is only used when having AF_XDP zero-copy
> on and in that case most packets will be directed to user space. This provides a
> little under 100k extra packets in throughput on my server when running l2fwd in
> xdpsock.
> 
> Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
> ---
>  drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
>
Tested-by: Vishakha Jambekar <vishakha.jambekar@intel.com>