bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH bpf-next 0/4] xsk: increase NAPI budget for AF_XDP zero-copy path
@ 2020-09-07 15:02 Björn Töpel
  2020-09-07 15:02 ` [PATCH bpf-next 1/4] xsk: add XSK_NAPI_WEIGHT define Björn Töpel
                   ` (3 more replies)
  0 siblings, 4 replies; 14+ messages in thread
From: Björn Töpel @ 2020-09-07 15:02 UTC (permalink / raw)
  To: ast, daniel, netdev, bpf
  Cc: Björn Töpel, magnus.karlsson, bjorn.topel, kuba,
	intel-wired-lan

The NAPI budget (NAPI_POLL_WEIGHT), meaning the number of received
packets that are allowed to be processed for each NAPI invocation,
takes into consideration that each received packet is aimed for the
kernel networking stack.

That is not the case for the AF_XDP receive path, where the cost of
each packet is significantly less. Therefore, this commit adds a new
NAPI budget, which is the NAPI_POLL_WEIGHT scaled by 4. Typically that
would be 256 in most configuration. It is encouraged that AF_XDP
zero-copy capable drivers use the XSK_NAPI_WEIGHT, when zero-copy is
enabled.

Processing 256 packets targeted for AF_XDP is still less work than 64
(NAPI_POLL_WEIGHT) packets going to the kernel networking stack.

Jakub suggested in [1] that a more generic approach was preferred over
"driver hacks". It is arguable if adding the budget as a define which
is a scaled NAPI_POLL_WEIGHT would classify as "generic", but it is a
bit further away from "driver hacks". ;-)

The first patch adds the actual define, and last three make the Intel
drivers use it.

The AF_XDP Rx performance for "rxdrop" is up ~8% on my machine.

Routing this series via bpf-next instead of Intel Wired/netdev, since
it is a core AF_XDP addition, and hopefully Nvidia will pick this up
for the mlx5 driver.


Björn

[1] https://lore.kernel.org/netdev/20200728131512.17c41621@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com/

Björn Töpel (4):
  xsk: add XSK_NAPI_WEIGHT define
  i40e, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  ice, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget

 drivers/net/ethernet/intel/i40e/i40e_xsk.c   | 2 +-
 drivers/net/ethernet/intel/ice/ice_xsk.c     | 2 +-
 drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
 include/net/xdp_sock_drv.h                   | 3 +++
 4 files changed, 6 insertions(+), 3 deletions(-)


base-commit: bc0b5a03079bd78fb3d5cba1ccabf0a7efb1d99f
-- 
2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH bpf-next 1/4] xsk: add XSK_NAPI_WEIGHT define
  2020-09-07 15:02 [PATCH bpf-next 0/4] xsk: increase NAPI budget for AF_XDP zero-copy path Björn Töpel
@ 2020-09-07 15:02 ` Björn Töpel
  2020-09-07 15:02 ` [PATCH bpf-next 2/4] i40e, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget Björn Töpel
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 14+ messages in thread
From: Björn Töpel @ 2020-09-07 15:02 UTC (permalink / raw)
  To: ast, daniel, netdev, bpf
  Cc: Björn Töpel, magnus.karlsson, kuba, intel-wired-lan

From: Björn Töpel <bjorn.topel@intel.com>

The NAPI budget (NAPI_POLL_WEIGHT), meaning the number of received
packets that are allowed to be processed for each NAPI invocation,
takes into consideration that each received packet is aimed for the
kernel networking stack.

That is not the case for the AF_XDP receive path, where the cost of
each packet is significantly less. Therefore, this commit adds a new
NAPI budget, which is the NAPI_POLL_WEIGHT scaled by 4. Typically that
would be 256 in most configuration. It is encouraged that AF_XDP
zero-copy capable drivers use the XSK_NAPI_WEIGHT, when zero-copy is
enabled.

Processing 256 packets targeted for AF_XDP is still less work than 64
(NAPI_POLL_WEIGHT) packets going to the kernel networking stack.

Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
---
 include/net/xdp_sock_drv.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/net/xdp_sock_drv.h b/include/net/xdp_sock_drv.h
index 5b1ee8a9976d..4fc8e931d56f 100644
--- a/include/net/xdp_sock_drv.h
+++ b/include/net/xdp_sock_drv.h
@@ -7,8 +7,11 @@
 #define _LINUX_XDP_SOCK_DRV_H
 
 #include <net/xdp_sock.h>
+#include <linux/netdevice.h>
 #include <net/xsk_buff_pool.h>
 
+#define XSK_NAPI_WEIGHT (NAPI_POLL_WEIGHT << 2)
+
 #ifdef CONFIG_XDP_SOCKETS
 
 void xsk_tx_completed(struct xsk_buff_pool *pool, u32 nb_entries);
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH bpf-next 2/4] i40e, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-07 15:02 [PATCH bpf-next 0/4] xsk: increase NAPI budget for AF_XDP zero-copy path Björn Töpel
  2020-09-07 15:02 ` [PATCH bpf-next 1/4] xsk: add XSK_NAPI_WEIGHT define Björn Töpel
@ 2020-09-07 15:02 ` Björn Töpel
  2020-09-07 15:02 ` [PATCH bpf-next 3/4] ice, " Björn Töpel
  2020-09-07 15:02 ` [PATCH bpf-next 4/4] ixgbe, " Björn Töpel
  3 siblings, 0 replies; 14+ messages in thread
From: Björn Töpel @ 2020-09-07 15:02 UTC (permalink / raw)
  To: ast, daniel, netdev, bpf
  Cc: Björn Töpel, magnus.karlsson, kuba, intel-wired-lan

From: Björn Töpel <bjorn.topel@intel.com>

Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
zero-copy path.

Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
---
 drivers/net/ethernet/intel/i40e/i40e_xsk.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
index 2a1153d8957b..a8018736ca32 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
@@ -272,7 +272,7 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget)
 	bool failure = false;
 	struct sk_buff *skb;
 
-	while (likely(total_rx_packets < (unsigned int)budget)) {
+	while (likely(total_rx_packets < XSK_NAPI_WEIGHT)) {
 		union i40e_rx_desc *rx_desc;
 		struct xdp_buff **bi;
 		unsigned int size;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH bpf-next 3/4] ice, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-07 15:02 [PATCH bpf-next 0/4] xsk: increase NAPI budget for AF_XDP zero-copy path Björn Töpel
  2020-09-07 15:02 ` [PATCH bpf-next 1/4] xsk: add XSK_NAPI_WEIGHT define Björn Töpel
  2020-09-07 15:02 ` [PATCH bpf-next 2/4] i40e, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget Björn Töpel
@ 2020-09-07 15:02 ` Björn Töpel
  2020-09-07 15:02 ` [PATCH bpf-next 4/4] ixgbe, " Björn Töpel
  3 siblings, 0 replies; 14+ messages in thread
From: Björn Töpel @ 2020-09-07 15:02 UTC (permalink / raw)
  To: ast, daniel, netdev, bpf
  Cc: Björn Töpel, magnus.karlsson, kuba, intel-wired-lan

From: Björn Töpel <bjorn.topel@intel.com>

Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
zero-copy path.

Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
---
 drivers/net/ethernet/intel/ice/ice_xsk.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index 797886524054..cb473ccdf613 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -561,7 +561,7 @@ int ice_clean_rx_irq_zc(struct ice_ring *rx_ring, int budget)
 	unsigned int xdp_xmit = 0;
 	bool failure = false;
 
-	while (likely(total_rx_packets < (unsigned int)budget)) {
+	while (likely(total_rx_packets < XSK_NAPI_WEIGHT)) {
 		union ice_32b_rx_flex_desc *rx_desc;
 		unsigned int size, xdp_res = 0;
 		struct ice_rx_buf *rx_buf;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-07 15:02 [PATCH bpf-next 0/4] xsk: increase NAPI budget for AF_XDP zero-copy path Björn Töpel
                   ` (2 preceding siblings ...)
  2020-09-07 15:02 ` [PATCH bpf-next 3/4] ice, " Björn Töpel
@ 2020-09-07 15:02 ` Björn Töpel
  2020-09-07 19:32   ` Jakub Kicinski
                     ` (2 more replies)
  3 siblings, 3 replies; 14+ messages in thread
From: Björn Töpel @ 2020-09-07 15:02 UTC (permalink / raw)
  To: ast, daniel, netdev, bpf
  Cc: Björn Töpel, magnus.karlsson, kuba, intel-wired-lan

From: Björn Töpel <bjorn.topel@intel.com>

Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
zero-copy path.

Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
index 3771857cf887..f32c1ba0d237 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
@@ -239,7 +239,7 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
 	bool failure = false;
 	struct sk_buff *skb;
 
-	while (likely(total_rx_packets < budget)) {
+	while (likely(total_rx_packets < XSK_NAPI_WEIGHT)) {
 		union ixgbe_adv_rx_desc *rx_desc;
 		struct ixgbe_rx_buffer *bi;
 		unsigned int size;
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-07 15:02 ` [PATCH bpf-next 4/4] ixgbe, " Björn Töpel
@ 2020-09-07 19:32   ` Jakub Kicinski
  2020-09-08  5:38     ` Björn Töpel
  2020-09-08  9:45   ` Eric Dumazet
  2020-09-08 10:12   ` [Intel-wired-lan] " Paul Menzel
  2 siblings, 1 reply; 14+ messages in thread
From: Jakub Kicinski @ 2020-09-07 19:32 UTC (permalink / raw)
  To: Björn Töpel
  Cc: ast, daniel, netdev, bpf, Björn Töpel, magnus.karlsson,
	intel-wired-lan

On Mon,  7 Sep 2020 17:02:17 +0200 Björn Töpel wrote:
> From: Björn Töpel <bjorn.topel@intel.com>
> 
> Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
> zero-copy path.
> 
> Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
> ---
>  drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
> index 3771857cf887..f32c1ba0d237 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
> @@ -239,7 +239,7 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
>  	bool failure = false;
>  	struct sk_buff *skb;
>  
> -	while (likely(total_rx_packets < budget)) {
> +	while (likely(total_rx_packets < XSK_NAPI_WEIGHT)) {

I was thinking that we'd multiply 'budget' here, not replace it with a
constant. Looks like ixgbe dutifully passes 'per_ring_budget' into the
clean_rx functions, not a complete NAPI budget.

>  		union ixgbe_adv_rx_desc *rx_desc;
>  		struct ixgbe_rx_buffer *bi;
>  		unsigned int size;


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-07 19:32   ` Jakub Kicinski
@ 2020-09-08  5:38     ` Björn Töpel
  0 siblings, 0 replies; 14+ messages in thread
From: Björn Töpel @ 2020-09-08  5:38 UTC (permalink / raw)
  To: Jakub Kicinski, Björn Töpel
  Cc: ast, daniel, netdev, bpf, magnus.karlsson, intel-wired-lan

On 2020-09-07 21:32, Jakub Kicinski wrote:
> On Mon,  7 Sep 2020 17:02:17 +0200 Björn Töpel wrote:
>> From: Björn Töpel <bjorn.topel@intel.com>
>>
>> Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
>> zero-copy path.
>>
>> Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
>> ---
>>   drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>> index 3771857cf887..f32c1ba0d237 100644
>> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>> @@ -239,7 +239,7 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
>>   	bool failure = false;
>>   	struct sk_buff *skb;
>>   
>> -	while (likely(total_rx_packets < budget)) {
>> +	while (likely(total_rx_packets < XSK_NAPI_WEIGHT)) {
> 
> I was thinking that we'd multiply 'budget' here, not replace it with a
> constant. Looks like ixgbe dutifully passes 'per_ring_budget' into the
> clean_rx functions, not a complete NAPI budget.
>

Correct, and i40e/ice does the same ("per_ring_budget").

As for budget << XSK_NAPI_MULT vs replacing; Replacing the budget is 
more in line with what the drivers do for the Tx cleanup 
(xxx_clean_tx_irq), where the napi budget is discarded completely; 
Again, with the idea that "this is much cheaper than a "per-packet 
through the stack".

Do you prefer the multiplier way that you describe?


Cheers,
Björn


>>   		union ixgbe_adv_rx_desc *rx_desc;
>>   		struct ixgbe_rx_buffer *bi;
>>   		unsigned int size;
> 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-07 15:02 ` [PATCH bpf-next 4/4] ixgbe, " Björn Töpel
  2020-09-07 19:32   ` Jakub Kicinski
@ 2020-09-08  9:45   ` Eric Dumazet
  2020-09-08 11:49     ` Björn Töpel
  2020-09-08 10:12   ` [Intel-wired-lan] " Paul Menzel
  2 siblings, 1 reply; 14+ messages in thread
From: Eric Dumazet @ 2020-09-08  9:45 UTC (permalink / raw)
  To: Björn Töpel, ast, daniel, netdev, bpf
  Cc: Björn Töpel, magnus.karlsson, kuba, intel-wired-lan



On 9/7/20 5:02 PM, Björn Töpel wrote:
> From: Björn Töpel <bjorn.topel@intel.com>
> 
> Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
> zero-copy path.
> 
> Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
> ---
>  drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
> index 3771857cf887..f32c1ba0d237 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
> @@ -239,7 +239,7 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
>  	bool failure = false;
>  	struct sk_buff *skb;
>  
> -	while (likely(total_rx_packets < budget)) {
> +	while (likely(total_rx_packets < XSK_NAPI_WEIGHT)) {
>  		union ixgbe_adv_rx_desc *rx_desc;
>  		struct ixgbe_rx_buffer *bi;
>  		unsigned int size

This is a violation of NAPI API. IXGBE is already diverging a bit from best practices.

There are reasons we want to control the budget from callers,
if you want bigger budget just increase it instead of using your own ?

I would rather use a generic patch.

diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 7bd4fcdd0738a718d8b0f7134523cd87e4dcdb7b..33bcbdb6fef488983438c6584e3cbb0a44febb1a 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -2311,11 +2311,14 @@ static inline void *netdev_priv(const struct net_device *dev)
  */
 #define SET_NETDEV_DEVTYPE(net, devtype)       ((net)->dev.type = (devtype))
 
-/* Default NAPI poll() weight
- * Device drivers are strongly advised to not use bigger value
- */
+/* Default NAPI poll() weight. Highly recommended. */
 #define NAPI_POLL_WEIGHT 64
 
+/* Device drivers are strongly advised to not use bigger value,
+ * as this might cause latencies in stress conditions.
+ */
+#define NAPI_POLL_WEIGHT_MAX 256
+
 /**
  *     netif_napi_add - initialize a NAPI context
  *     @dev:  network device
diff --git a/net/core/dev.c b/net/core/dev.c
index 4086d335978c1bf62bd3965bd2ea96a4ac06b13d..496713fb6075bd8e5e22725e7c817172858e1dd7 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -6608,7 +6608,7 @@ void netif_napi_add(struct net_device *dev, struct napi_struct *napi,
        INIT_LIST_HEAD(&napi->rx_list);
        napi->rx_count = 0;
        napi->poll = poll;
-       if (weight > NAPI_POLL_WEIGHT)
+       if (weight > NAPI_POLL_WEIGHT_MAX)
                netdev_err_once(dev, "%s() called with weight %d\n", __func__,
                                weight);
        napi->weight = weight;

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: [Intel-wired-lan] [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-07 15:02 ` [PATCH bpf-next 4/4] ixgbe, " Björn Töpel
  2020-09-07 19:32   ` Jakub Kicinski
  2020-09-08  9:45   ` Eric Dumazet
@ 2020-09-08 10:12   ` Paul Menzel
  2020-09-08 11:12     ` Björn Töpel
  2 siblings, 1 reply; 14+ messages in thread
From: Paul Menzel @ 2020-09-08 10:12 UTC (permalink / raw)
  To: Björn Töpel, ast, daniel, netdev, bpf
  Cc: kuba, Björn Töpel, intel-wired-lan, magnus.karlsson

Dear Björn,


Am 07.09.20 um 17:02 schrieb Björn Töpel:
> From: Björn Töpel <bjorn.topel@intel.com>
> 
> Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
> zero-copy path.

Could you please add the description from the patch series cover letter 
to this commit too? To my knowledge, the message in the cover letter 
won’t be stored in the git repository.

> Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
> ---
>   drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)

[…]


Kind regards,

Paul

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Intel-wired-lan] [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-08 10:12   ` [Intel-wired-lan] " Paul Menzel
@ 2020-09-08 11:12     ` Björn Töpel
  2020-09-08 11:20       ` Paul Menzel
  0 siblings, 1 reply; 14+ messages in thread
From: Björn Töpel @ 2020-09-08 11:12 UTC (permalink / raw)
  To: Paul Menzel, Björn Töpel, ast, daniel, netdev, bpf
  Cc: kuba, intel-wired-lan, magnus.karlsson

On 2020-09-08 12:12, Paul Menzel wrote:
> Dear Björn,
> 
> 
> Am 07.09.20 um 17:02 schrieb Björn Töpel:
>> From: Björn Töpel <bjorn.topel@intel.com>
>>
>> Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
>> zero-copy path.
> 
> Could you please add the description from the patch series cover letter 
> to this commit too? To my knowledge, the message in the cover letter 
> won’t be stored in the git repository.
>

Paul, thanks for the input! The netdev/bpf trees always include the 
cover letter in the merge commit.


Cheers,
Björn

>> Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
>> ---
>>   drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> […]
> 
> 
> Kind regards,
> 
> Paul

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Intel-wired-lan] [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-08 11:12     ` Björn Töpel
@ 2020-09-08 11:20       ` Paul Menzel
  2020-09-08 11:43         ` Björn Töpel
  0 siblings, 1 reply; 14+ messages in thread
From: Paul Menzel @ 2020-09-08 11:20 UTC (permalink / raw)
  To: Björn Töpel, Björn Töpel, ast, daniel, netdev, bpf
  Cc: kuba, intel-wired-lan, magnus.karlsson

Dear Björn,


Am 08.09.20 um 13:12 schrieb Björn Töpel:
> On 2020-09-08 12:12, Paul Menzel wrote:

>> Am 07.09.20 um 17:02 schrieb Björn Töpel:
>>> From: Björn Töpel <bjorn.topel@intel.com>
>>>
>>> Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
>>> zero-copy path.
>>
>> Could you please add the description from the patch series cover 
>> letter to this commit too? To my knowledge, the message in the cover 
>> letter won’t be stored in the git repository.
> 
> Paul, thanks for the input! The netdev/bpf trees always include the 
> cover letter in the merge commit.

Yes, for pull/merge requests. But you posted them to the list, so I’d 
assume they will be applied with `git am` and not merged, or am I 
missing something. Could you please point me to a merge commit where the 
patches were posted to the list?


Kind regards,

Paul

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [Intel-wired-lan] [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-08 11:20       ` Paul Menzel
@ 2020-09-08 11:43         ` Björn Töpel
  0 siblings, 0 replies; 14+ messages in thread
From: Björn Töpel @ 2020-09-08 11:43 UTC (permalink / raw)
  To: Paul Menzel, Björn Töpel, ast, daniel, netdev, bpf
  Cc: kuba, intel-wired-lan, magnus.karlsson

On 2020-09-08 13:20, Paul Menzel wrote:
[...]>> Paul, thanks for the input! The netdev/bpf trees always include the
>> cover letter in the merge commit.
> 
> Yes, for pull/merge requests. But you posted them to the list, so I’d 
> assume they will be applied with `git am` and not merged, or am I 
> missing something. Could you please point me to a merge commit where the 
> patches were posted to the list?
> 

An example: A series is posted to the list [1], and when merged the 
merge commit look like [2].


Thanks,
Björn


[1] 
https://lore.kernel.org/bpf/20200520192103.355233-1-bjorn.topel@gmail.com/
[2] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=79917b242c3fe0d89e4752bc25ffef4574c2194b

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-08  9:45   ` Eric Dumazet
@ 2020-09-08 11:49     ` Björn Töpel
  2020-09-08 15:12       ` Eric Dumazet
  0 siblings, 1 reply; 14+ messages in thread
From: Björn Töpel @ 2020-09-08 11:49 UTC (permalink / raw)
  To: Eric Dumazet, Björn Töpel, ast, daniel, netdev, bpf
  Cc: magnus.karlsson, kuba, intel-wired-lan

On 2020-09-08 11:45, Eric Dumazet wrote:
> 
> 
> On 9/7/20 5:02 PM, Björn Töpel wrote:
>> From: Björn Töpel <bjorn.topel@intel.com>
>>
>> Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
>> zero-copy path.
>>
>> Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
>> ---
>>   drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>> index 3771857cf887..f32c1ba0d237 100644
>> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>> @@ -239,7 +239,7 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
>>   	bool failure = false;
>>   	struct sk_buff *skb;
>>   
>> -	while (likely(total_rx_packets < budget)) {
>> +	while (likely(total_rx_packets < XSK_NAPI_WEIGHT)) {
>>   		union ixgbe_adv_rx_desc *rx_desc;
>>   		struct ixgbe_rx_buffer *bi;
>>   		unsigned int size
> 
> This is a violation of NAPI API. IXGBE is already diverging a bit from best practices.
>

Thanks for having a look, Eric! By diverging from best practices, do
you mean that multiple queues share one NAPI context, and the budget
is split over the queues (say, 4 queues, 64/4 per queue), or that Tx
simply ignores the budget? Or both?

> There are reasons we want to control the budget from callers,
> if you want bigger budget just increase it instead of using your own ?
> 
> I would rather use a generic patch.
>

Hmm, so a configurable NAPI budget for, say, the AF_XDP enabled
queues/NAPIs? Am I reading that correct? (Note that this is *only* for
the AF_XDP enabled queues.)

I'll try to rework this to something more palatable.


Thanks,
Björn


> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 7bd4fcdd0738a718d8b0f7134523cd87e4dcdb7b..33bcbdb6fef488983438c6584e3cbb0a44febb1a 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -2311,11 +2311,14 @@ static inline void *netdev_priv(const struct net_device *dev)
>    */
>   #define SET_NETDEV_DEVTYPE(net, devtype)       ((net)->dev.type = (devtype))
>   
> -/* Default NAPI poll() weight
> - * Device drivers are strongly advised to not use bigger value
> - */
> +/* Default NAPI poll() weight. Highly recommended. */
>   #define NAPI_POLL_WEIGHT 64
>   
> +/* Device drivers are strongly advised to not use bigger value,
> + * as this might cause latencies in stress conditions.
> + */
> +#define NAPI_POLL_WEIGHT_MAX 256
> +
>   /**
>    *     netif_napi_add - initialize a NAPI context
>    *     @dev:  network device
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 4086d335978c1bf62bd3965bd2ea96a4ac06b13d..496713fb6075bd8e5e22725e7c817172858e1dd7 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -6608,7 +6608,7 @@ void netif_napi_add(struct net_device *dev, struct napi_struct *napi,
>          INIT_LIST_HEAD(&napi->rx_list);
>          napi->rx_count = 0;
>          napi->poll = poll;
> -       if (weight > NAPI_POLL_WEIGHT)
> +       if (weight > NAPI_POLL_WEIGHT_MAX)
>                  netdev_err_once(dev, "%s() called with weight %d\n", __func__,
>                                  weight);
>          napi->weight = weight;
> 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH bpf-next 4/4] ixgbe, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget
  2020-09-08 11:49     ` Björn Töpel
@ 2020-09-08 15:12       ` Eric Dumazet
  0 siblings, 0 replies; 14+ messages in thread
From: Eric Dumazet @ 2020-09-08 15:12 UTC (permalink / raw)
  To: Björn Töpel, Eric Dumazet, Björn Töpel, ast,
	daniel, netdev, bpf
  Cc: magnus.karlsson, kuba, intel-wired-lan



On 9/8/20 1:49 PM, Björn Töpel wrote:
> On 2020-09-08 11:45, Eric Dumazet wrote:
>>
>>
>> On 9/7/20 5:02 PM, Björn Töpel wrote:
>>> From: Björn Töpel <bjorn.topel@intel.com>
>>>
>>> Start using XSK_NAPI_WEIGHT as NAPI poll budget for the AF_XDP Rx
>>> zero-copy path.
>>>
>>> Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
>>> ---
>>>   drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>>> index 3771857cf887..f32c1ba0d237 100644
>>> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>>> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
>>> @@ -239,7 +239,7 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector,
>>>       bool failure = false;
>>>       struct sk_buff *skb;
>>>   -    while (likely(total_rx_packets < budget)) {
>>> +    while (likely(total_rx_packets < XSK_NAPI_WEIGHT)) {
>>>           union ixgbe_adv_rx_desc *rx_desc;
>>>           struct ixgbe_rx_buffer *bi;
>>>           unsigned int size
>>
>> This is a violation of NAPI API. IXGBE is already diverging a bit from best practices.
>>
> 
> Thanks for having a look, Eric! By diverging from best practices, do
> you mean that multiple queues share one NAPI context, and the budget
> is split over the queues (say, 4 queues, 64/4 per queue), or that Tx
> simply ignores the budget? Or both?

For instance, having ixgbe_poll() doing :

return min(work_done, budget - 1);

is quite interesting. It could hide bugs like :

I got a budget of 4, and processed 9999 packets because my maths have a bug,
but make sure to pretend we processed 3 packets...


> 
>> There are reasons we want to control the budget from callers,
>> if you want bigger budget just increase it instead of using your own ?
>>
>> I would rather use a generic patch.
>>
> 
> Hmm, so a configurable NAPI budget for, say, the AF_XDP enabled
> queues/NAPIs? Am I reading that correct? (Note that this is *only* for
> the AF_XDP enabled queues.)
> 
> I'll try to rework this to something more palatable.
> 
> 
> Thanks,
> Björn
> 
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2020-09-08 20:08 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-07 15:02 [PATCH bpf-next 0/4] xsk: increase NAPI budget for AF_XDP zero-copy path Björn Töpel
2020-09-07 15:02 ` [PATCH bpf-next 1/4] xsk: add XSK_NAPI_WEIGHT define Björn Töpel
2020-09-07 15:02 ` [PATCH bpf-next 2/4] i40e, xsk: use XSK_NAPI_WEIGHT as NAPI poll budget Björn Töpel
2020-09-07 15:02 ` [PATCH bpf-next 3/4] ice, " Björn Töpel
2020-09-07 15:02 ` [PATCH bpf-next 4/4] ixgbe, " Björn Töpel
2020-09-07 19:32   ` Jakub Kicinski
2020-09-08  5:38     ` Björn Töpel
2020-09-08  9:45   ` Eric Dumazet
2020-09-08 11:49     ` Björn Töpel
2020-09-08 15:12       ` Eric Dumazet
2020-09-08 10:12   ` [Intel-wired-lan] " Paul Menzel
2020-09-08 11:12     ` Björn Töpel
2020-09-08 11:20       ` Paul Menzel
2020-09-08 11:43         ` Björn Töpel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).