[PATCH net-next v4 0/2] bnxt: add rx discards stats for oom and netpool

[PATCH net-next v4 0/2] bnxt: add rx discards stats for oom and netpool

[Jakub Kicinski]

Drivers should avoid silently dropping frames. This set adds two
stats for previously unaccounted events to bnxt - packets dropped
due to allocation failures and packets dropped during emergency
ring polling.

v4: drop patch 1, not needed after simplifications

Jakub Kicinski (2):
  bnxt: count packets discarded because of netpoll
  bnxt: count discards due to memory allocation errors

 drivers/net/ethernet/broadcom/bnxt/bnxt.c      | 18 +++++++++++++++++-
 drivers/net/ethernet/broadcom/bnxt/bnxt.h      |  2 ++
 .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c  |  4 ++++
 3 files changed, 23 insertions(+), 1 deletion(-)

-- 
2.31.1

[PATCH net-next v4 1/2] bnxt: count packets discarded because of netpoll

[Jakub Kicinski]

bnxt may discard packets if Rx completions are consumed
in an attempt to let netpoll make progress. It should be
extremely rare in practice but nonetheless such events
should be counted.

Since completion ring memory is allocated dynamically use
a similar scheme to what is done for HW stats to save them.

Report the stats in rx_dropped and per-netdev ethtool
counter. Chances that users care which ring dropped are
very low.

v3: only save the stat to rx_dropped on reset,
rx_total_netpoll_discards will now only show drops since
last reset, similar to other "total_discard" counters.

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
--
v2: don't count ret == 0 case [Michael]
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c         | 9 ++++++++-
 drivers/net/ethernet/broadcom/bnxt/bnxt.h         | 1 +
 drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 4 ++++
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index ee66d410c82c..95c1c4237fa8 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -2003,6 +2003,7 @@ static int bnxt_force_rx_discard(struct bnxt *bp,
 	struct rx_cmp *rxcmp;
 	u16 cp_cons;
 	u8 cmp_type;
+	int ret;
 
 	cp_cons = RING_CMP(tmp_raw_cons);
 	rxcmp = (struct rx_cmp *)
@@ -2031,7 +2032,10 @@ static int bnxt_force_rx_discard(struct bnxt *bp,
 		tpa_end1->rx_tpa_end_cmp_errors_v2 |=
 			cpu_to_le32(RX_TPA_END_CMP_ERRORS);
 	}
-	return bnxt_rx_pkt(bp, cpr, raw_cons, event);
+	ret = bnxt_rx_pkt(bp, cpr, raw_cons, event);
+	if (ret && ret != -EBUSY)
+		cpr->sw_stats.rx.rx_netpoll_discards += 1;
+	return ret;
 }
 
 u32 bnxt_fw_health_readl(struct bnxt *bp, int reg_idx)
@@ -10646,6 +10650,8 @@ static void bnxt_get_ring_stats(struct bnxt *bp,
 		stats->multicast += BNXT_GET_RING_STATS64(sw, rx_mcast_pkts);
 
 		stats->tx_dropped += BNXT_GET_RING_STATS64(sw, tx_error_pkts);
+
+		stats->rx_dropped += cpr->sw_stats.rx.rx_netpoll_discards;
 	}
 }
 
@@ -10660,6 +10666,7 @@ static void bnxt_add_prev_stats(struct bnxt *bp,
 	stats->tx_bytes += prev_stats->tx_bytes;
 	stats->rx_missed_errors += prev_stats->rx_missed_errors;
 	stats->multicast += prev_stats->multicast;
+	stats->rx_dropped += prev_stats->rx_dropped;
 	stats->tx_dropped += prev_stats->tx_dropped;
 }
 
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 7b989b6e4f6e..c8cdc770426c 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -939,6 +939,7 @@ struct bnxt_rx_sw_stats {
 	u64			rx_l4_csum_errors;
 	u64			rx_resets;
 	u64			rx_buf_errors;
+	u64			rx_netpoll_discards;
 };
 
 struct bnxt_cmn_sw_stats {
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 9f8c72d95228..5852ae3b26a2 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -307,6 +307,7 @@ static const char * const bnxt_cmn_sw_stats_str[] = {
 enum {
 	RX_TOTAL_DISCARDS,
 	TX_TOTAL_DISCARDS,
+	RX_NETPOLL_DISCARDS,
 };
 
 static struct {
@@ -315,6 +316,7 @@ static struct {
 } bnxt_sw_func_stats[] = {
 	{0, "rx_total_discard_pkts"},
 	{0, "tx_total_discard_pkts"},
+	{0, "rx_total_netpoll_discards"},
 };
 
 #define NUM_RING_RX_SW_STATS		ARRAY_SIZE(bnxt_rx_sw_stats_str)
@@ -603,6 +605,8 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,
 			BNXT_GET_RING_STATS64(sw_stats, rx_discard_pkts);
 		bnxt_sw_func_stats[TX_TOTAL_DISCARDS].counter +=
 			BNXT_GET_RING_STATS64(sw_stats, tx_discard_pkts);
+		bnxt_sw_func_stats[RX_NETPOLL_DISCARDS].counter +=
+			cpr->sw_stats.rx.rx_netpoll_discards;
 	}
 
 	for (i = 0; i < BNXT_NUM_SW_FUNC_STATS; i++, j++)
-- 
2.31.1

[PATCH net-next v4 2/2] bnxt: count discards due to memory allocation errors

[Jakub Kicinski]

Count packets dropped due to buffer or skb allocation errors.
Report as part of rx_dropped.

v2: drop the ethtool -S entry [Vladimir]

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
 drivers/net/ethernet/broadcom/bnxt/bnxt.c | 11 ++++++++++-
 drivers/net/ethernet/broadcom/bnxt/bnxt.h |  1 +
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 95c1c4237fa8..31e80d90ee98 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -1651,6 +1651,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
 		skb = bnxt_copy_skb(bnapi, data_ptr, len, mapping);
 		if (!skb) {
 			bnxt_abort_tpa(cpr, idx, agg_bufs);
+			cpr->sw_stats.rx.rx_oom_discards += 1;
 			return NULL;
 		}
 	} else {
@@ -1660,6 +1661,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
 		new_data = __bnxt_alloc_rx_data(bp, &new_mapping, GFP_ATOMIC);
 		if (!new_data) {
 			bnxt_abort_tpa(cpr, idx, agg_bufs);
+			cpr->sw_stats.rx.rx_oom_discards += 1;
 			return NULL;
 		}
 
@@ -1675,6 +1677,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
 		if (!skb) {
 			kfree(data);
 			bnxt_abort_tpa(cpr, idx, agg_bufs);
+			cpr->sw_stats.rx.rx_oom_discards += 1;
 			return NULL;
 		}
 		skb_reserve(skb, bp->rx_offset);
@@ -1685,6 +1688,7 @@ static inline struct sk_buff *bnxt_tpa_end(struct bnxt *bp,
 		skb = bnxt_rx_pages(bp, cpr, skb, idx, agg_bufs, true);
 		if (!skb) {
 			/* Page reuse already handled by bnxt_rx_pages(). */
+			cpr->sw_stats.rx.rx_oom_discards += 1;
 			return NULL;
 		}
 	}
@@ -1888,6 +1892,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
 			if (agg_bufs)
 				bnxt_reuse_rx_agg_bufs(cpr, cp_cons, 0,
 						       agg_bufs, false);
+			cpr->sw_stats.rx.rx_oom_discards += 1;
 			rc = -ENOMEM;
 			goto next_rx;
 		}
@@ -1901,6 +1906,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
 		skb = bp->rx_skb_func(bp, rxr, cons, data, data_ptr, dma_addr,
 				      payload | len);
 		if (!skb) {
+			cpr->sw_stats.rx.rx_oom_discards += 1;
 			rc = -ENOMEM;
 			goto next_rx;
 		}
@@ -1909,6 +1915,7 @@ static int bnxt_rx_pkt(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
 	if (agg_bufs) {
 		skb = bnxt_rx_pages(bp, cpr, skb, cp_cons, agg_bufs, false);
 		if (!skb) {
+			cpr->sw_stats.rx.rx_oom_discards += 1;
 			rc = -ENOMEM;
 			goto next_rx;
 		}
@@ -10651,7 +10658,9 @@ static void bnxt_get_ring_stats(struct bnxt *bp,
 
 		stats->tx_dropped += BNXT_GET_RING_STATS64(sw, tx_error_pkts);
 
-		stats->rx_dropped += cpr->sw_stats.rx.rx_netpoll_discards;
+		stats->rx_dropped +=
+			cpr->sw_stats.rx.rx_netpoll_discards +
+			cpr->sw_stats.rx.rx_oom_discards;
 	}
 }
 
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index c8cdc770426c..dc96dd6957c9 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -939,6 +939,7 @@ struct bnxt_rx_sw_stats {
 	u64			rx_l4_csum_errors;
 	u64			rx_resets;
 	u64			rx_buf_errors;
+	u64			rx_oom_discards;
 	u64			rx_netpoll_discards;
 };
 
-- 
2.31.1

Re: [PATCH net-next v4 0/2] bnxt: add rx discards stats for oom and netpool

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 409 bytes --]

On Fri, Aug 27, 2021 at 8:27 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> Drivers should avoid silently dropping frames. This set adds two
> stats for previously unaccounted events to bnxt - packets dropped
> due to allocation failures and packets dropped during emergency
> ring polling.
>
> v4: drop patch 1, not needed after simplifications

Thanks.
Reviewed-by: Michael Chan <michael.chan@broadcom.com>

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

Re: [PATCH net-next v4 0/2] bnxt: add rx discards stats for oom and netpool

[Edwin Peer]

Reviewed-by: Edwin Peer <edwin.peer@broadcom.com>

Regards,
Edwin Peer


On Fri, Aug 27, 2021 at 8:27 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> Drivers should avoid silently dropping frames. This set adds two
> stats for previously unaccounted events to bnxt - packets dropped
> due to allocation failures and packets dropped during emergency
> ring polling.
>
> v4: drop patch 1, not needed after simplifications
>
> Jakub Kicinski (2):
>   bnxt: count packets discarded because of netpoll
>   bnxt: count discards due to memory allocation errors
>
>  drivers/net/ethernet/broadcom/bnxt/bnxt.c      | 18 +++++++++++++++++-
>  drivers/net/ethernet/broadcom/bnxt/bnxt.h      |  2 ++
>  .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c  |  4 ++++
>  3 files changed, 23 insertions(+), 1 deletion(-)
>
> --
> 2.31.1
>

Re: [PATCH net-next v4 0/2] bnxt: add rx discards stats for oom and netpool

[Jakub Kicinski]

On Fri, 27 Aug 2021 11:23:47 -0700 Michael Chan wrote:
> On Fri, Aug 27, 2021 at 8:27 AM Jakub Kicinski <kuba@kernel.org> wrote:
> > Drivers should avoid silently dropping frames. This set adds two
> > stats for previously unaccounted events to bnxt - packets dropped
> > due to allocation failures and packets dropped during emergency
> > ring polling.
> >
> > v4: drop patch 1, not needed after simplifications  
> 
> Reviewed-by: Michael Chan <michael.chan@broadcom.com>

Thank you for the reviews, applied!