* [PATCH net-next 0/2] Report tc-taprio window drops in NXP ENETC driver @ 2021-06-02 12:21 Vladimir Oltean 2021-06-02 12:21 ` [PATCH net-next 1/2] net: enetc: manage ENETC_F_QBV in priv->active_offloads only when enabled Vladimir Oltean 2021-06-02 12:21 ` [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops Vladimir Oltean 0 siblings, 2 replies; 15+ messages in thread From: Vladimir Oltean @ 2021-06-02 12:21 UTC (permalink / raw) To: Jakub Kicinski, David S. Miller, netdev Cc: Claudiu Manoil, Michael Walle, Po Liu, Vinicius Costa Gomes, Vladimir Oltean From: Vladimir Oltean <vladimir.oltean@nxp.com> This series adds an ethtool statistics counter to the enetc driver which tracks the number of packets dropped due to insufficient transmit time in the tc-taprio schedule compared to the required time it takes the MAC to send that packet at the current link speed. Po Liu (1): net: enetc: count the tc-taprio window drops Vladimir Oltean (1): net: enetc: manage ENETC_F_QBV in priv->active_offloads only when enabled drivers/net/ethernet/freescale/enetc/enetc.c | 13 +++++++++++-- drivers/net/ethernet/freescale/enetc/enetc.h | 2 ++ .../net/ethernet/freescale/enetc/enetc_ethtool.c | 2 ++ drivers/net/ethernet/freescale/enetc/enetc_hw.h | 1 + drivers/net/ethernet/freescale/enetc/enetc_pf.c | 6 ++---- drivers/net/ethernet/freescale/enetc/enetc_qos.c | 6 ++++++ 6 files changed, 24 insertions(+), 6 deletions(-) -- 2.25.1 ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH net-next 1/2] net: enetc: manage ENETC_F_QBV in priv->active_offloads only when enabled 2021-06-02 12:21 [PATCH net-next 0/2] Report tc-taprio window drops in NXP ENETC driver Vladimir Oltean @ 2021-06-02 12:21 ` Vladimir Oltean 2021-06-02 12:21 ` [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops Vladimir Oltean 1 sibling, 0 replies; 15+ messages in thread From: Vladimir Oltean @ 2021-06-02 12:21 UTC (permalink / raw) To: Jakub Kicinski, David S. Miller, netdev Cc: Claudiu Manoil, Michael Walle, Po Liu, Vinicius Costa Gomes, Vladimir Oltean From: Vladimir Oltean <vladimir.oltean@nxp.com> All the other flags in priv->active_offloads are managed dynamically, except ENETC_F_QBV which is set statically based on the probed SI capability. This change makes priv->active_offloads & ENETC_F_QBV track the presence of a tc-taprio schedule on the port. Calling enetc_sched_speed_set() from phylink_mac_link_up() is kept unconditional as long as Qbv is supported at all on the port, because the tc-taprio offload does not re-trigger another link mode resolve, so the scheduler needs to be functional from the get go. Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com> --- drivers/net/ethernet/freescale/enetc/enetc_pf.c | 6 ++---- drivers/net/ethernet/freescale/enetc/enetc_qos.c | 6 ++++++ 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/freescale/enetc/enetc_pf.c b/drivers/net/ethernet/freescale/enetc/enetc_pf.c index 31274325159a..aced941e57e1 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_pf.c +++ b/drivers/net/ethernet/freescale/enetc/enetc_pf.c @@ -769,9 +769,6 @@ static void enetc_pf_netdev_setup(struct enetc_si *si, struct net_device *ndev, ndev->priv_flags |= IFF_UNICAST_FLT; - if (si->hw_features & ENETC_SI_F_QBV) - priv->active_offloads |= ENETC_F_QBV; - if (si->hw_features & ENETC_SI_F_PSFP && !enetc_psfp_enable(priv)) { priv->active_offloads |= ENETC_F_QCI; ndev->features |= NETIF_F_HW_TC; @@ -1022,7 +1019,8 @@ static void enetc_pl_mac_link_up(struct phylink_config *config, int idx; priv = netdev_priv(pf->si->ndev); - if (priv->active_offloads & ENETC_F_QBV) + + if (pf->si->hw_features & ENETC_SI_F_QBV) enetc_sched_speed_set(priv, speed); if (!phylink_autoneg_inband(mode) && diff --git a/drivers/net/ethernet/freescale/enetc/enetc_qos.c b/drivers/net/ethernet/freescale/enetc/enetc_qos.c index 4577226d3c6a..23ac6e1551ca 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_qos.c +++ b/drivers/net/ethernet/freescale/enetc/enetc_qos.c @@ -69,6 +69,9 @@ static int enetc_setup_taprio(struct net_device *ndev, enetc_wr(&priv->si->hw, ENETC_QBV_PTGCR_OFFSET, tge & (~ENETC_QBV_TGE)); + + priv->active_offloads &= ~ENETC_F_QBV; + return 0; } @@ -135,6 +138,9 @@ static int enetc_setup_taprio(struct net_device *ndev, dma_unmap_single(&priv->si->pdev->dev, dma, data_size, DMA_TO_DEVICE); kfree(gcl_data); + if (!err) + priv->active_offloads |= ENETC_F_QBV; + return err; } -- 2.25.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2021-06-02 12:21 [PATCH net-next 0/2] Report tc-taprio window drops in NXP ENETC driver Vladimir Oltean 2021-06-02 12:21 ` [PATCH net-next 1/2] net: enetc: manage ENETC_F_QBV in priv->active_offloads only when enabled Vladimir Oltean @ 2021-06-02 12:21 ` Vladimir Oltean 2021-06-02 17:19 ` Jakub Kicinski 1 sibling, 1 reply; 15+ messages in thread From: Vladimir Oltean @ 2021-06-02 12:21 UTC (permalink / raw) To: Jakub Kicinski, David S. Miller, netdev Cc: Claudiu Manoil, Michael Walle, Po Liu, Vinicius Costa Gomes, Po Liu, Vladimir Oltean From: Po Liu <Po.Liu@nxp.com> The enetc scheduler for IEEE 802.1Qbv has 2 options (depending on PTGCR[TG_DROP_DISABLE]) when we attempt to send an oversized packet which will never fit in its allotted time slot for its traffic class: either block the entire port due to head-of-line blocking, or drop the packet and set a bit in the writeback format of the transmit buffer descriptor, allowing other packets to be sent. We obviously choose the second option in the driver, but we do not detect the drop condition, so from the perspective of the network stack, the packet is sent and no error counter is incremented. This change checks the writeback of the TX BD when tc-taprio is enabled, and increments a specific ethtool statistics counter and a generic "tx_dropped" counter in ndo_get_stats64. Signed-off-by: Po Liu <Po.Liu@nxp.com> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com> --- drivers/net/ethernet/freescale/enetc/enetc.c | 13 +++++++++++-- drivers/net/ethernet/freescale/enetc/enetc.h | 2 ++ .../net/ethernet/freescale/enetc/enetc_ethtool.c | 2 ++ drivers/net/ethernet/freescale/enetc/enetc_hw.h | 1 + 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c index 3ca93adb9662..3f1cb921571a 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.c +++ b/drivers/net/ethernet/freescale/enetc/enetc.c @@ -170,7 +170,8 @@ static int enetc_map_tx_buffs(struct enetc_bdr *tx_ring, struct sk_buff *skb) } tx_swbd->do_twostep_tstamp = do_twostep_tstamp; - tx_swbd->check_wb = tx_swbd->do_twostep_tstamp; + tx_swbd->qbv_en = !!(priv->active_offloads & ENETC_F_QBV); + tx_swbd->check_wb = tx_swbd->do_twostep_tstamp || tx_swbd->qbv_en; if (do_vlan || do_onestep_tstamp || do_twostep_tstamp) flags |= ENETC_TXBD_FLAGS_EX; @@ -525,9 +526,9 @@ static void enetc_recycle_xdp_tx_buff(struct enetc_bdr *tx_ring, static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) { + int tx_frm_cnt = 0, tx_byte_cnt = 0, tx_win_drop = 0; struct net_device *ndev = tx_ring->ndev; struct enetc_ndev_priv *priv = netdev_priv(ndev); - int tx_frm_cnt = 0, tx_byte_cnt = 0; struct enetc_tx_swbd *tx_swbd; int i, bds_to_clean; bool do_twostep_tstamp; @@ -557,6 +558,10 @@ static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) &tstamp); do_twostep_tstamp = true; } + + if (tx_swbd->qbv_en && + txbd->wb.status & ENETC_TXBD_STATS_WIN) + tx_win_drop++; } if (tx_swbd->is_xdp_tx) @@ -610,6 +615,7 @@ static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) tx_ring->next_to_clean = i; tx_ring->stats.packets += tx_frm_cnt; tx_ring->stats.bytes += tx_byte_cnt; + tx_ring->stats.win_drop += tx_win_drop; if (unlikely(tx_frm_cnt && netif_carrier_ok(ndev) && __netif_subqueue_stopped(ndev, tx_ring->index) && @@ -2271,6 +2277,7 @@ struct net_device_stats *enetc_get_stats(struct net_device *ndev) struct enetc_ndev_priv *priv = netdev_priv(ndev); struct net_device_stats *stats = &ndev->stats; unsigned long packets = 0, bytes = 0; + unsigned long tx_dropped = 0; int i; for (i = 0; i < priv->num_rx_rings; i++) { @@ -2286,10 +2293,12 @@ struct net_device_stats *enetc_get_stats(struct net_device *ndev) for (i = 0; i < priv->num_tx_rings; i++) { packets += priv->tx_ring[i]->stats.packets; bytes += priv->tx_ring[i]->stats.bytes; + tx_dropped += priv->tx_ring[i]->stats.win_drop; } stats->tx_packets = packets; stats->tx_bytes = bytes; + stats->tx_dropped = tx_dropped; return stats; } diff --git a/drivers/net/ethernet/freescale/enetc/enetc.h b/drivers/net/ethernet/freescale/enetc/enetc.h index 08b283347d9c..2a5973aebc31 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.h +++ b/drivers/net/ethernet/freescale/enetc/enetc.h @@ -34,6 +34,7 @@ struct enetc_tx_swbd { u8 is_eof:1; u8 is_xdp_tx:1; u8 is_xdp_redirect:1; + u8 qbv_en:1; }; #define ENETC_RX_MAXFRM_SIZE ENETC_MAC_MAXFRM_SIZE @@ -70,6 +71,7 @@ struct enetc_ring_stats { unsigned int xdp_redirect_sg; unsigned int recycles; unsigned int recycle_failures; + unsigned int win_drop; }; struct enetc_xdp_data { diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c index ebccaf02411c..4d55e78fa353 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c +++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c @@ -204,6 +204,7 @@ static const char tx_ring_stats[][ETH_GSTRING_LEN] = { "Tx ring %2d frames", "Tx ring %2d XDP frames", "Tx ring %2d XDP drops", + "Tx window drop %2d frames", }; static int enetc_get_sset_count(struct net_device *ndev, int sset) @@ -279,6 +280,7 @@ static void enetc_get_ethtool_stats(struct net_device *ndev, data[o++] = priv->tx_ring[i]->stats.packets; data[o++] = priv->tx_ring[i]->stats.xdp_tx; data[o++] = priv->tx_ring[i]->stats.xdp_tx_drops; + data[o++] = priv->tx_ring[i]->stats.win_drop; } for (i = 0; i < priv->num_rx_rings; i++) { diff --git a/drivers/net/ethernet/freescale/enetc/enetc_hw.h b/drivers/net/ethernet/freescale/enetc/enetc_hw.h index 0f5f081a5baf..e7fa4fb85d0b 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_hw.h +++ b/drivers/net/ethernet/freescale/enetc/enetc_hw.h @@ -543,6 +543,7 @@ enum enetc_txbd_flags { ENETC_TXBD_FLAGS_EX = BIT(6), ENETC_TXBD_FLAGS_F = BIT(7) }; +#define ENETC_TXBD_STATS_WIN BIT(7) #define ENETC_TXBD_TXSTART_MASK GENMASK(24, 0) #define ENETC_TXBD_FLAGS_OFFSET 24 -- 2.25.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2021-06-02 12:21 ` [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops Vladimir Oltean @ 2021-06-02 17:19 ` Jakub Kicinski 2021-06-02 17:59 ` Vladimir Oltean 0 siblings, 1 reply; 15+ messages in thread From: Jakub Kicinski @ 2021-06-02 17:19 UTC (permalink / raw) To: Vladimir Oltean Cc: David S. Miller, netdev, Claudiu Manoil, Michael Walle, Po Liu, Vinicius Costa Gomes, Vladimir Oltean On Wed, 2 Jun 2021 15:21:14 +0300 Vladimir Oltean wrote: > From: Po Liu <Po.Liu@nxp.com> > > The enetc scheduler for IEEE 802.1Qbv has 2 options (depending on > PTGCR[TG_DROP_DISABLE]) when we attempt to send an oversized packet > which will never fit in its allotted time slot for its traffic class: > either block the entire port due to head-of-line blocking, or drop the the entire port or the entire queue? > packet and set a bit in the writeback format of the transmit buffer > descriptor, allowing other packets to be sent. > > We obviously choose the second option in the driver, but we do not > detect the drop condition, so from the perspective of the network stack, > the packet is sent and no error counter is incremented. > > This change checks the writeback of the TX BD when tc-taprio is enabled, > and increments a specific ethtool statistics counter and a generic > "tx_dropped" counter in ndo_get_stats64. Any chance we should also report that back to the qdisc to have a standard way of querying from user space? Qdisc offload supports stats in general, it shouldn't be an issue, and the stat seems generic enough, no? ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2021-06-02 17:19 ` Jakub Kicinski @ 2021-06-02 17:59 ` Vladimir Oltean 2021-06-02 21:30 ` Jakub Kicinski 0 siblings, 1 reply; 15+ messages in thread From: Vladimir Oltean @ 2021-06-02 17:59 UTC (permalink / raw) To: Jakub Kicinski Cc: David S. Miller, netdev, Claudiu Manoil, Michael Walle, Po Liu, Vinicius Costa Gomes, Vladimir Oltean On Wed, Jun 02, 2021 at 10:19:20AM -0700, Jakub Kicinski wrote: > On Wed, 2 Jun 2021 15:21:14 +0300 Vladimir Oltean wrote: > > From: Po Liu <Po.Liu@nxp.com> > > > > The enetc scheduler for IEEE 802.1Qbv has 2 options (depending on > > PTGCR[TG_DROP_DISABLE]) when we attempt to send an oversized packet > > which will never fit in its allotted time slot for its traffic class: > > either block the entire port due to head-of-line blocking, or drop the > > the entire port or the entire queue? I don't remember, I need to re-test. > > packet and set a bit in the writeback format of the transmit buffer > > descriptor, allowing other packets to be sent. > > > > We obviously choose the second option in the driver, but we do not > > detect the drop condition, so from the perspective of the network stack, > > the packet is sent and no error counter is incremented. > > > > This change checks the writeback of the TX BD when tc-taprio is enabled, > > and increments a specific ethtool statistics counter and a generic > > "tx_dropped" counter in ndo_get_stats64. > > Any chance we should also report that back to the qdisc to have > a standard way of querying from user space? Qdisc offload supports > stats in general, it shouldn't be an issue, and the stat seems generic > enough, no? You're thinking of something along the lines of tc_codel_xstats? How do you propose I pass this on to the taprio qdisc? Just call a function in enetc that is exported by net/sched/sch_taprio.c? If the skb is bound to a socket, I'm thinking it might be more useful to report a struct sock_extended_err similar to the SO_EE_TXTIME_MISSED stuff for tc-etf, what do you think? ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2021-06-02 17:59 ` Vladimir Oltean @ 2021-06-02 21:30 ` Jakub Kicinski 0 siblings, 0 replies; 15+ messages in thread From: Jakub Kicinski @ 2021-06-02 21:30 UTC (permalink / raw) To: Vladimir Oltean Cc: David S. Miller, netdev, Claudiu Manoil, Michael Walle, Po Liu, Vinicius Costa Gomes, Vladimir Oltean On Wed, 2 Jun 2021 20:59:37 +0300 Vladimir Oltean wrote: > On Wed, Jun 02, 2021 at 10:19:20AM -0700, Jakub Kicinski wrote: > > On Wed, 2 Jun 2021 15:21:14 +0300 Vladimir Oltean wrote: > > > From: Po Liu <Po.Liu@nxp.com> > > > > > > The enetc scheduler for IEEE 802.1Qbv has 2 options (depending on > > > PTGCR[TG_DROP_DISABLE]) when we attempt to send an oversized packet > > > which will never fit in its allotted time slot for its traffic class: > > > either block the entire port due to head-of-line blocking, or drop the > > > > the entire port or the entire queue? > > I don't remember, I need to re-test. > > > > packet and set a bit in the writeback format of the transmit buffer > > > descriptor, allowing other packets to be sent. > > > > > > We obviously choose the second option in the driver, but we do not > > > detect the drop condition, so from the perspective of the network stack, > > > the packet is sent and no error counter is incremented. > > > > > > This change checks the writeback of the TX BD when tc-taprio is enabled, > > > and increments a specific ethtool statistics counter and a generic > > > "tx_dropped" counter in ndo_get_stats64. > > > > Any chance we should also report that back to the qdisc to have > > a standard way of querying from user space? Qdisc offload supports > > stats in general, it shouldn't be an issue, and the stat seems generic > > enough, no? > > You're thinking of something along the lines of tc_codel_xstats? > How do you propose I pass this on to the taprio qdisc? Just call a > function in enetc that is exported by net/sched/sch_taprio.c? Check out red_dump_offload_stats()/TC_RED_STATS, this is the usual way of handling stats offload in TC. > If the skb is bound to a socket, I'm thinking it might be more useful to > report a struct sock_extended_err similar to the SO_EE_TXTIME_MISSED > stuff for tc-etf, what do you think? That's an interesting idea as well, dunno enough about the practical uses to be able to tell if applications care about per-socket state, per-interface state, or both. ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH net-next 0/2] Count tc-taprio window drops in enetc driver @ 2022-05-10 16:36 Vladimir Oltean 2022-05-10 16:36 ` [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops Vladimir Oltean 0 siblings, 1 reply; 15+ messages in thread From: Vladimir Oltean @ 2022-05-10 16:36 UTC (permalink / raw) To: netdev Cc: Jakub Kicinski, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang This series includes a patch from Po Liu (no longer with NXP) which counts frames dropped by the tc-taprio offload in ethtool -S and in ndo_get_stats64. It also contains a preparation patch from myself. Po Liu (1): net: enetc: count the tc-taprio window drops Vladimir Oltean (1): net: enetc: manage ENETC_F_QBV in priv->active_offloads only when enabled drivers/net/ethernet/freescale/enetc/enetc.c | 13 +++++++++++-- drivers/net/ethernet/freescale/enetc/enetc.h | 2 ++ .../net/ethernet/freescale/enetc/enetc_ethtool.c | 2 ++ drivers/net/ethernet/freescale/enetc/enetc_hw.h | 1 + drivers/net/ethernet/freescale/enetc/enetc_pf.c | 6 ++---- drivers/net/ethernet/freescale/enetc/enetc_qos.c | 6 ++++++ 6 files changed, 24 insertions(+), 6 deletions(-) -- 2.25.1 ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-10 16:36 [PATCH net-next 0/2] Count tc-taprio window drops in enetc driver Vladimir Oltean @ 2022-05-10 16:36 ` Vladimir Oltean 2022-05-11 8:17 ` Claudiu Manoil 2022-05-11 22:27 ` Jakub Kicinski 0 siblings, 2 replies; 15+ messages in thread From: Vladimir Oltean @ 2022-05-10 16:36 UTC (permalink / raw) To: netdev Cc: Jakub Kicinski, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu From: Po Liu <Po.Liu@nxp.com> The enetc scheduler for IEEE 802.1Qbv has 2 options (depending on PTGCR[TG_DROP_DISABLE]) when we attempt to send an oversized packet which will never fit in its allotted time slot for its traffic class: either block the entire port due to head-of-line blocking, or drop the packet and set a bit in the writeback format of the transmit buffer descriptor, allowing other packets to be sent. We obviously choose the second option in the driver, but we do not detect the drop condition, so from the perspective of the network stack, the packet is sent and no error counter is incremented. This change checks the writeback of the TX BD when tc-taprio is enabled, and increments a specific ethtool statistics counter and a generic "tx_dropped" counter in ndo_get_stats64. Signed-off-by: Po Liu <Po.Liu@nxp.com> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com> --- drivers/net/ethernet/freescale/enetc/enetc.c | 13 +++++++++++-- drivers/net/ethernet/freescale/enetc/enetc.h | 2 ++ .../net/ethernet/freescale/enetc/enetc_ethtool.c | 2 ++ drivers/net/ethernet/freescale/enetc/enetc_hw.h | 1 + 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c index d6930a797c6c..4470a4a3e4c3 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.c +++ b/drivers/net/ethernet/freescale/enetc/enetc.c @@ -172,7 +172,8 @@ static int enetc_map_tx_buffs(struct enetc_bdr *tx_ring, struct sk_buff *skb) } tx_swbd->do_twostep_tstamp = do_twostep_tstamp; - tx_swbd->check_wb = tx_swbd->do_twostep_tstamp; + tx_swbd->qbv_en = !!(priv->active_offloads & ENETC_F_QBV); + tx_swbd->check_wb = tx_swbd->do_twostep_tstamp || tx_swbd->qbv_en; if (do_vlan || do_onestep_tstamp || do_twostep_tstamp) flags |= ENETC_TXBD_FLAGS_EX; @@ -792,9 +793,9 @@ static void enetc_recycle_xdp_tx_buff(struct enetc_bdr *tx_ring, static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) { + int tx_frm_cnt = 0, tx_byte_cnt = 0, tx_win_drop = 0; struct net_device *ndev = tx_ring->ndev; struct enetc_ndev_priv *priv = netdev_priv(ndev); - int tx_frm_cnt = 0, tx_byte_cnt = 0; struct enetc_tx_swbd *tx_swbd; int i, bds_to_clean; bool do_twostep_tstamp; @@ -821,6 +822,10 @@ static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) &tstamp); do_twostep_tstamp = true; } + + if (tx_swbd->qbv_en && + txbd->wb.status & ENETC_TXBD_STATS_WIN) + tx_win_drop++; } if (tx_swbd->is_xdp_tx) @@ -873,6 +878,7 @@ static bool enetc_clean_tx_ring(struct enetc_bdr *tx_ring, int napi_budget) tx_ring->next_to_clean = i; tx_ring->stats.packets += tx_frm_cnt; tx_ring->stats.bytes += tx_byte_cnt; + tx_ring->stats.win_drop += tx_win_drop; if (unlikely(tx_frm_cnt && netif_carrier_ok(ndev) && __netif_subqueue_stopped(ndev, tx_ring->index) && @@ -2552,6 +2558,7 @@ struct net_device_stats *enetc_get_stats(struct net_device *ndev) struct enetc_ndev_priv *priv = netdev_priv(ndev); struct net_device_stats *stats = &ndev->stats; unsigned long packets = 0, bytes = 0; + unsigned long tx_dropped = 0; int i; for (i = 0; i < priv->num_rx_rings; i++) { @@ -2567,10 +2574,12 @@ struct net_device_stats *enetc_get_stats(struct net_device *ndev) for (i = 0; i < priv->num_tx_rings; i++) { packets += priv->tx_ring[i]->stats.packets; bytes += priv->tx_ring[i]->stats.bytes; + tx_dropped += priv->tx_ring[i]->stats.win_drop; } stats->tx_packets = packets; stats->tx_bytes = bytes; + stats->tx_dropped = tx_dropped; return stats; } diff --git a/drivers/net/ethernet/freescale/enetc/enetc.h b/drivers/net/ethernet/freescale/enetc/enetc.h index 68d806dc3701..29922c20531f 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc.h +++ b/drivers/net/ethernet/freescale/enetc/enetc.h @@ -36,6 +36,7 @@ struct enetc_tx_swbd { u8 is_eof:1; u8 is_xdp_tx:1; u8 is_xdp_redirect:1; + u8 qbv_en:1; }; #define ENETC_RX_MAXFRM_SIZE ENETC_MAC_MAXFRM_SIZE @@ -72,6 +73,7 @@ struct enetc_ring_stats { unsigned int xdp_redirect_sg; unsigned int recycles; unsigned int recycle_failures; + unsigned int win_drop; }; struct enetc_xdp_data { diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c index 60ec64bfb3f0..ff872e40ce85 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c +++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c @@ -204,6 +204,7 @@ static const char tx_ring_stats[][ETH_GSTRING_LEN] = { "Tx ring %2d frames", "Tx ring %2d XDP frames", "Tx ring %2d XDP drops", + "Tx window drop %2d frames", }; static int enetc_get_sset_count(struct net_device *ndev, int sset) @@ -279,6 +280,7 @@ static void enetc_get_ethtool_stats(struct net_device *ndev, data[o++] = priv->tx_ring[i]->stats.packets; data[o++] = priv->tx_ring[i]->stats.xdp_tx; data[o++] = priv->tx_ring[i]->stats.xdp_tx_drops; + data[o++] = priv->tx_ring[i]->stats.win_drop; } for (i = 0; i < priv->num_rx_rings; i++) { diff --git a/drivers/net/ethernet/freescale/enetc/enetc_hw.h b/drivers/net/ethernet/freescale/enetc/enetc_hw.h index ce5b677e8c2f..647c87f73bf7 100644 --- a/drivers/net/ethernet/freescale/enetc/enetc_hw.h +++ b/drivers/net/ethernet/freescale/enetc/enetc_hw.h @@ -543,6 +543,7 @@ enum enetc_txbd_flags { ENETC_TXBD_FLAGS_EX = BIT(6), ENETC_TXBD_FLAGS_F = BIT(7) }; +#define ENETC_TXBD_STATS_WIN BIT(7) #define ENETC_TXBD_TXSTART_MASK GENMASK(24, 0) #define ENETC_TXBD_FLAGS_OFFSET 24 -- 2.25.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* RE: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-10 16:36 ` [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops Vladimir Oltean @ 2022-05-11 8:17 ` Claudiu Manoil 2022-05-11 22:27 ` Jakub Kicinski 1 sibling, 0 replies; 15+ messages in thread From: Claudiu Manoil @ 2022-05-11 8:17 UTC (permalink / raw) To: Vladimir Oltean, netdev Cc: Jakub Kicinski, David S. Miller, Paolo Abeni, Eric Dumazet, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu > -----Original Message----- > From: Vladimir Oltean <vladimir.oltean@nxp.com> > Sent: Tuesday, May 10, 2022 7:36 PM [...] > Subject: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops > > From: Po Liu <Po.Liu@nxp.com> > > The enetc scheduler for IEEE 802.1Qbv has 2 options (depending on > PTGCR[TG_DROP_DISABLE]) when we attempt to send an oversized packet > which will never fit in its allotted time slot for its traffic class: > either block the entire port due to head-of-line blocking, or drop the > packet and set a bit in the writeback format of the transmit buffer > descriptor, allowing other packets to be sent. > > We obviously choose the second option in the driver, but we do not > detect the drop condition, so from the perspective of the network stack, > the packet is sent and no error counter is incremented. > > This change checks the writeback of the TX BD when tc-taprio is enabled, > and increments a specific ethtool statistics counter and a generic > "tx_dropped" counter in ndo_get_stats64. > > Signed-off-by: Po Liu <Po.Liu@nxp.com> > Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com> Reviewed-by: Claudiu Manoil <claudiu.manoil@nxp.com> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-10 16:36 ` [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops Vladimir Oltean 2022-05-11 8:17 ` Claudiu Manoil @ 2022-05-11 22:27 ` Jakub Kicinski 2022-05-11 22:57 ` Vladimir Oltean 1 sibling, 1 reply; 15+ messages in thread From: Jakub Kicinski @ 2022-05-11 22:27 UTC (permalink / raw) To: Vladimir Oltean Cc: netdev, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu On Tue, 10 May 2022 19:36:15 +0300 Vladimir Oltean wrote: > From: Po Liu <Po.Liu@nxp.com> > > The enetc scheduler for IEEE 802.1Qbv has 2 options (depending on > PTGCR[TG_DROP_DISABLE]) when we attempt to send an oversized packet > which will never fit in its allotted time slot for its traffic class: > either block the entire port due to head-of-line blocking, or drop the > packet and set a bit in the writeback format of the transmit buffer > descriptor, allowing other packets to be sent. > > We obviously choose the second option in the driver, but we do not > detect the drop condition, so from the perspective of the network stack, > the packet is sent and no error counter is incremented. > > This change checks the writeback of the TX BD when tc-taprio is enabled, > and increments a specific ethtool statistics counter and a generic > "tx_dropped" counter in ndo_get_stats64. Is there no MIB attribute in the standard for such drops? The semantics seem petty implementation-independent can we put it into some structured ethtool stats instead? ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-11 22:27 ` Jakub Kicinski @ 2022-05-11 22:57 ` Vladimir Oltean 2022-05-11 23:13 ` Jakub Kicinski 0 siblings, 1 reply; 15+ messages in thread From: Vladimir Oltean @ 2022-05-11 22:57 UTC (permalink / raw) To: Jakub Kicinski Cc: netdev, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu On Wed, May 11, 2022 at 03:27:40PM -0700, Jakub Kicinski wrote: > On Tue, 10 May 2022 19:36:15 +0300 Vladimir Oltean wrote: > > From: Po Liu <Po.Liu@nxp.com> > > > > The enetc scheduler for IEEE 802.1Qbv has 2 options (depending on > > PTGCR[TG_DROP_DISABLE]) when we attempt to send an oversized packet > > which will never fit in its allotted time slot for its traffic class: > > either block the entire port due to head-of-line blocking, or drop the > > packet and set a bit in the writeback format of the transmit buffer > > descriptor, allowing other packets to be sent. > > > > We obviously choose the second option in the driver, but we do not > > detect the drop condition, so from the perspective of the network stack, > > the packet is sent and no error counter is incremented. > > > > This change checks the writeback of the TX BD when tc-taprio is enabled, > > and increments a specific ethtool statistics counter and a generic > > "tx_dropped" counter in ndo_get_stats64. > > Is there no MIB attribute in the standard for such drops? > > The semantics seem petty implementation-independent can we put it into > some structured ethtool stats instead? My copy of IEEE 802.1Q-2018 talks about the following MIB table entries in clause 17.2.22 Structure of the IEEE8021-ST-MIB: ieee8021STMaxSDU subtree ieee8021STMaxSDUTable ieee8021STTrafficClass ieee8021STMaxSDU ieee8021TransmissionOverrun ieee8021STParameters ieee8021STParametersTable ieee8021STGateEnabled ieee8021STAdminGateStates ieee8021STOperGateStates ieee8021STAdminControlListLength ieee8021STOperControlListLength ieee8021STAdminControlList ieee8021STOperControlList ieee8021STAdminCycleTimeNumerator ieee8021STAdminCycleTimeDenominator ieee8021STOperCycleTimeNumerator ieee8021STOperCycleTimeDenominator ieee8021STAdminCycleTimeExtension ieee8021STOperCycleTimeExtension ieee8021STAdminBaseTime ieee8021STOperBaseTime ieee8021STConfigChange ieee8021STConfigChangeTime ieee8021STTickGranularity ieee8021STCurrentTime ieee802STConfigPending ieee8021STSupportedListMax The only entry that is a counter in the Scheduled Traffic MIB is TransmissionOverrun, but that isn't what this is. Instead, this would be a TransmissionOverrunAvoidedByDropping, for which there appears to be no standardization. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-11 22:57 ` Vladimir Oltean @ 2022-05-11 23:13 ` Jakub Kicinski 2022-05-11 23:17 ` Vladimir Oltean 0 siblings, 1 reply; 15+ messages in thread From: Jakub Kicinski @ 2022-05-11 23:13 UTC (permalink / raw) To: Vladimir Oltean Cc: netdev, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu On Wed, 11 May 2022 22:57:46 +0000 Vladimir Oltean wrote: > The only entry that is a counter in the Scheduled Traffic MIB is TransmissionOverrun, > but that isn't what this is. Instead, this would be a TransmissionOverrunAvoidedByDropping, > for which there appears to be no standardization. TransmissionOversized? There's no standardization in terms of IEEE but the semantics seem pretty clear right? The packet is longer than the entire window so it can never go out? ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-11 23:13 ` Jakub Kicinski @ 2022-05-11 23:17 ` Vladimir Oltean 2022-05-11 23:36 ` Jakub Kicinski 0 siblings, 1 reply; 15+ messages in thread From: Vladimir Oltean @ 2022-05-11 23:17 UTC (permalink / raw) To: Jakub Kicinski Cc: netdev, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu On Wed, May 11, 2022 at 04:13:46PM -0700, Jakub Kicinski wrote: > On Wed, 11 May 2022 22:57:46 +0000 Vladimir Oltean wrote: > > The only entry that is a counter in the Scheduled Traffic MIB is TransmissionOverrun, > > but that isn't what this is. Instead, this would be a TransmissionOverrunAvoidedByDropping, > > for which there appears to be no standardization. > > TransmissionOversized? There's no standardization in terms of IEEE but > the semantics seem pretty clear right? The packet is longer than the > entire window so it can never go out? Yes, so what are you saying? Become the ad-hoc standards body for scheduled traffic? ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-11 23:17 ` Vladimir Oltean @ 2022-05-11 23:36 ` Jakub Kicinski 2022-05-12 0:20 ` Vladimir Oltean 0 siblings, 1 reply; 15+ messages in thread From: Jakub Kicinski @ 2022-05-11 23:36 UTC (permalink / raw) To: Vladimir Oltean Cc: netdev, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu On Wed, 11 May 2022 23:17:46 +0000 Vladimir Oltean wrote: > On Wed, May 11, 2022 at 04:13:46PM -0700, Jakub Kicinski wrote: > > On Wed, 11 May 2022 22:57:46 +0000 Vladimir Oltean wrote: > > > The only entry that is a counter in the Scheduled Traffic MIB is TransmissionOverrun, > > > but that isn't what this is. Instead, this would be a TransmissionOverrunAvoidedByDropping, > > > for which there appears to be no standardization. > > > > TransmissionOversized? There's no standardization in terms of IEEE but > > the semantics seem pretty clear right? The packet is longer than the > > entire window so it can never go out? > > Yes, so what are you saying? Become the ad-hoc standards body for > scheduled traffic? We can argue semantics but there doesn't need to be a "standards body" to add a structured stat in ethtool [1]. When next gen of enetc comes out you'll likely try to use the same stat name or reuse the entire driver. So you are already defining uAPI for your users, it's only a question of scope at which the uAPI is defined. What I'm not sure of is what to attach that statistic to. You have it per ring and we famously don't have per ring APIs, so whatever, let me apply as is and move on :) [1] Coincidentally I plan to add a "real link loss" statistic there because AFAICR IEEE doesn't have a stat for it, and carrier_changes count software events so it's meaningless to teams trying to track cable issues. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-11 23:36 ` Jakub Kicinski @ 2022-05-12 0:20 ` Vladimir Oltean 2022-05-12 0:52 ` Jakub Kicinski 0 siblings, 1 reply; 15+ messages in thread From: Vladimir Oltean @ 2022-05-12 0:20 UTC (permalink / raw) To: Jakub Kicinski Cc: netdev, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu On Wed, May 11, 2022 at 04:36:55PM -0700, Jakub Kicinski wrote: > On Wed, 11 May 2022 23:17:46 +0000 Vladimir Oltean wrote: > > On Wed, May 11, 2022 at 04:13:46PM -0700, Jakub Kicinski wrote: > > > On Wed, 11 May 2022 22:57:46 +0000 Vladimir Oltean wrote: > > > > The only entry that is a counter in the Scheduled Traffic MIB is TransmissionOverrun, > > > > but that isn't what this is. Instead, this would be a TransmissionOverrunAvoidedByDropping, > > > > for which there appears to be no standardization. > > > > > > TransmissionOversized? There's no standardization in terms of IEEE but > > > the semantics seem pretty clear right? The packet is longer than the > > > entire window so it can never go out? > > > > Yes, so what are you saying? Become the ad-hoc standards body for > > scheduled traffic? > > We can argue semantics but there doesn't need to be a "standards body" > to add a structured stat in ethtool [1]. When next gen of enetc comes > out you'll likely try to use the same stat name or reuse the entire > driver. So you are already defining uAPI for your users, it's only > a question of scope at which the uAPI is defined. The trouble with over-standardization is that with a different driver that would use this ad-hoc structure for parts of it, you never know if a counter is 0 because it's 0 or because it's not implemented. As unstructured as the plain ethtool -S might be, at least if you see a counter there, you can expect that it actually counts something. > What I'm not sure of is what to attach that statistic to. You have it > per ring and we famously don't have per ring APIs, so whatever, let > me apply as is and move on :) It would probably have to be per traffic class, since the media reservation gates are per traffic class (TX rings have a configurable mapping with traffic classes). Although an aggregate counter would also be plausible. Who knows? I haven't seen this specific counter being reported by the LS1028A switch, for example (I'll have to check what increments on blocked transmission overruns). > [1] Coincidentally I plan to add a "real link loss" statistic there > because AFAICR IEEE doesn't have a stat for it, and carrier_changes > count software events so it's meaningless to teams trying to track > cable issues. I didn't quite get what's wrong with the carrier_changes sysfs counter, and how "real link loss" would be implemented differently/more usefully? At least with phylib/phylink users, netif_carrier_on() + netif_carrier_off() are called exactly on phydev->phy_link_change() events. Are there other callers of netif_carrier_*() that pollute this counter and make it useless for reliable debugging? ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops 2022-05-12 0:20 ` Vladimir Oltean @ 2022-05-12 0:52 ` Jakub Kicinski 0 siblings, 0 replies; 15+ messages in thread From: Jakub Kicinski @ 2022-05-12 0:52 UTC (permalink / raw) To: Vladimir Oltean Cc: netdev, David S. Miller, Paolo Abeni, Eric Dumazet, Claudiu Manoil, Vinicius Costa Gomes, Michael Walle, Xiaoliang Yang, Po Liu On Thu, 12 May 2022 00:20:18 +0000 Vladimir Oltean wrote: > > We can argue semantics but there doesn't need to be a "standards body" > > to add a structured stat in ethtool [1]. When next gen of enetc comes > > out you'll likely try to use the same stat name or reuse the entire > > driver. So you are already defining uAPI for your users, it's only > > a question of scope at which the uAPI is defined. > > The trouble with over-standardization is that with a different driver > that would use this ad-hoc structure for parts of it, you never know if > a counter is 0 because it's 0 or because it's not implemented. > As unstructured as the plain ethtool -S might be, at least if you see a > counter there, you can expect that it actually counts something. That's solved with the netlink ethtool stats. What's not repored by the driver is not reported to user space. Grep for ETHTOOL_STAT_NOT_SET. Maybe not beautiful but works. > > What I'm not sure of is what to attach that statistic to. You have it > > per ring and we famously don't have per ring APIs, so whatever, let > > me apply as is and move on :) > > It would probably have to be per traffic class, since the media > reservation gates are per traffic class (TX rings have a configurable > mapping with traffic classes). Although an aggregate counter would also > be plausible. Who knows? Well, users sometimes know what they want but the days when the kernel was written by its users are long gone. Or maybe that's just a perfect example of the "good old days" fallacy :) > I haven't seen this specific counter being reported by the LS1028A > switch, for example (I'll have to check what increments on blocked > transmission overruns). > > > [1] Coincidentally I plan to add a "real link loss" statistic there > > because AFAICR IEEE doesn't have a stat for it, and carrier_changes > > count software events so it's meaningless to teams trying to track > > cable issues. > > I didn't quite get what's wrong with the carrier_changes sysfs > counter, and how "real link loss" would be implemented > differently/more usefully? At least with phylib/phylink users, > netif_carrier_on() + netif_carrier_off() are called exactly on > phydev->phy_link_change() events. Are there other callers of > netif_carrier_*() that pollute this counter and make it useless for > reliable debugging? Yup, drivers will do a netif_carrier_off() to stop Tx and prevent the timeout watchdog from kicking in while they are doing some form of reconfig (ethtool -L / -G etc.). I guess we can add a special API for taking things down without bumping the counter. Since drivers I work with already report an ethtool -S stat from the device for "PHY really went down" my first instinct was a better ethtool stat... ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2022-05-12 0:52 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-06-02 12:21 [PATCH net-next 0/2] Report tc-taprio window drops in NXP ENETC driver Vladimir Oltean 2021-06-02 12:21 ` [PATCH net-next 1/2] net: enetc: manage ENETC_F_QBV in priv->active_offloads only when enabled Vladimir Oltean 2021-06-02 12:21 ` [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops Vladimir Oltean 2021-06-02 17:19 ` Jakub Kicinski 2021-06-02 17:59 ` Vladimir Oltean 2021-06-02 21:30 ` Jakub Kicinski 2022-05-10 16:36 [PATCH net-next 0/2] Count tc-taprio window drops in enetc driver Vladimir Oltean 2022-05-10 16:36 ` [PATCH net-next 2/2] net: enetc: count the tc-taprio window drops Vladimir Oltean 2022-05-11 8:17 ` Claudiu Manoil 2022-05-11 22:27 ` Jakub Kicinski 2022-05-11 22:57 ` Vladimir Oltean 2022-05-11 23:13 ` Jakub Kicinski 2022-05-11 23:17 ` Vladimir Oltean 2022-05-11 23:36 ` Jakub Kicinski 2022-05-12 0:20 ` Vladimir Oltean 2022-05-12 0:52 ` Jakub Kicinski
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.