[PATCH net] net: enetc: preserve TX ring priority across reconfiguration

[PATCH net] net: enetc: preserve TX ring priority across reconfiguration

[Vladimir Oltean]

In the blamed commit, a rudimentary reallocation procedure for RX buffer
descriptors was implemented, for the situation when their format changes
between normal (no PTP) and extended (PTP).

enetc_hwtstamp_set() calls enetc_close() and enetc_open() in a sequence,
and this sequence loses information which was previously configured in
the TX BDR Mode Register, specifically via the enetc_set_bdr_prio() call.
The TX ring priority is configured by tc-mqprio and tc-taprio, and
affects important things for TSN such as the TX time of packets. The
issue manifests itself most visibly by the fact that isochron --txtime
reports premature packet transmissions when PTP is first enabled on an
enetc interface.

Save the TX ring priority in a new field in struct enetc_bdr (occupies a
2 byte hole on arm64) in order to make this survive a ring reconfiguration.

Fixes: 434cebabd3a2 ("enetc: Add dynamic allocation of extended Rx BD rings")
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/ethernet/freescale/enetc/enetc.c  |  8 ++++---
 drivers/net/ethernet/freescale/enetc/enetc.h  |  1 +
 .../net/ethernet/freescale/enetc/enetc_qos.c  | 21 ++++++++++++-------
 3 files changed, 19 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c
index b4988fe6bb93..9c0ab983b6e9 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc.c
@@ -2061,7 +2061,7 @@ static void enetc_setup_txbdr(struct enetc_hw *hw, struct enetc_bdr *tx_ring)
 	/* enable Tx ints by setting pkt thr to 1 */
 	enetc_txbdr_wr(hw, idx, ENETC_TBICR0, ENETC_TBICR0_ICEN | 0x1);
 
-	tbmr = ENETC_TBMR_EN;
+	tbmr = ENETC_TBMR_EN | ENETC_TBMR_SET_PRIO(tx_ring->prio);
 	if (tx_ring->ndev->features & NETIF_F_HW_VLAN_CTAG_TX)
 		tbmr |= ENETC_TBMR_VIH;
 
@@ -2464,7 +2464,8 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
 		/* Reset all ring priorities to 0 */
 		for (i = 0; i < priv->num_tx_rings; i++) {
 			tx_ring = priv->tx_ring[i];
-			enetc_set_bdr_prio(hw, tx_ring->index, 0);
+			tx_ring->prio = 0;
+			enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
 		}
 
 		return 0;
@@ -2483,7 +2484,8 @@ int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
 	 */
 	for (i = 0; i < num_tc; i++) {
 		tx_ring = priv->tx_ring[i];
-		enetc_set_bdr_prio(hw, tx_ring->index, i);
+		tx_ring->prio = i;
+		enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
 	}
 
 	/* Reset the number of netdev queues based on the TC count */
diff --git a/drivers/net/ethernet/freescale/enetc/enetc.h b/drivers/net/ethernet/freescale/enetc/enetc.h
index ea3ceed5bfa9..40bcd68c0c49 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.h
+++ b/drivers/net/ethernet/freescale/enetc/enetc.h
@@ -95,6 +95,7 @@ struct enetc_bdr {
 		void __iomem *rcir;
 	};
 	u16 index;
+	u16 prio;
 	int bd_count; /* # of BDs */
 	int next_to_use;
 	int next_to_clean;
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_qos.c b/drivers/net/ethernet/freescale/enetc/enetc_qos.c
index a842e1999122..fcebb54224c0 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_qos.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_qos.c
@@ -137,6 +137,7 @@ int enetc_setup_tc_taprio(struct net_device *ndev, void *type_data)
 	struct tc_taprio_qopt_offload *taprio = type_data;
 	struct enetc_ndev_priv *priv = netdev_priv(ndev);
 	struct enetc_hw *hw = &priv->si->hw;
+	struct enetc_bdr *tx_ring;
 	int err;
 	int i;
 
@@ -145,16 +146,20 @@ int enetc_setup_tc_taprio(struct net_device *ndev, void *type_data)
 		if (priv->tx_ring[i]->tsd_enable)
 			return -EBUSY;
 
-	for (i = 0; i < priv->num_tx_rings; i++)
-		enetc_set_bdr_prio(hw, priv->tx_ring[i]->index,
-				   taprio->enable ? i : 0);
+	for (i = 0; i < priv->num_tx_rings; i++) {
+		tx_ring = priv->tx_ring[i];
+		tx_ring->prio = taprio->enable ? i : 0;
+		enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
+	}
 
 	err = enetc_setup_taprio(ndev, taprio);
-
-	if (err)
-		for (i = 0; i < priv->num_tx_rings; i++)
-			enetc_set_bdr_prio(hw, priv->tx_ring[i]->index,
-					   taprio->enable ? 0 : i);
+	if (err) {
+		for (i = 0; i < priv->num_tx_rings; i++) {
+			tx_ring = priv->tx_ring[i];
+			tx_ring->prio = taprio->enable ? 0 : i;
+			enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
+		}
+	}
 
 	return err;
 }
-- 
2.34.1

Re: [PATCH net] net: enetc: preserve TX ring priority across reconfiguration

[Alexander Lobakin]

From: Vladimir Oltean <vladimir.oltean@nxp.com>
Date: Tue, 22 Nov 2022 15:09:36 +0200

> In the blamed commit, a rudimentary reallocation procedure for RX buffer
> descriptors was implemented, for the situation when their format changes
> between normal (no PTP) and extended (PTP).
> 
> enetc_hwtstamp_set() calls enetc_close() and enetc_open() in a sequence,
> and this sequence loses information which was previously configured in
> the TX BDR Mode Register, specifically via the enetc_set_bdr_prio() call.
> The TX ring priority is configured by tc-mqprio and tc-taprio, and
> affects important things for TSN such as the TX time of packets. The
> issue manifests itself most visibly by the fact that isochron --txtime
> reports premature packet transmissions when PTP is first enabled on an
> enetc interface.
> 
> Save the TX ring priority in a new field in struct enetc_bdr (occupies a
> 2 byte hole on arm64) in order to make this survive a ring reconfiguration.
> 
> Fixes: 434cebabd3a2 ("enetc: Add dynamic allocation of extended Rx BD rings")
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
> ---
>  drivers/net/ethernet/freescale/enetc/enetc.c  |  8 ++++---
>  drivers/net/ethernet/freescale/enetc/enetc.h  |  1 +
>  .../net/ethernet/freescale/enetc/enetc_qos.c  | 21 ++++++++++++-------
>  3 files changed, 19 insertions(+), 11 deletions(-)

[...]

>  	err = enetc_setup_taprio(ndev, taprio);
> -
> -	if (err)
> -		for (i = 0; i < priv->num_tx_rings; i++)
> -			enetc_set_bdr_prio(hw, priv->tx_ring[i]->index,
> -					   taprio->enable ? 0 : i);
> +	if (err) {
> +		for (i = 0; i < priv->num_tx_rings; i++) {
> +			tx_ring = priv->tx_ring[i];
> +			tx_ring->prio = taprio->enable ? 0 : i;

Side note: is that `taprio ? 0 : i` correct? It's an error path
IIUC, why not just unconditional 0?
I guess it is, so

Reviewed-by: Alexander Lobakin <alexandr.lobakin@intel.com>

> +			enetc_set_bdr_prio(hw, tx_ring->index, tx_ring->prio);
> +		}
> +	}
>  
>  	return err;
>  }
> -- 
> 2.34.1

Thanks,
Olek

Re: [PATCH net] net: enetc: preserve TX ring priority across reconfiguration

[Vladimir Oltean]

On Wed, Nov 23, 2022 at 04:51:16PM +0100, Alexander Lobakin wrote:
> > +	if (err) {
> > +		for (i = 0; i < priv->num_tx_rings; i++) {
> > +			tx_ring = priv->tx_ring[i];
> > +			tx_ring->prio = taprio->enable ? 0 : i;
> 
> Side note: is that `taprio ? 0 : i` correct? It's an error path
> IIUC, why not just unconditional 0?

Yes, it is intended. On error path the priorities are restored.
If taprio->enable was false, but disabling failed, the ring priorities
need to be set as if taprio->enable was true.

Re: [PATCH net] net: enetc: preserve TX ring priority across reconfiguration

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net.git (master)
by Jakub Kicinski <kuba@kernel.org>:

On Tue, 22 Nov 2022 15:09:36 +0200 you wrote:
> In the blamed commit, a rudimentary reallocation procedure for RX buffer
> descriptors was implemented, for the situation when their format changes
> between normal (no PTP) and extended (PTP).
> 
> enetc_hwtstamp_set() calls enetc_close() and enetc_open() in a sequence,
> and this sequence loses information which was previously configured in
> the TX BDR Mode Register, specifically via the enetc_set_bdr_prio() call.
> The TX ring priority is configured by tc-mqprio and tc-taprio, and
> affects important things for TSN such as the TX time of packets. The
> issue manifests itself most visibly by the fact that isochron --txtime
> reports premature packet transmissions when PTP is first enabled on an
> enetc interface.
> 
> [...]

Here is the summary with links:
  - [net] net: enetc: preserve TX ring priority across reconfiguration
    https://git.kernel.org/netdev/net/c/290b5fe096e7

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html