[net-next] enetc: fix locking for one-step timestamping packet transfer

[net-next] enetc: fix locking for one-step timestamping packet transfer

[Yangbo Lu]

The previous patch to support PTP Sync packet one-step timestamping
described one-step timestamping packet handling logic as below in
commit message:

- Trasmit packet immediately if no other one in transfer, or queue to
  skb queue if there is already one in transfer.
  The test_and_set_bit_lock() is used here to lock and check state.
- Start a work when complete transfer on hardware, to release the bit
  lock and to send one skb in skb queue if has.

There was not problem of the description, but there was a mistake in
implementation. The locking/test_and_set_bit_lock() should be put in
enetc_start_xmit() which may be called by worker, rather than in
enetc_xmit(). Otherwise, the worker calling enetc_start_xmit() after
bit lock released is not able to lock again for transfer.

Fixes: 7294380c5211 ("enetc: support PTP Sync packet one-step timestamping")
Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
---
 drivers/net/ethernet/freescale/enetc/enetc.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c
index 4a0adb0b8bd7..65f1772c5740 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc.c
@@ -301,6 +301,15 @@ static netdev_tx_t enetc_start_xmit(struct sk_buff *skb,
 	struct enetc_bdr *tx_ring;
 	int count;
 
+	/* Queue one-step Sync packet if already locked */
+	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
+		if (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,
+					  &priv->flags)) {
+			skb_queue_tail(&priv->tx_skbs, skb);
+			return NETDEV_TX_OK;
+		}
+	}
+
 	tx_ring = priv->tx_ring[skb->queue_mapping];
 
 	if (unlikely(skb_shinfo(skb)->nr_frags > ENETC_MAX_SKB_FRAGS))
@@ -352,15 +361,6 @@ netdev_tx_t enetc_xmit(struct sk_buff *skb, struct net_device *ndev)
 			skb->cb[0] = ENETC_F_TX_TSTAMP;
 	}
 
-	/* Queue one-step Sync packet if already locked */
-	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
-		if (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,
-					  &priv->flags)) {
-			skb_queue_tail(&priv->tx_skbs, skb);
-			return NETDEV_TX_OK;
-		}
-	}
-
 	return enetc_start_xmit(skb, ndev);
 }
 

base-commit: 8ef7adc6beb2ef0bce83513dc9e4505e7b21e8c2
-- 
2.25.1

RE: [net-next] enetc: fix locking for one-step timestamping packet transfer

[Claudiu Manoil]

>-----Original Message-----
>From: Yangbo Lu <yangbo.lu@nxp.com>
>Sent: Tuesday, April 13, 2021 6:48 AM
[...]
>Subject: [net-next] enetc: fix locking for one-step timestamping packet
>transfer
>
>The previous patch to support PTP Sync packet one-step timestamping
>described one-step timestamping packet handling logic as below in
>commit message:
>
>- Trasmit packet immediately if no other one in transfer, or queue to
>  skb queue if there is already one in transfer.
>  The test_and_set_bit_lock() is used here to lock and check state.
>- Start a work when complete transfer on hardware, to release the bit
>  lock and to send one skb in skb queue if has.
>
>There was not problem of the description, but there was a mistake in
>implementation. The locking/test_and_set_bit_lock() should be put in
>enetc_start_xmit() which may be called by worker, rather than in
>enetc_xmit(). Otherwise, the worker calling enetc_start_xmit() after
>bit lock released is not able to lock again for transfer.
>
>Fixes: 7294380c5211 ("enetc: support PTP Sync packet one-step
>timestamping")
>Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
>---

Reviewed-by: Claudiu Manoil <claudiu.manoil@nxp.com>

Re: [net-next] enetc: fix locking for one-step timestamping packet transfer

[Jakub Kicinski]

On Tue, 13 Apr 2021 11:48:17 +0800 Yangbo Lu wrote:
> +	/* Queue one-step Sync packet if already locked */
> +	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
> +		if (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,
> +					  &priv->flags)) {
> +			skb_queue_tail(&priv->tx_skbs, skb);
> +			return NETDEV_TX_OK;
> +		}
> +	}

Isn't this missing queue_work() as well?

Also as I mentioned I don't understand why you created a separate
workqueue instead of using the system workqueue via schedule_work().

RE: [net-next] enetc: fix locking for one-step timestamping packet transfer

[Y.b. Lu]

Hi Jakub,

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: 2021年4月14日 1:11
> To: Y.b. Lu <yangbo.lu@nxp.com>
> Cc: netdev@vger.kernel.org; David S . Miller <davem@davemloft.net>; Richard
> Cochran <richardcochran@gmail.com>; Claudiu Manoil
> <claudiu.manoil@nxp.com>; Vladimir Oltean <vladimir.oltean@nxp.com>;
> Russell King <linux@armlinux.org.uk>
> Subject: Re: [net-next] enetc: fix locking for one-step timestamping packet
> transfer
> 
> On Tue, 13 Apr 2021 11:48:17 +0800 Yangbo Lu wrote:
> > +	/* Queue one-step Sync packet if already locked */
> > +	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
> > +		if
> (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,
> > +					  &priv->flags)) {
> > +			skb_queue_tail(&priv->tx_skbs, skb);
> > +			return NETDEV_TX_OK;
> > +		}
> > +	}
> 
> Isn't this missing queue_work() as well?
> 
> Also as I mentioned I don't understand why you created a separate workqueue
> instead of using the system workqueue via schedule_work().

queue_work(system_wq, ) was put in clean_tx. I finally followed the logic you suggested :)

See my reply in v2 discussion
https://patchwork.kernel.org/project/netdevbpf/patch/20210408111350.3817-3-yangbo.lu@nxp.com/

And v3 patch merged (7294380c5211 enetc: support PTP Sync packet one-step timestamping)
https://patchwork.kernel.org/project/netdevbpf/patch/20210412090327.22330-3-yangbo.lu@nxp.com/

Thanks.

Re: [net-next] enetc: fix locking for one-step timestamping packet transfer

[Jakub Kicinski]

On Wed, 14 Apr 2021 06:18:57 +0000 Y.b. Lu wrote:
> > On Tue, 13 Apr 2021 11:48:17 +0800 Yangbo Lu wrote:  
> > > +	/* Queue one-step Sync packet if already locked */
> > > +	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
> > > +		if  
> > (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,  
> > > +					  &priv->flags)) {
> > > +			skb_queue_tail(&priv->tx_skbs, skb);
> > > +			return NETDEV_TX_OK;
> > > +		}
> > > +	}  
> > 
> > Isn't this missing queue_work() as well?
> > 
> > Also as I mentioned I don't understand why you created a separate workqueue
> > instead of using the system workqueue via schedule_work().  
> 
> queue_work(system_wq, ) was put in clean_tx. I finally followed the
> logic you suggested :)

Ah, I didn't look close enough. I was expecting to see schedule_work(),
please consider sending a follow up, queue_work(system_wq, $work) is a
rare construct.

RE: [net-next] enetc: fix locking for one-step timestamping packet transfer

[Y.b. Lu]

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: 2021年4月15日 0:36
> To: Y.b. Lu <yangbo.lu@nxp.com>
> Cc: netdev@vger.kernel.org; David S . Miller <davem@davemloft.net>; Richard
> Cochran <richardcochran@gmail.com>; Claudiu Manoil
> <claudiu.manoil@nxp.com>; Vladimir Oltean <vladimir.oltean@nxp.com>;
> Russell King <linux@armlinux.org.uk>
> Subject: Re: [net-next] enetc: fix locking for one-step timestamping packet
> transfer
> 
> On Wed, 14 Apr 2021 06:18:57 +0000 Y.b. Lu wrote:
> > > On Tue, 13 Apr 2021 11:48:17 +0800 Yangbo Lu wrote:
> > > > +	/* Queue one-step Sync packet if already locked */
> > > > +	if (skb->cb[0] & ENETC_F_TX_ONESTEP_SYNC_TSTAMP) {
> > > > +		if
> > > (test_and_set_bit_lock(ENETC_TX_ONESTEP_TSTAMP_IN_PROGRESS,
> > > > +					  &priv->flags)) {
> > > > +			skb_queue_tail(&priv->tx_skbs, skb);
> > > > +			return NETDEV_TX_OK;
> > > > +		}
> > > > +	}
> > >
> > > Isn't this missing queue_work() as well?
> > >
> > > Also as I mentioned I don't understand why you created a separate
> > > workqueue instead of using the system workqueue via schedule_work().
> >
> > queue_work(system_wq, ) was put in clean_tx. I finally followed the
> > logic you suggested :)
> 
> Ah, I didn't look close enough. I was expecting to see schedule_work(), please
> consider sending a follow up, queue_work(system_wq, $work) is a rare
> construct.

Thank you Jakub. I sent another patch for this.
https://patchwork.kernel.org/project/netdevbpf/patch/20210415053455.10029-1-yangbo.lu@nxp.com/

RE: [net-next] enetc: fix locking for one-step timestamping packet transfer

[Y.b. Lu]

Hi,

I think this patch was reviewed and no objection now, right? (I see status is " Changes Requested ".)
Thanks.

> -----Original Message-----
> From: Yangbo Lu <yangbo.lu@nxp.com>
> Sent: 2021年4月13日 11:48
> To: netdev@vger.kernel.org
> Cc: Y.b. Lu <yangbo.lu@nxp.com>; David S . Miller <davem@davemloft.net>;
> Richard Cochran <richardcochran@gmail.com>; Claudiu Manoil
> <claudiu.manoil@nxp.com>; Jakub Kicinski <kuba@kernel.org>; Vladimir
> Oltean <vladimir.oltean@nxp.com>; Russell King <linux@armlinux.org.uk>
> Subject: [net-next] enetc: fix locking for one-step timestamping packet transfer
> 
> The previous patch to support PTP Sync packet one-step timestamping
> described one-step timestamping packet handling logic as below in commit
> message:
> 
> - Trasmit packet immediately if no other one in transfer, or queue to
>   skb queue if there is already one in transfer.
>   The test_and_set_bit_lock() is used here to lock and check state.
> - Start a work when complete transfer on hardware, to release the bit
>   lock and to send one skb in skb queue if has.
> 
> There was not problem of the description, but there was a mistake in
> implementation. The locking/test_and_set_bit_lock() should be put in
> enetc_start_xmit() which may be called by worker, rather than in enetc_xmit().
> Otherwise, the worker calling enetc_start_xmit() after bit lock released is not
> able to lock again for transfer.
> 
> Fixes: 7294380c5211 ("enetc: support PTP Sync packet one-step
> timestamping")
> Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
[...]