* [PATCH] gianfar: Fix TX ring processing on SMP machines
@ 2010-03-03 18:18 ` Anton Vorontsov
0 siblings, 0 replies; 8+ messages in thread
From: Anton Vorontsov @ 2010-03-03 18:18 UTC (permalink / raw)
To: David Miller
Cc: linuxppc-dev, netdev, Martyn Welch, Paul Gortmaker, Sandeep Gopalpet
Starting with commit a3bc1f11e9b867a4f49505 ("gianfar: Revive SKB
recycling") gianfar driver sooner or later stops transmitting any
packets on SMP machines.
start_xmit() prepares new skb for transmitting, generally it does
three things:
1. sets up all BDs (marks them ready to send), except the first one.
2. stores skb into tx_queue->tx_skbuff so that clean_tx_ring()
would cleanup it later.
3. sets up the first BD, i.e. marks it ready.
Here is what clean_tx_ring() does:
1. reads skbs from tx_queue->tx_skbuff
2. checks if the *last* BD is ready. If it's still ready [to send]
then it it isn't transmitted, so clean_tx_ring() returns.
Otherwise it actually cleanups BDs. All is OK.
Now, if there is just one BD, code flow:
- start_xmit(): stores skb into tx_skbuff. Note that the first BD
(which is also the last one) isn't marked as ready, yet.
- clean_tx_ring(): sees that skb is not null, *and* its lstatus
says that it is NOT ready (like if BD was sent), so it cleans
it up (bad!)
- start_xmit(): marks BD as ready [to send], but it's too late.
We can fix this simply by reordering lstatus/tx_skbuff writes.
Reported-by: Martyn Welch <martyn.welch@ge.com>
Bisected-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
Tested-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Tested-by: Martyn Welch <martyn.welch@ge.com>
Cc: Sandeep Gopalpet <Sandeep.Kumar@freescale.com>
Cc: Stable <stable@vger.kernel.org> [2.6.33]
---
drivers/net/gianfar.c | 5 ++++-
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
index 8bd3c9f..cccb409 100644
--- a/drivers/net/gianfar.c
+++ b/drivers/net/gianfar.c
@@ -2021,7 +2021,6 @@ static int gfar_start_xmit(struct sk_buff *skb, struct net_device *dev)
}
/* setup the TxBD length and buffer pointer for the first BD */
- tx_queue->tx_skbuff[tx_queue->skb_curtx] = skb;
txbdp_start->bufPtr = dma_map_single(&priv->ofdev->dev, skb->data,
skb_headlen(skb), DMA_TO_DEVICE);
@@ -2053,6 +2052,10 @@ static int gfar_start_xmit(struct sk_buff *skb, struct net_device *dev)
txbdp_start->lstatus = lstatus;
+ eieio(); /* force lstatus write before tx_skbuff */
+
+ tx_queue->tx_skbuff[tx_queue->skb_curtx] = skb;
+
/* Update the current skb pointer to the next entry we will use
* (wrapping if necessary) */
tx_queue->skb_curtx = (tx_queue->skb_curtx + 1) &
--
1.7.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH] gianfar: Fix TX ring processing on SMP machines
2010-03-03 18:18 ` Anton Vorontsov
@ 2010-03-04 8:41 ` David Miller
-1 siblings, 0 replies; 8+ messages in thread
From: David Miller @ 2010-03-04 8:41 UTC (permalink / raw)
To: avorontsov
Cc: martyn.welch, paul.gortmaker, Sandeep.Kumar, galak, linuxppc-dev, netdev
From: Anton Vorontsov <avorontsov@ru.mvista.com>
Date: Wed, 3 Mar 2010 21:18:58 +0300
> Starting with commit a3bc1f11e9b867a4f49505 ("gianfar: Revive SKB
> recycling") gianfar driver sooner or later stops transmitting any
> packets on SMP machines.
>
> start_xmit() prepares new skb for transmitting, generally it does
> three things:
>
> 1. sets up all BDs (marks them ready to send), except the first one.
> 2. stores skb into tx_queue->tx_skbuff so that clean_tx_ring()
> would cleanup it later.
> 3. sets up the first BD, i.e. marks it ready.
>
> Here is what clean_tx_ring() does:
>
> 1. reads skbs from tx_queue->tx_skbuff
> 2. checks if the *last* BD is ready. If it's still ready [to send]
> then it it isn't transmitted, so clean_tx_ring() returns.
> Otherwise it actually cleanups BDs. All is OK.
>
> Now, if there is just one BD, code flow:
>
> - start_xmit(): stores skb into tx_skbuff. Note that the first BD
> (which is also the last one) isn't marked as ready, yet.
> - clean_tx_ring(): sees that skb is not null, *and* its lstatus
> says that it is NOT ready (like if BD was sent), so it cleans
> it up (bad!)
> - start_xmit(): marks BD as ready [to send], but it's too late.
>
> We can fix this simply by reordering lstatus/tx_skbuff writes.
>
> Reported-by: Martyn Welch <martyn.welch@ge.com>
> Bisected-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
> Tested-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> Tested-by: Martyn Welch <martyn.welch@ge.com>
Applied.
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] gianfar: Fix TX ring processing on SMP machines
@ 2010-03-04 8:41 ` David Miller
0 siblings, 0 replies; 8+ messages in thread
From: David Miller @ 2010-03-04 8:41 UTC (permalink / raw)
To: avorontsov
Cc: linuxppc-dev, netdev, martyn.welch, paul.gortmaker, Sandeep.Kumar
From: Anton Vorontsov <avorontsov@ru.mvista.com>
Date: Wed, 3 Mar 2010 21:18:58 +0300
> Starting with commit a3bc1f11e9b867a4f49505 ("gianfar: Revive SKB
> recycling") gianfar driver sooner or later stops transmitting any
> packets on SMP machines.
>
> start_xmit() prepares new skb for transmitting, generally it does
> three things:
>
> 1. sets up all BDs (marks them ready to send), except the first one.
> 2. stores skb into tx_queue->tx_skbuff so that clean_tx_ring()
> would cleanup it later.
> 3. sets up the first BD, i.e. marks it ready.
>
> Here is what clean_tx_ring() does:
>
> 1. reads skbs from tx_queue->tx_skbuff
> 2. checks if the *last* BD is ready. If it's still ready [to send]
> then it it isn't transmitted, so clean_tx_ring() returns.
> Otherwise it actually cleanups BDs. All is OK.
>
> Now, if there is just one BD, code flow:
>
> - start_xmit(): stores skb into tx_skbuff. Note that the first BD
> (which is also the last one) isn't marked as ready, yet.
> - clean_tx_ring(): sees that skb is not null, *and* its lstatus
> says that it is NOT ready (like if BD was sent), so it cleans
> it up (bad!)
> - start_xmit(): marks BD as ready [to send], but it's too late.
>
> We can fix this simply by reordering lstatus/tx_skbuff writes.
>
> Reported-by: Martyn Welch <martyn.welch@ge.com>
> Bisected-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
> Tested-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> Tested-by: Martyn Welch <martyn.welch@ge.com>
Applied.
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] gianfar: Fix TX ring processing on SMP machines
2010-03-04 8:41 ` David Miller
@ 2010-03-04 16:34 ` Kumar Gala
-1 siblings, 0 replies; 8+ messages in thread
From: Kumar Gala @ 2010-03-04 16:34 UTC (permalink / raw)
To: Anton Vorontsov
Cc: linuxppc-dev list, Netdev, Martyn Welch, Paul Gortmaker,
Sandeep Gopalpet, David Miller
On Mar 4, 2010, at 2:41 AM, David Miller wrote:
> From: Anton Vorontsov <avorontsov@ru.mvista.com>
> Date: Wed, 3 Mar 2010 21:18:58 +0300
>
>> Starting with commit a3bc1f11e9b867a4f49505 ("gianfar: Revive SKB
>> recycling") gianfar driver sooner or later stops transmitting any
>> packets on SMP machines.
>>
>> start_xmit() prepares new skb for transmitting, generally it does
>> three things:
>>
>> 1. sets up all BDs (marks them ready to send), except the first one.
>> 2. stores skb into tx_queue->tx_skbuff so that clean_tx_ring()
>> would cleanup it later.
>> 3. sets up the first BD, i.e. marks it ready.
>>
>> Here is what clean_tx_ring() does:
>>
>> 1. reads skbs from tx_queue->tx_skbuff
>> 2. checks if the *last* BD is ready. If it's still ready [to send]
>> then it it isn't transmitted, so clean_tx_ring() returns.
>> Otherwise it actually cleanups BDs. All is OK.
>>
>> Now, if there is just one BD, code flow:
>>
>> - start_xmit(): stores skb into tx_skbuff. Note that the first BD
>> (which is also the last one) isn't marked as ready, yet.
>> - clean_tx_ring(): sees that skb is not null, *and* its lstatus
>> says that it is NOT ready (like if BD was sent), so it cleans
>> it up (bad!)
>> - start_xmit(): marks BD as ready [to send], but it's too late.
>>
>> We can fix this simply by reordering lstatus/tx_skbuff writes.
>>
>> Reported-by: Martyn Welch <martyn.welch@ge.com>
>> Bisected-by: Paul Gortmaker <paul.gortmaker@windriver.com>
>> Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
>> Tested-by: Paul Gortmaker <paul.gortmaker@windriver.com>
>> Tested-by: Martyn Welch <martyn.welch@ge.com>
>
> Applied.
Anton,
Once this makes it into Linus's tree can you make sure we get it added to -stable.
- k
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] gianfar: Fix TX ring processing on SMP machines
@ 2010-03-04 16:34 ` Kumar Gala
0 siblings, 0 replies; 8+ messages in thread
From: Kumar Gala @ 2010-03-04 16:34 UTC (permalink / raw)
To: Anton Vorontsov
Cc: linuxppc-dev list, Netdev, Martyn Welch, Paul Gortmaker,
Sandeep Gopalpet, David Miller
On Mar 4, 2010, at 2:41 AM, David Miller wrote:
> From: Anton Vorontsov <avorontsov@ru.mvista.com>
> Date: Wed, 3 Mar 2010 21:18:58 +0300
>=20
>> Starting with commit a3bc1f11e9b867a4f49505 ("gianfar: Revive SKB
>> recycling") gianfar driver sooner or later stops transmitting any
>> packets on SMP machines.
>>=20
>> start_xmit() prepares new skb for transmitting, generally it does
>> three things:
>>=20
>> 1. sets up all BDs (marks them ready to send), except the first one.
>> 2. stores skb into tx_queue->tx_skbuff so that clean_tx_ring()
>> would cleanup it later.
>> 3. sets up the first BD, i.e. marks it ready.
>>=20
>> Here is what clean_tx_ring() does:
>>=20
>> 1. reads skbs from tx_queue->tx_skbuff
>> 2. checks if the *last* BD is ready. If it's still ready [to send]
>> then it it isn't transmitted, so clean_tx_ring() returns.
>> Otherwise it actually cleanups BDs. All is OK.
>>=20
>> Now, if there is just one BD, code flow:
>>=20
>> - start_xmit(): stores skb into tx_skbuff. Note that the first BD
>> (which is also the last one) isn't marked as ready, yet.
>> - clean_tx_ring(): sees that skb is not null, *and* its lstatus
>> says that it is NOT ready (like if BD was sent), so it cleans
>> it up (bad!)
>> - start_xmit(): marks BD as ready [to send], but it's too late.
>>=20
>> We can fix this simply by reordering lstatus/tx_skbuff writes.
>>=20
>> Reported-by: Martyn Welch <martyn.welch@ge.com>
>> Bisected-by: Paul Gortmaker <paul.gortmaker@windriver.com>
>> Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
>> Tested-by: Paul Gortmaker <paul.gortmaker@windriver.com>
>> Tested-by: Martyn Welch <martyn.welch@ge.com>
>=20
> Applied.
Anton,
Once this makes it into Linus's tree can you make sure we get it added =
to -stable.
- k=
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] gianfar: Fix TX ring processing on SMP machines
2010-03-03 18:18 ` Anton Vorontsov
@ 2010-06-11 8:45 ` Esben Haabendal
-1 siblings, 0 replies; 8+ messages in thread
From: Esben Haabendal @ 2010-06-11 8:45 UTC (permalink / raw)
To: Anton Vorontsov
Cc: David Miller, linuxppc-dev, netdev, Martyn Welch, Paul Gortmaker,
Sandeep Gopalpet
On Wed, Mar 3, 2010 at 8:18 PM, Anton Vorontsov
<avorontsov@ru.mvista.com> wrote:
> Starting with commit a3bc1f11e9b867a4f49505 ("gianfar: Revive SKB
> recycling") gianfar driver sooner or later stops transmitting any
> packets on SMP machines.
>
> start_xmit() prepares new skb for transmitting, generally it does
> three things:
>
> 1. sets up all BDs (marks them ready to send), except the first one.
> 2. stores skb into tx_queue->tx_skbuff so that clean_tx_ring()
> would cleanup it later.
> 3. sets up the first BD, i.e. marks it ready.
>
> Here is what clean_tx_ring() does:
>
> 1. reads skbs from tx_queue->tx_skbuff
> 2. checks if the *last* BD is ready. If it's still ready [to send]
> then it it isn't transmitted, so clean_tx_ring() returns.
> Otherwise it actually cleanups BDs. All is OK.
>
> Now, if there is just one BD, code flow:
>
> - start_xmit(): stores skb into tx_skbuff. Note that the first BD
> (which is also the last one) isn't marked as ready, yet.
> - clean_tx_ring(): sees that skb is not null, *and* its lstatus
> says that it is NOT ready (like if BD was sent), so it cleans
> it up (bad!)
> - start_xmit(): marks BD as ready [to send], but it's too late.
>
> We can fix this simply by reordering lstatus/tx_skbuff writes.
>
> Reported-by: Martyn Welch <martyn.welch@ge.com>
> Bisected-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
> Tested-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> Tested-by: Martyn Welch <martyn.welch@ge.com>
> Cc: Sandeep Gopalpet <Sandeep.Kumar@freescale.com>
> Cc: Stable <stable@vger.kernel.org> [2.6.33]
> ---
> drivers/net/gianfar.c | 5 ++++-
> 1 files changed, 4 insertions(+), 1 deletions(-)
>
> diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
> index 8bd3c9f..cccb409 100644
> --- a/drivers/net/gianfar.c
> +++ b/drivers/net/gianfar.c
> @@ -2021,7 +2021,6 @@ static int gfar_start_xmit(struct sk_buff *skb, struct net_device *dev)
> }
>
> /* setup the TxBD length and buffer pointer for the first BD */
> - tx_queue->tx_skbuff[tx_queue->skb_curtx] = skb;
> txbdp_start->bufPtr = dma_map_single(&priv->ofdev->dev, skb->data,
> skb_headlen(skb), DMA_TO_DEVICE);
>
> @@ -2053,6 +2052,10 @@ static int gfar_start_xmit(struct sk_buff *skb, struct net_device *dev)
>
> txbdp_start->lstatus = lstatus;
>
> + eieio(); /* force lstatus write before tx_skbuff */
> +
> + tx_queue->tx_skbuff[tx_queue->skb_curtx] = skb;
> +
> /* Update the current skb pointer to the next entry we will use
> * (wrapping if necessary) */
> tx_queue->skb_curtx = (tx_queue->skb_curtx + 1) &
This patch also makes gianfar work stable on mpc8313 with 2.6.33/RT_PREEMPT.
WIthout it, I see exactly the same problems as reported by Anton on SMP.
/Esben
--
Esben Haabendal, Senior Software Consultant
DoréDevelopment ApS, Ved Stranden 1, 9560 Hadsund, DK-Denmark
Phone: +45 51 92 53 93, E-mail: eha@doredevelopment.dk
WWW: http://www.doredevelopment.dk
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] gianfar: Fix TX ring processing on SMP machines
@ 2010-06-11 8:45 ` Esben Haabendal
0 siblings, 0 replies; 8+ messages in thread
From: Esben Haabendal @ 2010-06-11 8:45 UTC (permalink / raw)
To: Anton Vorontsov
Cc: Paul Gortmaker, netdev, Martyn Welch, linuxppc-dev,
Sandeep Gopalpet, David Miller
On Wed, Mar 3, 2010 at 8:18 PM, Anton Vorontsov
<avorontsov@ru.mvista.com> wrote:
> Starting with commit a3bc1f11e9b867a4f49505 ("gianfar: Revive SKB
> recycling") gianfar driver sooner or later stops transmitting any
> packets on SMP machines.
>
> start_xmit() prepares new skb for transmitting, generally it does
> three things:
>
> 1. sets up all BDs (marks them ready to send), except the first one.
> 2. stores skb into tx_queue->tx_skbuff so that clean_tx_ring()
> =A0 would cleanup it later.
> 3. sets up the first BD, i.e. marks it ready.
>
> Here is what clean_tx_ring() does:
>
> 1. reads skbs from tx_queue->tx_skbuff
> 2. checks if the *last* BD is ready. If it's still ready [to send]
> =A0 then it it isn't transmitted, so clean_tx_ring() returns.
> =A0 Otherwise it actually cleanups BDs. All is OK.
>
> Now, if there is just one BD, code flow:
>
> - start_xmit(): stores skb into tx_skbuff. Note that the first BD
> =A0(which is also the last one) isn't marked as ready, yet.
> - clean_tx_ring(): sees that skb is not null, *and* its lstatus
> =A0says that it is NOT ready (like if BD was sent), so it cleans
> =A0it up (bad!)
> - start_xmit(): marks BD as ready [to send], but it's too late.
>
> We can fix this simply by reordering lstatus/tx_skbuff writes.
>
> Reported-by: Martyn Welch <martyn.welch@ge.com>
> Bisected-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
> Tested-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> Tested-by: Martyn Welch <martyn.welch@ge.com>
> Cc: Sandeep Gopalpet <Sandeep.Kumar@freescale.com>
> Cc: Stable <stable@vger.kernel.org> [2.6.33]
> ---
> =A0drivers/net/gianfar.c | =A0 =A05 ++++-
> =A01 files changed, 4 insertions(+), 1 deletions(-)
>
> diff --git a/drivers/net/gianfar.c b/drivers/net/gianfar.c
> index 8bd3c9f..cccb409 100644
> --- a/drivers/net/gianfar.c
> +++ b/drivers/net/gianfar.c
> @@ -2021,7 +2021,6 @@ static int gfar_start_xmit(struct sk_buff *skb, str=
uct net_device *dev)
> =A0 =A0 =A0 =A0}
>
> =A0 =A0 =A0 =A0/* setup the TxBD length and buffer pointer for the first =
BD */
> - =A0 =A0 =A0 tx_queue->tx_skbuff[tx_queue->skb_curtx] =3D skb;
> =A0 =A0 =A0 =A0txbdp_start->bufPtr =3D dma_map_single(&priv->ofdev->dev, =
skb->data,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0skb_headlen(skb), DMA_TO_D=
EVICE);
>
> @@ -2053,6 +2052,10 @@ static int gfar_start_xmit(struct sk_buff *skb, st=
ruct net_device *dev)
>
> =A0 =A0 =A0 =A0txbdp_start->lstatus =3D lstatus;
>
> + =A0 =A0 =A0 eieio(); /* force lstatus write before tx_skbuff */
> +
> + =A0 =A0 =A0 tx_queue->tx_skbuff[tx_queue->skb_curtx] =3D skb;
> +
> =A0 =A0 =A0 =A0/* Update the current skb pointer to the next entry we wil=
l use
> =A0 =A0 =A0 =A0 * (wrapping if necessary) */
> =A0 =A0 =A0 =A0tx_queue->skb_curtx =3D (tx_queue->skb_curtx + 1) &
This patch also makes gianfar work stable on mpc8313 with 2.6.33/RT_PREEMPT=
.
WIthout it, I see exactly the same problems as reported by Anton on SMP.
/Esben
--=20
Esben Haabendal, Senior Software Consultant
Dor=E9Development ApS, Ved Stranden 1, 9560 Hadsund, DK-Denmark
Phone: +45 51 92 53 93, E-mail: eha@doredevelopment.dk
WWW: http://www.doredevelopment.dk
^ permalink raw reply [flat|nested] 8+ messages in thread