* [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
@ 2015-01-26 21:12 ` Iyappan Subramanian
0 siblings, 0 replies; 15+ messages in thread
From: Iyappan Subramanian @ 2015-01-26 21:12 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Jan 22, 2015 at 2:50 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Thu, 2015-01-22 at 12:03 -0800, Iyappan Subramanian wrote:
>> This patch fixes the following kernel crash,
>>
>> WARNING: CPU: 2 PID: 0 at net/ipv4/tcp_input.c:3079 tcp_clean_rtx_queue+0x658/0x80c()
>> Call trace:
>
>>
>> Software writes poison data into the descriptor bytes[15:8] and upon
>> receiving the interrupt, if those bytes are overwritten by the hardware with
>> the valid data, software also reads bytes[7:0] and executes receive/tx
>> completion logic.
>>
>> If the CPU executes the above two reads in out of order fashion, then the
>> bytes[7:0] will have older data and causing the kernel panic. We have to
>> force the order of the reads and thus this patch introduces read memory
>> barrier between these reads.
>>
>> Signed-off-by: Iyappan Subramanian <isubramanian@apm.com>
>> Signed-off-by: Keyur Chudgar <kchudgar@apm.com>
>> Tested-by: Mark Langsdorf <mlangsdo@redhat.com>
>> ---
>> drivers/net/ethernet/apm/xgene/xgene_enet_main.c | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
>> index 83a5028..3622cdb 100644
>> --- a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
>> +++ b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
>> @@ -369,6 +369,8 @@ static int xgene_enet_process_ring(struct xgene_enet_desc_ring *ring,
>> if (unlikely(xgene_enet_is_desc_slot_empty(raw_desc)))
>> break;
>>
>> + /* read fpqnum field after dataaddr field */
>> + smp_rmb();
>> if (is_rx_desc(raw_desc))
>> ret = xgene_enet_rx_frame(ring, raw_desc);
>> else
>
> Reading your changelog, it looks like you need a plain rmb() here.
rmb() translates into dsb, which in arm64 serializes everything
including instructions and thus expensive compared to dmb.
Do you see any issue with smp_rmb() (which translates into dmb) ?
>
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
@ 2015-01-26 21:12 ` Iyappan Subramanian
0 siblings, 0 replies; 15+ messages in thread
From: Iyappan Subramanian @ 2015-01-26 21:12 UTC (permalink / raw)
To: Eric Dumazet
Cc: David Miller, netdev, linux-kernel, linux-arm-kernel, mlangsdo,
patches, Keyur Chudgar
On Thu, Jan 22, 2015 at 2:50 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Thu, 2015-01-22 at 12:03 -0800, Iyappan Subramanian wrote:
>> This patch fixes the following kernel crash,
>>
>> WARNING: CPU: 2 PID: 0 at net/ipv4/tcp_input.c:3079 tcp_clean_rtx_queue+0x658/0x80c()
>> Call trace:
>
>>
>> Software writes poison data into the descriptor bytes[15:8] and upon
>> receiving the interrupt, if those bytes are overwritten by the hardware with
>> the valid data, software also reads bytes[7:0] and executes receive/tx
>> completion logic.
>>
>> If the CPU executes the above two reads in out of order fashion, then the
>> bytes[7:0] will have older data and causing the kernel panic. We have to
>> force the order of the reads and thus this patch introduces read memory
>> barrier between these reads.
>>
>> Signed-off-by: Iyappan Subramanian <isubramanian@apm.com>
>> Signed-off-by: Keyur Chudgar <kchudgar@apm.com>
>> Tested-by: Mark Langsdorf <mlangsdo@redhat.com>
>> ---
>> drivers/net/ethernet/apm/xgene/xgene_enet_main.c | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
>> index 83a5028..3622cdb 100644
>> --- a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
>> +++ b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
>> @@ -369,6 +369,8 @@ static int xgene_enet_process_ring(struct xgene_enet_desc_ring *ring,
>> if (unlikely(xgene_enet_is_desc_slot_empty(raw_desc)))
>> break;
>>
>> + /* read fpqnum field after dataaddr field */
>> + smp_rmb();
>> if (is_rx_desc(raw_desc))
>> ret = xgene_enet_rx_frame(ring, raw_desc);
>> else
>
> Reading your changelog, it looks like you need a plain rmb() here.
rmb() translates into dsb, which in arm64 serializes everything
including instructions and thus expensive compared to dmb.
Do you see any issue with smp_rmb() (which translates into dmb) ?
>
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
2015-01-26 21:12 ` Iyappan Subramanian
(?)
@ 2015-01-26 21:27 ` Eric Dumazet
-1 siblings, 0 replies; 15+ messages in thread
From: Eric Dumazet @ 2015-01-26 21:27 UTC (permalink / raw)
To: Iyappan Subramanian
Cc: David Miller, netdev, linux-kernel, linux-arm-kernel, mlangsdo,
patches, Keyur Chudgar
On Mon, 2015-01-26 at 13:12 -0800, Iyappan Subramanian wrote:
> On Thu, Jan 22, 2015 at 2:50 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > On Thu, 2015-01-22 at 12:03 -0800, Iyappan Subramanian wrote:
> >> This patch fixes the following kernel crash,
> >>
> >> WARNING: CPU: 2 PID: 0 at net/ipv4/tcp_input.c:3079 tcp_clean_rtx_queue+0x658/0x80c()
> >> Call trace:
> >
> >>
> >> Software writes poison data into the descriptor bytes[15:8] and upon
> >> receiving the interrupt, if those bytes are overwritten by the hardware with
> >> the valid data, software also reads bytes[7:0] and executes receive/tx
> >> completion logic.
> >>
> >> If the CPU executes the above two reads in out of order fashion, then the
> >> bytes[7:0] will have older data and causing the kernel panic. We have to
> >> force the order of the reads and thus this patch introduces read memory
> >> barrier between these reads.
> >>
> >> Signed-off-by: Iyappan Subramanian <isubramanian@apm.com>
> >> Signed-off-by: Keyur Chudgar <kchudgar@apm.com>
> >> Tested-by: Mark Langsdorf <mlangsdo@redhat.com>
> >> ---
> >> drivers/net/ethernet/apm/xgene/xgene_enet_main.c | 2 ++
> >> 1 file changed, 2 insertions(+)
> >>
> >> diff --git a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> index 83a5028..3622cdb 100644
> >> --- a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> +++ b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> @@ -369,6 +369,8 @@ static int xgene_enet_process_ring(struct xgene_enet_desc_ring *ring,
> >> if (unlikely(xgene_enet_is_desc_slot_empty(raw_desc)))
> >> break;
> >>
> >> + /* read fpqnum field after dataaddr field */
> >> + smp_rmb();
> >> if (is_rx_desc(raw_desc))
> >> ret = xgene_enet_rx_frame(ring, raw_desc);
> >> else
> >
> > Reading your changelog, it looks like you need a plain rmb() here.
>
> rmb() translates into dsb, which in arm64 serializes everything
> including instructions and thus expensive compared to dmb.
>
> Do you see any issue with smp_rmb() (which translates into dmb) ?
What happens if you compile a kernel with CONFIG_SMP=n ?
Most drivers in drivers/net use rmb() in this case, not smp_rmb() or
barrier()
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
@ 2015-01-26 21:27 ` Eric Dumazet
0 siblings, 0 replies; 15+ messages in thread
From: Eric Dumazet @ 2015-01-26 21:27 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, 2015-01-26 at 13:12 -0800, Iyappan Subramanian wrote:
> On Thu, Jan 22, 2015 at 2:50 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > On Thu, 2015-01-22 at 12:03 -0800, Iyappan Subramanian wrote:
> >> This patch fixes the following kernel crash,
> >>
> >> WARNING: CPU: 2 PID: 0 at net/ipv4/tcp_input.c:3079 tcp_clean_rtx_queue+0x658/0x80c()
> >> Call trace:
> >
> >>
> >> Software writes poison data into the descriptor bytes[15:8] and upon
> >> receiving the interrupt, if those bytes are overwritten by the hardware with
> >> the valid data, software also reads bytes[7:0] and executes receive/tx
> >> completion logic.
> >>
> >> If the CPU executes the above two reads in out of order fashion, then the
> >> bytes[7:0] will have older data and causing the kernel panic. We have to
> >> force the order of the reads and thus this patch introduces read memory
> >> barrier between these reads.
> >>
> >> Signed-off-by: Iyappan Subramanian <isubramanian@apm.com>
> >> Signed-off-by: Keyur Chudgar <kchudgar@apm.com>
> >> Tested-by: Mark Langsdorf <mlangsdo@redhat.com>
> >> ---
> >> drivers/net/ethernet/apm/xgene/xgene_enet_main.c | 2 ++
> >> 1 file changed, 2 insertions(+)
> >>
> >> diff --git a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> index 83a5028..3622cdb 100644
> >> --- a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> +++ b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> @@ -369,6 +369,8 @@ static int xgene_enet_process_ring(struct xgene_enet_desc_ring *ring,
> >> if (unlikely(xgene_enet_is_desc_slot_empty(raw_desc)))
> >> break;
> >>
> >> + /* read fpqnum field after dataaddr field */
> >> + smp_rmb();
> >> if (is_rx_desc(raw_desc))
> >> ret = xgene_enet_rx_frame(ring, raw_desc);
> >> else
> >
> > Reading your changelog, it looks like you need a plain rmb() here.
>
> rmb() translates into dsb, which in arm64 serializes everything
> including instructions and thus expensive compared to dmb.
>
> Do you see any issue with smp_rmb() (which translates into dmb) ?
What happens if you compile a kernel with CONFIG_SMP=n ?
Most drivers in drivers/net use rmb() in this case, not smp_rmb() or
barrier()
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
@ 2015-01-26 21:27 ` Eric Dumazet
0 siblings, 0 replies; 15+ messages in thread
From: Eric Dumazet @ 2015-01-26 21:27 UTC (permalink / raw)
To: Iyappan Subramanian
Cc: David Miller, netdev, linux-kernel, linux-arm-kernel, mlangsdo,
patches, Keyur Chudgar
On Mon, 2015-01-26 at 13:12 -0800, Iyappan Subramanian wrote:
> On Thu, Jan 22, 2015 at 2:50 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > On Thu, 2015-01-22 at 12:03 -0800, Iyappan Subramanian wrote:
> >> This patch fixes the following kernel crash,
> >>
> >> WARNING: CPU: 2 PID: 0 at net/ipv4/tcp_input.c:3079 tcp_clean_rtx_queue+0x658/0x80c()
> >> Call trace:
> >
> >>
> >> Software writes poison data into the descriptor bytes[15:8] and upon
> >> receiving the interrupt, if those bytes are overwritten by the hardware with
> >> the valid data, software also reads bytes[7:0] and executes receive/tx
> >> completion logic.
> >>
> >> If the CPU executes the above two reads in out of order fashion, then the
> >> bytes[7:0] will have older data and causing the kernel panic. We have to
> >> force the order of the reads and thus this patch introduces read memory
> >> barrier between these reads.
> >>
> >> Signed-off-by: Iyappan Subramanian <isubramanian@apm.com>
> >> Signed-off-by: Keyur Chudgar <kchudgar@apm.com>
> >> Tested-by: Mark Langsdorf <mlangsdo@redhat.com>
> >> ---
> >> drivers/net/ethernet/apm/xgene/xgene_enet_main.c | 2 ++
> >> 1 file changed, 2 insertions(+)
> >>
> >> diff --git a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> index 83a5028..3622cdb 100644
> >> --- a/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> +++ b/drivers/net/ethernet/apm/xgene/xgene_enet_main.c
> >> @@ -369,6 +369,8 @@ static int xgene_enet_process_ring(struct xgene_enet_desc_ring *ring,
> >> if (unlikely(xgene_enet_is_desc_slot_empty(raw_desc)))
> >> break;
> >>
> >> + /* read fpqnum field after dataaddr field */
> >> + smp_rmb();
> >> if (is_rx_desc(raw_desc))
> >> ret = xgene_enet_rx_frame(ring, raw_desc);
> >> else
> >
> > Reading your changelog, it looks like you need a plain rmb() here.
>
> rmb() translates into dsb, which in arm64 serializes everything
> including instructions and thus expensive compared to dmb.
>
> Do you see any issue with smp_rmb() (which translates into dmb) ?
What happens if you compile a kernel with CONFIG_SMP=n ?
Most drivers in drivers/net use rmb() in this case, not smp_rmb() or
barrier()
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
2015-01-26 21:27 ` Eric Dumazet
(?)
@ 2015-01-26 21:32 ` Eric Dumazet
-1 siblings, 0 replies; 15+ messages in thread
From: Eric Dumazet @ 2015-01-26 21:32 UTC (permalink / raw)
To: Iyappan Subramanian
Cc: David Miller, netdev, linux-kernel, linux-arm-kernel, mlangsdo,
patches, Keyur Chudgar
On Mon, 2015-01-26 at 13:27 -0800, Eric Dumazet wrote:
> What happens if you compile a kernel with CONFIG_SMP=n ?
>
>
> Most drivers in drivers/net use rmb() in this case, not smp_rmb() or
> barrier()
Note that dma_rmb() was recently added as well.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
@ 2015-01-26 21:32 ` Eric Dumazet
0 siblings, 0 replies; 15+ messages in thread
From: Eric Dumazet @ 2015-01-26 21:32 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, 2015-01-26 at 13:27 -0800, Eric Dumazet wrote:
> What happens if you compile a kernel with CONFIG_SMP=n ?
>
>
> Most drivers in drivers/net use rmb() in this case, not smp_rmb() or
> barrier()
Note that dma_rmb() was recently added as well.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
@ 2015-01-26 21:32 ` Eric Dumazet
0 siblings, 0 replies; 15+ messages in thread
From: Eric Dumazet @ 2015-01-26 21:32 UTC (permalink / raw)
To: Iyappan Subramanian
Cc: David Miller, netdev, linux-kernel, linux-arm-kernel, mlangsdo,
patches, Keyur Chudgar
On Mon, 2015-01-26 at 13:27 -0800, Eric Dumazet wrote:
> What happens if you compile a kernel with CONFIG_SMP=n ?
>
>
> Most drivers in drivers/net use rmb() in this case, not smp_rmb() or
> barrier()
Note that dma_rmb() was recently added as well.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
2015-01-26 21:12 ` Iyappan Subramanian
@ 2015-01-26 22:34 ` David Miller
-1 siblings, 0 replies; 15+ messages in thread
From: David Miller @ 2015-01-26 22:34 UTC (permalink / raw)
To: isubramanian
Cc: eric.dumazet, netdev, linux-kernel, linux-arm-kernel, mlangsdo,
patches, kchudgar
From: Iyappan Subramanian <isubramanian@apm.com>
Date: Mon, 26 Jan 2015 13:12:23 -0800
>>> @@ -369,6 +369,8 @@ static int xgene_enet_process_ring(struct xgene_enet_desc_ring *ring,
>>> if (unlikely(xgene_enet_is_desc_slot_empty(raw_desc)))
>>> break;
>>>
>>> + /* read fpqnum field after dataaddr field */
>>> + smp_rmb();
>>> if (is_rx_desc(raw_desc))
>>> ret = xgene_enet_rx_frame(ring, raw_desc);
>>> else
>>
>> Reading your changelog, it looks like you need a plain rmb() here.
>
> rmb() translates into dsb, which in arm64 serializes everything
> including instructions and thus expensive compared to dmb.
>
> Do you see any issue with smp_rmb() (which translates into dmb) ?
smp_rmb() is not appropriate. You're not serializing accesses between
two cpus, you're serializing the cpu with the device.
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH] drivers: net: xgene: fix: Out of order descriptor bytes read
@ 2015-01-26 22:34 ` David Miller
0 siblings, 0 replies; 15+ messages in thread
From: David Miller @ 2015-01-26 22:34 UTC (permalink / raw)
To: linux-arm-kernel
From: Iyappan Subramanian <isubramanian@apm.com>
Date: Mon, 26 Jan 2015 13:12:23 -0800
>>> @@ -369,6 +369,8 @@ static int xgene_enet_process_ring(struct xgene_enet_desc_ring *ring,
>>> if (unlikely(xgene_enet_is_desc_slot_empty(raw_desc)))
>>> break;
>>>
>>> + /* read fpqnum field after dataaddr field */
>>> + smp_rmb();
>>> if (is_rx_desc(raw_desc))
>>> ret = xgene_enet_rx_frame(ring, raw_desc);
>>> else
>>
>> Reading your changelog, it looks like you need a plain rmb() here.
>
> rmb() translates into dsb, which in arm64 serializes everything
> including instructions and thus expensive compared to dmb.
>
> Do you see any issue with smp_rmb() (which translates into dmb) ?
smp_rmb() is not appropriate. You're not serializing accesses between
two cpus, you're serializing the cpu with the device.
^ permalink raw reply [flat|nested] 15+ messages in thread