* Regarding tx-nocache-copy in the Sheevaplug
@ 2014-10-13 10:52 Lluís Batlle i Rossell
2014-10-13 12:26 ` Eric Dumazet
` (2 more replies)
0 siblings, 3 replies; 13+ messages in thread
From: Lluís Batlle i Rossell @ 2014-10-13 10:52 UTC (permalink / raw)
To: linux-kernel, netdev; +Cc: Carles Pagès, linux-arm-kernel
Hello,
on the 7th of January 2014 ths patch was applied:
https://lkml.org/lkml/2014/1/7/307
[PATCH v2] net: Do not enable tx-nocache-copy by default
In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be
sent corrupted. I think this machine has something special about the cache.
Enabling back this tx-nocache-copy (as it used to be before the patch) the
transfers work fine again. I think that most people, encountering this problem,
completely disable the tx offload instead of enabling back this setting.
Is this an ARM kernel problem regarding this platform?
Thank you,
Lluís
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-13 10:52 Regarding tx-nocache-copy in the Sheevaplug Lluís Batlle i Rossell @ 2014-10-13 12:26 ` Eric Dumazet 2014-10-13 12:32 ` Lluís Batlle i Rossell 2014-10-13 14:21 ` Andrew Lunn 2014-10-15 21:57 ` Benjamin Poirier 2 siblings, 1 reply; 13+ messages in thread From: Eric Dumazet @ 2014-10-13 12:26 UTC (permalink / raw) To: Lluís Batlle i Rossell Cc: linux-kernel, netdev, Carles Pagès, linux-arm-kernel On Mon, 2014-10-13 at 12:52 +0200, Lluís Batlle i Rossell wrote: > Hello, > > on the 7th of January 2014 ths patch was applied: > https://lkml.org/lkml/2014/1/7/307 > > [PATCH v2] net: Do not enable tx-nocache-copy by default > > In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be > sent corrupted. I think this machine has something special about the cache. > > Enabling back this tx-nocache-copy (as it used to be before the patch) the > transfers work fine again. I think that most people, encountering this problem, > completely disable the tx offload instead of enabling back this setting. > > Is this an ARM kernel problem regarding this platform? Which NIC and driver is this exactly ? ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-13 12:26 ` Eric Dumazet @ 2014-10-13 12:32 ` Lluís Batlle i Rossell 0 siblings, 0 replies; 13+ messages in thread From: Lluís Batlle i Rossell @ 2014-10-13 12:32 UTC (permalink / raw) To: Eric Dumazet; +Cc: linux-kernel, netdev, Carles Pagès, linux-arm-kernel On Mon, Oct 13, 2014 at 05:26:11AM -0700, Eric Dumazet wrote: > On Mon, 2014-10-13 at 12:52 +0200, Lluís Batlle i Rossell wrote: > > Hello, > > > > on the 7th of January 2014 ths patch was applied: > > https://lkml.org/lkml/2014/1/7/307 > > > > [PATCH v2] net: Do not enable tx-nocache-copy by default > > > > In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be > > sent corrupted. I think this machine has something special about the cache. > > > > Enabling back this tx-nocache-copy (as it used to be before the patch) the > > transfers work fine again. I think that most people, encountering this problem, > > completely disable the tx offload instead of enabling back this setting. > > > > Is this an ARM kernel problem regarding this platform? > > Which NIC and driver is this exactly ? According to dmesg in 3.10.1: [ 7.858872] mv643xx_eth: MV-643xx 10/100/1000 ethernet driver version 1.4 [ 7.866001] mv643xx_eth_port mv643xx_eth_port.0 eth0: port 0 with MAC address 00:50:43:01:d1:bb Regards, Lluís. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-13 10:52 Regarding tx-nocache-copy in the Sheevaplug Lluís Batlle i Rossell 2014-10-13 12:26 ` Eric Dumazet @ 2014-10-13 14:21 ` Andrew Lunn 2014-10-13 14:31 ` Lluís Batlle i Rossell 2014-10-15 21:57 ` Benjamin Poirier 2 siblings, 1 reply; 13+ messages in thread From: Andrew Lunn @ 2014-10-13 14:21 UTC (permalink / raw) To: Lluís Batlle i Rossell Cc: linux-kernel, netdev, Carles Pagès, linux-arm-kernel On Mon, Oct 13, 2014 at 12:52:46PM +0200, Lluís Batlle i Rossell wrote: > Hello, > > on the 7th of January 2014 ths patch was applied: > https://lkml.org/lkml/2014/1/7/307 > > [PATCH v2] net: Do not enable tx-nocache-copy by default > > In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be > sent corrupted. I think this machine has something special about the cache. Hi Lluís Please could you describe your test setup. I would like to try to reproduce the problem. I have a machine based on kirkwood 6282 and the same ethernet. Thanks Andrew ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-13 14:21 ` Andrew Lunn @ 2014-10-13 14:31 ` Lluís Batlle i Rossell 2014-10-13 14:49 ` Eric Dumazet 0 siblings, 1 reply; 13+ messages in thread From: Lluís Batlle i Rossell @ 2014-10-13 14:31 UTC (permalink / raw) To: Andrew Lunn; +Cc: linux-kernel, netdev, Carles Pagès, linux-arm-kernel Enabling tx offload and disabling tx-nocache-copy, making the machine *send* a lot of ssh traffic (sftp for example) makes ssh fail HMAC. It's quite easy to reproduce here. As for the hardware, it's an old sheevaplug board. On Mon, Oct 13, 2014 at 04:21:56PM +0200, Andrew Lunn wrote: > On Mon, Oct 13, 2014 at 12:52:46PM +0200, Lluís Batlle i Rossell wrote: > > Hello, > > > > on the 7th of January 2014 ths patch was applied: > > https://lkml.org/lkml/2014/1/7/307 > > > > [PATCH v2] net: Do not enable tx-nocache-copy by default > > > > In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be > > sent corrupted. I think this machine has something special about the cache. > > Hi Lluís > > Please could you describe your test setup. I would like to try to > reproduce the problem. I have a machine based on kirkwood 6282 and the > same ethernet. > > Thanks > Andrew ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-13 14:31 ` Lluís Batlle i Rossell @ 2014-10-13 14:49 ` Eric Dumazet 2014-10-13 15:48 ` Lluís Batlle i Rossell 0 siblings, 1 reply; 13+ messages in thread From: Eric Dumazet @ 2014-10-13 14:49 UTC (permalink / raw) To: Lluís Batlle i Rossell, Ezequiel Garcia Cc: Andrew Lunn, linux-kernel, netdev, Carles Pagès, linux-arm-kernel On Mon, 2014-10-13 at 16:31 +0200, Lluís Batlle i Rossell wrote: > Enabling tx offload and disabling tx-nocache-copy, making the machine *send* a > lot of ssh traffic (sftp for example) makes ssh fail HMAC. It's quite easy to > reproduce here. > > As for the hardware, it's an old sheevaplug board. Have you tried disabling TSO only, and are you using the latest kernel ? Ezequiel Garcia added lot of changes recently. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-13 14:49 ` Eric Dumazet @ 2014-10-13 15:48 ` Lluís Batlle i Rossell 0 siblings, 0 replies; 13+ messages in thread From: Lluís Batlle i Rossell @ 2014-10-13 15:48 UTC (permalink / raw) To: Eric Dumazet Cc: Ezequiel Garcia, Andrew Lunn, linux-kernel, netdev, Carles Pagès, linux-arm-kernel On Mon, Oct 13, 2014 at 07:49:19AM -0700, Eric Dumazet wrote: > On Mon, 2014-10-13 at 16:31 +0200, Lluís Batlle i Rossell wrote: > > Enabling tx offload and disabling tx-nocache-copy, making the machine *send* a > > lot of ssh traffic (sftp for example) makes ssh fail HMAC. It's quite easy to > > reproduce here. > > > > As for the hardware, it's an old sheevaplug board. > > > Have you tried disabling TSO only, and are you using the latest kernel ? > > Ezequiel Garcia added lot of changes recently. > > Is TSO TCP segmentation offload? It's disabled. The kernel is 3.16.3 (debian). https://packages.debian.org/testing/kernel/linux-image-3.16-2-kirkwood ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-13 10:52 Regarding tx-nocache-copy in the Sheevaplug Lluís Batlle i Rossell 2014-10-13 12:26 ` Eric Dumazet 2014-10-13 14:21 ` Andrew Lunn @ 2014-10-15 21:57 ` Benjamin Poirier 2014-10-15 22:45 ` Eric Dumazet 2 siblings, 1 reply; 13+ messages in thread From: Benjamin Poirier @ 2014-10-15 21:57 UTC (permalink / raw) To: Lluís Batlle i Rossell Cc: linux-kernel, netdev, Carles Pagès, linux-arm-kernel On 2014/10/13 12:52, Lluís Batlle i Rossell wrote: > Hello, > > on the 7th of January 2014 ths patch was applied: > https://lkml.org/lkml/2014/1/7/307 > > [PATCH v2] net: Do not enable tx-nocache-copy by default > > In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be > sent corrupted. I think this machine has something special about the cache. > > Enabling back this tx-nocache-copy (as it used to be before the patch) the > transfers work fine again. I think that most people, encountering this problem, > completely disable the tx offload instead of enabling back this setting. > > Is this an ARM kernel problem regarding this platform? This is odd, only x86 defines ARCH_HAS_NOCACHE_UACCESS. On arm, skb_do_copy_data_nocache() should end up using __copy_from_user() regardless of tx-nocache-copy. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-15 21:57 ` Benjamin Poirier @ 2014-10-15 22:45 ` Eric Dumazet 2014-10-16 17:34 ` Benjamin Poirier 0 siblings, 1 reply; 13+ messages in thread From: Eric Dumazet @ 2014-10-15 22:45 UTC (permalink / raw) To: Benjamin Poirier Cc: Lluís Batlle i Rossell, linux-kernel, netdev, Carles Pagès, linux-arm-kernel On Wed, 2014-10-15 at 14:57 -0700, Benjamin Poirier wrote: > On 2014/10/13 12:52, Lluís Batlle i Rossell wrote: > > Hello, > > > > on the 7th of January 2014 ths patch was applied: > > https://lkml.org/lkml/2014/1/7/307 > > > > [PATCH v2] net: Do not enable tx-nocache-copy by default > > > > In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be > > sent corrupted. I think this machine has something special about the cache. > > > > Enabling back this tx-nocache-copy (as it used to be before the patch) the > > transfers work fine again. I think that most people, encountering this problem, > > completely disable the tx offload instead of enabling back this setting. > > > > Is this an ARM kernel problem regarding this platform? > > This is odd, only x86 defines ARCH_HAS_NOCACHE_UACCESS. On arm, > skb_do_copy_data_nocache() should end up using __copy_from_user() > regardless of tx-nocache-copy. kmap_atomic()/kunmap_atomic() is missing, so we lack __cpuc_flush_dcache_area() operations. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-15 22:45 ` Eric Dumazet @ 2014-10-16 17:34 ` Benjamin Poirier 2014-10-16 17:46 ` Lluís Batlle i Rossell 2014-10-16 17:48 ` Eric Dumazet 0 siblings, 2 replies; 13+ messages in thread From: Benjamin Poirier @ 2014-10-16 17:34 UTC (permalink / raw) To: Eric Dumazet Cc: Lluís Batlle i Rossell, linux-kernel, netdev, Carles Pagès, linux-arm-kernel On 2014/10/15 15:45, Eric Dumazet wrote: > On Wed, 2014-10-15 at 14:57 -0700, Benjamin Poirier wrote: > > On 2014/10/13 12:52, Lluís Batlle i Rossell wrote: > > > Hello, > > > > > > on the 7th of January 2014 ths patch was applied: > > > https://lkml.org/lkml/2014/1/7/307 > > > > > > [PATCH v2] net: Do not enable tx-nocache-copy by default > > > > > > In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be > > > sent corrupted. I think this machine has something special about the cache. > > > > > > Enabling back this tx-nocache-copy (as it used to be before the patch) the > > > transfers work fine again. I think that most people, encountering this problem, > > > completely disable the tx offload instead of enabling back this setting. > > > > > > Is this an ARM kernel problem regarding this platform? > > > > This is odd, only x86 defines ARCH_HAS_NOCACHE_UACCESS. On arm, > > skb_do_copy_data_nocache() should end up using __copy_from_user() > > regardless of tx-nocache-copy. > > kmap_atomic()/kunmap_atomic() is missing, so we lack > __cpuc_flush_dcache_area() operations. > You lost me there. 1) I don't see the link 2) It seems kmap_atomic and so on are there: $ grep kmap_atomic System.map-3.16-2-kirkwood c0014838 T kmap_atomic c001491c T kmap_atomic_pfn c00149a4 T kmap_atomic_to_page MACH_KIRKWOOD selects CPU_FEROCEON which has __cpuc_flush_dcache_area -> cpu_cache.flush_kern_dcache_area -> feroceon_flush_kern_dcache_area ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-16 17:34 ` Benjamin Poirier @ 2014-10-16 17:46 ` Lluís Batlle i Rossell 2014-10-17 20:55 ` Benjamin Poirier 2014-10-16 17:48 ` Eric Dumazet 1 sibling, 1 reply; 13+ messages in thread From: Lluís Batlle i Rossell @ 2014-10-16 17:46 UTC (permalink / raw) To: Benjamin Poirier Cc: Eric Dumazet, linux-kernel, netdev, Carles Pagès, linux-arm-kernel On Thu, Oct 16, 2014 at 10:34:01AM -0700, Benjamin Poirier wrote: > On 2014/10/15 15:45, Eric Dumazet wrote: > > On Wed, 2014-10-15 at 14:57 -0700, Benjamin Poirier wrote: > > > On 2014/10/13 12:52, Lluís Batlle i Rossell wrote: > > > > Hello, > > > > > > > > on the 7th of January 2014 ths patch was applied: > > > > https://lkml.org/lkml/2014/1/7/307 > > > > > > > > [PATCH v2] net: Do not enable tx-nocache-copy by default > > > > > > > > In the Sheevaplug (ARM Feroceon 88FR131 from Marvell) this made packets to be > > > > sent corrupted. I think this machine has something special about the cache. > > > > > > > > Enabling back this tx-nocache-copy (as it used to be before the patch) the > > > > transfers work fine again. I think that most people, encountering this problem, > > > > completely disable the tx offload instead of enabling back this setting. > > > > > > > > Is this an ARM kernel problem regarding this platform? > > > > > > This is odd, only x86 defines ARCH_HAS_NOCACHE_UACCESS. On arm, > > > skb_do_copy_data_nocache() should end up using __copy_from_user() > > > regardless of tx-nocache-copy. > > > > kmap_atomic()/kunmap_atomic() is missing, so we lack > > __cpuc_flush_dcache_area() operations. > > > > You lost me there. > 1) I don't see the link > 2) It seems kmap_atomic and so on are there: > $ grep kmap_atomic System.map-3.16-2-kirkwood > c0014838 T kmap_atomic > c001491c T kmap_atomic_pfn > c00149a4 T kmap_atomic_to_page > > MACH_KIRKWOOD selects CPU_FEROCEON which has > __cpuc_flush_dcache_area -> > cpu_cache.flush_kern_dcache_area -> > feroceon_flush_kern_dcache_area Hello all, it seems I was a bit wrong - although enabling back tx-nocache-copy makes the tx-errors happen much less often (ssh complaining about HMAC), they still happen. It seems that something was introduced in some recent kernels that broke the tx offload. I have no idea what it can be, but since 2.6 until at least 3.10 the network driver worked fine with tx offload in this sheevaplug board. Regards, Lluís. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-16 17:46 ` Lluís Batlle i Rossell @ 2014-10-17 20:55 ` Benjamin Poirier 0 siblings, 0 replies; 13+ messages in thread From: Benjamin Poirier @ 2014-10-17 20:55 UTC (permalink / raw) To: Lluís Batlle i Rossell Cc: Eric Dumazet, linux-kernel, netdev, Carles Pagès, linux-arm-kernel On 2014/10/16 19:46, Lluís Batlle i Rossell wrote: [...] > > Hello all, > > it seems I was a bit wrong - although enabling back tx-nocache-copy makes the > tx-errors happen much less often (ssh complaining about HMAC), they still > happen. It seems that something was introduced in some recent kernels that broke > the tx offload. > > I have no idea what it can be, but since 2.6 until at least 3.10 the network > driver worked fine with tx offload in this sheevaplug board. It's not the most pleasant alternative but if you can be sure enough whether the problem is occurring or not, you could try bisecting, possibly limiting the bisection to mv643xx $ git bisect start v3.16.3 v3.10 -- drivers/net/ethernet/marvell/mv643xx_eth.c Bisecting: 16 revisions left to test after this (roughly 4 steps) The problem might be outside of the driver though. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Regarding tx-nocache-copy in the Sheevaplug 2014-10-16 17:34 ` Benjamin Poirier 2014-10-16 17:46 ` Lluís Batlle i Rossell @ 2014-10-16 17:48 ` Eric Dumazet 1 sibling, 0 replies; 13+ messages in thread From: Eric Dumazet @ 2014-10-16 17:48 UTC (permalink / raw) To: Benjamin Poirier Cc: Lluís Batlle i Rossell, linux-kernel, netdev, Carles Pagès, linux-arm-kernel On Thu, 2014-10-16 at 10:34 -0700, Benjamin Poirier wrote: > On 2014/10/15 15:45, Eric Dumazet wrote: > > kmap_atomic()/kunmap_atomic() is missing, so we lack > > __cpuc_flush_dcache_area() operations. > > > > You lost me there. > 1) I don't see the link > 2) It seems kmap_atomic and so on are there: > $ grep kmap_atomic System.map-3.16-2-kirkwood > c0014838 T kmap_atomic > c001491c T kmap_atomic_pfn > c00149a4 T kmap_atomic_to_page > > MACH_KIRKWOOD selects CPU_FEROCEON which has > __cpuc_flush_dcache_area -> > cpu_cache.flush_kern_dcache_area -> > feroceon_flush_kern_dcache_area I meant to put a '?' instead of a '.' Note that tcp does a copy, using : ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2014-10-17 20:55 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-10-13 10:52 Regarding tx-nocache-copy in the Sheevaplug Lluís Batlle i Rossell 2014-10-13 12:26 ` Eric Dumazet 2014-10-13 12:32 ` Lluís Batlle i Rossell 2014-10-13 14:21 ` Andrew Lunn 2014-10-13 14:31 ` Lluís Batlle i Rossell 2014-10-13 14:49 ` Eric Dumazet 2014-10-13 15:48 ` Lluís Batlle i Rossell 2014-10-15 21:57 ` Benjamin Poirier 2014-10-15 22:45 ` Eric Dumazet 2014-10-16 17:34 ` Benjamin Poirier 2014-10-16 17:46 ` Lluís Batlle i Rossell 2014-10-17 20:55 ` Benjamin Poirier 2014-10-16 17:48 ` Eric Dumazet
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).